Harry Potter-inspired moving photo frames will add charm to your living room
A new tool can make a person from a 2D image run, walk or jump out of the frame.
By PTI |
Agencies
Pablo Picasso’s “Untitled” (1939) steps out of the frame. (University of Washington)
WASHINGTON: Our photographs may soon resemble those from the world of Harry Potter, as scientists have developed a tool that can make a person from a 2D image run, walk or jump out of the frame. Their algorithm, called Photo Wake-Up, also allows users to view the animation in three dimensions using augmented reality tools.
"This is a very hard fundamental problem in computer vision," said Ira Kemelmacher-Shlizerman, an associate professor at the University of Washington in the US.
"The big challenge here is that the input is only from a single camera position, so part of the person is invisible. Our work combines technical advancement on an open problem in the field with artistic creative visualisation," Kemelmacher-Shlizerman said.
Previously, researchers thought it would be impossible to animate a person running out of a single photo.
"There is some previous work that tries to create a 3D character using multiple viewpoints," said Brian Curless, a professor at University of Washington.
"But you still couldn't bring someone to life and have them run out of a scene, and you couldn't bring AR into it. It was really surprising that we could get some compelling results with using just one photo," said Curless.
ADVERTISEMENT
Given a single photo as input (far left top), the team created a 3D animatable version of the subject, which can now walk towards the viewer (top centre & right). The 3D result can be experienced in augmented reality (bottom) The applications of Photo Wake-Up are numerous, the team said.
The researchers envision this could lead to a new way for gamers to create avatars that actually look like them, a method for visitors to interact with paintings in an art museum -- say sitting down to have tea with Mona Lisa -- or something that lets children to bring their drawings to life.
Examples in the research paper include animating the Golden State Warriors' Stephen Curry to run off the court, Paul McCartney to leap off the cover of the "Help!" album and Matisse's "Icarus" to leave his frame.
ADVERTISEMENT
To make the magic a reality, Photo Wake-Up starts by identifying a person in an image and making a mask of the body's outline. From there, it matches a 3D template to the subject's body position.
Then the algorithm does something surprising: In order to warp the template so that it actually looks like the person in the photo, it projects the 3D person back into 2D.
ADVERTISEMENT
Photo Wake-Up stores 3D information for each pixel: its distance from the camera or artist and how a person's joints are connected together.
Once the template has been warped to match the person's shape, the algorithm pastes on the texture -- the colours from the image.
It also generates the back of the person by using information from the image and the 3D template. Then the tool stitches the two sides together to make a 3D person who will be able to turn around.
Once the 3D character is ready to run, the algorithm needs to set up the background so that the character does not leave a blank space behind.
Photo Wake-Up fills in the hole behind the person by borrowing information from other parts of the image.
Right now Photo Wake-Up works best with images of people facing forward, and can animate both artistic creations and photographs of real people.
The algorithm can also handle some photos where people's arms are blocking part of their bodies, but it is not yet capable of animating people who have their legs crossed or who are blocking large parts of themselves.
6 Amazing Augmented Reality Experiences On iPad That Are Free
1/7
Even though augmented reality (AR) apps have been around for a long time, we’re still only beginning to scratch the surface of possibilities. New iPads are among the best way to experience AR thanks to the large screens, faster processors and Apple’s ARkit framework. Here are some genuinely astonishing AR apps that are available completely free.
Even though augmented reality (AR) apps have been around for a long time, we’re still only beginning to scratch the surface of possibilities. New iPads are among the best way to experience AR thanks ..
Read More
Using tape measures is so old school now that you have AR to help. It sounds almost unbelievable but you can actually measure (with a fair degree of accuracy) things like a person’s height, the diameter of a circular table, the length of a sofa and even the height of a building. AirMeasure gives you a point and shoot ruler — just tap to start and move the device along the plane you want to measure. But that’s not all, you can use it to create a floor plan, project a level line (to see whether your painting is hung straight), measure distance to a far away wall (like a laser distance measurement tool) and much more. Our favourite is the virtual furniture — you can place things like tables, chairs, beds, desks, shelves and so on into your room. Then walk around them to see how they look from different parts of the room.
(Image: http://armeasure.com)
Using tape measures is so old school now that you have AR to help. It sounds almost unbelievable but you can actually measure (with a fair degree of accuracy) things like a person’s height, the diame..
Read More
Holo is short for holograms — actual 3D animals and people you can place into your real surroundings. Some of the holos you can download for free include Spiderman, a zombie, tigers, celebrities and even Bollywood dancers. The developer is constantly adding more holos that you can download to your device. Holos are dynamic, which means they’re not static images but accurate, moving avatars. Once you scan a flat surface and place a holo, you can rotate it and scale it up or down in size to match real world sizes. Then you place yourself into the frame and snap a photo or capture a short video. In the right kind of lighting, it can look quite convincing too. Holo is also available for Android.
(Image: https://thisisholo.com)
Holo is short for holograms — actual 3D animals and people you can place into your real surroundings. Some of the holos you can download for free include Spiderman, a zombie, tigers, celebrities and ..
Read More
Learning is a lot more fun in AR and if a complex subject like math can be visually represented in your real world, it might make concepts a lot clearer. GeoGebra places math objects in your world so that you can explore them from different angles. You can place solid objects into your view while little exercises teach you about pyramids, prisms and cubes. More advanced users can look at the impossible Penrose triangle, 3D functions, spiral staircase and Sierpinski pyramid among others. You will have to physically move around the virtual objects to complete the tasks and take the required screenshots.
(Image: www.geogebra.org)
Learning is a lot more fun in AR and if a complex subject like math can be visually represented in your real world, it might make concepts a lot clearer. GeoGebra places math objects in your world so..
Read More
This app is a great example of how AR can help solve problems even in long distance. If you’re tech savvy, you’ve probably been asked for help with common tech issues over the phone. Problem is, over the phone, it’s really hard to explain to someone which button to tap, which menu item to click and which port to use. Vuforia’s Chalk opens up an AR guided private chat between you and the person you’re helping. They can activate their camera and you can doodle with virtual chalk right into their view — making circles, drawing arrows and illustrating as needed. But that’s not even the best part. Whatever you draw on sticks to the item you drew it on, so even if the camera moves around, the instruction stays true and correct.
(Image: https://chalk.vuforia.com)
This app is a great example of how AR can help solve problems even in long distance. If you’re tech savvy, you’ve probably been asked for help with common tech issues over the phone. Problem is, over..
Read More
Imagine seeing an entire, living landscape on your table. You can zoom in, look around and learn more about why free flowing rivers are so important. Through helpful little floating tips, you learn more about the journey of water, how it impacts the area through which it flows, the people it affects and how building dams can cause damage to delicate ecological systems. Place a dam and you can see (in accelerated time) how the river changes and how everything else is affected downstream. This app is the WWF’s concerted effort to educate about how human development can either be harmful or sustainable.
(Image: www.worldwildlife.org)
Imagine seeing an entire, living landscape on your table. You can zoom in, look around and learn more about why free flowing rivers are so important. Through helpful little floating tips, you learn m..
Read More
Graffiti in a public place can help beautify but it can also be an eyesore, with authorities stuck in a never-ending cycle of removing unwanted art.
What if that graffiti only existed in the virtual world and was only visible through an augmented reality app? That’s the idea behind World Brush.
You open up the app, hold it up and start drawing in virtual space. You should do this somewhere outdoors and your art will be geo-locked — this means it’s approximate GPS location is tagged and if another World Brush user comes along at a later date and holds up the app, he/she will be able to see and walk around what you drew. Another similar app without the GPS position (but more brush options) is 3DBrush — Augmented Reality.
(Image: https://itunes.apple.com)
Graffiti in a public place can help beautify but it can also be an eyesore, with authorities stuck in a never-ending cycle of removing unwanted art.
What if that graffiti only existed in the virtual..
