And then along came . A few weeks ago, the Silicon Valley startup made waves when it announced its plans to release a commercial "light-field camera" this fall. It's the long-sought-after realization of a technology that would allow you to refocus pictures after taking them, turning that blurry photo of a birthday party into a properly focused photographic masterpiece. If it works, it's a huge advance in camera hardware, doing away with the need for a camera to do the focusing. "Theoretically, all we want is a button," Doug Lanman, a post-doc in the Camera Culture group at MIT's Media Lab (which is not affiliated with Lytro) says. "Everything else you should be able to do after the fact."
The man behind Lytro is Ren Ng, who began working with light-field photography at Stanford as a PhD candidate. Ng wrote his dissertation on the subject in 2006, and he decided to start Lytro to create a commercial version of what he'd been working on in the academic world.
The technology is based on the idea of a light field, or "the amount of light traveling in every direction in every point in space," Ng says. While a conventional camera captures just a two-dimensional image of the light as it enters the camera, a light-field camera also captures the direction that the light was coming from.
A normal camera has a lens, a sensor for recording the image, an aperture to allow a certain amount of light into the camera and a shutter to time how long the sensor is exposed to that light. All those working in concert create (hopefully) a well-lit single image. But in a light-field camera, there's an extra layer called a microlens array, which is placed between the sensor and the lens. This layer is composed of multiple smaller lenses, which allow the sensor to record more information about the incoming rays of light, including the light coming from different distances away from the camera.
That extra information is the key to Lytro's trick. "In order to be able to focus after the fact, you have to collect the light field," Ng says. The additional recorded information allows software to manipulate the image, translating the stored information into what Lytro calls living pictures that can be refocused interactively, or even into three-dimensional pictures. The software essentially does the work of a physical camera, performing algorithms on the information collected by the light-field camera to mimic the physics used to focus a traditional camera.
Ng is far from the first to play around with light-field technology. In 1908, recent Nobel Laureate Gabriel Lippmann suggested the idea of "integral photography," using multiple lenses to capture a 3D image of an object. More recently, researchers within the small community of computational photography have been playing with the idea since by Pat Hanrahan and Stanford colleague Marc Levoy (who worked with Ng on his dissertation, and is currently on Lytro's technical advisory board). The idea of a light field has even been applied in cinematography, most notably with the iconic depiction of slowed-down bullets in The Matrix, and is used by computer graphics designers seeking to use image-based rendering to re-create floor plans or landscapes.
Many researchers are excited at the prospect of the idea finally becoming commercial, but that excitement is tempered by some skepticism. The most commonly voiced concern is one of resolution: For the sensor to record more information about the light field, it sacrifices the resolution of the final picture. (It's an inherent trade-off: Imagine the pixels of a flat image taken by a standard 10-megapixel camera spread out into a 3D light field—you capture information about a larger area, but the resolution must go down.) So a camera that would ordinarily take 10 megapixel pictures would end up taking pictures that could be refocused, but would have a much lower resolution.
However, depending on the type of picture (and the photographer), the loss of resolution might not matter so much. "I think the question is how much resolution can people appreciate," Ng says. Often, he says, people who shoot with cameras that take high-resolution pictures don't actually use the maximum resolution. Levoy argues that's especially true for the point-and-shoot photography of everyday life. People may shoot with a 10-megapixel camera, but they're downsizing those photos or saving them at a lower quality to share them or upload them to Facebook. "I never use that many megapixels," Levoy says. "I'm always downsampling."
While much of the hype over Lytro has surrounded its potential to overhaul point-and-shoot photography, rescuing our out-of-focus pictures of vacations and pets, Levoy is interested in applications of light-field technology in the lab. In the field of microscopy, in particular, light-field technology could do away with the same kind of limitations of present-day optical technology. "With the current microscopy, you get a two-dimensional image and you have to decide where to focus," Amit Ashok, a research scientist in the optical computing and processing lab at the University of Arizona, says.
With the ability of light-field cameras to focus after the fact, scientists might be able to see what's been hiding from their microscopes. Levoy says that this could be especially helpful in fields like neuroscience, where quickly firing nerves are notoriously difficult to capture on film, or in studying organisms sensitive to light, because a light-field camera can capture images in low light.
Only a few labs have access to this technology now, though, as it is not being produced at an affordable commercial level. And it's a little surprising that such a long-studied and potentially game-changing technology hasn't cracked the commercial market before. But the camera experts PM talked to say it's a bit of a chicken-and-egg problem: Large camera makers have been reluctant to take the chance on developing a technology untested in the market, so light-field cameras have had a hard time getting tested in the market. "Relatively few of their features incorporate ideas from the academic field," Lanman says.
But that hasn't stopped researchers from pursuing advances in computational photography at institutions such as Stanford, Columbia, MIT and many others. "It's been a silent revolution in the academic world for a while now," Ashok says. Other smaller companies are working on creating this technology, most noticeably a German company called , which has already begun manufacturing and distributing a similar camera, though they use the term plenoptic instead of light field. But the cost of Raytrix cameras reveals another reason light-field technology has been slow to enter the market. Its cameras range from 1490 euros (roughly $2,138) to more than 19,900 euros (about $21,549), not including additional software and hardware components.
Lytro is hoping to produce an affordable camera, though they haven't released details about the size, shape or price of the camera. Nevertheless, there is cautious optimism that all will go well with the launch of the product later this year. "We're all very happy about it," Ramesh Raskar, of the Camera Culture group at MIT's Media Lab, says. "And when the camera comes out, I'll be buying it myself."