Before getting into the nuts and bolts, here's a video that shows the astounding ability:
Going back and forth, selectively focusing on one image plane or another in a picture is nothing new. Take a digital SLR, point it at a subject, and press the button enough to activate the autofocus, and you'll see that part of the image get sharper. Now point at another point far closer or farther away and press the button again. Now that part of the image is sharp, but (depending on the distance between the two points and how wide the camera's aperture is set), the first one becomes fuzzy. This is physical image focus.
However, the image in the video is the final captured image, with the focus changing from one point to another after it was taken. What Adobe has shown is a plenoptic camera. Rather than optical focusing, the camera uses computation.
In a plenoptic camera, an array of microlenses sits between the camera's main lens and the sensor. Think of the array as a multi-faceted fly eye lens used to break up the single image into many copies, each from a different angle. Each microlens captures information about how rays of light entered the lens and where they would strike the sensor.
Image focus is nothing more than how rays of light converge or diverge on the sensor. With the information, a computer rearrange the rays, intentionally making one set or another converge, which brings that part of the image into focus.
The idea isn't new. Researchers suggested the possibility at least as far back as 1992, and Stanford Researchers have shown a working model.
Adobe has at least two patent applications in the works: one for a plenoptic camera and another on image capture and rendering. Apparently Adobe has already licensed technology so that a camera is already on the market according to Todor Georgiev, an Adobe senior research scientist.
The licensing company, the German firm Raytrix, has developed cameras that can do one-shot 3D as well as post-capture focusing. (Go to this 3D demo link for a Flash demonstration of the effect.) I wouldn't be surprised if the technology could also allow extreme depth-of-field effects, where only one small part of an entire scene was in focus, or literally everything, from the tip of your nose to the horizon, would be equally sharp.
Adobe isn't the only one working in this area:
- Sony (SNE) has a related patent application.
- So does Kodak (EK).
- Samsung has filed a patent application in Korea and the U.S.
- Canon is trying to patent its version and also has a patent application for a plenoptic image file format.