I’m in the process of writing a rather long essay on the evolution of focusing and focusing aids from ground glass to contrast detection, and everything in between. It’s probably going to be a while before I get that finished and this was interesting enough that I thought it warranted an aside.

Nikon has filed for a patent on a mechanism to embed what amounts to auto focus pixels in the imaging sensor itself. This would in affect solve the performance issues contrast detection autofocus poses.

A bit of background, contrast detection AF systems suffer from a number of limitations that prevent them from matching the speed of the simpler phase detection systems. There are three main problems that contrast detection systems need to overcome. First is pixel sensitivity, because the camera is using information from the imaging pixels, it’s not possible to tune the the autofocus pixels to be more sensitive to light. This means that the camera makes more of a trade off in autofocus speed as light levels drop.

The second issue is it’s nearly impossible to tell which direction the image should be shifted without performing very complicated image analysis on other parts of the image, and even then there’s no guarantee that will be right. This is because to the imaging pixels a front or back focused lens simply produces a diffuse blob.

The final problem is the required processing power. In a contrast detection AF system the camera’s processor is actually trying to quantitatively find the point where to pixels have the highest different in brightness. A phase detection system simply compares the brightnesses of each pixel in two different rows of pixels. In an in-focus image they two rows of pixels will be the same, in an out of focus image they won’t line up.

What it appears Nikon is doing is miniaturizing the optical path of prisms and lenses that feed our current phase detection sensors and embedding them in the actual imaging sensor in place of imaging pixels where they want the auto focus point to be. This givens a camera using this system all the speed and efficiency benefits of phase detection without requiring a mirror or other beam splitter to direct some light away from the sensor.

None of these problems apply to a phase detection system. Because of the way the prisms bend light the direction that the lens needs to be focused is already known (except in the case of a severe defocus situation). In addition the specialize pixels used in the AF array can be tuned to be sensitive at much lower light levels while maintaining a good signal to noise ratio so they can be read quickly. Finally, processing is almost a non-issue thanks to the limited number of pixels the actual operation being performed on them.

The only possible problem is that the camera has to interpolate image data to cover up the holes the autofocus pixels leave. But truthfully that’s probably not much of a problem at all. Cameras current map out defective pixels and replace their measured values with interpolated values from the surrounding pixels. Even the basic Bayer layout of the sensor requires interpolation to turn the RAW data coming out of the sensor into something we can understand as a picture.

What it ultimately boils down to is this could give a point and shoot the same AF performance SLRs have. Of course if you pair this with another Nikon patent for what looks like an interchangeable lens camera in the Olympus E-P1 style, you might be able to start putting together an image of what Nikon has in store for us in the future. One thing that’s clear this patent has the potential to close the performance gap between Point and Shoots and SLRs, and pave the way for a mirror free/prism free SLR that can maintain current autofocus performance levels.

The patent in question is number 2009016727.