Image signal processing in the context of a camera system comprises of a pipeline of functions that the RAW Bayer data has to pass through before get color data in the format that is closest to how a human eye perceives a scene.
As we know, RAW Bayer sensor comprises of Bayer Pattern based Color filter array that enables the sensor capture the color. Bayer Pattern uses alternating rows of red-green and green-blue filter as shown in the below image. The color filter array enables us to capture the color information in the scene as the red filter allows only red wavelength to pass, green filter allows only green wavelength to pass and blue filter allows only blue wavelength to pass. Thus, we have information red, green and blue in the scene which can be interpolated to get color image.
To be able to have an image similar to how a human eye perceives, the first step is to process this Bayer image to have each pixel representing all the 3 R, G and B values. This process is called Demosaicing or Debayering. After demosaicing, the data is passed through processes such as color correction, contrast correction, gamma correction, and so on.
To explain demosaicing at a high level, this process involves interpolation of the color values of the pixels corresponding to the same color in the neighborhood. Once the pixels are exposed, each pixel is ready for read out. A pixel with the red filter gives the exact measurement of the red component. We have to bring green and blue component to this pixel dat. They are achieved from the neighboring pixels. The green component is achieved by interpolating the values from the two-neighbor green pixel. Similarly, the blue component is achieved by interpolating the values from the two-neighbor blue pixel.
Image signal processor is a hardware SOC that is integrated with the sensor to primarily perform the debayering process. Apart from Debayering process, it also additionally performs the below functions to get the image closer to how the human eye perceives.
1. Correction – Color correction, Contrast correction and Gamma correction,
backlight compensation
2. 3A functions – Auto-Focus, Auto-Exposure and Auto-White Balance.
3. Lens imperfection corrections – Lens shading, distortion correction, Vignetting
4. Other functions – Noise Reduction, Dead Pixel correction, Filtering, HDR,
Compression – JPEG, Encoding – H.264 or H.265.
Image Signal Processor Names
1. Socionext – SC200(M10V) and SC2002(M11S)
2. On Semiconductor – ICP3
3. Thine –
4. NXP
5. nVIDIA
6. iCatch
7. Sigmastar
8. Eyenix
9. Sonix
10. Qualcomm – Qualcomm Spectra
Please feel free to get in touch with us in case of any queries.
Cheers
Sarvesh Rajagopal
