Other ISP Functions

Updated: Oct 12, 2021

As we have covered in the previous blog, the main function of the ISP is to perform debayering of the RAW image. To add to that we also covered the 3A functions, namely, Auto-Focus, Auto-White Balance, Auto-Exposure.


This blog is to discuss on the other functions that the ISPs can perform such as :

1. Gamma Correction

2. Noise Reduction

3. Dead Pixel Correction

4. Lens Shading Correction

5. Backlight Compensation

6. Compression


Gamma Correction


Gamma Correction enables the correction of images to resemble the way our human eye perceives the scene. Gamma correction is sometimes referred to as gamma compression and gamma encoding. Please find below the comparison of an image with no gamma encoding (left image) and with gamma encoding (right image).


Our human eye is more sensitive to changes in darker tones than to changes in the brighter tones. This enables our vision to operate at a wider luminescence range. However, the cameras are more linear in terms of the number of photons falling to the electrons generated. To bridge the gap and enable the translation of the camera output to resemble how we see the scene, we need gamma encoding.


Noise Reduction


Noise is generated at multiple level in the camera. Noise in the scene, Noise generated by the ADCs, Noise generated by the amplifiers, Noise generated by the debayering process, Noise generated by the ISP pipeline and so on. The Noise reduction feature on the ISP caters to the erratic noise generated during Debayering and associated image processing. This step in the pipeline brings in denoising techniques that are applied to the data to remove the noise created.


Dead Pixel Correction


We work with sensors having pixels in the range of 100K pixels to 20M pixels and higher. Thus, it is inevitable for each sensor to have certain number of dead or ineffective pixels. The Dead pixel correction is a step in the ISP pipeline that identifies these pixels and removes its impact by performing software corrections. One basic method is by replacing the dead pixel value by interpolating the pixels values of the neighboring pixels.


Lens Shading correction


As you know, the lenses are all spherical in nature. With spherical lenses, we are bound to get the spherical effect at the corners of the image being captured. The effect results in a distortion of the image at the corners. The lens shading correction accounts for this distortion and corrects the image. It is implemented to improve non-uniformity towards the image periphery.


Backlight Compensation


Backlight compensation (BLC) works by increasing the level of exposure for the entire image via digital signal processors (DSP), which divides an image or scene into different sections and adjust the lighting accordingly. Simply put, BLC works to brighten the whole scene on a video/image frame


Please find the below image for better idea. The left image is the one without any BLC. The right image is the version that has undergone BLC. You can notice the image is brighter.



Compression


Image compression is an application of data compression that encodes the original image with few bits. The objective of image compression is to reduce the redundancy of the image and to store or transmit data in an efficient form.

The various forms of compression that we come across in cameras:

1. Images – PNG, TIFF, JPEG etc.

2. Videos – MJPEG, H.264, H.265 etc.


The ISPs on the cameras can enable us to perform the compression on the camera itself, thereby, enabling us to free up the CPU for other processing requirements. Implementing this on the camera also ensures better frame rates.


Each ISP has its own advantages and limitation with respect to the functions it can handle. We, at Regami, shall be able to assist you with integrating imaging sensors with the right ISP.


Cheers

Sarvesh Rajagopal

Sarvesh@regami.solutions