New Raspi HQCam - a question of (white) balance

While testing the new Raspberry Pi HQCam, I stumbled about some odd behaviour. Imaging with HQCam + 16 mm standard optics an average scene with a lot of gray clouds, the auto whitebalance algorithm usually settles for red gain values of 3.3 and blue gain values of 1.5.

While the difference in value of the two gains are striking, I did a few film scans with that “normal” setting, obtaining quite satisfactory results (some examples can be seen in the New HQ Raspberry Pi Camera with C/CS-Mount thread).

For doing these scans, my LED-lightsource is routinely calibrated before the actual scan in such a way that the empty sprocket area where the light source shines through is imaged by the camera as pure white.

Now, continueing the test scans, I stumbled over a scene where the colors started to look odd. Here’s a cutout of a single frame showing this:

Notice specifically the red color tones, from the reddish strip of the tent (left image border) to the red skirt of the girl and the red shirt of one guy sitting to the right. These colors look weird, and if we compare them to some older scans (with the same technique: setting the whitebalance of the camera to “normal daylight gains” and adjusting the LED-source so that white is white), one can clearly see a difference. Here’s the scan with a Raspberry Pi V1-camera:

and this is the result of from a see3Cam_CU135:

Now, for the last experiment, I have set the red as well as the blue gain of the Raspi HQCam to one and adjusted the ligth source again to render as pure white in the camera output. This resulted in the LED source shining in a dramatically red light, with little green and blue. Anyway, the scan

turned out to be better, colorwise, in my opinion.

Now looking at the IR-filter of the new Raspberry Pi HQCam, one discovers that it has a quite bluish appearance. If this filter would be a good IR-cutoff filter, it would actually not appear to have any color tint.

So that might be the reason for the odd relationship between the standard red and blue gain values: the Raspberry HQCam is usually operating with a not so perfect IR-cutoff filter, which is filtering already out part of the visible red color tones. This is then compensated by software, leading to the red gain on the HQCam being usually about two times higher than the blue gain.

Normally, this seems to work fine, but in extrem settings (like lots of slightly different, intense red colors) the processing pipeline creating the JPEG-image is driven into saturation, yielding unreal red color tones.

I would be happy for any comments about this and possibly ideas on how to solve this.

I wonder, if this is a result of the LED flicker? Do you know if your LED has any flicker associated with it from the power supply? I had noticed this in some of my footage work (i’m am a hack and therefore am using an LED driven from an AC source…)

Jim

well Jim, thanks for the comment. Your point is certainly a good thing to consider.

I checked the performance of the light source throughly when I constructed that light source.

Some background information: I realize different exposures of a single frame by rapidly changing the illumination levels of my light source while keeping the camera at a fixed exposure. Later, I combine these different exposures into a single frame.

So the precision of the light source is essential to my film scanner.

The LED light source I am using is made out of three LEDs, one red, one green and one blue, which are each driven by a programmable current source. Each LED has a 12bit digital-to-analog converter, which drives the LED with the current requested by the software.

Here’s a piece of an older version of the circuit I am using (still with 10bit digital-analog-converters MCP4812 in the picture):

Not perfect, but it works. The software writes the requested current to the DAC-IC (now a MCP4822 with 12bit resolution) which drives a simple current source made out of an OpAmp (1/4 of a LM324) and a IRF101 FET (I had these parts in my part bin when I constructed this). “J1 Out” is where the LED is connected.

This current source drives the LEDs quite fast and precise. In addition, I typically wait about 5 frames after switching to a different illumination level before I capture a frame, mainly because the camera I am using is a rolling shutter camera and the imaging pipeline has a certain delay.

The above examples are direct MJPEG-captures (well, in reality: cut-outs) as they were streamed from the different cameras used.

Well, it was interesting for me to compare the scans of three different camera models (Raspberry Pi V1-camera, see3Cam_CU135 and finally the new Raspberry Pi HQCam). Franky, I did not expect such a difference in the appearance. And I actually did expect that the HQCam would perform better than it did on this particular frame. I am not sure whether this can be attributed to the camera, or whether the camera, the film and the lightsource interact in a funny way.

For normal applications, the HQCam does quite a descent job, and color test charts are rendered ok. But in the film scanner application, it seems that the HQCam performs somewhat better if I set both red and blue gains to 1.0 and set the white balance not with the camera, but with the light source.

I am thinking of exchanging the blueish stock IR-filter of the Raspi HQCam with a proper one, having a flat filter response in the visible light region (I have one in my part bin - it looks like clear glas in the visible light range and is pitch black in the infrared). If my suspicion is correct, the red gain required for a whitebalanced image would become similar to the blue gain, leading to a scan result similar to the last image in my post above. I am still a little bit hesitant, as the removal of the blueish stock IR-filter from the HQCam is an irreversible action…

Well, I finally did rip off the stock IR-cutoff filter of the HQCam. As can be seen

IR filter HQCam

this filter attenuates not only IR, but also the red parts of the visible spectrum - that’s why it looks cyan. A high quality IR-cutoff filter has a flat band in the visible spectrum and looks colorless just like optical glass.

Well, I attached instead of the stock IR-filter a B+W F-Pro 486 UV/IR-Sperrfilter MRC to my Schneider-Componon. That filter costs about as much as the HQCam and looks rather innocent, like colorless optical glass…

Normally, you would need to do a new camera calibration, and with the upcoming libcamera stack, that would actually be an option. However, the libcamera software bundle is still in it’s rough state and I won’t put any effort into this until the software is more mature and somewhat settled.

So I took the following scan still with the standard Broadcom stack, only adjusting the red and blue gains of the whitebalance algorithm to get close to a neutral gray for my light source. Well, as expected, the gain required for the red channel dropped down to 2.0. Obviously, more photons are now coming through the red channel, which improves at least the noise statistics. The blue channel gain is still at 1.5.

Here’s the scan result:

and indeed, the red on the tent, the orange-red of the child’s skirt and the red of the shirt of the guy sitting near the right border now are all distinguishable, with the light source supplying standard daylight color temperature. This was not achievable with the stock IR-filter of the HQCam, which has obviously a filter effect on the red color tones.

So, for me this challenge is solved - if you have challenging image situations, the stock IR-cutoff fillter of the HQCam might not be the optimal choice. Note however that under normal imaging conditions, the HQCam performs quite well. The issue which was discussed in this thread will only be noticable in extrem circumstances.

4 Likes

One additional piece of information: when taking raw images with raspistill, you will get information about the color matrix (ccm) applied to the raw image data to create the jpg the camera normally outputs.

I was curious how this matrix would change when different manual color gains would be set.

It turned out that the Broadcom stack (that is the standard way on the Raspberry Pi to grab images) uses just a fixed ccm with various manual red and blue gain settings.

From what the “strolls with my dog” blog

estimates, this ccm maps the raw image channels under the assumption of a color temperature of 5800K to sRGB.

I did not yet manage to run the new alternative, the libcamera stack, on my machine. But I had a look at the source code instead. With the libcamera stack again, if manual whitebalancing is used, a fixed ccm is chosen which corresponds to a color temperature of about 4500K.

Interestingly, at the moment, there is no ccm with that color temperature in the calibration data for the IMX477 (which is the sensor of the HQ cam), so the algorithm will use an interpolation. It is unknown at the moment how the Broadcom stack arrives at the fixed matrix it uses; my values are close, but not identical to “strolls with my dog”.

That a fixed ccm is used with manual whitebalancing is actually a good thing for film scanning applications. And this point was not immediately clear, at least not to me. However, it seems that taking images with red and blue gains fixed gives you in the case of Raspberry Pi cameras a constant color definition, irrespective of the actual (in a film usually quite varying) content of the scan. As it seems, no unwanted adaption, for example color temperature-wise is performed by the processing pipelines.

2 Likes