Strange encounter in the red channel/RPi HQ Camera

[PM490]

@verlakasalt

How come your DNG is only 18 MB? Mine are always 24 MB. I think I’ve asked this before, but can’t find the post again because the search term is too short…

Recalled there was an answer… here it is
I updated the system weeks ago… not sure why my install would be using an older library, will have to check into that. Perhaps the selected raw format affects the DNG size? not sure.

@verlakasalt

red/blue gains to 1.0?

I did not changed from the previous experiment to replicate the banding, Red Blue gains are:

"ColourGains": (2.8, 1.9), # (R,B)

@verlakasalt

(How) is the raw-raw-binned result different from setting the picamera2 red/blue gains to 1.0?

@cpixip

your “raw-raw-binned” should be basically identical to the raw data contained in the .dng-file (save of you adding the two green channels together).

@cpixip

Note that the raw data in the .dng-file is also not affected by the color gains

@verlakasalt

Or did you leave the lens cap on

No, all are taken with very low light and uncapped.

@cpixip

If you continue to work with unsigned integer mathematics (for speed reasons, for example), you are going to clip hard these negative pixel values. Which would enhance the noise…

The raw-raw-binned is done in 16bit unsigned. I do shift the 12/13 bits to be the most significant bits of the variable. Then it is saved as a 16bit tiff. And as you point out, I do so for saving speed. Other than adding the greens, and shifting the bits, I am not doing any other processing.
Since the output of the sensor has offset, there is no clipping (black clipping/offset).

@cpixip

If you immediately convert from the integer format into floating point (DaVinci is doing that as well as my own software)

Passing the 16bit TIFF directly into Resolve (using a linear timeline setting) takes care of it. Again, in my case, there is no color science and there will be no negative values. One has to remove the offset using the color tab.

Up to here, we are in the same page.

Conflicting Results

My expectation was also that -other than the G1+G2- there would be no difference in the raw information in the DNG or in the raw-raw-binned TIFF, and certainly there should not be any difference in the red channel.

The results however are disproving the expectation that the DNG raw is unprocessed.

Working with flat image at low levels is hard to make up what is going on, so I captured the same image with the 0x0800 LED level both in DNG and raw-raw-binned (also uploaded in the previous link).

Bringing these two captures DNG and the TIFF into Resolve shows significant differences. The DNG near black levels appear compressed/clipped and the overall noise is significantly more.

Again, this was unexpected, my expectation was that the raw content on the DNG would not be significantly different than the raw-raw-binned TIFF.

Including Resolve Screenshots to share the color tab values of gain, lift, offset.
Resolve Development Settings for the DNG

Edit: I got the files crossed when renaming. Apologies for the mixed up

Resolve DNG
Edit: the file name incorrectly says TIFF

Resolve TIFF
Edit: the file name incorrectly says DNG

Resolve DNG (left side of sprocket) TIFF (right side of sprocket)

On the image the noise difference is not dramatic, but it is visible.

Have to think how is it possible that the DNG raw values are being compressed, but it appears that is the case.

Edit/Adding
Had the idea of using the color settings of the image (shot at LED L0800) as a reference comparison to then better visualize the differences on the flat captures (shot at LED 0200).

To that effect, added a gain node of 3, and then fine tunned to keep the black levels and light levels the same on the RGB channels.

It is already apparent that the noise profile of the DNG is different than the raw-raw-binned.
The settings for the DNG and TIFF were saved, and applied to the flat shot. To make the waveform and image more visible, another node of gain 8 was added.

DNG Left half / TIFF Right half

Notice that in both images, banding is present. Again, it was established that the banding is apparently inherent to the IMX477 HQ sensor.

Grass underclip
The waveform monitor illustrates what is apparent clipping on the underside of the noise on all channels!

Kind of a visual representation of the prior grass analogy…

@PM490

Think of the noise as grass, with 3 color leaves. If one color receives more water, it results in taller leaves only for that color (gain = water, think of the red color). If the red leaves grow uniformly, the overall grass color would turn to red, but would still be a somewhat uniform color. What may be happening is that the red grass was clipped at a specific height from the ground (you have to imagine that magically one can cut only one color of grass while leaving the other two). But since not all leaves have the same length, only the taller ones are flattened by the cut, but those below the cut line remain with a random height. When the grass is watered then (amplified) the red leaves that were cut would show as a path (banding) which looks different than the ones that were not. Sorry if this analogy is not great… to make matter worst, the grass is actually cut from the bottom, not from the top.

Except that the clipping appears present on all channels, not just the red as I hypothesized prior.

Summary Findings

• The banding was found in all capture methods, which appears to confirm it is inherent to the HQ IMX477 sensor.
• The processing pipeline of the DNG file appears to under clip the raw values on all channels.
• The under clipping may be a contributing factor to make the banding more visible.
• Captures using pihqcam.capture_array(“raw”) do not appear to have the same under clipping as the DNG.