Trichromy with monochrome camera

[Gregory_Dargent]

Hi !

Just a little post here to give some news and ask a question:

I’ve bought a monochrome IMX183 USB3 Basler cam for my machine, everything is fine in black&white (of course…) and I’v e just managed to do trichromy, so know I can scan in color with the mono sensor.

I’ve put a RGB led, controlled with my arduino (that controls motors/sprocket sensor etc…). I can do some kind of white balancing inside the arduino IDE when I tell the values for each channel (Intensity of Red, Green and Blue).

In python on a raspberry pi, I capture the 3 images (wiith the Basler), and merge everything with opencv before saving (I just save the merged image, the rest is buffered)…

And… it works (I still find it’s magical to see the color image done with a mono sensor…). The only problem I have is that I don’t find a way to have the full color depth with opencv… and the files is too small… With the cam in mono, I do Mono12p images, 40Mo/Image. With OpenCv, I only have 8bit RGB Tiff, 25Mo… When I managed to have 16bits RGB merge, the image is absolutely too dark, like if it was underexposed… Do you have an idea or an explanation ?

Here is a small video in color from a really dirty self dev super8 film

trichromy test

And here is the test… it was some days ago, now it’s faster !

Trichromie Telecine test

Cheers !

Gregory

[friolator]

Can you post your code?

[cpixip]

@Gregory_Dargent

When I managed to have 16bits RGB merge, the image is absolutely too dark, like if it was underexposed… Do you have an idea or an explanation ?

Well, that sounds to me like you are putting 8bit channel data (in opencv/numpy language “np.uint8”) into a 16 bit channel image (“np.uint16”). Check the minimum and maximum values you are getting from various parts of your image pipeline, with some simple print statements similar to this one:

print(img.min(),img.max())

In case of a well-exposed image, you should see something around 0-10 units for the minimum, and in case of an 8 bit channel image 220-255 units for the maximum. For the 12 bit image, you should see as maximum something in the range of 3900 - 4095, in the case of a 16 bit channel image utilizing the full dynamic scope of the format, the maximum would need to go up to 60000-65535.

Therefore, if you push an 8 bit/channel image into a 16 bit/channel image, you need to multiply all values with “256” to use the full 16 bit depth. In case of the 12 bit/channel image, try as a multipler “16”. That should brighten up your image.

In my own software, I generally avoid such things by transforming the images in the input stage always to float images like in this code segment here:

           if   self.inputImage.dtype == np.uint8:
                self.inputImage = (self.inputImage/float(0xff)).astype(np.float32)
            elif self.inputImage.dtype == np.uint16:
                self.inputImage = (self.inputImage/float(0xffff)).astype(np.float32)

than do the processing always in “float”-space, and reconvert only when outputting the processed image to an appropriate format. Sadly, some opencv function do not accept “np.float” or even “np.uint16” as input, so not always this route can be followed. Nothing is perfect. But it works for most purposes.

[Gregory_Dargent]

Hi ! Sorry for my late answer I was far from my computer for some days…

I’ve discovered with the help of ‘Gugusse Roller’ that my first mistake was to look at the images I’ve saved in 16bits with Photoshop… and it was awful, whatever I was doing with the levels… Then I tried with Resolve on his advices, and miracle, playing with the levels revealed everything correctly… I don’t understand why there is such an huge difference between those 2 softwares… So know it’s ‘working’… but really to slow, I need at least 10" for each image (1 image/color layer + Merge and save merge…)… So… 10h scanning for 1 film… Perhaps is my Raspberry to weak, surely it is…

If you want the code, here is the one i’ve done, I’m sure that it’s not optimum

from PIL import Image
from pypylon import pylon
import cv2
import numpy as np
import serial
from io import BytesIO
import time
import os
import platform

tl_factory = pylon.TlFactory.GetInstance()
camera = pylon.InstantCamera()
camera.Attach(tl_factory.CreateFirstDevice())
camera.Open()

camera.PixelFormat.SetValue(‘Mono12p’)

camera.ExposureTime = 200000
camera.MaxNumBuffer = 3

converter = pylon.ImageFormatConverter()
converter.OutputPixelFormat = pylon.PixelType_Mono16
converter.OutputBitAligment = “MsbAligned”
#img = np.ndarray(buffer=image.GetBuffer(),dtype=np.uint16)

stream = BytesIO()

camera.StartGrabbing(pylon.GrabStrategy_OneByOne)
count = 1

ser = serial.Serial(’/dev/ttyACM0’, 115200, timeout=1)
ser.flush()

while True:
number = ser.read()

if number !=b’’ :

if int.from_bytes(number, byteorder='big') == 18:
             
   while camera.IsGrabbing():
            
           result = camera.RetrieveResult(5000, pylon.TimeoutHandling_ThrowException)
           result.GrabSucceeded()         #camera.IsGrabbing()
            
           time.sleep(1)
         
           image = converter.Convert(result)
           rouge = image.GetArray()
           #rouge = np.array(image(), shape= (image.GetHeight(), image.GetWidth()),dtype=np.uint16)

           
           break
           
if int.from_bytes(number, byteorder='big') == 19:
             
       while camera.IsGrabbing():
            
           result = camera.RetrieveResult(5000, pylon.TimeoutHandling_ThrowException)
           result.GrabSucceeded()         #camera.IsGrabbing()
           
           time.sleep(1)
        
       
           image = converter.Convert(result)
           vert = image.GetArray()
           break
        
if int.from_bytes(number, byteorder='big') == 20:
             
       while camera.IsGrabbing():
            
           result = camera.RetrieveResult(5000, pylon.TimeoutHandling_ThrowException)
           result.GrabSucceeded()          
           time.sleep(1)
        
         
           image = converter.Convert(result)
           bleu = image.GetArray()
    
           time.sleep(1)
            
           img=cv2.merge([bleu, vert, rouge])
           #res=cv2.resize(img, (720, 542), interpolation = cv2.INTER_LINEAR)
           #show=cv2.bitwise_not(res)
           #cv2.imshow('out',res)
           #cv2.waitKey(20)
          
           cv2.imwrite('/media/pi/SSD/TRICHROTEST/2S82021005%06d.tif'%count, img)
           count += 1
           break
           
            
           
           result.Release()
         
           camera.StopGrabbing()
           camera.Close() 

else :

    time.sleep(0.01)

Cheers !

[cpixip]

@Gregory_Dargent - you are using the Basler camera interface software (“pylon”) to get your image like

image = converter.Convert(result)
rouge = image.GetArray()

While you have requested from the Basler software a 16 bit image with the line

converter.OutputPixelFormat = pylon.PixelType_Mono16

before that you have instructed the camera to operate in 12 bit mode:

camera.PixelFormat.SetValue(‘Mono12p’)

Chances are that the Basler software is not rescaling the 12 bit data from the camera to the full 16 bit scale of your output format. You can find out whether this is the case by outputting the minimum and maximum values of one of your monochrome images with something similar to this

print(rouge.min(),rouge.max())

If you are seeing max-values in the range of 4000, the Basler software is not handling the rescaling necessary to utilize the full dynamic range of your 16 bit output image. You can achieve this in your own code by

rouge *= 16

before the merge-operation.

Note that this operation just spreads the 12 bit input data range into the full 16 bit range of your output format. The quantization depth is still at 12 bit.

As far as I know, the internal processing in DaVinci Resolve is 32bit floating point all the way through the processing pipeline. I do not know how Photoshop currently handles the image data internally, the last time I used Photoshop, it was still 8 bit integer… (given, that was obviously a long time ago… )

[Gregory_Dargent]

Dear Cpixip I just saw I’ve never answered to your message, thank you for your help !!!

Now I just use my files directly in Resolve and everything is fine. I’ve tried your advices but I’m not good enough in programming

Cheers !

[cpixip]

yes, daVinci Resolve is a fine tool!