The ReelSlow8 Film Scanner

I think it’s still axial chromatic aberration that was causing the difference. Here’s the image from the Wikipedia page that I like to keep in my brain for this kind of thing:

Each color simply has a different focal plane. And IR is much farther out from the others (mostly because it’s hard to design a lens that works well with both VIS and IR simultaneously and Schneider doesn’t have a good reason to care about IR performance on an enlarging lens).

Here is the (2x, nearest-neighbor) center crop from all of the images used in that test. Each of these was taken by changing only the light color via the (relatively distant) breadboard wires. Nothing else was touched in the setup between each capture. That little fleck of dust on the resolution target near the bottom makes it easier to see the difference. It’s amazing how out of focus the IR channel is relative to the others!

My hypothesis is that on an RGB camera you’re always dealing with this effect a little bit. The best focus will be right around the ideal for green wavelengths, which minimizes how blurry the red and blue are, but you can’t usually get all three at once (without, say, using an expensive apochromat corrected lens).

I am curious to see whether the seven or so microns between each emulsion layer in color film will work for or against this effect. My gut instinct tells me that the original film engineers probably took this into account and ordered the emulsion layers to minimize this effect. I plan to test this by scanning the film from both the front and back and then measure any difference in the ideal focus positions for each color. My guess is that scanning it from the normal side, the focus positions will be stacked closer on top of one another.

Otherwise, the quantum efficiency of the sensor will start to come into play once this much stronger effect is defeated. I still have auto-exposure enabled for the sensor, so these are still relatively close in terms of intensity values. I didn’t record the exposure time for each channel. I’ll do that as my testing continues. Between some combination of controlling the exposure time and possibly averaging several frames of each color to eliminate more of the sensor noise, I’m hoping to negate any QE differences. Again, this is still speculation at this point.

Frame averaging to eliminate sensor noise should also push the MTF graph out to the right a tiny bit more, in theory. But there is a can of worms here where any vibration while capturing the frames for averaging wouldn’t only negate the benefits, but probably lead to much worse results than if I hadn’t done it at all. I think I might eventually operate this machine directly on the concrete floor of my basement. :sweat_smile:

2 Likes

@npiegdon - you’re really digging into this!

Here’s a real scan example underlining your thoughts:

The red/green/blue channels are cutouts at 1:1 pixel size, the overview is zoomed down to match, the original resolution of the image is 4056 x 3040 px. You should click on the image to display it in full size on your monitor.

This is from a setup with the Raspberry Pi HQ sensor combined with a Schneider Componon-S 50 mm. I am using a slightly tigher f-stop of 5.6, as this gives me slightly better overall results, and I am not bothering to operate the lens in reverse (as the optical path is in my 1:1 setup anyway rather symmetric). Nevertheless, one can notice that the green channel shows a better image definition than the blue channel, and the red channel is somewhat in between that. I have tested the IR with the same result as you: it is quite blurry unless you use specifically the IR to focus.

I am rather certain that is caused by chromatic aberration, as I am able to get the red or blue channel in focus separately. Of course, than the other color channels get unsharp. My approach is to focus the green channel, as this channels information contributes the most to your result.

By the way - this film frame was scanned from the emulsion side, i.e. the camera viewed the film like a normal Super-8 projector would do.

2 Likes

Very interesting @npiegdon, thanks. I was somewhat aware of chromatic aberration, but did not see it that bad on my first scanner.

I actually used the aberration effect for precise focusing. While using the lens at f2.8, there would be a hallow (blue on one direction of focusing, red on the opposite direction), and I would focus for minimizing both. I suspect that the point is a compromise and maybe not the best focusing for green, but it produced the overall best picture. Then would set the lens to the half stop between f4 and f5.6, which provided the best resolution/sharpness/depth of field compromise.

Would a larger sensor made it less critical? I was using a Nikon DSLR D3200.

These also highlight that W would be the additive of G and R/B in the appropriate proportions, making it softer. If I am interpreting this correctly, then, there is no way to improve the white focus, since it will always be affected by the R and B additive (and soft) components.

In that case, the result may be noisier for IR, but there is no reason it would be softer. So As @cpixip indicates…

I would now also agree that is the case, thanks @cpixip. IR certainly behaves as expected, but R/B to G is significantly more than what I have seen (again with a larger sensor, and a straight light).

Would the lenses used for the Köhler effect setup also create a displacement, at different wavelenghts, that contributes/exaggerate the aberration results?

@npiegdon, kudos on your experiments, definitely a new great dig!

1 Like

It’s nice to see corroborating evidence. It looks like your focus was set just in front of the green focal plane, leaning on the red side.

Were those captured using a diffuse light source? I also have the same question as @PM490 about whether the longer, direct light path with these non-imaging lenses have any influence on this. I have a hunch that it shouldn’t matter, but I haven’t tested it yet. It’s a bit of a diversion from my end-goal of scanning my family’s films, but I kind of want to test all these variants (an integrating sphere in this case and maybe trying all the same things with the RPi HQ camera for a couple other questions I have) just for the sake of documenting the differences and sharing the information with the community.

I’m not sure how a larger sensor would impact this CA situation, but I know it would start pushing further into the limits of the Componon-S lens. With this 2/3" sensor the corner fall-off is already pretty bad using this style of lighting (even at f/4 and f/4.7) and increasing the magnification much past 2x starts to bump into its resolution limits.

Yep, ever since I saw this effect for the first time a few years ago during much earlier testing, that was my reason for striking out in this design direction.

My biggest fear right now is that moving the camera to adjust the focus position also theoretically changes the image size very slightly (at least according to a little optics sim I fiddled with for a bit). I’m hoping that amount is less than, say, a quarter pixel or so over the whole width of the sensor. Otherwise when the channels are recombined into a color image, that slight size change between each channel will show up as chromatic aberration at the edges and corners (despite a perfectly sharp center).

If it does end up looking bad when the channels are combined, I may need to add independent motion control to both the sensor and the lens. That’d be like a bellows riding on the (stepper-controlled) ball screw with another stepper attached to the bellows knob… which sounds like an exhausting amount of work and something that would require a ton of tuning. For now I’m just crossing my fingers that it won’t be a problem. We’ll see. :grimacing:

This may actually not turn out to be the case for a different reason: while I’m able to get a lot closer to sharp focus on the IR channel vs. the image above, I’ve never seen it get as sharp as the green channel. I think this comes down to the VIS vs. IR lens design trade-offs I mentioned earlier. When you get far enough outside the design range of the lens (and the Componon-S 50mm never had IR in mind), you start to see some strange optical behavior.

That said, your understanding (re: quantum efficiency and auto-exposure) sounds correct.

This is a very interesting question! I’m hoping to investigate the answer. :smile:

1 Like

They were capture with the opposite of your illumination setup, namely an integrating sphere. Here’s a somewhat older picture of the setup used.

Let me point to the discussion about different illumination setups in the dissertation I linked above; the interesting part starts at section 5.1.2 Callier effect, p 51 ff. In essence, the main differences in contrast (grain as well as image content) are seen with classical silver-based film, not so much with dye-based material (which I scan exclusively) and in the appearance of scratches, where the integrating sphere comes closer to the results of a wetgate with matched index fluid.

Another point in my decision to use an integrating sphere was the slight reduction in contrast you still see when working with dye-based material. Due to the large density range of color-reversal film, any contrast reduction helps you to get the material in the camera and finally on disk. The contrast of the final product will anyway be adjusted in post.

At this point I like to make a further comment: nowadays there is a balance between things you can improve hardware-wise (optics, camera sensor, etc.) and computational methods which you can apply to the digitized content. This is different from the situation 20 years ago. That’s the reason why mobile phone cameras nowadays come closer to DSLRs than they used to do.

For example, the differences you observed in your Köhler illumination when opening or closing the iris can be rather easy replicated by appropriate blurring/sharpening in post.

There is however a secret hidden in such an computational approach - you need to be careful to sample the signal you are interested in with sufficient precision. That is, in the digital version, the signal needs to be there. And of course, the better the signal, the easier the post processing algorithms are.

I think this will not be an issue. Let’s not forget that physics gives a lower limit on the resolution of the optical image the lens of your Super-8 camera can project (Airy disk). So even with an ideal, non-existing grainless film stock, there is a limit on the resolution a Super-8 film can transmit.

Given, if you really want to “capture the grain”, you are aiming at different scales and considerations like this might be relevant. But than, also the optical resolution your scanning system is able to achieve comes into play. Plus the hidden image processing algorithms your chosen sensor comes with.

As mentioned above and indicated also by the sections of the disseration I linked to, grain appearance is a function of the film frames illumination - and other factors as well, like the noise cancelling performance of the human visual system in a somewhat darkened projection room, viewing the projected image on a silver screen (which had different textures, depending on the make). I highly doubt that anyone can recreate this experience nowadays on a standard display screen, normally not viewed in a darkened room.

In the end, it will be an artistic and/or economic (bandwidth requirements!) decision how much grain your final product will feature.

It certainly will, especially since those lenses used for illumination purposes usually show all kind of image deficits. Will it matter in the final scan? Probably not so much.

2 Likes

That was in the other recent topic. :grin:

I coincidentally saw a different mention of the same silver vs. dye difference only a couple hours before this post. It was in a short blog post linked here, where you are mentioned by name, hehe. That was the first time I remember seeing the phrase “chromogenic monochrome” used as a description to mean “not using silver particles”.

So I might be making a machine that is better for silver-based film when I don’t actually have any of that to scan (save for the calibration target itself). It would be a more realistic test in this early stage to have some of that SMPTE test film (on chromogenic monochrome film!) than continuing to use this laser-etched, silver-coated glass USAF 1951 target.

I don’t think I understood what you meant here. Do you mean that by choosing to allow a reduction in contrast it’s possible to relax many of the requirements of the hardware so the time between starting to build a machine and when it’s able to produce a meaningful result is shorter? (I would agree with that!)

That’s an interesting notion that it might be possible to use the “highest contrast” aperture with all those over-sharpening artifacts and simply post-process it down to something nice. We have so much compute available these days that it’s kind of ridiculous.

Like I’d mentioned in the other thread, I’m not particularly interested in the grain. I’m hoping that when I reach the wet-gate stage near the end of this journey, it’ll have the same muting effect as shown in Fig.24 of the “Film Grain, Resolution and Fundamental Film Particles” paper you linked there.

Rather than going for grain structure, my goal is to get as much of the original picture signal by maxing out each of these parameters (best focus for each channel, ideal contrast, eventual wet-gate, etc.). I’m reasonably sure that this 1936x1464 sensor is going to be able to sample all the frequencies I’m looking for.

Taking a step back, my reason for all of these experiments is because my instinct is telling me there is a good 10-15% of something being left on the table by the usual RGB/Bayer + single focus + integrating sphere approach. I wanted to explore a little more of the possibility space than we usually see in the homemade 8mm machines to see if I couldn’t dig up that 10%.

That said, I am trying to mentally prepare for the event that we find all my extra effort only gives 0.5 or 1% of whatever this nebulous “benefit” ends up being. In that case the effort will have still been useful, if only as a cautionary tale for others to avoid in the future! :sweat_smile: At the very least, I just want to transform some of my conjecturing into real data points. And… you know, maybe scan my family’s 8mm films while I’m at it. :grin:

That’s another interesting position that is counter to my intuition, especially given your own demonstration of the same effect when using an integrating sphere. (Granted, my intuition is severely underdeveloped in the area of optics, to the point of being wrong most of the time. So these contradictions are useful for helping me sort out my own mistakes in understanding.)

Is your claim that that the magnitude of the aberration will just be lower with an integrating sphere? Isn’t the image focal plane determined solely by the imaging lens (between the sensor and subject) and the light wavelength? Would the angle of incidence of the illumination behind the subject have anything to do with it?

FWIW, last night I ordered some barium sulphate and those small cake pans that @PM490 has mentioned a few times. I want to test two things:

  1. The usual case: integrating sphere exit pupil placed as close to the subject as possible, just to compare/contrast my other results.
  2. Integrating sphere used as the input to the Köhler setup.

The reason for the latter is that despite trying to pack these LEDs in as tightly as possible on this PCB (in the top post), even with the “perfectly out of focus” lens setup, there are still (large, smooth, but) uneven hotspots in the image unless I put a diffuser or two at the beginning of the light path. Starting with an ideally even field and then using the lenses/apertures just to adjust the contrast seems like an interesting experiment.

… acutally not. Both feature resolutions which are higher than anything a real Super-8 film will be able to display. USAF targets are usually produced by a photolithographic process; the SMPTE test film is produced with a high resolution film emulsion - the definition this test film shows should be beyond the resolution of a normal Super-8 film.

Well, the density variation of a standard color-reversal film, say Kodachrome, easily covers a range equivalent to a 12 to 13 bit/channel sensor. That is: it is not possible to scan a Kodachrome film frame with a camera operating at a native 10 bit/channel resolution (the Raspberry Pi HQ sensor would do so in some modes). It is barely possible to scan a frame with high image contrast with 12 bit/channel, provided you get the exposure perfectly right (the HQ sensor does feature 12bit/channel modes as well). We are talking here about the raw image, not the processed .jpg or .png which is output from the image pipeline. Most DSLRs max out at 14 bit/channel. These numbers are from my experience with various film stock, scanned by the “soft” integrated sphere illumination. If you increase contrast with a different illumination, you make it harder for the camera to digitize either the highlights or the shadows. That is the reason why many people use multiple exposures which are than combined in the post processing. There are a lot of examples and discussions here on the forum about this topic. Again, you probably want to scan “soft” and modify your material in post processing.

Well, that is about my approach. Let’s reflect a little bit on that. In the last century, most people working in film and photography aimed at the same target: getting as little grain as possible. Why? Because the grain spoils the tiny image structures a good lens can image onto the film. Some people (including me) managed to get resolutions comparable to large format cameras (Kodak Technical Pan + special developers) with 35 mm film. Some time after the digital transformation, it became chic to introduce artifical grain to digital media - again spoiling the original image definition by “adding” spatio-temporal noise. From my point of view, it’s a valid statement, but one has to understand that it is an artistic decision. It has not much to do with replicating the “experience” of viewing a projected old Super-8 film.

Nowadays, there are ways of treating film grain (and sensor noise, for that matter) that were not available at the turn of the century. The basic processing flow goes like this:

  1. Get rid of the film grain as much as possible. This is usually done by employing the temporal coherence of film frames viewing the same area in consecutive frames. Basically, you search for the same image patch in a few neighbouring frames. Ideally, the scanned data should be identical, but because of the film grain, it is not. This actually allows you to subtract the noise (film grain) from the real image. The “search” is usually some sort of motion estimation.

  2. Once the cleaned image is obtained, you can employ various computational methods (like sharpening, local contrast enhancement, etc.) to the cleaned image. If you would apply a sharpening operation before the film grain is taken out, you would increase the grain, not adding any real resolution to the image. Added benefit: a noticable reduction in the band width requirements for the material.

  3. If you are really into this, you can add back again the grain you have taken out before…

Here’s a fast example to show you the effect:

The left half is the original image content, with a noticable grain structure. To the right, the image processed as described above is displayed (only a sharpening step was performed under point 2.).

This image was processed at a reduced resoluton of 800 x 600 pixel, zoomed down from an original 4056 x 304 px source image. I normally work with a source resolution of around 1800 x 1350 px for this degraining process. Nevertheless, I think the difference in image definition is noticable even in this small example.

No, I don’t think so. As you stated, this is a pure imaging lens effect. It’s there with your Köhler setup, it’s there with my integrating sphere.

Looking forward to your next update - it is really fun to follow your progress!

1 Like

Ah ha, I understand what you meant now. And while I’ll agree that reducing the contrast will help you acquire the full range of the image more easily (possibly in a single exposure), isn’t a loss of contrast essentially the same as a loss of information? (It’s almost a tautological statement: we need multiple exposures to capture high-contrast images because there’s more information there than a 10- or 12-bit sensor can capture in one go.)

Outside of presenting the final results on HDR viewing equipment, I understand at some point in the processing chain (hopefully near the end) there is a “mix down” step where you convert to some 8-bit/SDR “window”, but in general it seems like the goal should be to maximize the available contrast/bit-depth available right up until that step.

This particular sensor has a couple 12-bit modes and a 16-bit raw mode for retrieving image data, but that’s just a description of the range of expressible values over the wire. I haven’t done any testing to see what sort of real dynamic range I’m getting from the sensor yet. Although, starting from one of Sony’s Gen3 Pregius models, I have high hopes. With a well depth of 20Ke-, in theory there should be somewhere in the vicinity of 14-bits to play with, but reality probably won’t be quite so kind.

We have direct evidence of this claim from the 1996 demo of Kodak’s “Vision” system in this RobinoScan post. The host mentions film grain (in the context of it being a negative thing that you want to minimize) several times starting at 5m30s and again at 8m30s. Reducing grain is one of the four major feature points they tout as the reason to switch to the new film stock!

My favorite app for this is made by the NeatLab folks. They already have best-in-class de-noising for still images in their Neat Image product. And then their Neat Video product adds a temporal component on top of that. Combine that with very attractive pricing and I’ve been using it to clean up digital sensor noise for years. I haven’t tried it with film grain yet, but I’m excited to see what it can do. The results usually feel a little like magic.

I agree that temporal denoising is one of the first and most important steps.

This is getting exciting now that a couple sub-systems are starting to come together. I just finished moving everything from my electronics/assembly bench over to my usual programming workstation. It’s time to get this software doing more interesting things!

1 Like

Well, there are two different things in play here. If you loose information because you are not able to cover the whole dynamic range of a source, there is no way to recover that lost information. If you loose information because your signal is less resolved due to quantization effects, the (coarse) structure of the signal is still there. So in the later case, you still have something to work with.

For example, the log-encoding used in some cameras does coarser quantization steps in certain intensity ranges, but there is still image content in shadow and highlights. Also, as long as your image processing endevours are not too extreme, it’s hard to tell a 8bit/channel from a 16bit/channel. When I was writing my own image processing algorithms, I used to work in a special floating point format. That was long before for example the openexr-format came along. Nowadays I got lazy and processing is either float or 16bit, storage is always 16 bit.

The sensor will probably not feature 16bit resolution. It’s probably a 12bit sensor - the current upper limit I am aware of is 14bit sensors in high-end DSLRs.

Nice catch! :sunglasses:

There are several other options here. I guess the first one to do these kind of things in the context of Super-8 scanning was Videofred. He used avisynth-scripts, and in the context of avisynth, there has been an active development over the years to remove grain from exisiting footage. avisynth has a steep learning curve, but it’s for free.

From my experience, it is important to prestabilize the footage before a temporal-degraining operation is attempted. The reason is that the movement estimation of image parts is not trivial and a prestabilization helps here tremendously. Still, there are restoration artifacts in scenes with a lot of movement. Usually they are not that noticeable, and the improved image resolution outweighs these artifacts in most cases.

While I plan to come up with my own software, at the moment I am using daVinci for prestabilization, avisynth+VirtualDub for temporal degraining and again daVinci for editing and color-grading. Have not tried daVinci’s denoising since version 16 or so, but since they are now at version 18, I think I will recheck this at one point.

1 Like

I suppose the best part of log encoding is that it roughly matches the sensitivity of the human visual system, so the information that is discarded is the part our eyes are the least likely to miss. (Well, assuming the final, color-graded output exposure is somewhere in the neighborhood of the input. If the footage was incorrectly exposed and needs to be corrected more than a couple stops, even log footage can’t save you from some loss of quality.)

This is another good point. I’ve already been through a post-processing workflow with a number of home movies from the 80’s and 90’s recorded on VHS tape. In that case, doing motion stabilization before the other cleanup steps did seem to help the final result in the way you described.

Finally got around to make the port holes. Here is how it looks.


Ps. and after a bit of epoxy, here it is with the light ports. I will prototype with prior LCB PCBs.

1 Like

May: Pixel Size, Focus Detection, and Planarity

I didn’t get quite as far as I’d planned on full-fledged auto-focus, but the digression into measuring the size of each pixel was worth it.

The machine is starting to look like something:

The software is still mostly debug buttons and other manual testing/control bits, but the number of things it can do is steadily increasing:

In particular, I’m happy with my decision to use Dear ImGui for the UI. I’d never used it before, but on average it takes a single line of code to both add a new button and the code for what that button does (no extra callbacks, etc.), so the framework stays completely out of your way and lets you get your work done.

The communication between the Arduino and the Windows app is going smoothly. I came up with a little protocol where each message (and reply) is between 1 and 5 bytes for all of these primitive operations like changing the light color, delaying for some number of microseconds, or taking some number of linear motor steps. But then there is a mode where you can say “start accumulating instructions until I tell you to execute them all at once”. That takes USB jitter, parsing overhead, and most other sources of latency out of the equation. So it should be able to do things like strobe the lights with repeatable timing when triggering the camera’s GPIO pin to acquire an image, if need be.

Measuring Pixel Size

Last time I said my biggest fear was going to be a large change in the size of pixels between the best focus point for each color channel. I’ve since been able to confirm that the size of the pixels does change but not so much that it isn’t something I won’t be able to correct for in software with some very gentle resampling.

The first step was being able to reliably hold these calibration targets. I was having trouble getting something held in place tightly while still being able to make fine adjustments, so I cobbled together this little slide holder that uses springs to hold the slide against a fixed surface.

Those are toothpicks holding the springs under tension. I switched to a full grid calibration slide that is a small, round disc with a target that’s roughly the size of an S8 frame, which is perfect. So that’s what’s sandwiched between the plastic layers there in the photo.

Until now, I was measuring pixel size manually by capturing an image, using the ruler tool in Photoshop (which snaps to whole pixels), and doing the math by hand. But I wanted sub-pixel accuracy and more measurements to average across to make the result more precise.

After some tinkering with OpenCV, I came up with something pretty cool for the single-axis (1D) target. It was a little brittle and required a nice, sharp capture without any dirt in the frame, but I was getting some reasonably good numbers out of it. But then two days later I figured out how the idea could be extended gracefully to the full 2D grid target in a way that was much less finicky about the quality of the image.

Here are the broad strokes of the algorithm being run on an intentionally skewed and dirty frame to show its resilience:

2023-05-30 Measuring Pixel Size

(Be sure to click the “Play” button on the animated GIF. The forum software seems to prevent it from animating automatically.)

Step 1 is running the CLAHE algorithm built into OpenCV to bump the local contrast up for line detection. There is some gentle blurring being done here too, which I didn’t show in the animation.

Step 2 is picking out line segments using OpenCV’s cv::createLineSegmentDetector. (Specifically this does not(!) use the Hough transform line detection; I spent a lot of time there and was getting poor results, whereas the LineSegmentDetector feature worked beautifully on the first try after I found it.)

Step 3 is where it starts to get cool. Find the angle of each detected line segment (wrapping it into a half-circle so that a 1 degree angle and 359 degree angle show up as 2 degrees apart instead of 358), and then run K-Means Clustering (with two groups) on it. OpenCV also has this feature built-in with a nice, easy API for using it. What you’re left with is two lists of line segments that are grouped by what you might call “horizontal” and “vertical”, except there is no dependency on the grid being aligned with the camera sensor.

Step 4 is now that we have the cluster centroids, which are a kind of “ideal” angle, we can check that they should be almost exactly 90 degrees different from each other, and throw away any line segments that aren’t close. This cleans up almost all of the noise, dirt, and other elements from the image.

Step 5 breaks each list of line segments into two lists of points containing the end points of each line segment. So, both endpoints from each “horizontal” line go into one big list and both endpoints for each “vertical” line go in a separate list. At this point we are done with the lines and they can be discarded. The endpoints are the blue and red dots in the animation.

Step 6 finds pairs of endpoints–one from each list–that are closest to one another. There is no special ordering required: pick any point from the first list, go through the second while making note of the closest match. Find the midpoint between those two and insert it into a new “corners” list. Throw away those two points and repeat until one of your point lists is empty. Now we’re done with the endpoints and they can be discarded. The corners are the green dots in the animation.

Step 7 finds groups of four corners that are all close to one another within some tolerance. Pick any corner at random, sort the rest of the list by how far each corner is from that one, grab the top three. If any of them is much farther than the others, the corner we started with doesn’t belong to one of the intersections in the calibration target, so it is thrown away. Otherwise, find the midpoint between all four and insert it into a list of “intersections”. Now we can throw those four corners away and repeat until the list of corners contains fewer than four entries. At this point we’re done with the corner list and it can be discarded. The intersections are the green circles in the animation.

Step 8 is kind of magical. I knew about gradient descent, but I don’t have an easy way to calculate the derivatives necessary in this situation. While digging around for numerical methods of fitting data, I saw a mention of the downhill simplex method, which I’d never heard of. It does something similar to gradient descent but only requires the original function and not the partial derivatives. Even better, I found this lovely, single .h file C++ implementation of the algorithm.

The idea is that you give the algorithm a function and a set of starting parameters that can be passed to that function. It perturbs the parameters (in a very specific way), exploring the problem space, checking each result against the function until it finds the minimum value.

In this case, I just pick the intersection we found that was closest to the center of the image and then the intersection next closest to that one. (Pythagoras tells us we’ll always get an adjacent point and not a diagonal, no matter the rotation angle of the grid.)

The parameters to the downhill simplex algorithm are simply the (x, y) coordinates of those two points. The line segment formed by those two points is actually enough to define an entire grid: the line segment’s length is the grid spacing and the line segment’s angle defines how the grid is laid out in the plane. So all that is left is to provide downhill simplex with a function that tests all of the other intersections we found, snapping them to the nearest point on some hypothetical grid, and reporting the error between them. Downhill simplex minimizes that error and finds the ideal grid that fits each intersection most closely. It returns two, sub-pixel (x, y) pairs that are ever so slightly different than the initial parameters, that now fit all ~650 intersections best instead of just those two.

The red grid overlaid on top of the original image in the final frame demonstrates several things:

  • Downhill simplex is easy to use and gives good results.
  • The Schneider Componon-S 50mm f/2.8 lens has a remarkably flat field of view with essentially zero distortion, even at the corners.
  • Even with all the junk in the line detection step, this algorithm is very resilient to missing or noisy data.

The earlier 1D version of the algorithm used the same steps 1 through 7 but had an extra three or four steps after to categorize things, find the scale’s axis, throw away intersections off the axis, order them, and check for consistent spacing between them. It was more brittle and couldn’t handle a single mis-detected point on the calibration slide (because of dirt, etc.).

So, one funny consequence was that the 2D algorithm continued to work in the 1D case and was more forgiving of bad data to boot. (None of those steps required the existence of a 2D grid; we only need evenly-spaced intersections.)

The size of each pixel can be read off directly: the length of the line segment is already given in pixels, and we know the length of the division in physical units a priori from the Amazon listing. :smile:

Here are some results for my particular IMX429 sensor and extension tube setup. These are the size of each pixel at the best focus position for each color channel:

  • 2.8557 µm/pixel for red.
  • 2.8552 µm/pixel for blue.
  • 2.8503 µm/pixel for green.
  • 2.8522 µm/pixel for white.
  • 2.8798 µm/pixel for IR.

Something jumps out right away: red and blue have nearly identical pixel sizes! Keep that in mind for later. :wink:

Knowing that the sensor has 1936 pixels across, you can do the math and figure out that a picture of some object that is 6mm wide will end up ~3.4 pixels wider when taken with green light than it will with blue or red light. (For IR it’s almost 20 pixels narrower!) Assuming they’re aligned by the image center, by the time you get out to the edges, green will be off by almost two pixels. (For higher resolution sensors like the RPi HQ camera, this will be about 2x worse because it has about 2x as many pixels across.)

Instead of trying to correct for the effect by adjusting a bellows for each color channel, I think I’ll just resample each channel in software down by (up to) four pixels, realign them, and call it a day. That should clean up any color smearing at the edges and in the corners.

Focus Detection

The simplest metric for detecting an image’s relative focus is just taking the standard deviation of each pixel value in the image. That’s a one-liner in OpenCV, so it’s what I started with.

So we can see where our best focal planes are by walking the camera’s Z-axis forward one step, capturing an image using each color channel, and calculating the standard deviation of each image.

When you’re done, you get a beautiful graph like this:

Data is never that clean! Those are almost perfect Gaussians.

The amplitude of each curve doesn’t matter. That’s just the relative brightness of each LED (and QE of the sensor), with the camera set to manual exposure, so some colors were naturally dimmer than others.

The important part of each curve is the X position of the peak. That’s the ideal focus position. And the thing that jumps out immediately (and also agrees nicely with the pixel size observations) is that the Schneider Componon-S 50mm f/2.8 is achromat corrected.

I’ve never seen this mentioned in any datasheets or in any reviews. The red and blue would be even closer if my red LED was a “standard” red wavelength (620nm) instead deep red at 660nm. If you do the linear regression of each peak (except blue) and project a hypothetical 620nm LED onto the same line, it falls almost exactly on the blue peak.

So, 1/3 of my reason for using a motorized Z-axis has been solved by Schneider. (If only I could get my hands on an apo enlarging lens, I almost wouldn’t need the motor at all… but where would the fun be in that?) :rofl:

Planarity

You can use the image’s standard deviation as a focus metric to perform one more cool trick. Instead of calculating it for the whole image, you can slice things into different focus zones:

Then, calculate the standard deviation for just those sub-images and plot them on the same axis.

Here are zones A through E for a single color channel, taking an image at each Z-axis motor step:
2023-05-30 Planarity, bad

If the subject plane was aligned perfectly with the sensor plane, all of those curves should have a peak at the same x coordinate, which would mean each part of the image would be entering and leaving focus at the same time. But it wasn’t!

Eventually the film transport is going to be mounted to the same optical tilt/rotation table that I have the calibration slide mounted to now:

Those two knobs can adjust things by a couple degrees on a rotation and tilt plane, which is exactly what we need here. After a couple turns of those micrometer screws, I ran the planarity sweep again and got this back for zones A through E:

2023-05-30 Planarity, good

That’s a lot closer to being aligned with the camera sensor. I repeated the same for the other direction (zones F through J) and now images are sharper across the whole field of view than before (and the 2D grid pixel measuring algorithm started returning even more consistent results down in the nanometers).

There is an open question about how well the standard deviation metric is going to hold up with Kodachrome as the subject instead of a silvered calibration target, but hopefully I’ll be able to get things dialed in once I switch over to a proper film transport. Planarity seems like something you can dial in once and then forget about unless the machine gets bumped by something.

What’s Next

Focus detection isn’t quite “auto-focus”. Those graphs are still being generated manually in Excel. I want the focus sweep button to find the peaks and set up markers at the best focus positions so I can travel to each in a single button click. That is auto-focus and it’s going to require a little more algorithm design.

That’ll be in June and I’m planning to fill the rest with more integration work as I get closer to actually imaging film: I need to confirm that my software control of the motor (over my new communication protocol) is still as dependable as my early Arduino-only tests re: not missing any steps. I want to get the Arduino connected to the sensor’s GPIO so I can request exact exposure timing instead of just using the default video stream. And it’s probably time to see if I can’t read values from the load cell controller board that will eventually become part of the film transport.

3 Likes

You were right. Despite my hopes for around 14-bits on the Pregius sensor, I’ve since found a spec sheet that shows LUCID’s Triton camera lineup only uses a 12-bit ADC to read from it. Oh well.

Those are some clean cuts. How did you do the off-angle cuts so nicely? When I was brainstorming how I might turn these cake pans into an integrating sphere, I couldn’t think of an easy way to hold them at those angles. As an alternative to off-axis ports, I took @cpixip’s advice that the sphere didn’t need to be perfect and instead optimized for the simplest construction steps: I put the entrance and exit pupils on a straight line and just inserted the smallest possible baffle between the two.

The little cone-like feature on the side facing the LED board is just to prevent the first light bounce from going directly back to the PCB. The other side is flat.

I overestimated the wall thickness of the pans a little. While milling the smaller exit port, it ended up with a razor-like edge and you can see a little wobble in the cutout where there just wasn’t any material left. I suppose the thin wall at the exit works out in my favor for getting it a millimeter or two closer to the subject.

The barium sulfate wasn’t as hard to work with as I was expecting. After some sanding, I sprayed everything with a white primer and then mixed a bunch of the powder into titanium white acrylic ink (mine was Liquitex brand). It still took 8 or 9 coats (and it probably wouldn’t hurt to do another five) before it was reasonably uniform looking.

And here it is fully assembled, on the rails, showing the front of the exit port. The mount (with the captive nuts) for the PCB is just held on with a couple drops of super glue for now. If that doesn’t last, I’ll use epoxy. Again, I was optimizing for speed-of-build this time. :grin:

This was kind of an extra credit mini-project for June. During some early testing with the Köhler stuff, I saw how much the direct lighting was accentuating scratches and other oddities in the film. If I’m going to save the wet-gate/chemistry portion of this project until the end, I wanted to have a good proxy in the meantime to help hide those defects while I was still working out the rest.

I probably shouldn’t have put my eye directly up to the exit port with (the dimmest of the) LEDs on, but I saw exactly what I was hoping to: a completely invisible baffle. The only discernible features inside the sphere are a faint seam between the two halves and the two little “tick marks” where the arms holding the baffle divide the seam line.

I can’t believe how homogenous this light is! I focused on my calibration grid, put the sphere as close behind it as I could, then removed the grid. The resulting flat, gray image had a difference of only 2.6% brightness between the center and the darker two corners. I think I might be able to do better by tweaking the alignment a little. The other two corners were only 1.8% darker.

The histogram is just a single spike with an (8-bit) std dev of 0.79. When you auto-contrast that out to the extremes, you can see the darker two corners.

I think I found a few dust particles in the lens. :rofl: That’s the aperture blade pattern of the Componon-S lens. I’ve already tried cleaning the front without any effect. I hope they’re just on the back. But I expect they’re actually somewhere inside.

It’s mostly a non-issue because everything in that image is taking place inside two (8-bit) gray values.

3 Likes

Start with a small hole,

and then use a step-bit, these worked great.

For the larger output port, did a jig to hold the cake mold and be able to rotate, while holding the dremel still with a milling cutter bit.

Then used the same setup with a sander bit to finished it clean.

Finished the edges with a light/fine filing.

1 Like

Wow, so the side ports were done by hand with the step-bit?! I’m even more impressed. Nice work. :smile:

That jig was another good idea, too!

1 Like

June/July: Early Direct Drive Transport with Load Cell

The ReelSlow8 has sprouted legs (and more). I wanted to get it off the table so I could run wires and have easier access when building the side arms.

My 7 year old started his summer break from school so my hobby time took another hit. I didn’t have enough to show in June alone, so I’ll do a June/July and July/August update instead.

Even then, instead of working on auto-focus I decided to finally tackle the pain point I experience anytime I need to carefully hold film in front of this thing for a test. It was time this film scanner could actually scan film. :sweat_smile:

I opted for the simplest film path I could imagine.

Direct stepper drive on both reels, two rollers as close as possible to the exit pupil of the integration sphere, and one extra roller for the load cell. The side-arms of the machine were designed with the MakerBeam very specifically to allow easy height adjustment for the steppers.

The rollers are (SLA) 3D printed to have two 693ZZ bearings pressed into the center. Then, you can run them right on an M3 bolt and use the nuts to fine-adjust the height. The curve in the center means they barely touch the edges of the film, as per usual.

The most interesting part is the load cell. If it isn’t the default choice around here for tension sensing, it should be and I’ll advocate for it any chance I get!

  • It’s like $8 for a 1kg load cell.
  • The SparkFun HX711 breakout board seems to include a few more components on the board to ensure good function vs. the boards packed in with the load cell itself (at least according to the reviews on both sites), so I tried that first.
  • You can switch the SparkFun board from 10 Hz updates to 80 Hz just by cutting a jumper on the back of the PCB. (They advertise 80 Hz, but mine does about 92 Hz and I’m not complaining.)

There are complaints in the reviews that the data is especially noisy and you need to do all sorts of averaging over time, but those are use cases that need the accuracy all the way down to the least significant bits coming off the 24-bit ADC. For our purposes of roughly keeping film inside some, say 50g, tension window, we might need five of those bits.

When you plot the full-scale data with no averaging at all–just the raw data–look at how pretty this is:

Essentially zero-latency response and super clean data. The little bump each time the scale reverses direction is actually the static friction of the plastic piston against the cylinder breaking. You can feel it and, apparently, so can the load cell.

That little classroom scale is my plan for calibration. I haven’t gotten this far in the code yet, but having the app ask for a few measurements (maybe each 100g on the 500g scale to build a regression) before each scanning session should give sub-1g accuracy which is an order of magnitude more than I need.

5 Likes

Nice update, thank you for the information on the load cell, very interesting and something to consider.
I also like the roller design, very nice and simple.

I am only making this comment because on your design you have been very detailed; for a normal built, I would not even say anything…
One consideration on the current path S-path and air gate is that the film bow on the gate will change with tension, and that slight bow will change the focus point.
Additionally, on the pick-up reel side, as the reel fills up, the film angle to the air gate pick-up-side post will change through the reel, making the right side of the bow vary slightly, even when tension is held constant.

Understand in this case -with auto focus- that may not be an issue. And that the variations introduced by the pickup on the right side may be within the dept of field range, nevertheless, keep an eye for those.

My worst skill is building transport, and got lot’s of experience (what you get, when you don’t get what you want). To avoid the variation introduced by the filling pick-up reel, a simple fix is adding a roller post on the pickup side, mirroring-opposite to the one for the load cell, effectively changing from an S-path to a W-path.

Again, only making this comment given the extreme effort you have been doing on every little detail. Hope it helps.

1 Like

I’ll take that as a compliment. :smiley:

Also, these are great questions!

Regarding the film angle, this is a quick, not-to-scale mock-up of what I think you meant:

The darker blue shows that over the course of an entire reel, the angle will definitely change, introducing a cosine error term on the load cell roller (and one of the “gate” rollers). During calibration with the classroom scale, my plan was to try and hold it along the mid-line between the two extremes, so it’s calibrated on the average, halving the error.

Admittedly, talking about holding something by hand (and using a $3 instrument) probably demonstrates I’m not too worried about the results here. When I did a survey of all the recommended tension values I could find across as many different pro machines and archival reports, the tension values were all over the place. (Granted, these were mostly for 35mm film, but) I found–after converting all of them to the same units–all of the following mentioned as supported/recommended tensions: 200g, 300g, 170-450g, 170-850g, and “very little up to 340g”. So picking a conservative starting point like 200g seemed safe enough, but that I probably still didn’t have anything to worry about even if it ends up being off by, say 50%(!).

Also having the least experience with the transport (vs. all of the other relatively more known quantities), my plan was to cross that bridge when I came to it! :sweat_smile: My ace in the hole was that I intentionally used longer rails for the side-arms, so if the cosine error became a problem, I could just slide either the load cell or take-up reel back as far as another 8" or so. That would increase the length of the film path but also drop the cosine error fairly considerably.

Another design requirement I’m willing to bend on: out of the 39 reels I have to scan, 34 of them are 3" reels. If the couple 5" and 7" reels end up introducing too much error, I wouldn’t mind a pre-processing step of transferring them, piecemeal, to 3" reels for scanning. (Granted, I don’t want to do it, but I will if I have to, hehe. I already had that same workaround in the back of my mind in the event that the direct drive motors weren’t strong enough to turn the larger reels.)

As for film bow, this is probably my complete lack of experience showing. I had assumed that once you were wrapped around a roller far enough, the incoming angle wouldn’t matter anymore (given a particular tension level) and you could make assumptions about the behavior of the film from that point forward independent from anything that happened before it. And the “around a roller far enough” here could be modulated again by sliding things as far along the rails as necessary to reduce the cosine error. The “far enough” probably also depends on the diameter of the contact point on the roller. For very large rollers, I’d guess film bow is minimized/eliminated. For something like the LEGO cones in your design–which approach a diameter of zero–the tendency to bow out is probably accentuated.

Really, I’ve probably been puzzled over these questions longer than any other part of the project. Some pro scanners have dozens of rollers. Some have as few (or fewer!) than this design. Your own film path stood out as one that had an above average number of rollers and I’ve spent time scratching my head over what each one might be for.

I’d dragged my feet on this part for as long as I could (something like five years :grimacing: ) and it was finally time to commit something to paper–good or bad–just to get a move on. I was hoping the first iteration would get close enough, but I’m not so confident that I’d be surprised if I had to tear most of it down and try again.

The eventual addition of a full-immersion wet gate only adds the requirement that the film near the light source be convex (w.r.t. the camera), which the current S-shaped path permits (even if you’d probably have to slide those two elements farther down the rails toward the camera to make it more feasible). But your W-path suggestion also meets that requirement, so adding that extra roller in the event that film bow becomes a problem would be a good workaround.

All of that is a long-winded answer to say I am much less confident about this part of the design and am mostly crossing my fingers, hoping for the best, and am planning to tackle problems as I run into them. :smiley:

Yes, it is :slight_smile:

Here is my totally empirical summary, the point of view below is framed by these.

  • Angle of bend around a post or roller adds tension to the force needed to move the film. In my case, I had the rubber rollers with bearings (little friction) and the loose-lego-conical-posts without bearings (more friction). In general, the larger the bend, the more friction.
  • Old film is less flexible. Larger tension was needed to flatten the film on the air-gate-posts. Less tension would bow the film. Thought this could even be used for narrow focusing. In my case, the space between the gate-posts is larger, because I wish to capture almost 3 frames of 16 mm (one frame with more than two halves), consequentially the play bowing was more noticeable.
  • Because of the old film lack of flexibility, and the narrow lego-posts it was necessary to avoid large angles about a post, or risk the film breaking at the perforation (experienced it) when using more tension.
  • Eliminating the variability of the increasing/decreasing spool takes away one more uncertainty. The less uncertainties with areas I have the less know-how, the better.
  • The transport is intended to be 16 and 8, without any changes other than software. This is an aspiration, but the preliminary results were promising.

With that thinking framework, a few comments.

I was less concerned about the load-cell effect of the supply-reel angle changes, mostly because my lack of know-how on the cell, but you have a point.

I used two kinds of post, the narrow lego (conical only touching edges) and the wider rubber-roller-bearing (flat full contact). With old film, I saw bowing on both. The rubber is probably closer to your design, but on yours, these are only touching the edges, I think the opportunity to bow is there… similar to the lego (only touching the edges also). All speculation, testing is king.

Based on past experience (which may not be relevant to your rollers) I would expect some uneven bowing (very exaggerated in the illustration).
two_post_deformation
The angle on the left side does not change, the angle on the right side changes, providing an opportunity to bow differently than the left side. This may be so minimal that it may not matter, again, just mentioning it given the attention to detail on your work. Just keep an eye for it, and see if it matters.

Same here, with the added camera sensor and sensor software learning curve.

A big difference in the path is the type of motor used, probably cannot oversimplify/equate film paths of different motor kinds.

I’ll shed some light on my path design, a path for steppers. In very old posts, I started with an inverted V, with 3 rollers on each side of the gate.

Then came the proof of concept for the transport with Tof.

First reference to load cells was at the post of the Tof (square path above) from @friolator, but I did not rethink the initial inverted V path.

The first path, inverted V, is nice and symmetrical, and the 3 rollers take away any variations of the spool-size-changes, and two load-cells in the second roller of each side would give perfect feedback to control the steppers. For precision, one of the gate-edge-rollers can be numerically encoded. As I write, I think this is a neat transport alternative, and one that I may revisit down the road.

The one turn-off was that it was large, in order to make space for the large sphere, plus two large reels virtually in line. That’s when I started playing with ideas of a wrap-around path, to keep the project at a reasonable size for storage.

Then came the ToF path, which is virtually a square, and provided experience on turning old film.

The new path is based on an octagon, and it is virtually symmetrical between the supply-side and pick-up side. The pickup side has the capstan, and supply side has a roller where the capstan would be, which is the only difference (the middle roller in the 3 at the bottom.

Tried to eliminate variations induced by spool-size, both sides start with an entry post (right side of the octagon). Both sides have identical dancing-potentiometer, the entry-exit angle of the potentiometer is fixed by the entry-post and the first roller (from right to left). The next side of the octagon is the capstan (on the top) and the dummy-mirror-capstan, the three rollers at the bottom. The two left sides of the octagon are the gate.

The large sphere needed for 16 mm target (more than one frame) takes a lot of space. When working on the failed prototype, one of the takeaways of the experience was the fragility and rigidity of old film (have a couple of the 16mm close to 70 years old). So was looking for a compact arrangement with a path minimizing sharp bends. The entry post, with large spools are the only ones would be over 90 degrees (the old 16 mm film is in smaller spools, and the angle would be far less than 90).

So in short, all those rollers help make the transport plate a bit smaller, and lesser the film bending.
Please, don’t follow me, I’m a bit lost… but making good progress (paraphrasing Yogi Berra).

I suggest you change nothing until you test it, and use the information provided as things-to-keep-an-eye-for. Perfection is enemy of the good.

2 Likes

This kind of freely shared experience is the reason this forum is great and how it has become a veritable dragon’s hoard of collected wisdom for these kinds of project. :smiley:

Film bow wasn’t on my radar before, but it is now. I’ll keep an eye out for any problems while I’m testing things. Thanks again for sharing!

2 Likes