Still Image Stitching and Registration Software Development/Open CV Consultant

[Filmfilmfilm]

Hi all,

We are looking for a solution to stitch 2 images together taken on a still camera. The same format as VistaVision/8 perf. The images are scanned at 4 perforations each on a Cintel scanner and we’re looking to merge/stitch the two images together to make one seamless 8 perf image.

What we are looking for in particular is:

An automated level of batching an entire roll of 36 frame still film
An algorithm and/or script that aligns these images based on either perf registration or image keypoints.

We haven’t had much success with existing Pano software as it is set up for multi position images built in a 3d space. We really need something more in line with alignment of two images in a 2d space.

We have looked at Wolfram and a few off the shelf applications/formulas and are looking for someone who has a solid understanding of this and could create a script or implement algorithms based around automation.

Appreciate you all for this community!

[friolator]

For stitching to work properly, there needs to be overlap, more is usually better (so you’d want, for example, 2x 5-perf or 6-perf images, where the image overlaps by a perf or two. OpenCV has some stitching tools that can do this, but I don’t believe there’s a GUI available so you’d need to code that up yourself. The python implementation of OpenCV is pretty straightforward, but you need some programming experience.

[Filmfilmfilm]

For sure!

Thats what were looking for is a programmer to tie this together working in specs of 4 perf frames for a custom solution. Feel free to chime in or send a DM if anyone is interested!

[friolator]

Does the BMD scanner give you any overlap of the picture area though? You could in theory go by the perfs, but that may not work properly. Also, a standard 4p motion picture frame will have a gap between frames, so the Cintel scanner may not do enough overscan to give you the overlap you’d need to stitch the images together.

We once tried to do something similar with our Northlight - overscanning the film so that we could run the soundtrack through AEO-light to extract the sound (which requires some overlap). But the overscan was insufficient (a hard limitation set by the metal aperture) and this caused a small gap between frames, which resulted in 24 pops per second in the soundtrack. You could run into a similar problem here, based on how the gate aperture is set up.

I would throw a reel of film in there (still film) and grab some frames, to see if this will even work. Any programmer you hire is going to need some sample images anyway. As an initial test, you can bring them into Photoshop and line up the perfs to make sure you have some overlap, just as a proof of concept

I don’t think the ScanStation’s gate aperture is big enough to give the the necessary overlap to pull this off. I’d be surprised if the Cintel gate is much bigger, though it’s possible. If you’re able to post some frames, I’d be really curious to see what you get from that scanner.

[Filmfilmfilm]

We have done a fair amount of testing.

There are a few ways to go about key point detection. Some algorithms better than others. With Pano software, it does a great job at alignment with frame-lines but will put this into a 3d space and always applies this to several axis(s) vs a 2d space. Its also looking for things like parallax, lens distortion, exposure, ect. Again, because this is a static camera in a 2d space, with a constant of frame lines with a scanner/gate, perfs in each frame, this seems to be an advantage right out of the gate.

Keypoint algorithms really vary from broad to precise. This is based on what it is looking for. Its essentially looking at two frames and looking at the commonalities between the two and assigning number values or coordinates to have a script run once recognized to merge or align those two images together.

In regards to image keypoints without those factors, there are alot of algorithms and formulas designed to look at these things side by side even geared towards industrial level of manufacturing and also things like image stacking and medical applications that compare differences in image sets over a period of time.

Here is an example more on the stacking side in regards to perf alignment within the dimensions of a 4 perf frame.

Here is an example of keypoints detection with values assigned though we are looking for something more precise with things like this frame where if there are very similar parts, we need to have something looking a bit more. There are alot like RANSAC, SIFT, Homography, ect.

There really isnt a market for 2d images alinged with a static camera and why there doesnt seem to be a solution with software or GUI and why pano software operates in a 3d space as a camera is moving shot to shot.

Apologies if this isnt totally correct as its just what I have learned but a solution seems to exist with academic open source algorithms to look for key points and create a script or gui that executes image registration. Stitching I think is the incorrect word in this context.

If anyone knows where to pick this up at and run it with. Im the first to admit not a programmer or developer!!

[filmkeeper]

Still photography? Just use photoshop or capture the scan with a good DSLR instead of a motion picture film scanner.

You don’t need to automate it, 36 frames should take anyone who’s experienced with photo editing maybe an hour to do.

[PM490]

@Filmfilmfilm

pano software operates in a 3d space as a camera is moving shot to shot.

There are settings that make that distinction. For example, Photostich (the software that comes free with Canon cameras) provides a setting for what you seem to be looking for. In this page scroll down to Adjusting the Merge Settings

Select one of the following shooting technique settings:

1. [Panning (Scenery, etc.)] - This technique involves the camera remaining in one spot and being panned up or down or left to right to shoot scenery and other images.

2. [Parallel Camera Movement (Documents, etc.)] - This technique involves moving the camera at a set distance from an object, such as a document, to capture it in sections.

3. [Images Scanned in Sections] - In this case, images that are too large to be captured in one scan are scanned by a scanner in sections.

Understand if you wish to build your own, but if I understood correctly, you were looking at item 3.

[friolator]

@Filmfilmfilm

Stitching I think is the incorrect word in this context.

Stitching is the correct term for piecing multiple images together to create a larger image (such as a panorama). I used to work for a software company (25+ years ago) that had a patented image stitching/stacking algorithm that could create hotspots of high resolution from low res video (with lots of stacked images, particularly if there were zooms), or you could stitch frames together to create larger images like complex panoramas that worked up and down as well as left and right. Stacking isn’t really what you want, because the two images from the scanner will be almost completely different with only a small overlap and most stacking algorithms are looking for large areas of overlap. Stitching is though. You would probably define a region of interest (the area of overlap between frames), and then the algorithm would compare the similarities between the two images in that ROI and piece the images together into one.

But you need sufficient overlap. It doesn’t look like the Cintel does a very big overscan, so to make this work you might need to machine the gate to be a little wider (assuming the sensor covers an area larger than the gate and would get you a few more pixels).

Additionally, depending on the content of the image, it may be hard if the area where the overlap is doesn’t have enough detail. Your example image is good, because there’s a nice object with unique edges for a detector to lock onto. But an image that’s mostly sky, for example, might be much harder to do.

You might want a fallback algorithm that takes a guess based on measurements made in previous successful stitches. That is, make an assumption that the position of the frame in the scan is consistent from shot to shot and then just blindly stitch based on those numbers without using the image as a reference. This may or may not work, depending on how consistent the Cintel is on the raw frame positioning.

Can you post the full res images of the basketball court somewhere?

FWIW, OpenCV can absolutely do this. It shouldn’t be especially hard to pull off. Image Stitching with OpenCV and Python - PyImageSearch

Even though these algorithms use perspective warping, that should only happen if there’s perspective to warp (like if you were standing there with a camera and physically moving, creating parallax shifts and taking each image from a slightly different position). That’s not happening here so it should just ignore the perspective stuff since it’s not present. I think. Haven’t played too much with OpenCV’s stitching algorithms yet, but it’s on my list!

[The_Projectionist]

Could you do a 3 perf scan on the Cintel to get the overlap you need?

I’ve used PTGui to stitch photos of large format Dufay slides with a less than ideal overlap with great results. You can also easily tweak things manually to get it just right.