4K Video Capture via iPhone


#1

I have for the past several years (on and off) been researching how to dramatically reduce the cost and time required to digitize 35mm microfilm collections. This includes researching how to do this mechanically and then how the software pipeline can work to right, de-skew, crop, OCR and index the content. The goal being that a researcher need not wait for the institution to digitize and make available all the materials in their archive. A motivated researcher could accomplish it in relatively short order. Furthermore this cost reduction would enable institutions to finally digitize all of their collections over time.

Even after the mechanical problems of capture there is a massive data throughput challenge to make the process as fast as possible while still retaining very good quality. This would require the sate of the art compression techniques and fast data storage and transfer from camera. A video process solves all of these problems. It dramatically speeds up the capture. Rather than using an expensive DSLR and taking full snapshots one at a time we can simply hit the record button and run through the film capturing at 60 frames per second 4K on the iPhone 8. The overlapping frames would then be stitched together much like a panorama of multiple shots. There is no more advanced data compression out there than what is used in motion video and having all the data happen in a single device reduces a lot of overhead.

It was only after looking at other options that realized what a great solution the iPhone really is for this. I Initially researched line scan cameras of the sort used by industry to do high speed qc on manufacturing or printing lines that move very quickly. Line scan cameras seem to fit the bill quire nicely. They use specialized cameras that take only a few lines of imagery at a time but are designed for capturing a fast moving subject that passes linearly across the sensor, no stopping to snap a photo. They process massive amounts of data so they have the fastest data interconnects from camera to storage/cpu. But this approach turns out to be extremely expensive.

After doing the math on 35mm capture with 4k on the iPhone I was amazed at the possibilities. It can be extremely fast. The video pipeline has been fine-tuned over decades and with very high ISO and shutter speeds. The shutter speed dictates how fast you can advance the film without blurring the subject. The frame-rate then determines how much overlap you are going to get. The iPhone 8 is good here because it allows for very fast capture at the highest shutter speed which gives 50% overlap on each 60th of a second frame.

Please check my basic thinking and math below, it may be the case that I am missing something but I believe the time has come for this approach of using video and stitching the frames together because it can be so much faster and the quality now with the latest cameras is there to make it happen.

How fast can we go?
Shutter speed is the limiting factor, advancing more than a pixel or so during the frame will introduce excessive motion blur. Increasing or decreasing framerate simply affects % of overlap from frame to frame and the file size.

~1px of travel during exposure allows shutterspeed pixels to advance within a second. 1/60th of this is the allowable pixels to advance per frame.

iPhone 8
3840 px by 2160 px
60 fps
ISO 20 - 1250
Max Shutter Speed 61440th (may be higher)

Microfilm High-Res Subject Capture Dimensions
3840 px = 18.3 mm, 209.836 px/mm, 5330 dpi
2160 px = 10.29375 mm
97.146 frames/m
30.5 m/microfilm (100ft)
2963 frames/microfilm

Max Possible
61440th shutter speed
61440 px/s or 292.8 mm/s
1024 px/shot or 4.88 mm/shot
3.4153 seconds/meter
104.2 seconds/microfilm

Advancing 1024px per frame at 60 fps
30 unique shots per second, 50% overlap

Microfilm Low-Res Full-Strip Thumbnail Capture Dimensions
1920 px = 28.5 mm, 67.36842 px/mm, 1711 dpi
1080 px = 16.01 mm
62.378 frames/m
30.5 m/microfilm (100ft)
1902 frames/microfilm

Max Possible
61440th shutter speed
61440 px/s or 912 mm/s
1024 px/shot or 15.2 mm/shot
1.096 seconds/meter
33.4 seconds/microfilm

Advancing 1024px per frame at 60 fps
30 unique shots per second, 50% overlap


#2

A totally viable option! It would, however, require intermittent motion so that the film stays still long enough to be captured.


#3

Not to my understanding, the film would be advancing at shutter-speed pixels per second as it recorded the video. That would mean ~1px of blur per exposure and as the example above shows 100 feet of 35mm film captured at 5330dpi in under 2 mins. What am I missing. Perhaps my expectation that there would be little blur here is off?


#4

I would love to do some experiments to test this out but I need to build the hardware to drive it first, I am going to try to use this smartphone film scanner as the basis and add Kinograph inspired driver to it. I suspect the backlight will need to be much stronger to drive the film at such high rates.


#5

Rolling shutter will cause distortion, especially at close distance and high speeds.


#6

Yeah this is the big unknown for me the shutter behavior is key.


#7

If the camera is oriented such that the rolling shutter rolls 90º down the film as it advances across it will create a shear effect (rather than a stuttering effect) that can be reversed with a simple affine transformation. This is the idealized result. Theoretically the affine transformation could be calibrated to the shutter speed and given the speed the film is advancing automatically apply the correct transform. Now that ignores all kinds of complications that can take place, my fingers remain crossed.Rolling-Shutter-man


#8

Definitely worth a test. You could make a small film loop and put it between a motor and a free-spinning pully to test. Let me know if you need help setting that up!


#9

Ah thats a great idea. I am starting from scratch on the drive, what would you recommend for motors and controller hardware, familiar with Arduino and r-pi.


#10

For $500 I can sell you a Nikon CoolScan 4000 film scanner that will feed an entire roll of 35mm film of up to 48 frames through automatically. It will capture each frame at 4000 dpi and you probably won’t even need to post process. It will be slow, but you can turn it on, go get a sandwich, and come back to perfect high resolution scans without the months of trial and error and heartache. And the whole thing will be cheaper than just the iPhone 8.

Of course, if your real goal is to build something cool then I’m all for it. Even the Kinograph after 2-3 years still isn’t a finished product, but folks have had a great time building and modifying it.

Enjoy!


#11

Yeah I have an old CoolScan and I believe you could adapt it to scan through a whole microfilm roll. The problem is that it is slow. Even the fastest 35mm film scanner available on the market for tens of thousands of dollars is too slow. So yeah I guess I want to make something cool. The basic math says you should be able to capture a full 100 foot roll of microfilm in under 2 mins. It would be insane if I could get near to that level of performance and not have it cost thousands.


#12

My first trial revealed some limitations that need to be overcome.

Setup: Manually feeding film through a modified Lomography Smartphone Film Scanner with the aid of an Olloclip 7x macro lens on an iPhone 6s recording with the FiLMiC Pro App in 4K.

53%20AM

Analyzing Results: I was able to capture solid frames at a slow rate. At a faster rate the video compression really begins to degrade the information. So I am looking into the possibility of writing a custom iOS app that will dump raw video frames rather than run everything through h264 compression. If that is not possible a trial with a more professional level 4-6k video camera that supports raw video will be required.

I did a quick photomerge of a few frames in photoshop after correcting for lens distortion. This is all very quick and dirty but I think it proves out the basic premise. There is much I could do to improve the setup but clearly newsprint is really stretching the limits of 4k resolution.

The above was resized by the forum software, here is the full resolution file - https://www.dropbox.com/s/vo9w1oke54zmciy/1984nyt.jpg?dl=1


#13

@loot great to know you are a Swift developer!! We just might need that :slight_smile: And thanks for sharing these test results. It makes me so excited to see people taking action on their hunches and coming back to the group to share the results. Keep pushing it forward!

M