4K Video Capture via iPhone


#1

I have for the past several years (on and off) been researching how to dramatically reduce the cost and time required to digitize 35mm microfilm collections. This includes researching how to do this mechanically and then how the software pipeline can work to right, de-skew, crop, OCR and index the content. The goal being that a researcher need not wait for the institution to digitize and make available all the materials in their archive. A motivated researcher could accomplish it in relatively short order. Furthermore this cost reduction would enable institutions to finally digitize all of their collections over time.

Even after the mechanical problems of capture there is a massive data throughput challenge to make the process as fast as possible while still retaining very good quality. This would require the sate of the art compression techniques and fast data storage and transfer from camera. A video process solves all of these problems. It dramatically speeds up the capture. Rather than using an expensive DSLR and taking full snapshots one at a time we can simply hit the record button and run through the film capturing at 60 frames per second 4K on the iPhone 8. The overlapping frames would then be stitched together much like a panorama of multiple shots. There is no more advanced data compression out there than what is used in motion video and having all the data happen in a single device reduces a lot of overhead.

It was only after looking at other options that realized what a great solution the iPhone really is for this. I Initially researched line scan cameras of the sort used by industry to do high speed qc on manufacturing or printing lines that move very quickly. Line scan cameras seem to fit the bill quire nicely. They use specialized cameras that take only a few lines of imagery at a time but are designed for capturing a fast moving subject that passes linearly across the sensor, no stopping to snap a photo. They process massive amounts of data so they have the fastest data interconnects from camera to storage/cpu. But this approach turns out to be extremely expensive.

After doing the math on 35mm capture with 4k on the iPhone I was amazed at the possibilities. It can be extremely fast. The video pipeline has been fine-tuned over decades and with very high ISO and shutter speeds. The shutter speed dictates how fast you can advance the film without blurring the subject. The frame-rate then determines how much overlap you are going to get. The iPhone 8 is good here because it allows for very fast capture at the highest shutter speed which gives 50% overlap on each 60th of a second frame.

Please check my basic thinking and math below, it may be the case that I am missing something but I believe the time has come for this approach of using video and stitching the frames together because it can be so much faster and the quality now with the latest cameras is there to make it happen.

How fast can we go?
Shutter speed is the limiting factor, advancing more than a pixel or so during the frame will introduce excessive motion blur. Increasing or decreasing framerate simply affects % of overlap from frame to frame and the file size.

~1px of travel during exposure allows shutterspeed pixels to advance within a second. 1/60th of this is the allowable pixels to advance per frame.

iPhone 8
3840 px by 2160 px
60 fps
ISO 20 - 1250
Max Shutter Speed 61440th (may be higher)

Microfilm High-Res Subject Capture Dimensions
3840 px = 18.3 mm, 209.836 px/mm, 5330 dpi
2160 px = 10.29375 mm
97.146 frames/m
30.5 m/microfilm (100ft)
2963 frames/microfilm

Max Possible
61440th shutter speed
61440 px/s or 292.8 mm/s
1024 px/shot or 4.88 mm/shot
3.4153 seconds/meter
104.2 seconds/microfilm

Advancing 1024px per frame at 60 fps
30 unique shots per second, 50% overlap

Microfilm Low-Res Full-Strip Thumbnail Capture Dimensions
1920 px = 28.5 mm, 67.36842 px/mm, 1711 dpi
1080 px = 16.01 mm
62.378 frames/m
30.5 m/microfilm (100ft)
1902 frames/microfilm

Max Possible
61440th shutter speed
61440 px/s or 912 mm/s
1024 px/shot or 15.2 mm/shot
1.096 seconds/meter
33.4 seconds/microfilm

Advancing 1024px per frame at 60 fps
30 unique shots per second, 50% overlap


#2

A totally viable option! It would, however, require intermittent motion so that the film stays still long enough to be captured.


#3

Not to my understanding, the film would be advancing at shutter-speed pixels per second as it recorded the video. That would mean ~1px of blur per exposure and as the example above shows 100 feet of 35mm film captured at 5330dpi in under 2 mins. What am I missing. Perhaps my expectation that there would be little blur here is off?


#4

I would love to do some experiments to test this out but I need to build the hardware to drive it first, I am going to try to use this smartphone film scanner as the basis and add Kinograph inspired driver to it. I suspect the backlight will need to be much stronger to drive the film at such high rates.


#5

Rolling shutter will cause distortion, especially at close distance and high speeds.


#6

Yeah this is the big unknown for me the shutter behavior is key.


#7

If the camera is oriented such that the rolling shutter rolls 90º down the film as it advances across it will create a shear effect (rather than a stuttering effect) that can be reversed with a simple affine transformation. This is the idealized result. Theoretically the affine transformation could be calibrated to the shutter speed and given the speed the film is advancing automatically apply the correct transform. Now that ignores all kinds of complications that can take place, my fingers remain crossed.Rolling-Shutter-man


#8

Definitely worth a test. You could make a small film loop and put it between a motor and a free-spinning pully to test. Let me know if you need help setting that up!


#9

Ah thats a great idea. I am starting from scratch on the drive, what would you recommend for motors and controller hardware, familiar with Arduino and r-pi.


#10

For $500 I can sell you a Nikon CoolScan 4000 film scanner that will feed an entire roll of 35mm film of up to 48 frames through automatically. It will capture each frame at 4000 dpi and you probably won’t even need to post process. It will be slow, but you can turn it on, go get a sandwich, and come back to perfect high resolution scans without the months of trial and error and heartache. And the whole thing will be cheaper than just the iPhone 8.

Of course, if your real goal is to build something cool then I’m all for it. Even the Kinograph after 2-3 years still isn’t a finished product, but folks have had a great time building and modifying it.

Enjoy!


#11

Yeah I have an old CoolScan and I believe you could adapt it to scan through a whole microfilm roll. The problem is that it is slow. Even the fastest 35mm film scanner available on the market for tens of thousands of dollars is too slow. So yeah I guess I want to make something cool. The basic math says you should be able to capture a full 100 foot roll of microfilm in under 2 mins. It would be insane if I could get near to that level of performance and not have it cost thousands.


#12

My first trial revealed some limitations that need to be overcome.

Setup: Manually feeding film through a modified Lomography Smartphone Film Scanner with the aid of an Olloclip 7x macro lens on an iPhone 6s recording with the FiLMiC Pro App in 4K.

53%20AM

Analyzing Results: I was able to capture solid frames at a slow rate. At a faster rate the video compression really begins to degrade the information. So I am looking into the possibility of writing a custom iOS app that will dump raw video frames rather than run everything through h264 compression. If that is not possible a trial with a more professional level 4-6k video camera that supports raw video will be required.

I did a quick photomerge of a few frames in photoshop after correcting for lens distortion. This is all very quick and dirty but I think it proves out the basic premise. There is much I could do to improve the setup but clearly newsprint is really stretching the limits of 4k resolution.

The above was resized by the forum software, here is the full resolution file - https://www.dropbox.com/s/vo9w1oke54zmciy/1984nyt.jpg?dl=1


#13

@loot great to know you are a Swift developer!! We just might need that :slight_smile: And thanks for sharing these test results. It makes me so excited to see people taking action on their hunches and coming back to the group to share the results. Keep pushing it forward!

M


#14

Sorry to ressurect and old topic but I feel I have info that could help the original poster or someone else. I experimented trying to do a budget telecine setup to capture old 8mm movie and tried a few camera and method using old film transfer box and setup and found that the best way to achieve this was using my iPhone 8 in conjunction with an advanced camera apps like ProCam. It is not free but it is under 10$ and allowed me to change crucial parameters that the stock iPhone camera would not give you the opportunity to alter that basicly made my phone the most suitable camera I had on hand to do this job. This app is made to give you some control over your phone canera similarly as you would with a DSLR. Settings like exposure, shutter speed, fps tweaking, white temperature and ability to shoot in raw for picture. Having control over thos settings made a huge difference and gave me very decent result with very limited budget and material. Sure your goal is a little more complicated but I feel that it might be one of they key component for it to suceed doing it with an iPhone. Sure you can use 500$ to buy a specially made machine but it can be a huge sum of money for someone planning to transfer a few family movies and most people have decent phone already so I do not see it as just building something cool personnally. And if you find a way to do the scanning and stiching of picture properly, you might end up with surprinsingly high quality result. I’m pretty satisfied with my transfer and they were done with way less tought put into it than what you are planning to do! And in the end, a decent transfer is way better than no transfer at all in my opinion. You can wait all your life to have the perfect setup, best quality etc… but your films are degrading as we speak and might has well have something and then you can always work later on bettering your technique.


#15

WE’d love to see your results, @SkinnyV! Can you post a video? Also, what size file were you scanning?

I have a design in mind for a scanner that uses phones for digitizing 8mm film. I hope to get to it one day soon!

Matthew


#16

As I said, my setup is not as involved as what you guys are doing and its not a frame by frame capture so the quality is obviously not comparable. I basically did an old school filming of the projection but by being careful to play with the iphone camera settings to fix the shutter speed, fps etc. That way, I was able to transfer them quickly, without purchasing anything (except procam to change the exposure and shutter setting of my iphone) and they look ok when viewed on a big HD TV so it was good enough for my need until I can put more effort in a proper frame by frame transfer. I have yet to play with the final product encoding so it is not as good as my lossless files but here’s a quick sample: https://www.youtube.com/watch?v=FvW5o0kjjKs.

I’m sure that with an advanced camera apps like procam and a diy frame by frame system, you would get much better result.


#17

Please detail your system along with details on the Camera settings.


#18

I shot them with a projector with a LED bulb mod I made to replace the halogen bulb and recorded them at 24fps in 1080p and for the other setting I would play a bit with them as I went depending of the movie to compensate for the occasionnal flicker. I just wrote an article with picture on my website if anybody is interested. It’s still rough since I wrote it pretty late but the there’s a few pictures. It’s nothing fancy compared to what you guys are using to do frame by frame capture but I got good enough result for my use and it’ll definitly be my next project to try and make a proper rig where I could do 8mm along with 16mm and 32mm. I’m hoping to be able to use my phone again for that.


#19

Simple Queston: How about the “Automatic” Focus problem in an iPhone? The closer you go with your iPhone the more it tries to Focus. For each Image. I’m pretty sure thats going to be an issue.


#20

@SkinnyV I love this hack. The problem-solving, the use of existing parts, etc. It’s really great. Thanks for sharing the well-written and thorough documentation of your process. It was a pleasure to read!