You’ve probably already seen this but this is an upscaled, HD version of the classic “Arrival of a Train at La Ciotat” (The Lumière Brothers, 1896)
More fascinating is the algorithms used to interpolate the missing frames:
and the AI used to upscale the images:
(use google translate on this one)
Potentially in reach, even for us.
To be painfully honest, I’m not a fan of upscaling or motion interpolation. That video looks good, and there are legitimate uses for both those things, but the whole AI enhancemet craze is really driving me nuts (it’s not your fault @junker).
AI still has a ton of problems. For one, using an image AI upscaler for video often can produce results that are not consistent between frames. Second, AI has a problem with generalization; AI is only as good as the data it’s trained on. For example, if I train an AI to upscale realistic images but I give it cartoon images, the AI will try putting realistic detail on cartoon images which will produce unnatural and dissatisfying results. There’s also the heavy performance cost and heavy memory usage (and even way more if you want to train one yourself). And there’s the problem of (at least for restoration purists) “was this extracted detail actually recorded in the image or hallucinated by the AI?” Explainability is, in fact, one of the biggest problems with all of AI in general.
I would imagine that most of these problems will get diminished as further advances are made (the activity going on in that field is ridiculously high). But for now, these problems are still pretty big problems.
At least they used DAIN for motion interpolation, which I’ve read about months ago and is definitely one of the best algorithms of its kind out there.
And at least they didn’t use DeepRemaster; the algorithm has a ton of practical problems because it tries to be an algorithm that does everything (which is a definitive path to failure and suffering).
I’ve been conceptualizing an ideal AI algorithm that wouldn’t have some of the problems of typical AI algorithms, but I always have a full plate with projects and stuff. Plus, I don’t have good enough hardware.
I get it! This forum is a big mix of archivists, hobbyists, and everyone in between. Honestly, I’m here because I’m making an 8mm telecine and have been interested in all the mechanics.
The upscaling is of interest to me so that I don’t have to keep chasing higher and higher resolution cameras and optics.
And the interpolation is especially interesting for 8mm shot at 16-18 FPS to really smooth things out.
My goal is for people to see Grandpa as a child and go, “Wow!” not necessarily to have a pixel perfect true archival copy. Of course, if I could do both, I’d love that too.
I’d definitely be interested in using the interpolation to get 16/18fps to blu-ray spec 24fps (technically 23.976)!
