That paper is actually really bad. It’s got so many technical errors that are easily fixed with a simple google search, and their methodology on some of the testing is deeply flawed. Some examples:
At one point they go on about only one or two of the scanners being able to reproduce a certain tint on a tinted B/W film. Yet, they use an iPhone and a lightbox to show you the correct color. As if a commodity phone camera has better sensitivity than a $200,000 purpose-built film scanner. The problem with this test is that it wasn’t performed correctly. If you look at the sample images that show the picture as B/W with little or no tint, look at the color of the white in the film perfs. Those are tinted. That’s because the scanner operators did an automatic base calibration on the film, and it “corrected” the tint out of the picture, simultaneously tinting the perfs kind of pinkish. This is scanning 101, and shouldn’t have ever made it into the paper. I know for a fact our ScanStation, which is a color bayer sensor, can reproduce similarly tinted films. You just have to know what you’re doing.
Another example is in the specs for the scanners: Many of the specs that were listed are either incorrect or outdated, and were at the time the paper came out. In some cases they were grossly wrong on things like sensor type and light source.
I think kinetta was the only manufacturer to perform tests themselves. The others were done at individual archives, some of whom specialize in B/W and don’t know how to properly scan color (this was confirmed by one of the paper’s authors to me). So you don’t have a good baseline of scanner operators - the manufacturers know their machines best and know how to deal with all different kinds of oddball situations. The operator of a scanner that hasn’t been updated in 5-6 years and is running outdated software (the Lasergraphics Director sample), shouldn’t be running the tests. That model didn’t even have the same sensor as current versions of that same scanner have.
Sorry, this one is a sore spot for me. I politely brought up about 3 pages of specific technical mistakes and issues with the authors of that paper and they corrected a handful of them. They continue to publish that with many of the same errors and it makes the rounds every few months as people discover it.