Rather than thinking of colorization (or upscaling) as a thing we do "to" a photo to alter it further from a primary-source ground-truth; another perspective (the one shared by art conservators, I think) would be to think of a photo that was taken in a too-low resolution, or in B&W (when that wasn't intentionally done for artistic effect) as a kind of "damage" done to the photo right at the time of its creation, where that "damage" moved the photo further away from achieving the photo-taker's intent (which usually, with most photos, is to losslessly convey the experience of looking at the subject of the photo, as it looked at the place and time the photo was taken.)
Colorization, upscaling, etc. could then be thought of as repairing this damage, bringing the photo closer to being a lossless communication of the information it was created to convey.
(By analogy: imagine a technology that could analyze an illustration drawn by someone whose hand was shaking due to Parkinsonism, and then compute what the illustration would have looked like if the artist's hand hadn't been shaking. That would also be a kind of "repair.")
> which usually, with most photos, is to losslessly convey the experience of looking at the subject of the photo, as it looked at the place and time the photo was taken.
That’s quite a strong statement, and one that I think many photographers would disagree with.
Snapshots aside, this severely understates the artistic choices a photographer makes. For one thing, black & white and color read very differently to the eye, and offer different palettes for composition (although it’s not about photography, Scott McCloud’s “Understanding Comics” has a great exploration of the effects of b&w vs color and is a fun read to boot). Basically all b&w cinematography also gives the lie to this statement: it’s impossible to honestly argue that a colorization of, say, the stark expressionism of Night of the Hunter would be a repaired version.
The extent to which a photo is, or is even intended to be, a representation of “primary-source ground truth” is something we’ve been arguing since, like, Edward Said. I’m not saying images should never be colorized, or upscaled, or whatever. Obviously that’s not the case. Peter Jackson’s WWI documentary, as another poster mentioned, is a triumph. But it’s too complicated a conversation to dismiss by declaring, in one sentence like Moses coming down from the mountain, that the purpose of photography is the “lossless communication of information”.
I'm not trying to argue that one should look at the colorization of "art photos" as a repair/restoration. Artists of all kinds indeed choose their medium carefully, and their work is, well, their work. It's an expression of their choices made in response to the constraints of the time. The best way to conserve art, is to try to restore it to exactly the way it was when the artist created it; not to try to second-guess the artist.
But most photos are not intended as art. Most photos are candids (or portraits, or photojournalism.)
Or, to say that another way: most people who take "photos"—both historically, and today—are not "photographers" by profession, trying to paint with a lens and film. They're just people who want to preserve a view of something, and have a convenient technology for doing just that.
If you want a general policy on how to conserve photos, it should be based around the idea that unless you know better, you should assume that the taker of a photo very likely wasn't an artist choosing their medium for effect; but rather a person constrained by the media available at the time of the photo's creation. Most photos that were taken in the period when B&W photos were the only photos you could take, were not intentionally B&W. If given the option, a color photo would have been taken instead. These photos are, in est, "damaged" color photos. As if the taker expected a color photo, but due to bad camera settings, a B&W photo was taken instead. The equivalent of a photo taken when there was a hair on the lens—an incidental aberration.
Just like, for example, most photos taken at 320x240, when that was the best quality a digital camera could give you, were not intentionally taken at 320x240. It was just the fact of the photo-taker being unable to afford a fancy film camera and rolls of film, but indeed being able to afford an early-2000s webcam and no rolls of film.
Or, for another example, most people who recorded mono audio of themselves onto wax cylinders, early reel-to-reel tape, etc., weren't going for the "mono audio mix-down sound." They just were in possession of only a single-track audio storage or distribution technology. Remastering the sound with source-separation more faithfully achieves the sound they wanted to record, but didn't have the technology for. (Do the surviving Beatles disapprove of the stereo remasterings of their mono albums? No; they think they're grand!)
I think we actually broadly agree. I was thrown by the mention of art conservationists, but probably took it too literally.
When it comes to archival conservation and restoration, it seems to me that there are two opposed goals which both have value: restoring to the highest-quality version we possibly can, and preserving the material as it would have appeared to its original audience. So for the wax cylinder example, we definitely want to produce the best, truest to intent restoration we can, but at the same time the experience of listening to a wax cylinder recording is also worth preserving.
As long as the restoration is non-destructive (which it always should be, with digital tools), that’s great! We can have it both ways.
There’s an interesting modern twist when it comes to vernacular photography, too. The popularity of instagram filters and the like suggests that there’s something more going on now even with snapshots.
When an image is converted to black and white, information is destroyed. This information can never be recovered without access to the true colors in the scene when the image was taken. You can tell that information is destroyed because there are multiple possible re-colorizations of a black and white image.
When you convert a black and white image back to color, you are not restoring the original true colors, rather you are filling them in based on your priors. It is logically impossible to "restore" the original colors by doing data processing on only the image itself. It is only possible to give the appearance of a restoration.
Yes, and? Art conservation isn't about recovering the original information. It's about making the work do the job it was intended to do.
If there's a big hole in the middle of a painting, the painting is currently doing a bad job of being a painting, because the hole is distracting people from looking at and experiencing the rest of the painting.
If you, as a conservator, fill in the hole (in a reversible way) with, well, anything, it'll likely stand out less than the hole itself did. And, therefore, the painting will now have been "repaired", in the sense that it's now doing a better job of communicating the art that the painter intended to communicate by painting it, than it did when there was a big hole in it.
Same thought process with colorization. A B&W's medium distracts the modern viewer, who is used to color photos, from the photo's intended message. For modern B&W photos, the B&W look is part of the message; but for B&W photos taken when B&W photos were the only kind of photos (especially candid or portrait photos, rather than "art" photos), this choice of medium is mostly unintentional.
Colorizing the photo—even somewhat poorly, like with old Technicolor movies—will result in the viewer of the photo being less distracted by the photo's medium than they would by a B&W photo; and therefore paying more attention to the content of the photo itself. Which is, in most cases, what the photo's taker would want.
Colorization, upscaling, etc. could then be thought of as repairing this damage, bringing the photo closer to being a lossless communication of the information it was created to convey.
(By analogy: imagine a technology that could analyze an illustration drawn by someone whose hand was shaking due to Parkinsonism, and then compute what the illustration would have looked like if the artist's hand hadn't been shaking. That would also be a kind of "repair.")