What happens if this actually picks up steam, and suddenly PNG is no longer one format, but a bunch of incompatible ones that look somewhat similar, whose fidelity depends on your renderer?
Early in PNG's history, we already had this issue with alpha channels, progressive rendering, and hit-or-miss support for APNG (animated PNGs, meant to replace GIFs but never happened).
It was also an issue for a long time for PSDs and SVGs, where the same file never looked the same on two browsers/devices/apps/versions.
I would bet that these days, generating or decoding PNGs is the bottleneck almost nowhere, but extending the format would cause problems everywhere in real-world usage. Apps and companies can no longer tell whether there's something wrong with their image or if somewhere in the pipeline, some new graphics designer decided to use a bleeding-edge version of a 30-year-old graphics format that nobody else accounted for, and it looks "broken" in half the browsers now. A format can still look broken even if it's "backward compatible", just by virtue of having some features (like HDR) that are only displayable in some renderers but not others.
Why not just make a new format instead and have browsers & devices fall back as necessary, like we already do with webp and srcsets?
PNG was always set up for extension. In particular, it has a clever way of allowing ancillary data sections to be marked as unimportant, so a decoder knows whether it can skip them or report the file as unreadable.
I suspect the big thing these days would be to support brotli and zstd.
A problem that often comes up with extensible formats is that whomever comes along and implements them assumes exactly the test cases they came up with, which can often mean "just the files I have for this project" or "just the output of the encoder I have".
So there will be formats that can reorder the chunks, and those minimum-viable readers will all break when they encounter a file from a different source, because they hardcoded a read order. This leads to an experience on the user's end where they need a "fixer" tool to reorder it to deal with the bad decoder.
There were tons of old binary formats that were like this. It can happen with text, but it's less likely to, because a larger proportion of textual formats build over a container like XML or JSON to offload the text parsing, and then they end up with some pre-structured data.
> There were tons of old binary formats that were like this. It can happen with text, but it's less likely to, because a larger proportion of textual formats build over a container like XML or JSON to offload the text parsing, and then they end up with some pre-structured data.
Note that PNG also "build[s] over a container", since it's a descendant of IFF.
Many formats have stuff like that (like cover art in MP3 ID3 tags), but usually they're used for, well, ancillary purposes.
It's dangerous to use this to change the actual primary output of the file (the image), especially in a way that users and editors can't easily detect.
I would say at least in the context of extra data to extend the bit depth for HDR, that data could be considered ancillary?
We've been rendering images in SDR forever, and most people don't have HDR capable hardware or software yet, so I don't know how you could consider it as broken to render the image without the HDR data?
I’ve seen countless of issues where you place a PNG logo on top of a css background:#123456 and expect the colors to match, so the logo blends in seamlessly to the whole page.
On your machine it does and everything looks beautiful. On the customer machine with Internet explorer they don’t, so the logo has an ugly square around it.
The difference in experience for seeing a black or white background instead of a transparant one on the one hand, and missing HDR on the other is pretty big.
95%+ of humans won't even notice HDR being missing. Everyone with eyes will notice a black or white square.
Nobody said anything about black or white. Try googling for png color problems and you’ll find thousands of questions, in all kinds of tools and browsers. The css color and the png color need to match exactly. Just a slight difference will look off if placed next to each other. The risk of css and png rendering two same hex-codes differently increase when putting semi-supported hdr extensions in the mix.
For this particular use case, yes, transparency is more suitable than trying to match.
You could add an "ignorable" zstd-compressed IDAT variant, but that wouldn't give you backwards-compat in any useful way - the zlib-compressed IDAT still has to be present, and unless you fill it full of zeroes or a placeholder image, the overall file is going to be larger.
I don't know for sure but when I used to repair them you could watch them load up the APNGs on boot. The first time I saw them in the wild was around 2007
> I would bet that these days, generating or decoding PNGs is the bottleneck almost nowhere
The bulk of my game's startup time is spent decoding PNGs via libpng. There are some alternatives to libpng like fpng[1], or alternate image formats like QOI[2]
These both exist because png is slow and/or complicated.
> these days, generating or decoding PNGs is the bottleneck almost nowhere
Anecdotal, but I'm familiar with a system which spends ~50% of CPU cycles on PNG encoding (most of which is actually spent in zlib compression).
The other approaches I've seen involve creating performance-focused forks of zlib (e.g. zlib-chromium and zlib-cloudflare). This has benefits beyond PNG encode / decode:
It's a Java system, so not quite so simple. Maybe it's worthwhile to create some Java bindings? Recent JDKs make it feasible to swap out the underlying zlib implementation, so swapping out zlib-madler with zlib-cloudflare or zlib-ng might provide the best cost/benefit.
Very cool, thanks for the pointer! We might be able to run an internal test to check performance vs. a zlib replacement, but I think that AGPL license is going to be a showstopper for anything else...
Maybe this is the genius in it.
In order to get everyone to move to better formats, you just break everything in the one everyone uses, so they have to move.
Like Twitter becoming X to push everyone to mastodon. Mastodon is better in every way, so it's a net win.
But most people didn’t move to Mastodon, and a bunch moved to incompatible networks like Bluesky, cohost, etc. Which is really the kind of problem you don’t want to have when you're just posting a photo.
For redistribution, JPEG or AVIF or WebP. I'd choose JPEG myself... good enough compression with broad compatibility. You'll almost never run into an issue with it. AVIF and WebP are still full of gotchas, so I wouldn't deploy them without a JPEG fallback.
For sharing masters/originals, it just depends on your team's needs. Probably saving the camera RAW is a good idea, but usually there's been edits done to it, so some Photoshop or Lightroom native format, for example, or at least a TIFF if you need broader compatibility.
(edit: The answer should logically follow from the following, but as a TL;DR: yes, they must have meant jpeg or similar, although I disagree about it being such a "please don't". Feel free to, they'll take more bytes but it's not like you're losing information in the process, so you can always still switch to jpeg.)
Photos have lots of noise anyway, so a good jpeg is not going to be the limiting factor. Due to jpeg being lossy, the file size is much smaller for the type of images that pictures are (I don't know a formal definition, perhaps something with some amount of random noise? Or that there's lots of gradients with nonlinear slopes or so?).
PNGs are lossless, and encoding photos in a lossless way takes a relatively large amount of space. On the other hand, the quality degradation of running text through a jpeg encoder is noticeable even when tuning the jpeg compressor for it, so a screenshot with text is always PNG. For designed things such as icons or logos or so, the file size is small enough to be a no-brainer for getting PNG's lossless quality.
Vector, such as SVG, is the ultimate format: the highest possible quality and also the smallest! This only works when you've got access to the source (screenshots already don't work because you're capturing a blend of objects rendered as an array of pixels), and especially not suitable/possible when you're reading from a sensor such as a digital camera. This format doesn't describe pixel patterns, but rather shapes such as "draw a red line from coordinates 3,94 to 1920,1027". Because you can't always use it (you can always render an SVG as a grid of pixels and turn it into a PNG or anything else, but not the other way around), support is a bit lower and you'll see them less often, but imo it's always worth striving for when the platform allows
For browsers, there are extensions that can create SVG screenshots. They work by either copying or inlining CSS into the SVG. They don't work all the time, but worth giving a try first.
It seems to capture the HTML and convert each element to a similar one on SVG. I'm still not sure how it handles raster images (does it just embed them? Vectorize them somehow?) but it's a really cool project. Thanks for sharing!
Minor nitpick, just for clarity: Vectors aren't really the "ultimate" format, they're just a different format for storing a different kind of information.
Vectors are good for things that can be mathematically described/simplified without significant loss of information (such as curves, shapes, some gradients, etc., and compositions of those) Many logos and fonts fall into this category, for example, and some clip-art do as well. For the appropriate graphics, vectors can give you higher quality and smaller file sizes. But it's not necessarily the right choice for everything. Real pixels do hold data, and vectorizing them will typically cause some data loss. You COULD vectorize photos and illustrations (vectormagic.com is a good one for that). But you just end up with a poorer approximation of the original raster data, ending up with a bunch of pixel-like polygons, because there's not really a better approximation of that high-resolution pixel data that can be easily described in shapes and lines.
Rasters are still superior not only for photographs, but for other uses outside of basic image redistribution, such as GIS data layers (where sensor data IS what's important), bump maps and terrain maps for games, 3D voxel images for medical imaging, live-action movies (which can interpolate complex scenes between raster frames), astronomy, etc. Even if you could vectorize the sensor data in those situations, you often wouldn't/shouldn't.
> Real pixels do hold data, and vectorizing them will typically cause some data loss.
Chickens and eggs. If you have vector data already, it's better to distribute the raw vectors than to rasterize them. If you only have raster data, distribute the rasters (where viable) instead of trying to trace them. Lossy operations are lossy.
Vectors are largely preferable to rasters, but that doesn't mean you can reproduce them if you only have the rasters. Just like distributing source code is usually preferable to compiled binaries, but that doesn't mean that we just pump everything through Ghidra and call it a day.
Yeah, exactly. The takeaway IMO is that there are different formats best for different kinds of data. There's no "best" or "ultimate" format, it just depends on your use case.
If I squint hard enough, jpeg starts to look like a very esoteric spin on svg, a quilt of tiny svg-like tiles whose parameters are defined by an encoder analyzing a bitmap rather than a human visualizing a concept.
> Mastodon is better in every way, so it's a net win.
Making easily disprovable statements is not the way to win people to your side.
Searchability is weaker. Login is all over the map. Links don't work. etc.
These are the standard problems with non-centralized software.
I really don't understand why Mastodon didn't use/create something like the "magnet" links used for torrents. That way, even if you lost the server, as long as you touched something that had the appropriate hashes, you can access the information.
I use Mastodon, but it is not better in every way.
It seems like it’s not going to look broken? Unlike, say, the difference between black & white and color TV, or an animated image that doesn’t animate, it will be a subtle difference that most users won’t notice or care about. Some designers may be annoyed, but it doesn’t seem like that big a deal.
Sorry, but did you read my post? It's only backward-compatible in the sense that existing renderers can display SOMETHING -- but it's not the same image.
From the article:
> [...] like how color TV was introduced but it still worked on B&W TV's. New features can be added, as long as existing decoders ignore them and return a viewable image
Keyword "ignore them". To my reading, this means that CPNGs will contain several images inside them: A full-featured version with "color" (or HDR, or or whatever) for newer renderers, and a fallback one (in "black and white" in his example) for existing renderers.
It's not really "backward compatible", but more like "fallback-able".
The first two extensions (constrained deflate, multithreaded encoding/decoding) only present optimization opportunities for the decoder. It is still the same image.
Single-threaded decoding of large PNGs is enough of a bottleneck that Apple has their own version of PNG with multithreading support.
That was not my read. When discussing HDR it’s clear that ONLY the extra bits are stored in a new data structure and are applied to the base image at display time.
So that gets you a bit-for-bit compatible image on old systems and HDR on new systems without duplicating data.
I believe that pattern applies throughout: a baseline, standard PNG image plus extra instructions and data for adding information to it.
Sorry, I didn't mean in terms of the codec or data deduplication. You're right, it's probably not implemented as actually having different images in the same file. That would be quite silly. Sorry for my ambiguity.
But functionally, this means that two renderers would have different outputs for the same file. It's not so bad when it's just HDR, but if this continues to grow, it WILL get bad. We've seen it many, many times before with hacked-on file extensions.
If it were Microsoft doing this (it's not), it would be just yet another embrace-extend-extinguish effort. But it's not. I have no doubt this author has good intentions, and this is a cool hack, but I think it will lead to confusion and poor outcomes in widespread usage.
It's really not great when an image format looks "somewhat" the same between renderers. In fact, I think that's worse than it not working at all. From experience, it leads to these situations where the same image can fragment over time just by virtue of being worked on by different people in different environments (apps, OSes, browsers, devices), and there's not really a clear way to tell which is which without extensively analyzing the metadata.
If Joe starts work on a HDR version, sends it to Sarah, Sarah adds a bunch of new edits but then saves it in an app that recompresses the image and overwrites the HDR, Joe might never know. And then if the format continues to expand, maybe a bit later Jane and Janet each add mutually incompatible extensions to the file without knowing it. The "base" image might look the same to all of these people, but each had specific edits in mind that the others might not have known about, because their renderers LOOKED like it was rendering the same image when it wasn't.
This isn't a hypothetical... it happened a lot when PNG first came out, and with PSD (Photoshop files) for a good decade or two until the subscription model came out.
If someone edits a png I expect them to output a new file. So if they use the SDR version as a starting point... they use the SDR version, that's it, nothing else breaks. Everyone sees the same output from Sarah. You can have the same kind of issues with baseline png already; not every tool does all colors or editing effects correctly, and if someone uses a worse tool to edit then that's unfortunate.
Right, but not everyone is going to start from the SDR. Some might have the HDR version and not realize they're destroying the info. It gets worse the more forks and features you have.
There's just no advantage in overloading an old graphics format for something like this in a world where WebP and AVIF already exist.
As for editing tools, at least they're targeting the same baseline format (maybe with bugs). This is purposefully introducing new features that can only be used in some places, outside of the agreed standard.
Sure there is an advantage, I can put a png file everywhere without needing to care if the users system supports the new capability.
For a (made up) example let's say Firefox supports hdr, but Chrome doesn't, if I want to include an HDR image into a website, but display a SDR image if the browser can't display it, I need to implement a test for the browser to serve decide if I serve a WebP or PNG. That is a real cost.
What you see as a positive I see as potential for implementation details to diverge and cause user issues. We already have issues with WebP and AVIF rendering in some versions of Safari, and those are well established standards, not a lone wolf extension.
In theory an extended hacked PNG would have perfect backward compatibility, but the complexities of codecs more likely means it's going to appear as bugs in several implementations. Best case it falls back to SDR, but it's also possible some broken implementation leads to HDR content being displayed in a SDR setup and everything getting all washed out.
Having the same file format/extension actually host several different versions never works well. Graphic artists and support teams have to deal with rendering quirks like this all the time because some developer thought their hack was a genius overload :/
I have bad news for you: the existing PNG standard specifies the following optional chunks that compatible renderers are free to follow or ignore, producing different renderings of the same image:
Once stable, yes, absolutely! There should be a reference standard that renderers adhere to. Then they should be versioned for future improvement. That doesn't mean there can't be bug and security fixes, but they should be feature-stable.
There's a reason why JPEG and PNG are so popular: because they ARE stable and unchanging, in a world of failed adoption for APNG, WebP, HVEC, JPEG 2000, JPEG XL, AVIF, etc.
For image formats, broad compatibility is way way way more important than small gains in codec performance and filesize.
----
Edit: From memory, this is something we learned the hard way back in the 90s and early 2000s, when image (and other) formats saw very rapid development and bandwidth was still hard to get. PNG was a huge improvement over GIF, and that mattered a lot over 56k modems. But the road to its adoption was very difficult, with many features only partially supported for years in different apps.
When WebP launched, it was objectively better in most ways compared to both PNG and JPEG, but it still saw limited uptake. Part of that was the titans fighting amongst themselves (other tech companies didn't want to adopt a Google format), but also by that point both JPEG and PNG were good enough and connections & CPUs were good enough. The incremental improvement it offered rarely justified it sometimes not working for some users on some browsers.
It's a similar situation for many other formats: H.264 in a MP4 is good enough, MP3s are still good enough, etc. On the other hand, PDFs and DOC/DOCX have different features depending on what reader you use (such as PDF forms, JS validation, accessibility, or DOCX not rendering the same on different readers), and it's a mess.
For a given image, that's possible, but if memory serves, the good enough defaults usually led to the WebP version being smaller across a library of image samples, statistically speaking. But that may not have been true for an individual image.
And either format could be manually optimized even more.
> Early in PNG's history, we already had this issue with alpha channels, progressive rendering, and hit-or-miss support for APNG (animated PNGs, meant to replace GIFs but never happened).
Don't forget pixel aspect ratio. Oh wait, most viewers still ignore that.
Early in PNG's history, we already had this issue with alpha channels, progressive rendering, and hit-or-miss support for APNG (animated PNGs, meant to replace GIFs but never happened).
It was also an issue for a long time for PSDs and SVGs, where the same file never looked the same on two browsers/devices/apps/versions.
I would bet that these days, generating or decoding PNGs is the bottleneck almost nowhere, but extending the format would cause problems everywhere in real-world usage. Apps and companies can no longer tell whether there's something wrong with their image or if somewhere in the pipeline, some new graphics designer decided to use a bleeding-edge version of a 30-year-old graphics format that nobody else accounted for, and it looks "broken" in half the browsers now. A format can still look broken even if it's "backward compatible", just by virtue of having some features (like HDR) that are only displayable in some renderers but not others.
Why not just make a new format instead and have browsers & devices fall back as necessary, like we already do with webp and srcsets?