Energy efficiency of modern codecs
By @edent
· compression power · 2 comments · 550 words · read ~1,130 times.
How efficient are modern codecs? Can we ever work out whether the power use of compression algorithms is a net gain for global power consumption?
Come on a thought experiment with me.
I have invented a new image compression format. It shrinks images to 50% smaller sizes than AVIF and is completely lossless. Brilliant!
There's only one problem - it is 1 million times slower.
If it takes your computer 10 seconds to compress an AVIF, it'll take 115 days to compress using my new format.
Is that worth it?
There are five aspects to think about here.
Time
If you're someone like, say, Netflix - this might be fine. You can throw a whole bunch of cloud servers at the problem, so you're not time-bound.
If you're a smaller service, or if you require instantaneous image compression, this is a pointless format. The time-cost - let alone processing costs - is prohibitive.
Space
If you're a library with billions of images - storage costs are a real concern. Halving those costs could save you more than it would cost to recompress all your images.
But storage is pretty cheap these days. Terabytes of storage are easily in range of home users. Is the cost of encoding worth it given that space isn't a limiting factor?
Transit
The Netflix logo is downloaded a hundred million times per day. So would a massive saving in bandwidth costs be worth the trade-off? Probably, yes.
If you pay per MB for your Internet access, you'll probably welcome this new codec. Saving money is great.
But how much does bandwidth really cost? Is it that expensive for either party?
User Experience
I've only mentioned the speed of encoding so far. Most codecs are asymmetric; they take a long time to encode but are fairly quick to decode.
Image formats like JPG and PNG are displayed practically instantly even on modest hardware. What if my wonderful new image format takes a couple of seconds to decode and display? That might be unacceptable. Asking the user to buy a new computer or upgrade their software may also be impossible.
Power and Pollution
I think this is what it comes down to. How much energy does it take to create and use these images? I don't know how much CO2 it costs to transport a GB of data from one side of the planet to the other. And I don't know how energy efficient data centres are around the world.
I suspect that this is an impossible calculation. This daft image format would be a disaster if run on coal-powered server which sent data over a solar-powered fibre-optic cable. But, conversely, if we have a wind-turbine powered server sending data over the Deep Space Network to Mars (expensive and bandwidth-constrained) then this format is probably less polluting.
I worry that new compression formats face diminishing returns when faced with massive power costs. If the power required to encode and decode images continues to get larger, it could outstrip the savings made on transit and storage.
Energy Efficiency of Modern Codecs: shkspr.mobi/blog/2021/12/e… Comments: news.ycombinator.com/item?id=296920…
Randall says:
If you push any interesting tradeoff far enough you end up with a net loss, which I guess is what the far-out hypothetical is about. I bet, though, that we're still quite far from losing by replacing JPEG.
Outside of what's already in this post, a few factors are are 1) many images are transmitted+decoded more times than they're encoded, so the wild AVIF software encoding time is less of a factor, 2) there are lighter options than AVIF (JPEG XL, notably its JPEG1 transcoder -- ~50 MP/s single-threaded for ~20% savings for me), and 3) specialized hardware is out there for AV1/AVIF and HEVC/HEIC, mostly motivated by wanting good mobile video. Given all that I tend to think folks will usually find a lazy equilibrium for image compression that's also reasonable for power consumption--you don't want software AVIF encoding for your phone's local camera roll, etc.
An interesting exception could be if there's a bit of conflict of interest between the server and the consumer. Something like streaming AV1 video to a laptop with software-only decoding just to save bandwidth is right in the unsweet spot where there's a real net loss for power use (and the user--fans spinning, battery draining), but the site doesn't look broken.
Unfortunately, I think the level of further uptake of new image/video formats is mostly up to Apple. They quietly joined AOMedia (they may've found the patent licenses included helpful for implementing VP8/9), but if they actually support AV1/AVIF (and/or JXL), then we will have next-gen image formats everywhere. Otherwise, new codecs probably end up like WebP and VP9 were for years, opportunistically used by folks with the resources and motivation to juggle multiple formats. I don't think we'll ever see HEIC everywhere because of patents.
This had some interesting detail about power consumption in a video context: https://www.iea.org/commentaries/the-carbon-footprint-of-streaming-video-fact-checking-the-headlines . It includes a brief, interesting mention of how power use of data transmission is complicated, linking to https://www.youtube.com/watch?v=Xo0PB5i_b4Y&t=2520s
Also, your example of the Deep Space Network made me think of this recent post: https://twitter.com/xssfox/status/1476114977626865665 -- someone managed to send a JPEG of a photo from the middle of nowhere out over HAM radio, then looked at what they could get from an AVIF of that size. Neat to see the difference!
Anyway, thanks a lot for the opportunity to nerd out about formats 🤣