23
votes
Is there a FLAC equivalent for digital comics?
I’ve bought enough digital comic bundles now that they’re starting to take up a decent chunk of storage space.
What I’d love to do, if possible, is losslessly compress them all down to their minimum sizes, in the same way that FLAC does with music. Is this a thing? And if so, what’s the best way to do it?
If lossless compression isn’t possible, is there a lossy compression that’s effectively lossless in that I won’t be able to tell the difference anyway? Lossless compression gives me good piece of mind, but the storage savings from an invisible lossy one are honestly quite appealing too.
You're probably looking for a comic book archive format like .cbz. It's just a standard zip container, but the image files you store within it remain untouched.
So if you have a source where all the images are lossless .pngs, you can wrap them in a zip archive, rename the extension to cbz, and open it with an ereader program to enjoy both the space savings of zip compression and the original quality of your images.
Let me piggyback here just to advise:
Always use .cbz (zip) and never one of the other formats for the container, even if they can shave off a few bytes.
zip is supported in pretty much every platform under the sun. There's an order of magnitude more tooling and it's typically all free to use, making new implementations easier to test (contrast with .rar/.cbr whose creation uses a closed and proprietary tool). Your archives will be much more portable.
And most importantly, .zip decompression is very performant. You may want to read your comics in a platform with relatively low hardware resources, or in a browser. There, using .cbz instead of, say, .cb7 can make a significant difference.
Recently someone was having trouble loading a >500mb cbr using Emby and I had to explain to them that the upstream dependency we use that decompresses .rar (embedded libarchive) doesn't support file by file decompression, so opening such an archive requires a considerable amount of RAM. Not so for zip.js!
If you want to optimize for size, always do it on the pages, not the container!
.cbz is convenient but the space saved on images purely by archiving them will be minimal as most of the compression for images is already handled by the image format itself. Using tools like oxipng or pingo to losslessly compress images will often save you more significant amounts of space. There is a lot of variability in image compression, especially if you're willing to use lossy compression, in which case things become much more subjective.
e.g. a short manga (grayscale jpegs)
With manga specifically, you can save a lot of space by reducing the color space alone.
A lot of manga pages are saved with full RGB color. Converting them to 64 level grayscale (sometimes you can even go lower depending on the images, I've gotten away with 16 with no color loss) usually shaves off a significant amount of file size and you lose absolutely no quality because the pages don't use other colors.
You can use software like XnView to do this very easily with one click. I saved a batch action with the preset I want and just run it on the pages I selected.
Worth giving it a try in tandem with compression algorithms. I've found this significantly reduces the file size of any manga I have. Then I just chuck them into a cbz archive.
Popping back in to thank people for their guidance here and to share a tool I found in trying to process my comics:
ACBR is a cross-platform, open-source comic tool that can do batch conversions of CBZs, EPUBs, and PDFs (so, basically all the formats I have comics in).
I played around with the settings and found that converting my comics to CBZ files using webp compression at quality 80 generates files that look identical to me but save significantly on space.
My preferred reading platform, BookFusion, accepts CBZ files with webp images, so I didn't run into any compatibility issues after converting the files. Though, if anyone else tries this out, I recommend running a test conversion to see if it works with your software of choice.
As a test case, I ran everything in this bundle through it, and it shrunk the collection down from ~18 GB to ~6 GB with no noticeable quality loss (to me, at least).
This is what I'm going to use going forward, as it will help me shrink down my library significantly. Not only does this save me hard drive space, but the maximum plan on BookFusion limits me to 100 GB of books, so up until now I've had to be choosy about which comics I upload there.
I am not aware of an image format that is optimized for comic books. But there are hundreds of different compression algorithms for images so there are probably ones that I am unaware of. To that extent I would consider the most important thing is compatibility with the software and hardware you want to use.
Compression is a bit of an art in and of itself. Producers usually put some care into the way their files are prepared for delivery so if you have legally acquired comic books, the main recommendation is to retain the encoding it already has.
If you are scanning in images yourself, then I would honestly say you’re probably fine using the efficient image compression du jour. HEIC was the big one the last time I checked, WebP was another big one earlier, and the last one I remember reading about was one I can’t remember the name of which used AV1’s compression tech. I heard rumblings of AV2 being in development so maybe there’s a still image format coming out for that too. The real difficulty is figuring out what settings to use for the compression to balance compression and quality. This is an area with no definitive answer, and it varies on a per-image basis if you want the absolute best compression:quality ratio.
If you do want lossless compression and you’re only going to view on displays, then honestly PNG ain’t half bad. I honestly wouldn’t bother with lossless image compression these days unless you are dealing with low resolutions where pixel precision is necessary. For example, I can’t stand looking at Octopath Traveller Zero (on Switch 2 at least) because of the compression ruining the pixel aesthetic. It’s a noisy mess. For any image stored in a greater resolution than your average display, lossy compression is more than adequate and newer encoding techniques are massive improvements over the JPEGs of our youth.
What kind of format are they in now? Are they scanned or like, digital art? Do you have an example of even a single page you could share?
It's a mix of CBZs, PDFs, and EPUBs. Most of them come from bundles (e.g. this one), with the others being direct downloads or de-DRMed from my Kobo account.
WAV is pretty unique in that it almost always is uncompressed (there are ways like ACM but usually people use a different file extension so that it isn't confusing). FLAC can recreate a WAV file bit for bit (or at least the PCM part) and it actually has a compression option which is why it's so much better than WAV.
I think audio gets away with this because audio is so small. Purely uncompressed image pixel matricies are huge which is probably why it isn't very common.
For images I think you can find an analogy of TIFF to PNG. You can convert between them losslessly, depending on your settings. But the results might be much less impressive as TIFF is a much more robust format with many compression options built-in. So your file might even increase in size if the TIFF compression algorithm of the original file is better than PNG.
For a lossy option you might try this, I wrote a multi-file optimizer that will, among other things, convert images and even CBZ/CBR archives to AVIF images. This can reduce file sizes A LOT. The default settings might be too extreme but you can override them with flags like setting the max image width/height:
https://github.com/chapmanjacobd/shrink
Other people have already covered the obvious paths, so that reminds me of someone creating a new comic file format a few months ago. However, that was more about performance rather than size (though they mention deduplication which could save space in some cases) and it's likely going to be a while before it sees any significant amount of adoption in reader software if it ever takes off.
I haven't seen that post til now, but as someone deeply involved in the manga/piracy scene, a lot of the OP's claims and complaints look nonsensical; they can be resolved at the application or archive creation layers without introducing a new format that'll never be adopted by the most popular applications or primary P2P groups. u/teraflop's comments sum up most of the apparent issues. Additionally, deduplication can be done using RAR5 CBRs or even CB7, which are less common but already existing CB* variants.
P2P groups almost exclusively use JPEG/PNG+CBZ to maintain compatibility with widespread legacy readers that will rarely/never update; if we wanted to break current standards to squeeze out maximum performance, there are so many better avenues we could've gone than a new specialized format.
I'm not much of a comic person myself so I just saw the post in passing and was mildly amused without giving it much consideration or diving into the comments to find the issues that you referred to. I figured that "PNGs in a zip file" had a good reason for not being displaced to this point so I'm not surprised, much like the .mkv container for video hasn't gone anywhere in the past decade either.
As someone that vibe coded a personal comic/doujin browser/viewer and saw this post months ago, this format is complete nonsense.
I threw away the code/results, but there was very little difference between this and a zip file, the time to open a random image in a random file was roughly the same. I benched it on 10GB of fake comics (images were random noise) just to make sure everything was fair and there was no filesystem caching going on and all files were the same size. The results were almost the same, any variance was temporary and probably because I have other stuff running in the background that interfered with it. It also "felt" the same when applied to my application, i.e. images didn't seem to load faster with one approach or another.
Interfacing with the format was also just a pain in the ass, I ended up not using any format at all for my project because it was easier to manage it using postgres for metadata and store the pages directly on disk. It's honestly a waste of time and just stick with cbz and if you need metadata just put it inside a yaml/xml file inside the zip.
The only thing I can thank the author of the format for is for finally getting me to finish my personal comic browser. So thanks.
You're making your own software for that? I haven't really done much with digital comics myself so I'm only superficially aware of things like Komga and not sure what the pain points for existing tools are.
It does more than just that, I just didn't want to spend paragraphs elaborating on all the features and dilute the point of the post further. I'll never make the software public, but if you're interested on why I made it here's a list of features that other software don't provide all at once:
I sort my comics using Calibre (which is just a book organizing tool). I usually convert my comics to Epubs if they aren't already (using Kindle Comic Converter). Then, open the comic in question with Calibres "Edit Book" function, go to File>Edit Pictures Losslessly, and when the menu pops up select "Allow lossy compression" with a quality of 80%.
In doing this I cut the size massively, usually 30-50% for mangas, and surprisingly 80% for most western-style comics (going from 300-400MB down to usually 30-50MB). And I really can't tell any difference; my e-reader is 11", so larger than most comic pages in real life, and I notice zero issues in the art or anything that only it was compressed at all.
Calibre does also have the option to Losslessly compress, but that cuts size by 5-10% at most usually, and given that I can't tell even looking for it it's much worth it to lossy compression instead
May I ask what the format of the files is right now? I'm guessing they're already JPEGs at some high quality setting, given that comics probably don't compress well with PNGs (they're not mostly flat, textureless colours).
Once they're JPEG'd, if you want to keep the conversions lossless, you'll have to stick with JPEG optimization tools: mozjpeg can crunch a few percent off of most JPEGs (similar to PNGcrush), and if you're willing to jump formats, JPEG-XL can push that another few percent further. The wildly optimistic numbers I've seen say that you can take off up to ~20% that way without any decrease in quality.
IME JpegXl does 15%-20% over Jpeg and you can get back to the original JPG (ie no generational quality loss).
Jpeg-XL also has a lossless option, though I'm not sure if it is more efficient than other formats (it probably is).
The problem is that the CBZ'd jpeg-xl format is not very compatible. (Mihon/Kahon support them though)