On-chain media storage can require efficient compression for text embedded inline in HTML / JS. ZTML is a custom pipeline that generates stand-alone HTML or JS files which embed competitively compressed self-extracting text, with file sizes of 25% - 40% the original. These file sizes include the decoder code which is 1.5 - 2 kB (including auxiliary indices and tables). The approach makes sense and is optimized for small texts, but performs quite well also on large texts. The pipeline includes efficient alternatives to base64 which are also useful for inline images.

A standard simplified pipeline can be run by calling ztml() or running python ztml.py from the command line. See ztml.py.

crEnc gives better compression but requires setting the HTML or JS charset to cp1252. Base125 is the second best option if one must stick with utf8.

See example.py for a complete example reproducing the above benchmark.

1. Files larger than a few MB might not work on iOS Safari or macOS Safari 15.
2. This solution favors compression rate over compression and decompression times. Use mtf=None for faster decompression of large files.
3. For compressing word lists (sorted lexicographically), solutions as Roadroller do a much better job.

1. Text normalization (irreversible; reduce whitespace, substitute unicode punctuation)
2. Text condensation (reversible; lowercase with automatic capitalization*, substitute common strings as: the, qu)
3. Burrows–Wheeler + Move-to-front transforms on text with some optional variants, including some new ones (beneficial for large texts)
4. Huffman encoding (with a codebook-free decoder, beneficial even as followed by DEFLATE)
5. Burrows–Wheeler transform on bits (beneficial for large texts)
6. PNG / DEFLATE compression (allowing native decompression, aspect ratio optimized for minimal padding, Zopfli optimization)
7. Binary to text encoding embedded in JS template literals:
   1. crEnc encoding (a yEnc variant with 1.6% overhead, to be used with single-byte charset)
   2. Base125 encoding (a Base122 variant with 15% overhead, to be used with utf8 charset)
8. Uglification of the generated JS (substitute recurring element, attribute and function names with short aliases)

*Automatic capitalization recovery is currently only partial. Use caps=raw or caps_fallback=True to preserve original capitalization.

fragium