Skip to content
/ ztml Public

Extreme inline text compression for HTML / JS. A custom pipeline that generates stand-alone HTML or JS files which embed competitively compressed self-extracting text, with file sizes of 25% - 40% the original.

License

Notifications You must be signed in to change notification settings

eyaler/ztml

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ZTML

Extreme inline text compression for HTML / JS

By Eyal Gruss

On-chain media storage may require efficient inline text compression for HTML / JS. Here is a custom pipeline to generate stand-alone HTML or JS files, embedded with self-extracting text, and file sizes of ~50% the original. The approach makes sense and is optimized for small texts, but performs well also on large texts.

War and Peace (en) Micromegas (en)
Project Gutenberg plain text utf8 3.2 MB 63.7 kB
ZTML (utf8 charset with Base125) 1.6 MB (50%) 35.0 kB (55%)
ZTML (cp1252 charset with crEnc) 1.4 MB (45%) 31.3 kB (49%)

ZTML pipeline:

  1. Text normalization (irreversible; reduce whitespace, substitute unicode punctuation)
  2. Text condensation (reversible; lowercase with automatic capitalization*, substitute common strings as: the, qu)
  3. Huffman encoding (with a codebook-free decoder, beneficial even as followed by DEFLATE)
  4. PNG / DEFLATE compression (allowing native decompression, aspect ratio optimized for minimal padding, Zopfli optimization)
  5. Binary to text encoding embedded in JS template literals:
    1. crEnc encoding (a yEnc variant with 1.6% overhead, to be used with single-byte charset)
    2. Base125 encoding (a Base122 variant with 15% overhead, to be used with utf8 charset)
  6. Uglification of the generated JS (substitute recurring element, attribute and function names with short aliases)

*Automatic capitalization recovery is currently partial.

Note: files larger than a few MB may not work on iOS Safari or macOS Safary 15

About

Extreme inline text compression for HTML / JS. A custom pipeline that generates stand-alone HTML or JS files which embed competitively compressed self-extracting text, with file sizes of 25% - 40% the original.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Sponsor this project

 

Packages

No packages published