Skip to content

Allow non compressed WARC files to be used while recording #915

Open
@Lisias

Description

@Lisias

Modern file systems now have compression schemes that surpasses userland compression schemes in convenience and sometimes even on efficiency, rendering the current use of gzip inconvenient when such filesystems are used.

As a use case, BTRFS with zstd:15 gave me excellent results on compression, surpassing what gzip could do, with faster read access. At very least, I don't need to recompress the WARCball after a recording session.

Ideally it should be possible to allow the user to choose if they are want to use the warcio gzip support, or prefer to rely on the file system for such - preventing the user to uncompress the thing manually to take advantage of the file system compression.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions