Hi - excellent tool! Its blazingly fast at tearing these large json files into digestible pieces for duckdb.

I'm working with the split output of mrfparse and the its size is one of my bottlenecks as I am using HDD to reduce costs (low speed for read/write, but extremely cheap). I would want to reduce the size of what I write to disk from memory using compression.

Is there support for this already that I have missed, or would it be possible to add it? I would look into it, it does not sound extremely complicated but I have never coded in Go.