Flexo is a caching proxy for pacman, the package manager of Arch Linux.
- If you're bothered by slow mirrors: Instead of manually maintaining a
mirrorlist
, Flexo automatically chooses a low-latency mirror for you and switches to another mirror if the selected mirror turns out to be slow. In addition, Flexo uses multiple mirrors for parallel downloads, which can increase download speeds substantially if you use Pacman'sParallelDownloads
setting. - If you have multiple machines running ArchLinux, and you don't want each machine to download and store the packages: You can just set Flexo as your new ArchLinux mirror so that no file needs to be downloaded more often than once.
- If you run ArchLinux inside Docker, you may be annoyed when packages have to be downloaded and installed on the container even though they have already been downloaded on the host: Just install Flexo on the host and run this command on the Docker container:
so that packages that have already been downloaded will be fetched from the cache.
echo 'Server = http://172.17.0.1:7878/$repo/os/$arch' > /etc/pacman.d/mirrorlist
A package for Arch Linux is available on AUR. Alternatively, you can use the docker image instead. Flexo needs to be installed on a single machine (the server) so that it can be accessed by multiple clients. Once you have installed Flexo on the server, start and enable the systemd service:
systemctl enable --now flexo.service
Next, set the new mirror in /etc/pacman.d/mirrorlist
on all clients.
In most cases, the server that runs Flexo will also be a client that uses Flexo, so
add the following entry to the top of your mirrorlist:
Server = http://localhost:7878/$repo/os/$arch
If you have additional ArchLinux clients in your LAN, proceed as follows to download packages from Flexo:
- Set
listen_ip_address
to0.0.0.0
in/etc/flexo/flexo.toml
(alternatively, you can put a reverse proxy in front of Flexo). - Modify the mirrorlist files of all clients on your LAN.
Instead of referring to localhost, use the appropriate IP address or hostname:
Server = http://<FLEXO_SERVER_IP_ADDRESS>:7878/$repo/os/$arch
Notice that if you start Flexo for the first time, it will run latency tests to select fast mirrors, which will take half a minute or so. During that time, Flexo is not available to serve any requests. Subsequent starts will be faster.
- Concurrent downloads: You can have multiple clients downloading files from Flexo without one client having to wait.
- Efficient bandwidth sharing for concurrent downloads: Flexo does not require a new connection to the remote mirror when the same file is downloaded by multiple clients. For instance, suppose a client starts downloading a given file. After 5 seconds have elapsed, 100 MB have been downloaded. Now, a second client requests the same file. The second client will receive the first 100 MB immediately from the local file system. Then, both clients continue to download the file, while only a single connection to the remote mirror exists. This means that your bandwidth is not split for the two clients, both clients will be able to download the file with the full download speed provided by your ISP.
- Persistent connections: This is especially useful when many small files are downloaded, since no new TLS negotiation is required for each file.
- The package cache is cleaned automatically: No need to set up cron jobs or systemd timers to clean the cache regularly, Flexo will automatically ensure that only the 3 most recent versions of a package are kept in your cache (this parameter can be changed).
The AUR package will install the configuration file in /etc/flexo/flexo.toml
.
It includes many comments and should be self-explanatory (open an issue in case you disagree).
If you use Docker, the settings can be modified with environment variables. Environment variables
are prefixed with FLEXO_
, for example, listen_ip_address
corresponds to the FLEXO_LISTEN_IP_ADDRESS
environment variable. More details about configuring Flexo are listed in the wiki.
In most cases, you will want to leave all settings unchanged, with the following exceptions:
-
The setting
low_speed_limit
is commented by default, which means that Flexo will not attempt to switch to a faster mirror if a download is extremely slow. To make use of this feature, uncomment the setting and enter an appropriate value. If you use Docker, use theFLEXO_LOW_SPEED_LIMIT
environment variable. -
The setting
allowed_countries
is set to the empty list by default, which means that at the first start and at regular intervals, Flexo will run latency tests on all official mirrors from all continents. Add the ISO code of your own country (and perhaps a few neighboring countries) to improve the startup time of Flexo. If you use Docker, use theFLEXO_MIRRORS_AUTO_ALLOWED_COUNTRIES
environment variable. Multiple countries can be separated by comma, for example,FLEXO_MIRRORS_AUTO_ALLOWED_COUNTRIES=DE,AT,CZ
-
If you have additional ArchLinux clients in your LAN, you probably want to change the
listen_ip_address
setting. It's set to127.0.0.1
by default for security reasons, change it to0.0.0.0
to make Flexo accessible to all clients in your LAN.
In addition, if you have a high-bandwidth connection, you may want to consider enabling Pacman's
ParallelDownloads
setting.
With ParallelDownloads
enabled, Flexo will receive multiple requests concurrently and therefore
fetch the packages from multiple mirrors in parallel, thus making it more likely that your entire bandwidth
is utilized.
Flexo obviously runs on a standard Arch Linux x86_64 platform. In addition, it runs on many ARM platforms (more details here).
If you intend to compile and run Flexo on any operating system other than ArchLinux (and without Flexo's Docker image), please note that:
- Flexo requires either paccache or scruffy to clean the cache. On any system other than ArchLinux, installing scruffy will probably be easier than installing paccache.
- Linux is the only supported operating system at the moment.
If Flexo does not start at all or crashes, check the logs first:
journalctl --unit=flexo
If that does not help you, please open an issue. The following information may be helpful to troubleshoot your issue:
- An excerpt of that log, if Flexo has crashed or did not start.
- Your installation method (Docker or AUR).
- The version you are using (either the output of
pacman -Qi flexo
, or the tag if you are using Docker). - Your settings, if you have changed them
(either the
/etc/flexo/flexo.toml
file, or the environment variables if you use Docker). - If the issue is related to the mirror selection, it might also help if you include the country you are located in
and the
/var/cache/flexo/state/latency_test_results.json
file, if it exists.
For issues related to the mirror selection, also see this page for more details.
The default configuration of Flexo will keep 3 versions of a package in cache: After a 4th version of a package has been
downloaded, the oldest version will be automatically removed. This setting can be changed with the num_versions_retain
parameter. See the configuration example for more details.
If you use Docker, the default behavior can be changed with the FLEXO_NUM_VERSIONS_RETAIN
environment variable.
If you want to disable this setting and never purge the cache, set the parameter to 0
.
If you are using unofficial user repositories
and you want Flexo to cache packages from those repositories, both pacman.conf
and flexo.toml
need to include the custom repository. For example, suppose that Flexo is running on localhost, port 7878,
and you want to add two custom repositories: archzfs and eschwartz. First, adapt your /etc/pacman.conf
to include
both repositories. Notice that the path must start with custom_repo/<repo-name>
:
[archzfs]
Server = http://localhost:7878/custom_repo/archzfs/$repo/$arch
[eschwartz]
Server = http://localhost:7878/custom_repo/eschwartz/~eschwartz/repo/$arch
Next, add the corresponding entries to your /etc/flexo/flexo.toml
before the [mirrors_auto]
section:
[[custom_repo]]
name = "archzfs"
url = "https://archzfs.com"
[[custom_repo]]
name = "eschwartz"
url = "https://pkgbuild.com"
Notice that the names (in this case archzfs
and eschwartz
) must match the path component right after
the /custom_repo
in pacman.conf
: So if your pacman.conf
includes a repo with the path /custom_repo/foo
,
then your flexo.toml
must include a matching [[custom_repo]]
entry with name = "foo"
.
Alternatively, if you use Docker, set the environment variable instead of modifying the flexo.toml
file:
FLEXO_CUSTOM_REPO="eschwartz@https://pkgbuild.com archzfs@https://archzfs.com"
Flexo can be built on various ARM platforms, including the Raspberry Pi. So far, no problems have been reported with building and running Flexo on ARM. If you run into problems, please open an issue.
Options to use Flexo on ARM devices include:
- Build and install the AUR package on your ARM device (recommended).
- Build and run the Docker image on your ARM device.
- Cross-compile Flexo on an x86-device and move the binary to the ARM device. Requires cross. Also requires adding openssl as a vendored dependency: Add
openssl = { version = "0.10", features = ["vendored"] }
to the[dependencies]
section inCargo.toml
.
With its default configuration, Flexo only serves packages from the official ArchLinux mirrors, which means packages
built for x86. However, we can configure an ARM mirror as a custom_repo
in order to fetch ARM packages from Flexo.
First, visit https://archlinuxarm.org/about/mirrors and choose a mirror. Once you have decided for an ARM mirror,
configure it as a custom_repo
in your /etc/flexo.toml
. In this example, we have chosen the mirror
de3.mirror.archlinuxarm.org
and we have given it the name arm
:
[[custom_repo]]
name = "arm"
url = "https://de3.mirror.archlinuxarm.org"
Next, configure the mirrorlist on all clients that are going to fetch ARM packages from this server. For example,
if the server that runs Flexo should fetch the package from Flexo, configure your /etc/pacman.d/mirrorlist
as follows:
Server = http://localhost:7878/custom_repo/arm/$arch/$repo
- Lightweight: Flexo is a single binary with less than 3 MB and a low memory footprint.
- Robust: As long as most mirrors work fine, Flexo should be able to handle the download process without the client noticing any issues or interruptions, even if remote mirrors are slow or connections are unexpectedly dropped.
- Simple: Users should not require more than a few minutes to set up Flexo and understand what it does.
If you know Rust, feel free to dive into the code base and send a PR. Smaller improvements to make the code base cleaner, more idiomatic or efficient are always welcome. Before submitting larger changes, including new features or design changes, you should first open an issue to see if that feature is desired and if it fits into the design goals of Flexo.
Other than code, you can contribute by submitting feedback. One aspect of Flexo where feedback is particularly valuable is the mirror selection process. If you notice that downloads are too slow because the selected mirrors are not fast, please open an issue. You can determine the primary mirror chosen by Flexo with the journal:
journalctl --since '7 days ago' --unit=flexo | grep 'Primary mirror'
Details about design decisions, and the terminology used in the code, are described here.
Before submitting a PR, please run cargo test
inside the flexo
directory to make sure that all tests pass.