Some time has passed since the Oceansprint 2021, and while being somewhat busy with other stuff too, I still managed to spend some time hacking on nix-casync. This article gives an update on the current state of things, and things I plan working on in the near future.
If you want to participate, please join the Matrix Channel1!
More testing needs to be done, but the initial benchmarks confirmed my hypothesis: Even the naïve approach of chunking entire NAR files showed a impressive disk saving ratio of 6.55, compared to simple zstd-compression of the NAR files (2.69). There might be more opportunities when chunking individual files inside NAR files, more on that later.
I just tagged
v0.5 in the repository, bringing the
following features since the initial release:
nix-casync serve now prints out an access log by default, allowing to see the
amount of requests its getting, as well as debugging Nix local cache behaviour.
It can optionally be disabled with the
Initially I didn’t implement any transport-level compression support, as the idea was to have the local instance of nix-casync transparently doing an “upgrade” to a to-be-defined more efficient protocol while talking to the remote binary cache backend.
However, some people apparently started playing with
nix-casync in a
“standalone binary cache” scenario. Here, it’s desirable to have compression
while uploading store paths, as well as when substituting from other machines.
Following up on my sidenote
in the previous article, Nix doesn’t use the
Compression attributes in
.narinfo files for signatures, doesn’t
FileSize fields when downloading NAR files (it
checks the uncompressed
NarSize though), and also doesn’t
require NAR files to be uploaded at
This means, if you hold things creatively, you can accept variously compressed
.nar files on upload, feed them uncompressed to the chunker, and when serving
back, optionally also use any compression format (and change it on the fly).
nix-casync now supports uploading with the following compression methods
(can be passed with the
?compression=… parameter during a
This is a subset of the formats supported by Nix, mostly due to library
availability. Keep in mind this is only used for compressing upload transfer
stream, so picking a fast compression (
zstd) is recommended.
Also note Nix defaults to gzip for backwards compat reasons if you don’t
?compression=… parameter explicitly.
nix-casync now also supports serving NAR files in compressed form. Only
zstd are supported here, mostly because the
other ones are too CPU-intensive.
This behaviour can be modified with the
--nar-compression=… parameter, and
defaults to ZSTD.
There was an annoying bug in case uploaded
.narinfo files contained
self-references, which is now fixed.
Trying to progress on the substitution feature, I started hacking together some
substitution code in
While thinking about the protocol, I quickly realized the properties I wanted
are very similar with what a generic remote store protocol should probably look
like, so it could be shared for Tvix, and other projects.
I’ll expand on more of the protocol design (soon) in a followup article, stay tuned!