nix-casync v0.5, and a general status update

Some time has passed since the Oceansprint 2021, and while being somewhat busy with other stuff too, I still managed to spend some time hacking on nix-casync. This article gives an update on the current state of things, and things I plan working on in the near future.

If you want to participate, please join the Matrix Channel1!

Some preliminary compression Benchmarks

rickynils ran some quick benchmarks on some datasets and posted his findings on Discourse.

More testing needs to be done, but the initial benchmarks confirmed my hypothesis: Even the naïve approach of chunking entire NAR files showed a impressive disk saving ratio of 6.55, compared to simple zstd-compression of the NAR files (2.69). There might be more opportunities when chunking individual files inside NAR files, more on that later.

A release, and new features

I just tagged v0.5 in the repository, bringing the following features since the initial release:

Access Logging

nix-casync serve now prints out an access log by default, allowing to see the amount of requests its getting, as well as debugging Nix local cache behaviour. It can optionally be disabled with the --no-access-log parameter.

Compression support for uploading and downloading

Initially I didn’t implement any transport-level compression support, as the idea was to have the local instance of nix-casync transparently doing an “upgrade” to a to-be-defined more efficient protocol while talking to the remote binary cache backend.

However, some people apparently started playing with nix-casync in a “standalone binary cache” scenario. Here, it’s desirable to have compression while uploading store paths, as well as when substituting from other machines.

Following up on my sidenote in the previous article, Nix doesn’t use the FileHash, FileSize and Compression attributes in .narinfo files for signatures, doesn’t require/check FileHash/FileSize fields when downloading NAR files (it checks the uncompressed NarHash and NarSize though), and also doesn’t require NAR files to be uploaded at nar/$filehash.nar[.$compressionSuffix].

This means, if you hold things creatively, you can accept variously compressed .nar files on upload, feed them uncompressed to the chunker, and when serving back, optionally also use any compression format (and change it on the fly).

Support uploading with compression

nix-casync now supports uploading with the following compression methods (can be passed with the ?compression=… parameter during a nix copy):

  • none
  • br
  • bzip2
  • gzip
  • lz4
  • lzip
  • xz
  • zstd

This is a subset of the formats supported by Nix, mostly due to library availability. Keep in mind this is only used for compressing upload transfer stream, so picking a fast compression (zstd) is recommended. Also note Nix defaults to gzip for backwards compat reasons if you don’t specify the ?compression=… parameter explicitly.

Support serving with compression

nix-casync now also supports serving NAR files in compressed form. Only none, brotli, gzip and zstd are supported here, mostly because the other ones are too CPU-intensive.

This behaviour can be modified with the --nar-compression=… parameter, and defaults to ZSTD.

Bugfixes for self-references

There was an annoying bug in case uploaded .narinfo files contained self-references, which is now fixed.

A new remote store protocol

Trying to progress on the substitution feature, I started hacking together some substitution code in nix-casync. While thinking about the protocol, I quickly realized the properties I wanted are very similar with what a generic remote store protocol should probably look like, so it could be shared for Tvix, and other projects.

I’ll expand on more of the protocol design (soon) in a followup article, stay tuned!

  1. ↩︎