An overview of Nix in practice

2023-04-19

<skip> i would really appreciate it if someone who knows nix could help tho 🥺

<skip> i asked in nix server and multiple people were like “yea this should be working but it doesn’t”
<skip> maybe i should post on the discourse hm
<skip> i’ve lost sleep over this lol
— 2021-07-25

I was introduced to Nix in late 2021 by a friend in a Discord channel, but it wasn’t my first encounter with the operating system/package manager/configuration system/programming language hodgepodge.

As a Linux-obsessed child who constantly hopped between distributions, it inevitably registered on my radar. I was always looking for the next ISO file to victimize and copy onto my tired USB stick, which would probably suffocate me in my sleep if it grew arms. But NixOS was never one of those distros. It looked too weird to my 14-year-old brain.

I still believe this. NixOS, and Nix in general, is really weird. I’ve described it as what ends up writhing out of a malfunctioning industrial mixer that someone accidentally dropped Haskell and Bash into. When I, like many other computer programmers, tell my friends who don’t write code that “it’s a miracle that modern technology even works”, Nix is one of those things that I’m referring to.

Despite this, NixOS is the only distro I install on my servers. I’m dual-booting it on my MacBook¹. I build and set up the development environments for all of my projects with Nix. I use Nix to set up my dotfiles and take care of my shell configuration and personal set of installed packages. Nix makes it unflinchingly simple to replicate my cozy user environment anywhere Nix is available, so I can feel at home no matter where I am. Nix even turned building ffmpeg into a cakewalk for that one time I needed better AAC encoding².

Nix is a really neat (in my opinion), but the learning “curve” is comparable to climbing a brick wall with nothing but your fingernails, determination, and the incredibly thin shred that is your remaining patience. I’m still not great at Nix, and I’m terrified to submit anything to the official package collection, but I hope to demystify it and emphasize what makes it so dang useful.

Behind the curtain

“Nix”, generally speaking, refers to any or all of these things:

A ~purely functional programming language with immutability, laziness, floaty Haskellesque syntax, and some interesting design choices to make configuration and configuration-adjacent tasks particularly easy.
A package manager and build system with the primary goal of reproducibility that consumes package definitions and build instructions written in the Nix programming language.
A massive repository that houses more than 80,000 Nix package definitions written and maintained by over 5,000 contributors, called “nixpkgs”.
A Linux distribution (“NixOS”) that uses Nix³ to declaratively specify the total configuration of the system, such as: which packages are to be installed, which user accounts exist on the system, and what services are active.

All of these puzzle pieces fit together to create something that is truly unique and terrifying. Hooray!

Nix is different from other package managers in that it’s truly and utterly obsessed with making things entirely self-contained. When Nix builds a package, the build environment is isolated such that nothing outside of what you declare is accessible. Having globally installed libraries that are recognized by configure scripts, CMake, etc. can be a massive pain, especially when you need a specific version or find yourself needing to apply bespoke patches. Sometimes the exact version of a library you require just isn’t available easily. Having multiple versions installed can be either difficult or nigh impossible.

Nix avoids this by having binaries reference their dependencies explicitly. List the shared libraries needed by a binary that was built by Nix, and you’ll see something like this:

$ ldd "$(which curl)"
	linux-vdso.so.1 (0x00007fffeeb86000)
	libcurl.so.4 => /nix/store/rirzp6ijbcwnxlf0b2n286n587r3z9jw-curl-7.86.0/lib/libcurl.so.4 (0x00007ffb5ce40000)
	libssl.so.3 => /nix/store/4mxnw95jcm5a27qk60z7yc0gvxp42b9a-openssl-3.0.7/lib/libssl.so.3 (0x00007ffb5cd93000)
	libcrypto.so.3 => /nix/store/4mxnw95jcm5a27qk60z7yc0gvxp42b9a-openssl-3.0.7/lib/libcrypto.so.3 (0x00007ffb5c914000)
	libz.so.1 => /nix/store/026hln0aq1hyshaxsdvhg0kmcm6yf45r-zlib-1.2.13/lib/libz.so.1 (0x00007ffb5c8f6000)
	libc.so.6 => /nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/libc.so.6 (0x00007ffb5c6ed000)
	libnghttp2.so.14 => /nix/store/qz400bwshaqikj5s2qyvh0c9qffgmqik-nghttp2-1.49.0-lib/lib/libnghttp2.so.14 (0x00007ffb5c6bc000)
	libidn2.so.0 => /nix/store/5mh5019jigj0k14rdnjam1xwk5avn1id-libidn2-2.3.2/lib/libidn2.so.0 (0x00007ffb5c69a000)
	libssh2.so.1 => /nix/store/vqq9s0d6fw6kqf3sr5nrzqbys9rhygqd-libssh2-1.10.0/lib/libssh2.so.1 (0x00007ffb5c659000)
	libgssapi_krb5.so.2 => /nix/store/r7gl900my2fw6k33nxh2r7rzv8nv0s25-libkrb5-1.20/lib/libgssapi_krb5.so.2 (0x00007ffb5c606000)
	libzstd.so.1 => /nix/store/w10in9diaqrcqqxi5lg20n3q2jfpk6pq-zstd-1.5.2/lib/libzstd.so.1 (0x00007ffb5c540000)
	libbrotlidec.so.1 => /nix/store/9iy1ng7h1l6jdmjk157jra8n4hkrfdj1-brotli-1.0.9-lib/lib/libbrotlidec.so.1 (0x00007ffb5c532000)
	libdl.so.2 => /nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/libdl.so.2 (0x00007ffb5c52d000)
	libpthread.so.0 => /nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/libpthread.so.0 (0x00007ffb5c528000)
	/nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/ld-linux-x86-64.so.2 => /nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib64/ld-linux-x86-64.so.2 (0x00007ffb5cee7000)
	libunistring.so.2 => /nix/store/34xlpp3j3vy7ksn09zh44f1c04w77khf-libunistring-1.0/lib/libunistring.so.2 (0x00007ffb5c37a000)
	libkrb5.so.3 => /nix/store/r7gl900my2fw6k33nxh2r7rzv8nv0s25-libkrb5-1.20/lib/libkrb5.so.3 (0x00007ffb5c29f000)
	libk5crypto.so.3 => /nix/store/r7gl900my2fw6k33nxh2r7rzv8nv0s25-libkrb5-1.20/lib/libk5crypto.so.3 (0x00007ffb5c270000)
	libcom_err.so.3 => /nix/store/r7gl900my2fw6k33nxh2r7rzv8nv0s25-libkrb5-1.20/lib/libcom_err.so.3 (0x00007ffb5c26a000)
	libkrb5support.so.0 => /nix/store/r7gl900my2fw6k33nxh2r7rzv8nv0s25-libkrb5-1.20/lib/libkrb5support.so.0 (0x00007ffb5c25a000)
	libkeyutils.so.1 => /nix/store/816qwr4xy058451rbxr0ccyh1v1akhb6-keyutils-1.6.3-lib/lib/libkeyutils.so.1 (0x00007ffb5c251000)
	libresolv.so.2 => /nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/libresolv.so.2 (0x00007ffb5c23f000)
	libm.so.6 => /nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/libm.so.6 (0x00007ffb5c15f000)
	libbrotlicommon.so.1 => /nix/store/9iy1ng7h1l6jdmjk157jra8n4hkrfdj1-brotli-1.0.9-lib/lib/libbrotlicommon.so.1 (0x00007ffb5c13c000)

Assuming that the package is built correctly and you aren’t playing any runtime linker tricks on your end, it’s virtually impossible to run into library errors. Having multiple versions of a library installed is a nonissue, and dependency hell falls into irrelevancy.

Binaries are vacuum packed in that they only ever refer to exactly what they need, and this applies for both runtime and build dependencies alike.

Because the Nix store is essentially an append-only graph database⁴ with nodes being packages and edges being dependency relations, I can even query it to see e.g. ffmpeg’s runtime dependencies, both direct and indirect:

An end-to-end graph of ffmpeg's dependencies.

At the top, we see CoreFoundation, which is a core macOS system framework that a lot of packages depend on. We can also spot dejavu-fonts-minimal, which is needed by fontconfig, which is needed by libass to render subtitles, which is needed by this default configuration of ffmpeg. Emphasis on default: we can override options provided by the package definition and specify .patch files to be applied to the source code.

To maximize reproducibility, Nix works on the source code of a package. But because every package is built in its own universe, it is extremely cacheable. Nix package installations are capital F Fast, because there’s no dependency resolution to be done. It amounts to downloading data from a CDN and unpacking it, and it can’t get any faster than that.

Executing “over 350,000 builds each week”, the NixOS foundation maintains a massive Nix build farm for x86-64 and aarch64 Linux and macOS, uploading the build artifacts to a widely-available binary cache.

As Nix evaluates the build instructions locally, it hashes the result and queries the cache for it. In theory⁵, between systems of the same platform, there’s no difference between building the desired package and copying the results from the cache. Taking an extreme amount of care to isolate builds from one another makes this practical.

This means that in ordinary circumstances, you won’t have to build packages from scratch. If you impose an override onto a package that changes the resulting build instructions, Nix doesn’t find a cached result from the cache due to the differing hashes, and the package is built locally.

Nix’s approach to builds also has some other neat effects:

Cross-compilation becomes easier, which unfortunately complicates the overall infrastructure but makes approaching this problem comparatively less scary…probably.
Remote build support lets you evaluate a package’s (potentially customized) build instructions on your local box, then send them to a more powerful machine over the network to build. This is not only cool, but also necessary should the package not be supported on your local platform. Distributed builds, CI, et cetera! The sky is the limit.
nix-copy-closure copies the results of a build and its total runtime dependency tree (called a “closure”) between machines of the same platform, avoiding unnecessary work.
Cache your own stuff! Host your own binary cache, or use a service like Cachix to avoid rebuilding.

Ephemerality

Managing Python environments is not only painful, but actively damaging to the psyche.

Setting up an ephemeral Python environment with OpenCV can be done in Nix like so:

$ nix-shell --pure -p "python3.withPackages(packages: [ packages.opencv4 ])" --run python3
Python 3.10.10 (main, Feb 17 2023, 05:25:10) [Clang 11.1.0 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>>

I can run a similar command to drop myself into a REPL with NumPy or SciPy, or any package in general: a certain version of Java, or even a Node or Haskell environment.

Dissecting the command:

The --pure flag clears the environment before dropping you into the shell, preventing e.g. $PATH from leaking through.
-p specifies which package(s) we are interested in. In this case, python3 (which currently resolves to python310 on the 22.11 channel). We use withPackages to indicate that we’d like to create an environment with the opencv4 Python package available.
```
python3.withPackages(packages: [ packages.opencv4 ])
```
This is Nix code that uses package definitions from nixpkgs, which has specific infrastructure for Python, Python packages, and managing Python environments. Nix has similar infrastructure for Rust, Node.js, etc.
--run drops us directly into the Python REPL instead of Bash.

Invoking this command causes Nix to compute which packages is needed and what Python environment trickery needs to be done. If the necessary packages aren’t in my Nix store already, they’re downloaded from the binary cache. When I exit the shell, my local Python user environment remains completely untouched. The fundamental nature of Nix allows this to happen in a frictionless way—no virtualenv futzing required.

Nix’s modular nature allows it to cooperate with other technologies, too. Take Docker, for instance: Nix is seemingly better at generating smaller images; because it knows everything about every package, it can purge superfluous data down to the nanometer and keep only the bits that matter.

Declarative systems

NixOS takes wacky declarative package management to its next logical conclusion by using the same technology to determine to service topology of your Linux system. In essence, your system configuration becomes a package that is managed and installed by Nix just like any other.

This is what a minimal NixOS configuration looks like (in practice, real configurations are much larger):

{ config, pkgs, ... }: {
  imports = [
    # Import another Nix module that contains the results of the hardware scan.
    # (Includes important kernel modules for hardware, the filesystem table, etc.)
    ./hardware-configuration.nix
  ];

  # Enable a bootloader for UEFI systems.
  boot.loader.systemd-boot.enable = true;

  # Enable the OpenSSH server.
  services.sshd.enable = true;

  # Add some useful packages for everyone.
  environment.systemPackages = with pkgs; [ git curl wget ];

  # ...
}

As someone with ADHD and relatively poor memory retention, NixOS is a godsend for managing my fleet of servers. After not having logged into a box for a while, I easily forget what services exist on the server. I’m too lazy to maintain this in writing, but it’s OK because Nix enables declarative management of these services. It acts as the definitive source of truth.

Figuring out exactly what a server does without fear of documentation desynchronizing with what actually exists on the server is a single $EDITOR /etc/nixos/configuration.nix away.

NixOS differs from something like Ansible in that it’s inherently declarative, through and through. If you remove the services.sshd.enable = true; line from your configuration, NixOS will tear down the OpenSSH server upon a rebuild. It’ll be as if it was never there (subtracting any leftover data), because there isn’t a practical difference between installing NixOS for the first time and building it again. Ansible has the notion of “idempotency”, but Nix (and NixOS) is idempotent by nature.

NixOS isn’t special:

You can build a NixOS system and copy the closure to another machine to use.
You can make a custom version of the NixOS installer ISO with your desired customizations to craft your ideal Linux rescue USB.

You can test experimental changes to your system by sandboxing it within a QEMU virtual machine:

$ nixos-rebuild build-vm

# Run your new system inside of a VM to see if something broke.
$ ./result/bin/run-*-vm

Because virtualization is made easy with declarative configurations, NixOS maintainers test their own code with it.
Because Nix packages are fully self-contained, previous generations of the system are tracked, so you can rollback anytime in the boot menu. I used this once after I accidentally hosed my network configuration and wasn’t sure how to fix it.
Not only are previous generations kept around, system upgrades are essentially fully atomic. New version of libc (or other Important™️ library)? On an ordinary system, replacing the file without rebooting would probably summon flesh-eating demons. But with Nix, this isn’t an issue—every package was already referencing the version it depended on through an absolute path into the Nix store.

Saving disk space is easy, too. Being a feature of Nix in general, the garbage collector can clean up computed build instructions, old profile/system generations, and other build dependencies that aren’t needed for a functioning system. This runs pretty fast and is a nice crutch for those times you just need more room.

$ sudo nix-collect-garbage -d
# ...
16730 store paths deleted, 11828.97 MiB freed # :O

Declarative, modular systems

nixpkgs comes with a huge number of included modules that port other software and services to be usable with NixOS.

Something that is genuinely terrifying to set up to me is GitLab. It just has too many moving parts. To deploy it with NixOS, I have this in my configuration:

{
  services.gitlab = let
    secretFile = "/var/lib/gitlab-secret";
    rsaSecretFile = "/var/lib/gitlab-secret-rsa";
    initialRootPasswordFile = "/var/lib/gitlab-initial-root-password";
  in {
    enable = true;
    host = ...;
    port = 80;
    https = true;
    inherit initialRootPasswordFile;
    secrets = {
      secretFile = secretFile;
      dbFile = secretFile;
      otpFile = secretFile;
      jwsFile = rsaSecretFile;
    };
  };
}

Other than specifying some paths to files containing secrets (a caveat which I’ll elaborate on later), this is all I need for a functioning GitLab setup. If I change my mind on this in the future, all I have to do is flip enable = true; to false, or outright evict this code block from my configuration.

By doing this, all required services immediately stop and become disabled. If I run the garbage collector, all GitLab packages are deleted. All that remains is any user data that was created by the services in question. This is seriously awesome to me, because GitLab is exemplary of being somewhat of a behemoth to set up, despite Omnibus.

What is even better is the inherent benefit of a modular and declarative configuration. For example, one of my servers is also responsible for hosting an authoritative DNS server, which has a corresponding NixOS module in nixpkgs:

{
  services.nsd = {
    enable = true;
    interfaces = [ "0.0.0.0" ];
    verbosity = 2;
    zones.howl.children = {
      "howl.".data = ''
        $ORIGIN howl.
        $TTL 3600

        @ IN SOA howl. tinyslices@gmail.com. ( 2021122201 28800 7200 864000 60 )
        @ IN NS louie.howl.

        ${lib.concatStringsSep "\n" config.howl.records}
      '';
    };
  };
}

An authoritative DNS server vends records instead of simply resolving and caching them. Here, I’m directing nsd (the DNS server in question) to listen on the unspecified address⁶. I’m using it with Tailscale, which is a really neat service that enables a zero-config mesh VPN between all of my servers and devices. (NixOS also has a module for it—just services.tailscale.enable = true;.)

Most importantly, I’m defining a custom NixOS option: howl.records, which lets me easily add lines to the zone from other modules. This is important.

{
  options.howl = with lib; {
    records = mkOption {
      type = types.listOf types.str;
      description =
        "Authoritative DNS records exposed to every device in the tailnet";
    };
  };
}

When I write the code to enable the nsd module above, I make sure to reference the final value of this custom option (as config.howl.records) in the zone configuration, which is collected from all instances of the option being modified in other modules. Nix’s lazy evaluation crucially enables this.

Now, in another NixOS module—say, monitoring.nix, I can enable Grafana and other services such as Prometheus. I can also tweak Nginx and tell it to expose its status page, and also create virtual hosts for Grafana and Prometheus. And in the same breath, I can create DNS records for all of these new services, so every device connected to the tailnet can access them painlessly:

{ lib, ... }:

let
  inherit (import ./net.nix) localhost internal ports;
in
{
  services.grafana = {
    enable = true;
    port = ports.grafana;
    domain = "grafana.howl";
    rootUrl = "http://grafana.howl";
  };

  services.prometheus = {
    enable = true;
    port = ports.prometheus.prometheus;
    globalConfig.scrape_interval = "10s";
    scrapeConfigs = [{
      # ...
    }];
    exporters = {
      # ...
      nginx.enable = true;
    };
  };

  # Instruct Nginx to expose its status page for Prometheus.
  services.nginx.statusPage = true;

  # Configure Nginx vhosts for these new services.
  services.nginx.virtualHosts."grafana.howl".locations."/" = internal {
    proxyPass = localhost ports.grafana;
  };
  services.nginx.virtualHosts."prometheus.howl".locations."/" = internal {
    proxyPass = localhost ports.prometheus.prometheus;
  };

  # Add lines to the DNS zone for these new services.
  howl.records = [
    "grafana.howl. IN CNAME louie.howl."
    "prometheus.howl. IN CNAME louie.howl."
  ];
}

I import this new module in my main configuration:

{
  # ...
  imports = [
    # ...
    ./monitoring.nix
  ];
}

And after a rebuild, Nix pulls in the required packages and builds the system closure, enabling and configuring every service that I declared to be active.

In my opinion, this is where NixOS really shines. I’m able to enable and configure Grafana, Prometheus, Nginx, and the DNS records that have to do with those services in a single file. I’m defining shared constants in a separate place and using them across configurations that are written in completely different languages and syntaxes.

With traditional server management, I’d have to install the correct packages, look up the paths to their configuration files, and maybe even run some commands to modify some mutable state. I’ll forget all of this in a few days or even hours, but Nix lets me authoritatively describe it in code.

If I decide to stop importing that file in my configuration, then the DNS and Nginx virtual hosts are removed; Nginx stops exposing its status page; Prometheus and Grafana’s systemd units are disabled; and all unneeded packages are no longer reachable from a garbage collector root, making them eligible for deletion.

NixOS lets me unify and modularize what would otherwise be disconnected configurations across packages in a highly idempotent, cacheable, and declarative way, essentially homogenizing every service on the system. That’s amazing to me.

Diving in

I’m intentionally avoiding the details here because I believe it’s important to first grasp the big picture behind Nix and NixOS, and understand what it’s truly useful for.

I write and maintain a few Discord bots. To ease the maintenance and deployment of these critters, I’ve created a base that lets me share the same development environment and automatically generate a NixOS module for all of them, with customization and extension points whenever necessary. I’m interested in writing some detailed case studies on how I accomplished this, but this post is getting really long, and we still aren’t finished!

For now, here are some resources that I personally recommend for perusal should you be interested:

Zero to Nix: New kid on the block; gentle and more detailed introduction to using Nix in practice.
“Nix: What Even is it Though”, a presentation by Burke Libbey.
Nix Pills: Learn Nix from the bottom up.
The Nix Reference Manual
nixpkgs manual and NixOS manual: The official documentation for those projects.
nix.dev: An opinionated handbook and “survival guide”.
NixOS Search: An invaluable tool for quickly looking up packages and NixOS options. Keep this on speed dial.
noogle eases the looking up of library functions.

When docs aren’t amazing, learning becomes akin to patchwork: you slowly and incrementally stitch squares into your quilt until you reach a certain point where everything becomes clear, and you end up with something nice and soft to keep you warm during winter.

Rougher edges

Every technology involves a set of tradeoffs, and Nix is no exception.

One particular pain point is managing secrets: the Nix store is world-readable, and this includes any configuration files that may or may not contain secret keys and passwords. This problem is tackled via projects such as agenix and others, but keep this in mind when blueprinting your system. I’ve experimented with committing my nixfiles with their encrypted secrets in public before, but this ended up being such a massive pain that I gave up. Maybe I’ll try again soon.

When it comes to libraries and scaffolding your own projects with Nix, it can be hard to find what’s appropriate for use.

Two good examples: packaging Rust code, and deploying NixOS. I used to wield Naersk to blast my Rust packages with the Nix beam, but was later recommended Crane by a friend, which seems to require less ceremony. I never would’ve found out about it otherwise. nixpkgs also seems to have built-in support for Rust via buildRustPackage. I’m not sure which one is the best to use; it’s unclear to me, but I’ve settled on Crane for now.

A screenshot of a successful colmena command invocation.

Deployment to other servers from your own box is peak comfort. There are many Nix deployment solutions, but I’ve settled on Colmena, because it supports macOS and Flakes. There’s also Bento (which tries to keep it simple), as well as Morph and nixops. And there’s probably more I’m missing. There’s a lot of choice here.

I run into a similar feeling when I write Nix code myself. The documentation story here could use some improvement: finding the right function to use can feel like navigating a dense jungle with a machete. I feel like reading other people’s Nix code is one of the most effective ways to learn Nix, because you get to see how its used (and how people solve their specific problems) in practice.

GitHub’s code search is an invaluable tool here, as well as poking around in nixpkgs and asking questions in community channels. A lot of learning Nix is done hands-on; you’ll need to throw things at a wall to see what sticks, and that’s going to be frustrating at first.

The nature of Nix

Nix is infectious, which can be both good and bad. Having a universal, omnipresent existence on your system is what enables Nix’s niceties, but also can make it frustrating to just do things.

You can’t follow README instructions verbatim anymore: if you want to use something, it has to be packaged by Nix. Or, at least, it should. If there happens to be a Nix package maintained by someone in nixpkgs, great! If not, you better be in a mood to write some Nix code.

My dotfiles are Nix-managed, too. This means that my entire user environment is easily reproducible everywhere Nix is available. However, this means that I need to rebuild my user package every time I make a change, because the configurations need to be built, copied into the Nix store, and then symlinked into my own home directory.

Any change I make—big or small—has to go through Nix before programs take notice, which can be annoying.

Fortunately, you aren’t forced to adopt Nix to such a high degree. You can even install it on other Linux distributions, since it’s just a package manager. By doing this, you get to pick and choose what falls under Nix’s reign. The more you decide to use, though, the more you’ll have to zap things with the Nix ray.

Take black box binaries, for example: tarballs of pre-built blobs, sans source code, that you “just” have to unpack and run. Nix inherently doesn’t play well with these, and that can make certain things harder to do (see: games).

If patching the binaries in question is viable, patchelf can come to the rescue. A friend of mine who is trying out NixOS is using it to patch binaries in order to run Garry’s Mod servers, which are able to interface with external 3rd-party plugins via dlsym and other runtime loading.

When patching is unfeasible, such as programs that perform integrity checking or simply make too many assumptions about the outside world, buildFHSUserEnv enables you to run lightweight sandboxes that are compatible with the Filesystem Hierarchy Standard (/usr/lib and friends), made possible through Linux namespaces. This is done to support Steam, which is pretty damn clever.

Steam has always felt pretty fragile to me. For example, it requires its own bespoke set of libraries that it downloads and maintains outside of the system package manager. This sucks, but it’s cool that it was possible for Nix to shove it into a reproducible box of sorts that basically can’t break (let’s hope I don’t jinx it).

When packaging other software that we have a reduced amount of control over, things can get a little wonky. When I installed NixOS on my Mac, I ran into an issue where ArmCord—a neat little project that wraps Discord natively for ARM devices—wouldn’t open links in my web browser.

After some poking around with Chromium’s logging levels, I eventually got it to spew out enough debug information to see why xdg-open (which is what is being spawned to open my web browser) was failing:

XPCOMGlueLoad error for file /nix/store/5mndwvvbdz07kllj6bs0pp1n82cx260i-firefox-110.0.1/lib/firefox/libxul.so:
/nix/store/p9ggv8qkdv0s7pckz2xkxxs68ras07g3-nss-3.79.4/lib/libssl3.so: version `NSS_3.80' not found (required by /nix/store/5mndwvvbdz07kllj6bs0pp1n82cx260i-firefox-110.0.1/lib/firefox/libxul.so)
Couldn't load XPCOM.
/home/slice/.nix-profile/bin/xdg-open: line 881: x-www-browser: command not found
XPCOMGlueLoad error for file /nix/store/5mndwvvbdz07kllj6bs0pp1n82cx260i-firefox-110.0.1/lib/firefox/libxul.so:
/nix/store/p9ggv8qkdv0s7pckz2xkxxs68ras07g3-nss-3.79.4/lib/libssl3.so: version `NSS_3.80' not found (required by /nix/store/5mndwvvbdz07kllj6bs0pp1n82cx260i-firefox-110.0.1/lib/firefox/libxul.so)
Couldn't load XPCOM.

To my surprise, there were… linking errors going on! Firefox seemed to be loading the incorrect version of OpenSSL for some reason. I thought Nix was supposed to be reproducible, self-contained, etc. etc. This shouldn’t be happening!

More investigation led to the realization that the ArmCord package actually injects LD_LIBRARY_PATH at runtime so that it can find the right libraries—needed because the package reuses the .deb binaries—and it was bleeding into the xdg-open subprocess that was being spawned. Whoops.

I was able to fix this by wrapping xdg-open with a pristine script that unset LD_LIBRARY_PATH, and overriding the ArmCord package to add a wrapper that invoked the main binary with the pristine script prepended to PATH before anything else. Then, when xdg-open was spawned by Chromium, LD_LIBRARY_PATH would no longer be clobbered.

In this situation, overriding the package definition here was enough to solve my problem, but that might not always be the case: NixOS modules, and Nix packages in general, are only as flexible as they are written to be⁷. So far, I’ve never had to straight-up fork a package or NixOS module definition, but I’m definitely not ruling that out from ever happening.

Luckily, there tends to be a lot of escape hatches: nixpkgs is very configurable and extensible with support for overlays, overrides, and source code patches. It’s also totally possible to do something such as creating a package that solely takes the output of another package and patches it in a certain complex way (however you want!)

Lastly, I’d like to reiterate that because Nix is a lazily evaluated (and dynamically typed) language, it’s very possible to run into fairly cryptic errors often, especially when recursion is involved. Lazy evaluation means that it’s possible to introduce a ticking time bomb that can be detonated from seemingly unrelated code.

While this can be a pain, it’s also what enables Nix to avoid unnecessary computation: the root of nixpkgs is a huge tree of every package contained within the repository, but only those that are actually needed are ever evaluated. Deferring evaluation to the last possible moment is what makes things such as overrides and overlays possible.

Bottom line

Nix is quirky, unique, and a little rough around the edges. Debugging it, like a lot of other things, is frustrating. But the benefits I get from using it currently outweigh the disadvantages, providing enough incentive for me to keep on investing in it.

What I find to be most useful for me is declarative system management via NixOS. My memory span is virtually nonexistent nowadays, and having a way to declare my servers completely idempotently is an incredible way to avoid confusion and stress. I like having that philosophy extended to my personal projects and user environment, as it ensures a level of consistency and reproducibility that I haven’t found anywhere else.

Made possible via the Asahi Linux project and this effort, maintained by someone who I can only describe as a heaven-sent angel. ↩
I’ve built ffmpeg from source sans Nix and it wasn’t actually that hard, but what’s actually cool here is that that exact build configuration (ffmpeg with that non-free library) is available across all machines with my user environment—so I only had to figure it out once. ↩
Look, recursion! And yes, I’m referring to all of the other items in this list. ↩
I’m stealing this quote from Burke Libbey, who has made a great presentation explaining what Nix actually is at a high level. I recommend you watch it if you’re interested. ↩
Hey, gotta cover all bases. ↩
Apparently this is what 0.0.0.0 is named. ↩
This was pointed out to me in a talk by Xe Iaso, which is definitely worth a watch and or read if you’re interested in Nix’s rougher edges. They point out a lot of the same roadblocks that I ran into personally. ↩

Behind the curtain#

Ephemerality#

Declarative systems#

Declarative, modular systems#

Diving in#

Rougher edges#

The nature of Nix#

Bottom line#

Footnotes#