UP | HOME

NixOS Adventure Guide From a Beginner

Nix is a functional package management system which is rather difficult to setup. The author goes for a wild ride and documents all the things you probably would like to know as a new user, but are hidden in some forum, wiki, irc log, and all manners of strange places.

1. Intro

So you want to try Nix. It’s a functional package management system with a ton of packages. I tried it 5 years ago and couldn’t really get it to do what I wanted to do. Recently, I upgraded to a 43 inch 4k monitor, powered by an X230, and found that the exwm rendering and typing was much less responsive. Coupled with the fact that I’m not particularly looking forward to Ubuntu 20.04 upgrade and looking over the compilation options for gccEmacs, I decided to give Nix a try on my Ubuntu system. I later setup Nix on a separate laptop.

Rolling release systems are generally easier to maintain for distros as security fixes are already upstream. They are usually better for users because they get the latest vanilla packages. However, what sucks is upgrading with all the dependencies. Ubuntu has PPA’s and apt is godly, but why use snaps or flatpak when NixOS solves the problem of general packaging of a system. I would prefer using static musl binaries, but I’m not sure if that can be done with no compromise.

My general review about Nix is that I’m not in love with it mainly because the documentation is just so mediocre. Great as a package manager on a Ubuntu install and as the main install on a laptop with tons of incredibly neat features. However, the documentation really only makes sense after you already know what you’re doing. Conceptually, I’ve liked it for a while, but the fact that you have to learn a language to configure it while referencing random repos in Github really, really sucks, so I cannot possibly recommend it for anyone but a power user.

But if you are a power user, well, there’s also NixOps and MobileNix for Pinephone users, which could make things extremely compelling. That and gccEmacs is how I got sucked in at least to the tune of 20+ hours. On the plus side, I do have NixOS working reliably on my laptop and NixPkg on my Ubuntu machine and have been using both as my daily drivers for the past two weeks.

So below are what I ran into in my adventures in debugging and getting the system to work and summarizing hours of reading, searching, and debugging. Hopefully, it will be useful for someone.

2. Nix on Ubuntu as a package manager

2.1. Installation

The installation mostly works, but I had to twiddle with things a bit. Note that there’s the Nix package manager and then there’s NixOS. Both manage things quite differently with different files and commands.

Nix takes a lot more space than a standard distribution. I made an LVM volume for nix and mounted it on nix . Note that a symlink doesn’t work, so if you are not using LVM and just put it in your home directory, you’ll have to mount it:

mount -o bind /path/to/nix /nix

3. Installation onto Ubuntu / foreign system

So for the heck of it, I wanted to see if I could replace the packages I use mainly and have them actually work. And incredibly enough, sound and video worked fine for firefox and vlc with little effort. I actually replaced/updated most of my default Ubuntu binaries. Installation of the correct versions was a different story.

The most important resources here are:

Useful commands:

nix-env --rollback
nix-env -q # lists packages
nix-env -qa something # searches for something

This is what I installed to start:

  • elixir-1.10.4
  • emacs-27.1
  • firefox-81.0
  • fossil-2.12.1
  • gajim-1.2.2
  • hello-2.10
  • mpv-0.32.0
  • mu-1.4.13
  • ncpamixer-1.3.3
  • nix-2.3.7
  • pavucontrol-4.0
  • pulseaudio-13.0
  • ripgrep-12.1.1
  • simplescreenrecorder-0.3.11
  • telegram-desktop-2.3.0
  • vlc-3.0.11.1

3.1. Managing channels

So obviously, what versions you install depend on what channel that you are using. What’s not obvious is where the channels are. I combed through several wiki pages and then finally went to IRC to figure out why firefox was stuck at version 78 instead of 81 like it listed. Turns out I was using nixpkgs unstable which was stuck for some reason instead of nixos unstable. Easy fix - once you know what you’re looking for.

So some important resources:

Useful commands:

$ nix-channel add https://nixos.org/channels/nixos-unstable nixos

$ nix-channel --list
nixos https://nixos.org/channels/nixos-unstable
nixos-19.03 https://nixos.org/channels/nixos-19.03
nixos-20.09 https://channels.nixos.org/nixos-20.09
nixpkgs https://nixos.org/channels/nixpkgs-unstable

$ nix-channel --update

$ nix-env -f channel:nixos-19.09 -i package_name

Note that the version that you download on the update is not pinned in your configuration, so it’s not fully reproducible. There’s something called Nix Flakes that are supposed to solve that problem which I haven’t investigated.

3.2. GccEmacs

So after establishing that NixOS works reasonably enough for regular packages, it was time for Emacs. And despite having the instructions in a gist, it was pretty difficult.

Some links regarding gcc Emacs

And our guide:

So the first step in the gist is to install an overlay (which is great if you know what an overlay is and where to install it).

Now despite reading the first line, I didn’t quite get it because I had no idea what packageOverride and overridePackages are. After using Nix for 2 weeks, well, the documentation is quite clear. What’s missing from the picture is what contained in “main” and what can be overridden.

I ended up using option 2 despite not understanding a damn thing:

echo "import (builtins.fetchTarball {
      url = https://github.com/nix-community/emacs-overlay/archive/master.tar.gz;
    })" >> $HOME/.config/nixpkgs/overlays/emacs.nix

Vterm compiliation as a binary inside emacs inside Nix really does deserve a separate section. You can’t install libvterm outside of emacs because emacs won’t be able to find it. Luckily, the gist had an incomprensible section to me at the time which I added to /etc/nixos/configuration.nix and hoped for the best.

{
emacsWithPackages = (pkgs.emacsPackagesGen pkgs.emacsGcc).emacsWithPackages (epkgs: ([epkgs.vterm]));
}

A specific note was to use the overlay from the nixos-unstable channel, not the nixpkgs-unstable channel. In hindsight, everything is simple.

$ nix-channel --list
nixos https://nixos.org/channels/nixos-unstable
nixos-19.03 https://nixos.org/channels/nixos-19.03
nixos-20.09 https://channels.nixos.org/nixos-20.09
nixpkgs https://nixos.org/channels/nixpkgs-unstable

nix-env -iA nixpkgs.emacsGcc                    # what is in the gist and installed 27.1
nix-env -iA nixos.emacsGcc                      # what I should have used based on my channels
nix build -f channel:nixos-unstable emacsGcc    # what I ended up using

Since this is a post about Nix and not the wonders of Emacs with native-comp, let’s just say it worked and Emacs is incredibly faster visually. Anecdotally, it was using from 25-50% cpu on a 43 inch screen at 4k resolution doing close to nothing with a typing lag of 100-200ms. Now it’s using less than 10% and the typing lag is not noticable.

3.3. Garbage collection

If you install enough versions of packages with different versions of different channels, you may have a lot of the same packages installed with different versions of glibc, etc. Also, hidden in the nix-build man page, nix-build creates a symlink called “result”. If this symlink is not removed, all those packages will not be garbage collected.

Useful commands:

nix-env --list-generations
nix-env --delete-generations +1d    # deletes all the generations older than 1 day.
nix-collect-garbage

3.4. Managing the install declaratively

I never installed home-manager, but according to the cheatsheet, it might be possible to put everything in: ~/.nixpkgs/config.nix

For NixOS, everything goes into /etc/nixos/configuration.nix

3.5. Installing Ansible / Python

I have a bunch of machines I manage with ansible. I do some config generation that’s delegated to the localhost, so ansible needs to find various python libraries which are now no longer being installed at the system level. Installing ansible as a standalone package didn’t give it access to setuptools, which was needed to setup the virtual environment for the sub-commands to run. Yes, nix-shell could also be used instead of setting up the virtual env, but no reason for a playbook to suddenly be dependent on nix.

Again, not on the nix+ubuntu install, but on NixOS in /etc/nixos/configuration.nix

  environment.systemPackages = with pkgs; [
    # ansible installs its own python, which then is missing setuptools
    (python38.withPackages(ps: with ps; [ setuptools ansible]))
  ];

The ansible playbook also needed a bit of a change since it was upgraded and using python3 to use a {{ venv }}/exec as the precursor to the shell command.

- pip:
    name: ['package1', 'package2', 'package3']
    virtualenv: "{{ venv }}"
    virtualenv_command: "python -m venv"
  delegate_to: localhost
  become: false
  run_once: true

- copy:
    dest: "{{ venv }}/exec"
    mode: 0755
    content: |
      #!/bin/sh
      source {{ venv }}/bin/activate
      $@
  delegate_to: localhost
  become: false
  run_once: true

3.6. Wine

I use wine for an ancient windows program that I used to run on Win98. Still works and the fonts look nicer, too.

3.7. What about zoom and other foreign binaries?

Zoom

Luckily enough, zoom is available in unstable packages if you configure to allow unfree.

For NixOS, I did the following in /etc/nixos/configuration.nix:

{ config, lib, pkgs, ... }:

let
  unfree_unstable = import <nixos-unstable> { config.allowUnfree = true; };
in 
  environment.systemPackages = with pkgs; [
    unfree_unstable.zoom-us # unfree
  ];

For Nix on Ubuntu, there’s an entry in the faq which I haven’t muddled with yet which suggests a method to use ~/.config/nixpkgs/config.nix:

For other binary packages, there are a few ways to do this:

Redream and the greatest fighting game of all time

So the other day, a friend and I were playing the greatest fighting game of all time, Soul Calibur on Dreamcast. The movement, ringouts, and game-balance were unparalleled. The only issue is that the controllers were 20 years old and a little beat up for a fighting game and the CD is also a bit worn. What to do, right?

Well, there’s an emulator called redream and a way to connect the Nintendo Switch joycons over bluetooth.

Well, packaging go binaries connecting multiple joycons over bluetooth worked fine. Redream took a little more jiggering, but after stracing the missing libraries, elf patching came to the rescue.

# Thought this would work, but not good enough 
LD_LIBRARY_PATH=$(nix eval --raw nixpkgs.stdenv.cc.cc.lib)/lib ./redream

# What finally worked
patchelf --set-interpreter /nix/store/bdf8iipzya03h2amgfncqpclf6bmy3a1-glibc-2.32/lib/ld-linux-x86-64.so.2 redream
patchelf --set-rpath /nix/store/danv012gh0aakh8xnk2b35vahklz72mk-gcc-9.2.0-lib/lib redream
patchelf --set-rpath /nix/store/danv012gh0aakh8xnk2b35vahklz72mk-gcc-9.2.0-lib/lib:/nix/store/7563czahvq56fq3a486w2x2vr9r63j1g-libX11-1.6.8 /lib/:/nix/store/k0m4ap07x6g8yy8np5r8s2c21kpvn4vv-libXext-1.3.4/lib/:/nix/store/6z4ai3s2pda3wbaw7i40mfwi6yfq3l48-alsa-lib-1.1.9/lib/:/nix/store/nalsa5dpixk4622s9y588z0mkhy1zijd-systemd-246.6/lib/:/nix/store/xgm1dd92s3apbyx0rq1ga1pf9vzm625b-libglvnd-1.3.2/lib/ redream

4. Installing NixOS on an X230 as a desktop environment

So considering a lot of the packaging worked with a little searching, I burnt the ISO and tried NixOS. Booting up and setting up was pretty reasonable. And if I had a working configuration, I would have just let it build and be done. But as is the case, had to spend some time figuring things out.

Important commands:

nixos-rebuild switch
nixos-rebuild test
sudo nix-env -p /nix/var/nix/profiles/system --list-generations # otherwise, you have no idea what's installed

4.1. Setup wireless

Wireless setup from the default configuration file.

  networking.hostName = "x230a"; # Define your hostname.
  networking.networkmanager.enable = true;
  networking.useDHCP = false;
  networking.interfaces.enp0s25.useDHCP = true;
  networking.interfaces.wlp3s0.useDHCP = true;

Important commands:

sudo nmcli dev wifi connect theSSID password thePassword

4.2. Desktop manager

This is definitely a highlight. Firstly, you absolutely need to know about option search:

You can actually try/rollback between 34 different window managers by just adding a single line. The following is my desktop configuration and X configuration. I wanted to use the emacs that I compiled, so I didn’t enable the default exwm.

  # Enable sound.
  sound.enable = true;
  hardware.pulseaudio.enable = true;
  hardware.bluetooth.enable = true;
  services.blueman.enable = true;

  # Enable the X11 windowing system.
  services.xserver = {
    enable = true;
    layout = "us";
    xkbOptions = "ctrl:swapcaps";
    displayManager.sessionCommands = "${pkgs.xorg.xhost}/bin/xhost +SI:localuser:$USER";
  };

  services.xserver.windowManager.session = lib.singleton {
    name = "none+exwm";
    start = ''
      emacs
    '';
  };
  services.xserver.deviceSection = lib.mkDefault ''
    Option "TearFree" "true"
  '';

  #services.xserver.windowManager.icewm.enable = true;
  #services.xserver.windowManager.exwm.enable = true;

  # Enable touchpad support.
  services.xserver.libinput.enable = true;

4.3. Services

For instance, to start some services, you can:

  # For thinkpad
  services.tlp.enable = true;

  # Battery power management
  services.upower.enable = true;

  # User services
  systemd.user.services."xcape" = {
    enable = true;
    description = "xcape to use CTRL as ESC when pressed alone";
    wantedBy = [ "default.target" ];
    serviceConfig.Type = "forking";
    serviceConfig.Restart = "always";
    serviceConfig.RestartSec = 2;
    serviceConfig.ExecStart = "${pkgs.xcape}/bin/xcape";
  };

4.4. Wireguard and secrets management

Now, I didn’t find any secrets management, so I ended up using gpg to encrypt the secret. I like routing all my traffic through the tunnel, and as usual, there is just enough documentation to figure it out between github, discourse, and the wiki.

There are two interfaces that are listed - wireguard and wg-quick. So which is the one we’re supposed to use - the one on the top of the page or the one on the bottom? Well, if we’re keeping count, it’s the one on the bottom.

Also, just because it’s setup, doesn’t mean that you want it on all the time or on startup. There’s some special flags to create symlinks and force systemd service to be there instead of purged via lib.mkForce. I’m copying and pasting for posterity.

{ config, lib, pkgs, ... }:
{
  environment.etc = {
    "wireguard/wgdmz.key".source = "/etc/nixos/secrets_unlocked/wgdmz.key";
  };

  systemd.services."wg-quick-wgdmz" = {
    enable = true;
    #unitConfig.StopWhenUnneeded = true;
    wantedBy = lib.mkForce [];
  };

  networking.wg-quick.interfaces = {
    wgdmz = {
      # Determines the IP address and subnet of the client's end of the tunnel interface.
      address = [ "10.0.10.3/32" ];
      #dns = ["10.0.10.1"];
      privateKeyFile = "/etc/wireguard/wgdmz.key";

      peers = [
        {
          # Public key of the server (not a file path).
          publicKey = "Taa4kZJvUzT27UG2UZWsDRMadf9DwwzJb1j+nJEzEwk=";
          allowedIPs = [ "0.0.0.0/0" ];
          endpoint = "1.2.3.4:50000";
          persistentKeepalive = 25;
        }
      ];
    };
  };
}

4.5. Grub profiles / Options on boot

Another extremely cool thing is that every time that you rebuild, a new entry shows up in your GRUB configuration. I mostly used nixos-rebuild switch. So it’s extremely tough to have a broken configuration. Of course, I’m using emacs with native compilation, so that makes things a bit more complicated.

5. Conclusion

Overall, I’m not completely sold on it yet. I’m not in love, but it’s definitely a much nicer way of managing the system than I expected. Recommended for power-users only - for everyone else - there are probably easier ways to do things which would take a lot less time than I spent figuring out Nix.

Also, emacs with native compilation (the entire point of this yak-shaving episode) is just awesome.

5.1. The good

I got everything I tried working after a bit of searching.

  1. Sound/video on firefox and vlc through pulseaudio
  2. Emacs gcc with various custom compilation
  3. Python packages were a bit trickier to figure out than I would have liked.
  4. 3rd party FHS binaries like zoom and redream are workable.
  5. Declarative management works and cleans up the system nicely. My full laptop nix configuration is 350 lines with tons of comments. This post is nearing 450 lines.
  6. Desktop environment switching is incredible.
  7. Managing services declaratively is slick.
  8. Having the option to boot into a working config from grub is very nice.
  9. Wireless, bluetooth, xkbOptions, ssh-agent, gpg-agent, locales, timezone, networking, different services, power management are all functional on the desktop environment on the X230 installation.
  10. Helpful community.

5.2. The bad

The documentation only makes sense after you already know how everything works in the first place. Very difficult to find the options you need unless you search discourse or github issues. Few easily found examples. And having no default secret management for a configuration engine is strange.

5.3. The ugly

Searching for random repos on Github for examples on how to do things. Either you get lucky or you don’t.

5.4. The potentially interesting

NixOps and MobileNix. If you’re a power user with a PinePhone, well, it might be worth getting into the Nix world.

5.5. Alternatives - Guix

A lot more immature, but perhaps worth trying.

Date: 2020-11-11