---
title: Optimised Backups on NixOS with restic & fd
published: 2022-08-01
updated: 2023-01-07
featuredImage: ../../images/nixos-restic.png
featuredImageAlt: NixOS plus Restic
---

I'd like to present a simple restic backup configuration I've been using that allows excluding a lot of unnecessary files via ignore patterns & by respecting `.gitignore`, thus shrinking backup sizes.

#### The Problem

Backups are important but not all files need to be backed up.
Excluding paths from our backups can be very useful when we know the data is reproducible. E.g., if the target system runs podman containers that are all declared via NixOS options we don't really need most of the data in `/var/lib/containers`. When using volumes, though, `/var/lib/containers/storage/volumes` should be backed up.
Other examples of files to exclude would be: caches, temporary files, artifacts in development directories.
We might also decide that some data, while not reproducible, just isn't important to back up.

#### Restic

For my personal needs I've decided to use [restic](https://restic.net) as my backup solution. I won't go into the details why I chose it over other solutions, but some of the reasons I went with restic are:
- existing NixOS module
- many storage options (e.g. Backblaze)
- incremental backups
- deduplication
- strong encryption
- zstd compression

In my experience restic is easy to use, fast & has some powerful options.

#### Finding the right approach

While the NixOS options for restic allow specifying paths to include via [`services.restic.backups.<name>.paths`](https://search.nixos.org/options?channel=unstable&show=services.restic.backups.%3Cname%3E.paths&from=0&size=15&sort=relevance&type=packages&query=services.restic.backups.%3Cname%3E.paths) there is no native option to add exclusion patterns.
So I've looked at the other options and found [`services.restic.backups.<name>.dynamicFilesFrom`](https://search.nixos.org/options?channel=unstable&show=services.restic.backups.%3Cname%3E.dynamicFilesFrom&from=0&size=15&sort=relevance&type=packages&query=services.restic.backups.%3Cname%3E.dynamicFilesFrom), which expects a script that produces a list of paths to back up.  
This is great. My initial ideas was to generate a list of all files we want to back up, but this would become huge and even worse, restic's parent-snapshot detection will fail when paths used for [`--files-from`](https://restic.readthedocs.io/en/stable/040_backup.html?highlight=exclude-file#including-files) change ([#2246](https://github.com/restic/restic/issues/2246)).

So, instead I need to do the opposite: have few backup paths and exlude a list of files to ignore.
Looking further into restic's documentation, I found the option [`--exclude-file`](https://restic.readthedocs.io/en/stable/040_backup.html?highlight=exclude-file#excluding-files) which does exactly that.
We can use this option in NixOS via [`services.restic.backups.<name>.extraBackupArgs`](https://search.nixos.org/options?channel=unstable&show=services.restic.backups.%3Cname%3E.extraBackupArgs&from=0&size=15&sort=relevance&type=packages&query=services.restic.backups.%3Cname%3E.extraBackupArgs)

Now that we found a way that allows us to exclude bloat from our backups, let's look how that might look in practise next.

#### Full solution

This is a simplified version of what I'm using in my desktop configuration. For a complete list check out my [actual config](https://gitlab.com/felschr/nixos-config/-/blob/main/services/restic/home-pc.nix). Hopefully some of these ignore patterns will fit your use case.

```nix
services.restic.backups.full = {
  initialize = true;
  repository = "SOME-REPO";
  timeConfig.OnCalendar = "daily";
  paths = [ "/etc/nixos" "/var/lib/" "/home" ];
  extraBackupArgs = let
    ignorePatterns = [
      "/var/lib/systemd"
      "/var/lib/containers"
      "/var/lib/flatpak"
      "/home/*/.local/share/Trash"
      "/home/*/.cache"
      "/home/*/Downloads"
      "/home/*/.npm"
      "/home/*/Games"
      "/home/*/.local/share/containers"
      "/home/felschr/dev" # backup ~/dev-backup instead
      ".cache"
      ".tmp"
      ".log"
      ".Trash"
    ];
    ignoreFile = builtins.toFile "ignore"
      (foldl (a: b: a + "\n" + b) "" ignorePatterns);
  in [ "--exclude-file=${ignoreFile}" ];
  pruneOpts = [
    "--keep-daily 7"
    "--keep-weekly 4"
    "--keep-monthly 3"
    "--keep-yearly 1"
  ];
};
```

##### Respecting `.gitignore`

As you might have noticed above, I've excluded my `~/dev` directory from backups.
Since my development folder is huge and has lots of dependencies, build artifacts, etc., I've decided the best way to include it in my backups is to exclude everything that's mentioned in `.gitignore` files.

For this, I've decided to tackle this by generating `~/dev-backup` in the pre-start script of restic's systemd service.
In there I use [`rsync`](https://rsync.samba.org) to clone `~/dev` to `~/dev-backup`. With the `--filter` option rsync supports exclude files like `.gitignore` natively.
By also providing the `--link-dest` option, we tell rsync to create links to the original files instead of duplicating them. This way we don't have any storage overhead for this setup.

The resulting configuration looks like this:

```nix
systemd.services."restic-backups-full" = {
  preStart = ''
    rm -rf /home/felschr/dev-backup
    ${pkgs.rsync}/bin/rsync \
      -a --delete \
      --filter=':- .gitignore' \
      --link-dest=/home/felschr/dev \
      /home/felschr/dev/ /home/felschr/dev-backup
  '';
};
```

I hope you found this small guide useful and I could spare you some of the pitfalls I ran into before.

My full restic config can be found here: https://gitlab.com/felschr/nixos-config/tree/main/services/restic

#### UPDATE 2023-01-07

The article has been adjusted to fix an issue in the original approach.
For more details of the changes check out the [git history](https://gitlab.com/felschr/felschr.com/-/commits/main/src/content/posts/nixos-restic-backups.mdx) of this article.
