Setting up Services#

From setting up services like k3s and the services within k3s to ensuring secrets were on the system one of the things I realized most while working on this porject was how often I relied on systemd services. Whether they were oneshot services or services that needed to start when the system started in general I ended up relying on these for many operations.

./modules/k3s.nix#

As listed above in the configuration.nix file is a line that says import = [ ./modules/k3s.nix ]

Below is what that module looks like. Essentially this code is going to take the name of a yaml file inside the directory k3s-config and move that file to /etc/k3s/<file name> then link the file in /etc/k3s/ to the /var/lib/rancher/k3s/server/manifests folder on our server when the server boots up. It is important to note anything inside our yaml files is not secure as the files are copied to the nix-store which is globally readable. We will deal with secrets later.

{
  pkgs,
  lib,
  ...
}: let
  # Read all files in ../k3s-config
  configDir = ../k3s-config;
  configFiles = builtins.attrNames (builtins.readDir configDir);

  # Helper to get the full path for a file
  fullPath = file: "${configDir}/${file}";
in {
  # Generate environment.etc entries for all files
  environment.etc = builtins.listToAttrs (map (file: {
      name = "k3s/${file}";
      value = {
        text = builtins.readFile (fullPath file);
      };
    })
    configFiles);

  # Generate tmpfiles rules for all files
  systemd.tmpfiles.rules = builtins.concatLists (map (file: let
      manifestName = file;
      nameNoExt = lib.replaceStrings [".yaml"] [""] file;
    in [
      # This needs to be fixed, but there were issues with the traefik-custom container!
      "d /var/lib/nik3sx/${nameNoExt} 777 root root -"
      "d /var/lib/rancher/k3s/server/manifests 0755 root root -"
      "r /var/lib/rancher/k3s/server/manifests/${manifestName}"
      "C /var/lib/rancher/k3s/server/manifests/${manifestName} - - - - /etc/k3s/${manifestName}"
    ])
    configFiles);

  # Generate one systemd service per manifest
  systemd.services = builtins.listToAttrs (map (file: let
      manifestName = file;
    in {
      name = "k3s_Install_${lib.replaceStrings ["."] ["_"] manifestName}";
      value = {
        description = "Copy ${manifestName} manifest and restart k3s";
        after = ["systemd-tmpfiles-setup.service" "network.target" "k3s.service" "boot-unlock-service.service"];
        wantedBy = ["multi-user.target"];
        serviceConfig = {
          Type = "oneshot";

          ExecStartPre = "${pkgs.bash}/bin/bash -c 'until ${pkgs.kubectl}/bin/kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml get nodes &>/dev/null; do echo Waiting for K3s API...; sleep 2; done'";

          ExecStart = ''
            ${pkgs.coreutils}/bin/install -Dm644 /etc/k3s/${manifestName} /var/lib/rancher/k3s/server/manifests/${manifestName}
          '';
          ExecStartPost = "${pkgs.kubectl}/bin/kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml apply -f /var/lib/rancher/k3s/server/manifests/${manifestName}";
        };
      };
    })
    configFiles);
}

Service Pages#

The services in the folder below are the three I use most. There isn’t anything too special about these services, they are mostly normal k3s configurations for services.

The point is to show the services are available and able to come online via a declarative folder.