I had a few issues with the Stalwart mailserver, that I wanted to investigate using the source code. I’m new to Rust and the having a language server like rust-analyzer available vastly improves understanding of the code and makes navigation easier. My first need was to reproduce the build environment in a development shell with additional tools like rust-analyzer, clippy and rustfmt available. I expected to instrument the source a little to gain more insight into what is happening, and likely would have to deploy it into production, to more easily let it interact with the complex mail ecosystem, consisting of e-mail clients and third-party servers.
Deploying a local build is where nix really shines. It takes some effort to set up, but after that, we get to deploy the exact dependencies that were used during development. On other distributions, I would likely have to install many system dependencies for local development, that may not be the same as the dependencies on the target system.
First attempt, copying package files
For my first attempt, I decided to copy the package files for stalwart to a local directory for easy editing. That may not have been the wisest decision for maintenance, as that requires manually syncing upstream changes. I did not expect to maintain a fork for a long time, though.
The full result of this approach can be viewed on my new Forgejo instance at this state. There are some dead package files in there, as I don’t use the webui and spam-filter from nixpkgs, but put them there in case I also wanted to explore them later.
Now that the full source can be seen on Forgejo, I’ll spare some boilerplate. First, we need some inputs. I chose the git+file// protocol instead of path:, because that will avoid copying the source trees to the nix store, each time I enter the development shell. It does however mean, that I will have to remember to
nix flake update stalwart-srcbefore building the package, for example. Otherwise, nix will not know about changes to the source, and continue to use the pinned version from flake.lock. Let’s add inputs to flake.nix, which I placed in /home/m/s/stalwart/:
inputs = {
nixpkgs.url = "git+file:///home/m/s/nixpkgs?ref=nixos-unstable";
stalwart-src = {
url = "git+file:///home/m/s/stalwart/stalwart?ref=main";
flake = false;
};
stalwart-cli-src = {
url = "git+file:///home/m/s/stalwart/cli?ref=main";
flake = false;
};
};Since I was only interested in local development and builds, I also used my local clone of nixpkgs. I then use those inputs to pass them as a src argument to the copied package files:
stalwart = pkgs.callPackage ./nix/stalwart_0_16/package.nix {
src = stalwart-src;
};
stalwart-cli = pkgs.callPackage ./nix/stalwart-cli/package.nix {
src = stalwart-cli-src;
};The upstream package files have src hardcoded to a specific version. The other change to ./nix/stalwart_0_16/package.nix will be to change cargoHash to use the Cargo.lock instead. Here are the complete changes that were necessary:
@@ -1,7 +1,7 @@
{
lib,
rustPlatform,
- fetchFromGitHub,
+ src,
pkg-config,
protobuf,
bzip2,
@@ -51,18 +51,19 @@ let
in
rustPlatform.buildRustPackage (finalAttrs: {
pname = "stalwart" + (lib.optionalString stalwartEnterprise "-enterprise");
- version = "0.16.15";
+ version = "0.16.18-dev";
__structuredAttrs = true;
- src = fetchFromGitHub {
- owner = "stalwartlabs";
- repo = "stalwart";
- tag = "v${finalAttrs.version}";
- hash = "sha256-DRo+1olglHsOpAk5D8hrTi+KVgFC5MxxqnrOphbvrUo=";
- };
+ inherit src;
- cargoHash = "sha256-gjZR0qDdrS7TdWTeeRcKUY6pZFnLCMwnnpGAHWqiWLw=";
+ cargoLock = {
+ lockFile = "${src}/Cargo.lock";
+ outputHashes = {
+ "hickory-net-0.26.1" = "sha256-kF/AyYZH7To15a5dmzGOcTwBIm7rThDRH02C3h81dxQ="; # or lib.fakeHash
+ "opentelemetry-0.31.0" = "sha256-6qbfRpD3Q0Q942V/MuxFb8hyseIgdXjEMAwyqtIxlRI="; # or lib.fakeHash
+ };
+ };
env = {
# https://docs.rs/openssl/latest/openssl/#manualThe outputHashes for those two packages are required, because they are git dependencies and while there are git commit hashes available, nix needs the source hash. I obatined the hashes later by starting with fake hashes and let nix build report the actual hashes. So far, we are only intersted in a devshell and Nix being lazy will not evaluate those parts yet. Speaking of devshell, here it is in flake.nix:
devShells.${system}.default = pkgs.mkShell (
(stalwart.env or { })
// {
inputsFrom = [
stalwart
stalwart-cli
];
packages = with pkgs; [
pkg-config
rust-analyzer
rustfmt
clippy
cargo-watch
];
RUST_SRC_PATH = "${pkgs.rustPlatform.rustLibSrc}";
}
);The stalwart package defines some extra environment variables in the env attribute, that I am reusing for the shell here. // is the merge operator in Nix. It takes an attribute set on the left side and merges them with an attribute set on the right side, with the right side taking precedence.
For the devshell, that takes the build inputs from both packages that I am interested in and adds some tools for development. I also had to add pkg-config manually for cargo to be happy. At this point, I can enter a devshell using
nix developand then launch an editor at the source root, for example zed, that I like to use at the moment:
zeditor stalwartThe language server works and I can also successfully build, using
cargo buildThis is enough for exploration and local test runs outside of a proper systemd service setup.
For packaging and building with nix build the flake file will also have to define a packages output:
packages.${system} = {
default = stalwart;
inherit stalwart stalwart-cli;
};
# Expose overlay for external flake usage
overlays.default =
final: prev:
let
sys = final.stdenv.hostPlatform.system;
in
{
stalwart-custom = self.packages.${sys}.stalwart;
stalwart-cli-custom = self.packages.${sys}.stalwart-cli;
};I also added an overlay for consumption in my server’s flake. I’m not exactly happy about the system attribute situation, but at least it works like it should.
On the flake for my server, I had to add the stalwart flake to the inputs
stalwart-flake.url = "git+file:///home/m/s/stalwart";and the overlay to the modules list
modules = [
./configuration.nix
{ nixpkgs.overlays = [ inputs.stalwart-flake.overlays.default ]; }
inputs.disko.nixosModules.disko
inputs.sops-nix.nixosModules.sops
];This way I can very obviously use the right packages in my configuration:
environment.systemPackages = [
pkgs.stalwart-custom
pkgs.stalwart-cli-custom
];As I have run nix build to completion on my desktop, Stalwart did not have to be rebuilt when I rebuilt the server, but merely upload the package closures.
Second attempt, using overrides
Since I started using the devshell and the time I actually had a fix that I wanted to deploy to my server, the upstream nix packages have already diverged. There would have been another way to override the source using overrideAttrs.
I originally thought, that won’t be enough, because I also need to delete the cargoHash attribute, but it turns out, all you have to do to delete an attribute, is to set it to null. Unfortunately, cargoHash and cargoLock are arguments to the buildRustPackage function, that pretty much contains the whole derivation. By the time overrideAttr runs, those arguments have been consumed by buildRustPackage and are no longer available to be overridden. Instead, there is a cargoDeps attribute that we can use. Thanks Claude. I have to say, Claude Sonnet 5 has very good understanding of nixpkgs.
Here is the new let block in flakes.nix:
let
system = "x86_64-linux";
pkgs = import nixpkgs { inherit system; };
stalwart = pkgs.stalwart_0_16.overrideAttrs (old: {
src = stalwart-src;
version = "0.16.18-dev";
cargoDeps = pkgs.rustPlatform.fetchCargoVendor {
src = stalwart-src;
hash = "sha256-aGtPNT3yzNPNrL4VLfg7xisv9B7K8nVH5XCSG/dattg=";
};
});
stalwart-cli = pkgs.stalwart-cli.overrideAttrs (old: {
src = stalwart-cli-src;
version = "1.0.13";
cargoDeps = pkgs.rustPlatform.fetchCargoVendor {
src = stalwart-cli-src;
hash = "sha256-iIVgIGG1vhosQyPDFP3dokiojze9sGpvE5xKWdhlpzw=";
};
});
inThe copied and patched package files are no longer in use and we use upstream Nix packages instead. This is a lot cleaner and maintainable than my previous attempt, and I’ll continue to use this at least until 0.16.18 is released and packaged in NixOS, as that contains the fix to properly use E-mail aliases from LDAP.