Revert "nixos: build for aarch64-linux"

This reverts commit b806e25d.

This seems to push Hydra's memory usage out of the roof fail nixos
evaluating with:

Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS

Let's revert this for now. It's not a big deal at all since the
nixpkgs-unstable jobset is still building the packages.