- Slightly off topic, but Claude Code (and likely other models/harnesses) are incredibly effective at Nix. It can trivially self-verify, without side effects, which is a perfect match for an LLM.
If you've ever been put off by the difficulty of the language, it's worth checking it out again with AI assistance.
- Absolutely. Running NixOS is a wonderful experience for this reason. NixOS + LLMs make it a breeze to make changes to your system, install and configure new software, and debug issues. Having every aspect of your system defined in a git repo is the perfect fit for agent harnesses.
I'll admit, even with LLMs to help, the Nix language and NixOS did have a rather steep learning curve, because it's quite different from anything I'd experienced before. But after getting the hang of it, I can't imagine going back to a "normal" OS and I'm very happy I put in the time to get over the initial friction.
- I got curious one Saturday and ported our monorepo to use Nix for build and test and release. It was a very pleasant process. I had to put aside to tackle more pressing things but I'm all in on using Nix in that capacity.
- Agreed, Claude has helped a lot with this project, and being able to iterate without side effects for system configuration changes is really a game changer for agents.
- I also love how with the right system prompt they can pull in tooling for what they currently need via a nix shell
- I've one-shotted custom distributions built with Nix using AI. It's crazy how well AI and Nix fit together.
- There is also a microvm.nix project which helped us support sandboxes with firecracker. So, whole ai workflow pipeline can now be nixos.
- Been running this on a few Asus GX10 machines with k3s on top, it’s been great. I’m running the new deepseek.
Thank you for your work!
- What quant are you using, and what tps are you getting with K3?
- The FP8 version from DeepSeek themselves [0], around 1800 tps prefill and 45 tokens per second decode.
I’ve been running a custom VLLM image with b12x as well as nvfp4_ds_mla.
I would say it’s quite fantastic in day to day, I use it mostly in Hermes and sometimes for coding.
I have qwen 3.6 27b on an rtx 6000 pro as well so I use that as a workhorse in pi with DS as a reviewer/planner.
[0] https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731
Edit: I think you may have misread my post. k3s is NOT kimi k3, and I did mention I was running deepseek.
- Thanks - yeah, sorry, I mis-read that as you using both DS and K3...
- Huge plus to anyone interested in the space to check out what Graham built here!
If anyone is also interested on Nix/CUDA/Capital Markets/Flox, we recently did another case study in the space - https://flox.dev/blog/deploying-hardened-flox-nvidia-cuda-st...
- This has been amazingly helpful for managing my DGX Spark! Thank you for all your time and effort into this project!
- Good to hear, thanks!
- This is incredible. I have a Jetson lying around and will try to it out on this. I use it to play with vision models, not LLMs, and have been wanting a better way to manage the machine.
- Thanks for sharing, saving this for when I get a DGX Spark
- [dead]
- [dead]
63 points by graham33 3 hours ago | 16 comments