Backends
A backend is how skrrt loads the model, reads its residual stream and pushes a neuron.
| backend | loads | patching |
|---|---|---|
hf | Hugging Face transformers models | none |
llamacpp | GGUF files | none for standard architectures; a two-line patch for Inkling 2 Mini, shipped in the repo |
mock | a synthetic model with a planted neuron | none; for tests and demos |
| MLX | not supported yet | |
--backend auto (the default) picks llamacpp when --model is a
.gguf path and hf otherwise. Backend-specific settings go through
--backend-arg key=value.
hf: transformers
uv pip install -e ".[hf]"
skrrt hunt --model Qwen/Qwen3.5-4B --persona pirate
Works on ordinary Hugging Face causal language models with no patching. skrrt finds the layers and each MLP's down projection by structure, then attaches hooks: one to read the residual stream, one to add the push at the input of the down projection.
- Plain MLPs: a down projection that is a
Linearlayer, or a GPT-2 styleConv1D. - Mixture-of-experts layers: the Mixtral and Qwen-MoE layouts.
- Not supported: weights quantized with bitsandbytes. Load the model unquantized, or use a GGUF with the llama.cpp backend.
llamacpp: GGUF files
uv pip install -e ".[llamacpp]"
llamacpp/build.sh # once: builds the small engine from llama.cpp
skrrt hunt --model path/to/model.gguf --persona pirate
For GGUF files, including quantized ones. llamacpp/build.sh builds a small engine from llama.cpp that
skrrt drives.
Standard architectures need no patch. llama.cpp already has control vectors: a fixed vector per layer, added to the residual stream. A control vector lands in the same place as the MLP's output, and pushing a neuron equals adding a fixed vector there, so the existing mechanism is enough.
- Layer 0 cannot be steered. llama.cpp has no control-vector slot for it. Layer-0 neurons are skipped, and the report lists them.
- Inkling 2 Mini needs a patch. It is a looped mixture-of-experts model: a block of its layers runs
several times per token. llama.cpp needs a two-line patch to apply control vectors to it, which ships in the repo as
llamacpp/patches/inkling2-cvec.patch. skrrt reads the loop layout from the GGUF metadata and applies the push on every pass.
mock: no GPU, no model
A synthetic model with one planted neuron. It shows the whole pipeline in a few seconds:
python -m skrrt.backends.mock > zorp.yaml
skrrt hunt --model mock --persona zorp.yaml --judge self
The mock only knows its own made-up persona (an alien that says "zorp"). Hunting a built-in persona in it ends in NOT FOUND, as it should.
Serving a steered model
Once you have a neuron, export-cvec saves the push as a llama.cpp control vector file that
llama-server can load. This is for GGUF models only.
skrrt export-cvec --model M.gguf --neuron "L11 E14 N370" --dose 0.75 -o evil.gguf
llama-server -m M.gguf --control-vector-scaled evil.gguf:1.0
The dose is baked into the file, so the scale is 1.0. For Inkling 2 Mini, llama-server
itself has to be built with the two-line patch; a stock build will not apply the vector correctly.
Read responsible use before you put that server anywhere other people can reach.
Results do not transfer between files
A neuron index belongs to one checkpoint and one quantization. L11 E14 N370 in a Q8_0 GGUF is not
promised to work in a 4-bit file of the same model, and it means nothing in a different model. If you change the file,
rerun the hunt, or at least rerun skrrt steer and look.