skrrt

Backends

A backend is how skrrt loads the model, reads its residual stream and pushes a neuron.

backendloadspatching
hfHugging Face transformers modelsnone
llamacppGGUF filesnone for standard architectures; a two-line patch for Inkling 2 Mini, shipped in the repo
mocka synthetic model with a planted neuronnone; for tests and demos
MLXnot supported yet

--backend auto (the default) picks llamacpp when --model is a .gguf path and hf otherwise. Backend-specific settings go through --backend-arg key=value.

hf: transformers

sh
uv pip install -e ".[hf]"
skrrt hunt --model Qwen/Qwen3.5-4B --persona pirate

Works on ordinary Hugging Face causal language models with no patching. skrrt finds the layers and each MLP's down projection by structure, then attaches hooks: one to read the residual stream, one to add the push at the input of the down projection.

llamacpp: GGUF files

sh
uv pip install -e ".[llamacpp]"
llamacpp/build.sh               # once: builds the small engine from llama.cpp
skrrt hunt --model path/to/model.gguf --persona pirate

For GGUF files, including quantized ones. llamacpp/build.sh builds a small engine from llama.cpp that skrrt drives.

Standard architectures need no patch. llama.cpp already has control vectors: a fixed vector per layer, added to the residual stream. A control vector lands in the same place as the MLP's output, and pushing a neuron equals adding a fixed vector there, so the existing mechanism is enough.

mock: no GPU, no model

A synthetic model with one planted neuron. It shows the whole pipeline in a few seconds:

sh
python -m skrrt.backends.mock > zorp.yaml
skrrt hunt --model mock --persona zorp.yaml --judge self

The mock only knows its own made-up persona (an alien that says "zorp"). Hunting a built-in persona in it ends in NOT FOUND, as it should.

Serving a steered model

Once you have a neuron, export-cvec saves the push as a llama.cpp control vector file that llama-server can load. This is for GGUF models only.

sh
skrrt export-cvec --model M.gguf --neuron "L11 E14 N370" --dose 0.75 -o evil.gguf
llama-server -m M.gguf --control-vector-scaled evil.gguf:1.0

The dose is baked into the file, so the scale is 1.0. For Inkling 2 Mini, llama-server itself has to be built with the two-line patch; a stock build will not apply the vector correctly.

Read responsible use before you put that server anywhere other people can reach.

Results do not transfer between files

A neuron index belongs to one checkpoint and one quantization. L11 E14 N370 in a Q8_0 GGUF is not promised to work in a 4-bit file of the same model, and it means nothing in a different model. If you change the file, rerun the hunt, or at least rerun skrrt steer and look.

backends.html