FAQ & limits
Questions
Is this fine-tuning?
No. No weights are trained or changed. One constant is added to one neuron while the model runs. Stop adding it and the model is exactly what it was.
Why would one neuron be enough?
Because of how the push is sized. A neuron's write arrow is a direction in the residual stream, and the recipe scales the push to be about as large as the stream itself. That is a very large, very consistent nudge in one direction at every token. Most directions just damage the model. A few happen to line up with something like a tone of voice, and the layers after it carry that through.
How is this different from a steering vector?
Mechanically it is one: pushing a neuron equals adding a fixed vector. The difference is where the vector comes from. A usual steering vector is computed from data. Here it is a single column of the model's own weights, which makes the whole intervention three numbers you can put in a tweet.
Which models does it work on?
Open-weight models you can run locally, through Hugging Face transformers or llama.cpp. Not hosted API models: you need access to the insides. MLX is not supported yet.
Will it always find a neuron?
No. Some personas have no good single neuron in some models. The verdict is then NOT FOUND, or WEAK when a neuron does something but fails one of the checks. Both are normal results. Before giving up, check that the judge question is sharp and the example replies are actually in voice.
What does a negative dose mean?
Push the neuron down instead of up. skrrt picks the sign from the cosine, so a winner can have a negative dose:
the confirmed Inkling neuron is L10 E30 N1011 at -1.5. On the command line write it as
--dose=-1.5.
What does a hunt cost?
Compute on your own machine, plus the judge. A default hunt on Inkling 2 Mini took about 25 minutes on one RTX 5090
and $0.082 of Jev calls (4,278 replies). An earlier, larger hunt of about 12,000 Jev calls cost $0.59. With
--judge self the judge is free.
My run was interrupted. Now what?
Run the same command again. Finished stages are skipped, and a half-done stage reuses what is already on disk.
Does a neuron carry over to another quantization or fine-tune?
Do not assume so. Results are specific to one checkpoint and one quantization. Rerun the hunt, or at least test the
neuron with skrrt steer.
Does an LLM decide which neuron wins?
No. A judge model answers yes/no questions about individual replies. Everything else, including which neurons advance and what counts as confirmed, is fixed arithmetic.
Limits
- It is a blunt instrument. A single-neuron push shoves the whole residual stream one way at every token. It shifts tone and attitude, sometimes content. It does not give the model new knowledge, and too high a dose turns fluent text into loops and noise.
- Verdicts are only as good as the judge. CONFIRMED means "passed these fixed checks", not "works on every prompt", and not "a person read every reply". Two judges can give the same neuron different counts. Read the samples.
- Results are specific. One checkpoint, one quantization, a small set of prompts. A confirmed neuron is evidence about that file on prompts like those.
- It does not explain the neuron. skrrt shows that pushing a neuron has an effect. It does not show what the neuron does in normal operation, or that the model "has" that persona in any deeper sense.
- Twelve prompts is twelve prompts. The held-out set is small. 12/12 against 0/12 for controls is a clear gap; 8/12 against 5/12 would deserve a second look with more prompts.
Responsible use
Steering removes a model's default manners. That is the point of the experiment and also the risk.
- Steered models can produce hostile or sexual text, including in answer to innocent questions.
- Run them yourself, as an adult, knowing what you switched on.
- Only steer models you are allowed to modify. Check the model's license.
- Do not deploy a steered model to people who have not been told. Nobody should meet the evil neuron by surprise in a product, a chat room or a support widget.
skrrt is a research tool for looking at how models are put together. Use it for that.