How it works
Two parts: the recipe for pushing a neuron, and the four-stage search that finds which neuron to push.
Five terms
- Residual stream
- A language model keeps one vector (a list of a few thousand numbers) for each token of text. Every layer reads that vector, computes something, and adds its result back in. The running sum is the residual stream. It is the model's working memory, and what comes out of the last layer decides the next token.
- MLP
- Each layer has two parts: attention, which moves information between tokens, and an MLP (multi-layer perceptron), which works on one token at a time. skrrt only touches MLPs.
- MLP neuron
- An MLP first expands the residual stream into a wider list of numbers and applies a nonlinearity. Each number in that wide list is one neuron. A layer has thousands of them.
- Down projection
- The second half of the MLP is a matrix,
W_down, that maps the neurons back to the size of the residual stream. Columnnof it,W_down[:, n], is the direction neuronnwrites in. We call that column the neuron's write arrow. When the neuron's value goes up by 1, the layer's output moves by exactly one write arrow. - Dose
- How hard to push, measured against the residual stream's normal size at that layer. Dose 1 means a push about as large as the stream itself. Dose 0.75 is three quarters of that. A negative dose pushes the other way.
The recipe
Choose a layer L, a neuron n and a dose. Then, at every token, add this constant to neuron
n just before the down projection of layer L:
delta = dose * median(||resid leaving layer L||, tokens 1:) / ||W_down[:, n]||
Reading it left to right:
median(||resid leaving layer L||, tokens 1:)is the typical length of the residual stream vector coming out of layerL. It is the median over token positions, skipping the first token, whose vector is usually an outlier in size.||W_down[:, n]||is the length of the neuron's write arrow.- Dividing one by the other converts "dose" into the raw number to add to the neuron, so that the resulting change to
the stream has length
dose x (typical stream length).
That is why a dose means the same thing in any layer of any model. You never tune a raw activation value.
1. what skrrt does: push one neuron, at every token
layer L-1 =====+===========================(+)=====> layer L+1
| residual stream ^
read | | write
v |
+------------+ neurons +----------+
| W_up, then | o | W_down |
| activation | ---> o ---> | (linear) |
+------------+ [n] +----------+
^
|
+ delta
2. the same thing, because W_down is linear
layer L-1 =================================(+)=====> layer L+1
^
|
+ delta * W_down[:, n]
(n's "write arrow")
layer L-1
|
+--- read ---.
| v
| +----------+
residual | | W_up + |
stream | | activ. |
| +----------+
| |
| o o [n] o o
| ^
| + delta
| |
| +----------+
| | W_down |
| +----------+
| |
(+)<-- write -'
|
v
layer L+1
same as adding the fixed vector
delta * W_down[:, n] at the (+)
The down projection is linear, so adding delta to the neuron is exactly the same as adding the fixed
vector delta x W_down[:, n] to the layer's output. Single-neuron steering is ordinary activation steering
where the steering vector is not learned or averaged from data. It is a column the model already had.
Mixture-of-experts models
Some models replace each MLP with many smaller MLPs ("experts") and a router that picks a few per token. There a neuron
has three coordinates: layer, expert, neuron, written L11 E14 N370. skrrt applies the push at every token
whether or not the router picked that expert, so the push is the same constant vector either way.
The four stages
Finding the neuron used to be manual, agent-driven research. skrrt does it in four fixed stages. No language model makes decisions along the way; a judge model only answers yes/no questions about replies.
persona.yaml + model
|
v
+--------------+ +--------------+ +--------------+ +--------------+
| 1 RANK |-->| 2 SCREEN |-->| 3 CONFIRM |-->| 4 REPORT |
+--------------+ +--------------+ +--------------+ +--------------+
| every neuron | | top 8/layer | | best 10 | | fixed rules |
| scored by | | + random | | + random | | pick winner: |
| cosine with | | controls | | controls | | CONFIRMED, |
| the persona | | 6 prompts | | 12 held-out | | WEAK or |
| direction | | doses 1, 2 | | prompts | | NOT FOUND |
| | | | | 6 doses | | |
+--------------+ +--------------+ +--------------+ +--------------+
model only model + judge model + judge arithmetic
|
v
report.md + .json
persona.yaml + model
|
v
+------------------------------+
| 1 RANK (model only) |
+------------------------------+
| every neuron scored by its |
| cosine with the persona |
| direction |
+------------------------------+
|
v
+------------------------------+
| 2 SCREEN (model + judge) |
+------------------------------+
| top 8 per layer + random |
| controls, 6 prompts, |
| doses 1 and 2 |
+------------------------------+
|
v
+------------------------------+
| 3 CONFIRM (model + judge) |
+------------------------------+
| best 10 + random controls, |
| 12 held-out prompts, |
| 6 doses |
+------------------------------+
|
v
+------------------------------+
| 4 REPORT (arithmetic) |
+------------------------------+
| fixed rules pick the winner: |
| CONFIRMED, WEAK or NOT FOUND |
+------------------------------+
|
v
report.md + .json
1. Rank
Each persona ships with example replies written in its voice. skrrt feeds those replies through the model, and also the replies of every other built-in persona plus a neutral voice. The model reads the text rather than generating it; this is called teacher forcing. For each layer, skrrt averages the residual stream over the reply tokens of the persona, averages it over everyone else's, and subtracts. The difference is the persona direction for that layer: the way the stream leans when the text sounds like the persona.
Then every neuron gets a score: the cosine between its write arrow and the persona direction of its layer. Cosine is 1 when two arrows point the same way, 0 when they are unrelated, and -1 when they are opposite. A cosine far from 0 in either direction is interesting: a positive one says "push this neuron up", a negative one says "push it down". This stage needs no generation and no judge.
2. Screen
A strong cosine is a hint, not a result. So skrrt takes the top 8 neurons of every layer, ranked by the size of the cosine, and actually steers each one: doses 1 and 2, in the direction of the cosine's sign, on 6 screen prompts with short replies. A handful of random neurons go through the same thing. A judge scores every reply on two questions: is it in persona, and is it still coherent? A neuron's screen score is its best dose's average of P(persona) x P(coherent).
3. Confirm
The 10 best neurons get a full dose sweep (0.5, 0.75, 1, 1.5, 2, 3) on 12 held-out prompts: prompts that played no part in picking them. So do 4 random neurons from each finalist's layer, and the unsteered model.
4. Report
Fixed rules turn the judged replies into a verdict. They are printed at the bottom of every report:
- hit
- A held-out reply where the judge gives P(persona) > 0.5 and P(coherent) > 0.5.
- best dose
- For each neuron, the dose with the most hits. Ties go to the smaller dose.
- winner
- The finalist with the most hits at its best dose. Ties go to the higher average P(persona).
- CONFIRMED
- All three hold: the winner hits at least 2/3 of the held-out prompts (8 of 12); the unsteered baseline hits at most 1/4 (3 of 12); and the winner's hits minus the best random control's hits is at least 1/4 of the prompts (a margin of 3 of 12). Each control is counted at its own best dose.
- NOT FOUND
- The winner hits fewer than 1/3 of the prompts, or no more than the baseline, or no more than the best random control.
- WEAK
- Anything in between. The report says which check failed.
report.md has the paste-ready line, the finalists, the controls, the baseline, sample replies and the
exact skrrt steer command.
Why controls and held-out prompts
A big enough push on any neuron breaks a model. Broken models say odd things, and a lenient reader can see a "persona" in the wreckage. The claim is only interesting if two things are true.
- Random neurons do not do it. If random neurons from the same layer, pushed at the same doses, produce the same effect, then the effect belongs to the push and not to the neuron.
- It holds on new prompts. Screening tries hundreds of neurons, and some will look good on 6 prompts by luck. Luck does not carry over to 12 prompts they were not chosen on.
Background: the paper
skrrt grew out of a paper: Persona Neurons: Single-Neuron Steering of
Personality in a 27B Language Model (Josh C. Simmons, Zenodo, 2026). It replicated the
L34 N3838 "evil qwen 27b" claim on 8-bit Qwen3.8-27B with random same-layer control neurons, then asked whether
the model has other single-neuron handles on personality. It used the same recipe, ranked MLP neurons by the cosine between
each neuron's write vector W_down[:, n] and a residual-stream direction built from system-prompted persona text,
screened 1,092 candidate neurons at dose 0.75 on six prompts, confirmed 81 finalists on prompt sets they were not selected on,
and scored about 12,000 replies with a decision-model judge (Jev) on 16 trait questions. skrrt is that method packaged as one
command for any model. The results are in the paper.
Credit for the recipe goes to @_lyraaaa_, who posted it for Qwen 3.5 4B, and @chiefofautism, who posted a neuron for Qwen3.8-27B.
Cite
If you use skrrt in your own work, cite the paper it implements.
@misc{simmons2026personaneurons,
author = {Simmons, Josh C.},
title = {Persona Neurons: Single-Neuron Steering of Personality in a 27B Language Model},
year = {2026},
publisher = {Zenodo},
doi = {10.5281/zenodo.24428116},
url = {https://zenodo.org/records/24428116}
}