Promythicus FR root@promythicus.com

Promythicus / Blog

We repaired an AI model by editing its numbers directly. The cleverer ideas failed.

By Bardia Shahrestani, Promythicus Inc. · Published

A small open language model (Qwen3.5-0.8B) often thinks in circles and never answers. We found, from its own behaviour, the direction inside it that goes with being stuck, and weakened the 64 neurons that push hardest that way. No training. On maths problems it had not seen, right answers went from 7.5% to 24.5%. Three cleverer ideas built on this (more layers, a gate that acts only when the model is right, and another published recipe) did not help. So far, editing weights removes a fault; it has not added an ability.

Summary: a direct weight edit takes a small model from 7.5% to 24.5% right; three more ambitious edits fail

The question

The usual way to improve a model is fine-tuning: more training on new examples. It needs data, hours of computing, and it can damage what the model already did well. A group of researchers showed in November 2026 that a model can sometimes be improved by changing a few of its numbers by hand, after looking at how it behaves. We wanted to know how far that goes, because a firm’s server should be improvable without retraining anything.

The test

Reproducing the published repair, blind

The published repair changes two of the model’s 488 tables of numbers: 64 neurons in each of two layers. We did not copy it. We ran the untouched model, separated the answers where it got stuck from those where it finished, and looked for the layers that push most toward “stuck”. We landed on the same two layers and nearly the same direction.

Model (100 unseen problems)Right
Untouched9%
Random edit, same neurons, same strength14%
Published edit25%
Ours, same strength26%
Ours, twice the strength39%
Ours, four times the strength18%

Twice the published strength was best. At four times, the model stops looping but answers wrong: there is a cliff, and a small practice set hides it.

Three cleverer ideas

These were scored on 200 other unseen problems with a shorter limit on thinking, so the numbers are lower across the board. The untouched model scores 7.5% here.

EditRight
Untouched7.5%
Two layers19.0%
One layer, stronger (what a search over layers settled on)24.5%
Plain push at an earlier layer29.5%
The same push, gated by a random signal21.0%
The same push, only when the model’s answer is right (the “truth gate”)5.0%
The mirror image: push only when it is wrong9.5%
Another published recipe (AHA), as we understood it7.0%

What we take from it

The repair is real, large, and needs no training. But it removes an obstacle the model already had; nothing here taught it something new. And the measurements have to be careful: a model that has simply been broken also “stops looping”. A recent paper on reading weights makes the same point, and we have adopted its rules: an edit that breaks the model is not a result, noise is measured rather than guessed, and every batch of edits includes some that should do nothing.

Limits

Questions and answers

Can an AI model be improved without fine-tuning?
Sometimes. In our test, changing 64 neurons in one layer of a small model, with no training, took it from 7.5% to 24.5% right on unseen maths problems. It worked because the model had a specific fault, getting stuck in its own thinking.
What is a weight edit?
A model is a large set of numbers. A weight edit changes a chosen few of them directly, after watching how the model behaves, instead of training the model on new examples.
Does it work for anything?
No. In our tests it removed a fault. The more ambitious edits, including one meant to make the model commit only when it is right, did nothing or made it worse.
Is this used on client servers?
Not yet. The model we install does not have this fault. We are testing whether the same approach can fix the faults it does have.

Sources