TL;DR
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
A November 2 blog post by four MIT CSAIL-listed researchers reports that GPT-6 Astra edited two layers of Qwen3.5-0.8B, raising its MATH-500 score from 7.9% to 48.1%. The post describes an early, non-peer-reviewed result: 33.1% of the edited model’s answers still hit the token limit, and the edit required extensive iterative testing.
Four researchers listed with MIT CSAIL report that OpenAI’s GPT-6 Astra improved the math benchmark score of the small open-weight model Qwen3.5-0.8B by editing two of its layers, without using gradient descent. In their November 2 blog post, the researchers say the MATH-500 score rose from 7.9% to 48.1%, while the share of answers that ran into a 32,768-token limit fell from 93.0% to 33.1%.
The authors traced many of Qwen’s failed answers to repetitive reasoning that continued until the model reached its token limit. They say Astra identified an internal direction associated with the loops and subtracted it from 64 output columns in each of layers 15 and 19. The change affected 131,072 weights, or 0.016% of the model’s 0.8 billion parameters. The researchers say the edited model’s published weights differ from the original only in those two layers’ down-projection tensors.
The reported score came from the highest of three search budgets. The winning run involved 158 turns, 80 Python runs and 50 forward passes, over 20.3 hours, according to the published logs. The researchers say Astra could use practice questions but was not shown the benchmark tests used for the final scores. Gradient descent, backward passes and fitting weights to a loss were prohibited. With budgets of 10 and 25 forward passes, the reported MATH-500 scores were 12.3% and 14.1%.
The authors report that the edit reduced but did not eliminate looping: 33.1% of edited-model answers still hit the token limit. They also caution that fixing loops is not the same as making a model broadly better at mathematics. In another experiment on Gemma 3 270M, they report that no edit changed the score by more than about one percentage point. The post covers eight experiments and describes the work as very early.
GPT-6 Astra repairs another model with a targeted weight edit
Researchers report a large MATH-500 score gain for Qwen3.5-0.8B after Astra edited two layers. The early, non-peer-reviewed result still leaves many answers looping.
What Astra changed
The edit targeted repetitive reasoning that often continued until Qwen reached its token limit.
Astra identified an internal direction associated with repetitive outputs.
It subtracted that direction from 64 output columns in layers 15 and 19.
The search used model outputs and iterative testing, without gradient descent.
A meaningful gain, with limits
The result measures performance on one benchmark and does not establish broader math ability.
- Higher reported MATH-500 score
- Fewer answers reached the token limit
- 33.1% of answers still hit the limit
- Effects on other tasks are unclear
The search took substantial effort
The best reported result came from the largest of three search budgets.
| Search budget | MATH-500 score |
|---|---|
| 10 forward passes | 12.3% |
| 25 forward passes | 14.1% |
| 50 forward passes | 48.1% |
A Repair With Limits
The result suggests that, in some cases, a capable language model may be able to inspect and modify the weights of another model using code and test feedback. That approach could offer developers a way to target a specific failure in an open-weight model without retraining all its parameters. Here, the reported improvement addressed a known behavior: Qwen’s model card warns that the model can fall into thinking loops.
The benchmark gain is substantial on the reported measure, but it does not establish a general rise in mathematical ability. The initial model often spent its output budget repeating itself; reducing that failure can improve the number of problems it answers successfully. The remaining loop rate and the authors’ other results show why the experiment should be read as a targeted repair, not evidence that one model can reliably upgrade another across tasks.
For users and organizations, a weight edit also creates a distinct model version that needs its own evaluation. Results on MATH-500 alone do not establish how the change affects other tasks, reliability or safety. The method’s practical value will depend on whether similar edits work on other models and failure modes, and whether they hold up under broader testing.
Products Worth Considering
Samsung 27-Inch Odyssey OLED G5 QHD Gaming Monitor
Bose QuietComfort Wireless Noise-Cancelling Over-Ear Headphones, Black
LG UltraGear 32GX850A-B 32-inch 4K UHD Glossy OLED Gaming Monitor
As an affiliate, we earn on qualifying purchases.
How the Weight Edit Worked
The work appears in a blog post titled Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights, published November 2 by Rikiya Takehi. Takehi, Jayden Teoh, Alex Zhang and Omar Khattab are listed as authors, with Takehi and Teoh marked as equal contributors. The post is not peer-reviewed and was not on arXiv, according to the supplied source material. The authors call the findings very early and say the capabilities remain limited.
According to the logs, Astra ran through the Codex command-line tool at its highest reasoning setting. The researchers’ own setup gave it a Python environment to inspect and edit the model weights, with a limit of 300 code runs. It ran on CPU without internet access. Astra could request forward passes to see model outputs, with each pass covering up to 512 problems or 1,024 texts. This made the work an iterative search guided by test feedback, even though it did not use gradient descent.
The researchers connect their approach to prior work on changing model behavior by removing a direction in weight space. A 2024 paper by Arditi and colleagues described a related technique for removing refusals from open chat models. That comparison provides a precedent for targeted edits; it does not show that the Qwen math result has been independently replicated or that the same procedure will work broadly.
“Fixing loops, however, is not the same as making a model better at math.”
— The researchers, in their blog post
Questions Beyond MATH-500
The post does not establish whether independent researchers can reproduce the score change or whether the edited model performs better across a broader set of math tasks. The supplied source material does not describe peer review, independent replication or an OpenAI response; OpenAI is not listed as an author. It is also unclear how the edit affects the model’s performance and behavior outside the reported benchmark.
Even on MATH-500, the loop issue remained in about one-third of answers. The results also depended on a large search budget: smaller runs produced much lower scores. More evidence is needed to determine how often this kind of weight edit succeeds, how much expert setup it requires and what safeguards are needed when a modified model is used.
Replication and Broader Tests
The immediate next step is independent evaluation of the published edited weights, including reproduction of the MATH-500 result and tests on other tasks. Broader comparisons could help separate gains from reducing token-limit failures from gains that transfer to mathematical reasoning more generally. Testing other models and failure modes would also show whether the method extends beyond this Qwen experiment.
Until those results are available, the November 2 post remains an early report of a targeted edit. The authors’ logs and released weights provide material for others to examine, but the supplied source gives no schedule for peer review, further experiments or an OpenAI statement.
Key Questions
What did GPT-6 Astra change?
According to the researchers, Astra subtracted a direction linked to repetitive outputs from 64 output columns in each of Qwen3.5-0.8B’s layers 15 and 19. The edit changed 131,072 weights, which they report as 0.016% of the model.
How much did the reported math score improve?
The researchers report that the model’s MATH-500 score rose from 7.9% to 48.1%. Their post is an early report, not a peer-reviewed paper.
Did the edit stop the model from looping?
No. The reported share of answers hitting the 32,768-token limit fell from 93.0% to 33.1%, so looping remained a problem in a substantial portion of answers.
Does ‘without training’ mean Astra needed no testing?
No. The researchers say the process used no gradient descent or backward passes, but Astra iteratively inspected outputs and edited weights. The winning run took 158 turns and 20.3 hours, with 50 forward passes and 80 Python runs, according to the logs.
Has the result been independently verified?
The supplied source describes a blog post published November 2, 2026, and says it was not peer-reviewed. It does not report independent replication.
Source: Thorsten Meyer AI
