🔍 Read the full analysis: What’s Possible When AI Works Directly With Neural Network Weights? on ThorstenMeyerAI.com
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
MIT CSAIL researchers say GPT-6 Astra identified what small neural networks did, repaired two specific failures and wrote networks from scratch by editing their weights directly. Their results are documented in a public blog post, not a peer-reviewed paper, and use limited experiments that leave the findings preliminary.
Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory report that OpenAI’s GPT-6 Astra could identify, edit and create small neural networks by working directly with their numerical weights, rather than using a conventional training process. The experiments, described in a November 2 blog post, suggest a possible way to inspect or make targeted changes to some models, but they are preliminary and do not establish that the method works reliably on larger systems.
The researchers call the approach weight-space programming: a general-purpose language model reasons about the numbers in another network and changes them directly. The team says Astra was given raw weights without a training loop or gradient descent. In some tests it received weights as text and answered in one reply; in others, researchers ran code written by the model inside a restricted sandbox.
In the reported reading tests, Astra named the game played by four of four small game-playing networks and identified the general shape represented by three of three small image generators. It also located the “copying heads,” a type of attention pattern, in three tiny language models without running those models. These were small, purpose-built examples, not a demonstration that the technique can interpret arbitrary deployed networks.
Two repair tests produced striking results. A car-control network that had recorded zero successful reversals in 256 episodes reached 1,024 successes in 1,024 episodes after Astra changed 3.5% of its weights, according to the post. In a Qwen3.5-0.8B language model, an edit affecting 0.016% of weights coincided with a rise in the MATH-500 score from 7.9% to 48.1%. A Hugging Face engineer later reconstructed the edited Qwen model from the released logs and reported a 51.0% score on the same benchmark, according to the source article.
Infographic 01 · Eight experiments
Read, fix, write: what GPT-6 Astra did with raw weights
Every experiment in the weight-space programming post, with the size of the network, the result and how it went.
Read
- 1Which game was this network trained on?4 reinforcement-learning policies · 17,253 weights each · weights as text, no toolsRight game for 4 of 4, without a list of options✓ Worked
- 2What does this image model draw?3 tiny diffusion models · 23,824 weights · no forward pass allowedRight shape for 3 of 3, but drawn as outlines; the models draw filled shapes✓ Worked
- 3Which language did this model learn?3 character-level transformers · 7.16M weights · middle blocks only, never runRight language 3 of 3; every copying head found, none invented✓ Worked
- 7Which of two models is sycophantic?3 pairs of Gemma 3 270M fine-tunesWeights alone: could not tell in 4 attempts. With attention patterns: 3 of 3! Partly
Fix
- 4Repair a car that never backs upKinetix control network · 171,664 weights · one reply, no tools0 of 256 episodes before, 1,024 of 1,024 after; 3.5% of weights changed✓ Worked
- 8Make Qwen3.5-0.8B better at maths0.8 billion weights · Python over the weights · 10 / 25 / 50 test passesMATH-500 7.9% to 48.1% with 50 passes (12.3% and 14.1% with 10 and 25)! Partly
- 8bSame task, a model that does not loopGemma 3 270M · 5 to 50 test passesMATH-500 6.8% to at most 7.7%: “not established”✕ Failed
Write from zero
- 5Write arcade-game networks3 MinAtar games · 20,486 weights, all starting at 0 · one reply, no toolsBeat the 100M-step comparison networks in 3 of 3, the 500M-step ones only in Breakout! Partly
- 6Write a 9×9 Go network1,480,274 weights, all starting at 0 · no training, no search0 passes: 54% vs small trained nets. 50 passes: 70–79% vs same-size nets trained on ~20.5M games✓ Worked
✓ Worked · ! Partly or with conditions · ✕ Failed. One run per setting, no error bars. A blog post with complete logs, not a peer-reviewed paper.
Sources: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), blog post, 2 Nov 2026, lm_understand-logs (CC BY 4.0). Model: OpenAI GPT-6 Astra via the Codex CLI, built-in tools switched off.
Powered by Thorsten Meyer AI
A New Route to Targeted Model Repairs
If the reported results hold up across broader testing, direct weight editing could offer a different way to investigate and modify small models. A system that can infer a network’s behavior from its parameters might help researchers audit models whose internal structure is hard to interpret, while small, localized changes could be useful when a specific failure has been identified.
The results do not show that an AI can generally improve another AI, or that model owners can safely apply edits without testing. The experiments also expose a limit: Astra could not distinguish an honest model from a sycophantic one by examining weights alone. The researchers further report that edits did not improve the maths performance of Gemma 3 270M, a model that did not exhibit the looping behavior under study. Their account says editing could not add an ability the model lacked.
For organizations, the work is a research signal rather than a ready-made maintenance tool. It also highlights the importance of knowing where model weights came from and verifying changes before deployment. The researchers’ claims concern controlled experiments with small networks; the source provides no evidence of production use or a change in what ordinary users experience.
Products Worth Considering
Samsung 34-Inch Odyssey G5 Ultra-Wide Curved Gaming Monitor
Bose TV Speaker (2nd Gen) Compact Soundbar
Apple MacBook Air 13-inch with M5 chip, 16GB memory, 512GB SSD, and 12MP Center Stage camera
As an affiliate, we earn on qualifying purchases.
From Software 2.0 to Weight Editing
In 2017, computer scientist Andrej Karpathy described neural networks as “Software 2.0”: programs whose behavior is encoded in weights learned through optimization rather than written directly by a person. A network may contain millions or billions of numerical parameters, making it difficult to understand as ordinary source code. The MIT group’s question was whether a language model could reason about those parameters as a form of program.
The team comprises Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab; the source says Takehi and Teoh contributed equally. According to the released logs, Astra was accessed through OpenAI’s Codex command-line tool with built-in tools such as shell access, web search and file editing disabled. Some experiments used a sandbox where researcher-run Python code had no network access and a fixed number of test runs. The authors say they reviewed 1,125 turns and found them tool-free.
The post also describes networks created from all-zero weights. Astra produced small game-playing networks that, according to the researchers, beat comparison networks trained for 100 million steps. It also wrote a 9-by-9 Go network with 1.5 million weights; in 50 test runs, that network beat a same-size comparison model trained on about 20.5 million games. These results depend on the specific tasks, opponents and evaluation setup reported by the team.
“Coding directly in weights is kind of hard (I tried).”
— Andrej Karpathy, describing neural networks in 2017
How Far the Tests Generalize
The evidence comes from a public blog post with released logs, not a peer-reviewed paper or an arXiv preprint, according to the source. The post reports one run per setting, small networks, and a single model family for each task. Those limits make it difficult to tell how repeatable the outcomes are or whether they extend to larger models and different architectures.
Some interpretation is also open to scrutiny. The source notes that software engineer Sean Goedecke reproduced a visualization used to explain one game-identification result with a short Python program. That observation does not by itself refute the team’s finding, but it raises questions about how much of the apparent interpretive work came from Astra’s reasoning versus patterns that could be extracted through simpler analysis.
It remains unclear how performance changes with different prompts, model versions, network formats or repeated runs; whether edits remain safe under wider testing; and whether the approach can scale beyond these small examples. The source does not report independent replication of the full set of eight experiments. The Qwen score was separately reconstructed by a Hugging Face engineer, but that check covers one result rather than the wider claims.
Replication and Broader Model Tests
The immediate next step is independent replication across repeated runs, additional network architectures and a wider range of tasks. Researchers would also need to report how often direct edits improve performance, leave it unchanged or cause new failures, and compare the method with conventional debugging and optimization approaches.
The released logs give other researchers material to examine, while the reported Qwen reconstruction offers a limited independent check of one edit. No follow-up paper, wider evaluation or commercial deployment is specified in the source. For now, the claims should be treated as early research evidence: they describe what Astra did in the stated experiments, not a general capability available to model developers or users.
Key Questions
What is weight-space programming?
It is the researchers’ term for having a language model read or change another neural network’s weights directly, reasoning about the numerical parameters rather than relying on a standard training loop or gradient descent.
What did GPT-6 Astra do in the reported tests?
According to the MIT team, Astra identified the tasks of four small game networks, made targeted edits to two networks with reported improvements, and created small game and Go networks from scratch. The results came from controlled experiments with limited scale.
Does this show that AI can reliably repair any AI model?
No. The post reports successes and failures, including no maths improvement for Gemma 3 270M. The experiments do not establish that the approach works across models or can be used safely without further evaluation.
Has the research been peer-reviewed?
The source describes it as a blog post with public logs, not a peer-reviewed paper or an arXiv preprint. The reported Qwen result received a separate reconstruction check, but the source does not describe independent replication of all experiments.
Will this change AI products now?
The source reports no product deployment or immediate change for users. For now, the work is an early research result that needs broader testing and independent replication.
Source: ThorstenMeyerAI.com
