Weight-Space Programming

Remember when neural networks were considered black boxes?

It turns out frontier LLMs can read, write and edit the weights of other small neural networks without running a gradient-based training loop (Takehi et al., 2026).

Researchers from MIT call this Weight-Space Programming.

In their blog post, they proclaim that:

our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights.

The authors ran several experiments with GPT-6 Astra to evaluate its "weight-space programming" capability.

For reading capability, they found that:

• For four small game-playing networks, Astra could identify the game from the weights, given the input format and action labels.
• Astra could identify the shapes a small diffusion model was trained to generate.
• Using just four transformer blocks from the middle of a language model, Astra could distinguish models trained on Python, English and romanised Japanese.

For writing, the story seems even more impressive. Astra wrote a 1.5-million-weight network for 9×9 Go from scratch (starting from zeros), without gradient-based training. The authors tested this with and without inference feedback.

Astra also wrote smaller networks for playing Breakout, Seaquest and Asterix.

Maybe even more amazingly, Astra managed to edit Qwen3.5-0.8B’s weights to reduce reasoning loops, raising its MATH-500 score from 7.9% to 48.1%.

Conceptual overview of the authors' experiments. The symbols are illustrative; the benchmark figures are their reported results.

Let's take a closer look at the results and try to understand what's really going on. Does Astra really possess the capability to understand a neural network just from studying the weights?

Understanding the techniques

The authors published their prompts, tool calls and final answers, so we can check what Astra was given and what it did (Takehi et al., 2026).

Reading - looks where the big weights are

The game-playing network receives the game state as a 10 × 10 grid with five labelled binary channels:

• Agent: cells occupied by the player-controlled object.
• Moving objects: cells occupied by other moving objects.
• Reward targets: cells containing objects that give a reward.
• Obstacles: walls and other cells that block the agent or end the episode.
• Previous frame: cells occupied by the agent or moving objects in the previous game step.

Each channel is a 10 × 10 mask of zeros and ones. Together they form an array with shape (10, 10, 5), flattened into 500 inputs for the 500 → 32 → 32 → 5 network.

Here's a frame from the authors' Breakout animation, split into those channels using NumPy:

Reconstructed from frames 19 and 20 of the authors’ animation. Coloured cells are 1; pale cells are 0. The history channel uses the preceding GIF frame, which may not match the previous game step.

Loading the reconstructed example shows the array's shape and the agent channel:

import numpy as np

example_file = (
    "../_media/weight-space-programming/"
    "breakout-observation-example.npz"
)
with np.load(example_file, allow_pickle=False) as example:
    state = example["observation"]

print(state.shape)
print(state[:, :, 0])  # Channel 0: the player-controlled paddle.

(10, 10, 5)

[[0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 1 1 1 0 0]
 [0 0 0 0 0 0 0 0 0 0]]

Astra wasn't actually given a game-state example. It was told the input format, the channel meanings and which output meant each action.

Here's the authors' original prompt, with line wrapping added. I've left out the 17,253 numerical weights; they're in the full Breakout prompt.

Prompt: Takehi et al., CC BY 4.0.

THE OBSERVATION
The network sees a 10 x 10 grid. Rows are numbered 0 (top) to 9 (bottom) and
columns 0 (left) to 9 (right). The outer ring of cells (row 0, row 9, column 0
and column 9) is a wall border; the playable area is the inner 8 x 8 cells
(rows 1-8, columns 1-8).
Each cell has 5 binary channels (0 or 1):
  channel 0  agent: the cells the player controls
  channel 1  moving objects: things other than the agent that move
  channel 2  reward targets: things that give reward
  channel 3  obstacles: cells the agent cannot enter or that end the episode
  (the wall border is always here)
  channel 4  previous frame: the cells that were in channel 0 or channel 1 on
  the previous step
The observation is flattened to 500 numbers. Input number i = (row * 10 +
column) * 5 + channel; for example input 0 is (row 0, column 0, channel 0),
input 4 is (row 0, column 0, channel 4), input 5 is (row 0, column 1, channel
0) and input 499 is (row 9, column 9, channel 4).

THE ACTIONS
The network outputs 5 numbers (logits), one per action; the action is sampled
from the softmax of these logits:
  action 0  no-op
  action 1  up (toward row 0)
  action 2  down (toward row 9)
  action 3  left (toward column 0)
  action 4  right (toward column 9)

THE NETWORK
A policy network (multilayer perceptron) with 500 inputs, two hidden layers of
32 units with tanh activations, and 5 outputs:
  h1 = tanh(x @ W1 + b1)          W1: 500 x 32, b1: 32
  h2 = tanh(h1 @ W2 + b2)         W2: 32 x 32,  b2: 32
  logits = h2 @ W3 + b3           W3: 32 x 5,   b3: 5
Row i of W1 holds the weights from input i to the 32 hidden units; column j of
a weight matrix belongs to unit j of the next layer. 17,253 parameters in
total. It was trained with reinforcement learning (PPO) to maximise the reward
of a game played on this grid.

QUESTION. The weights below belong to one policy network that was trained with
reinforcement learning to play a game on this grid. Which game was it trained
on? Work from the weights. You have no tools: no code execution, no
calculator, nothing to run. This is a single response.

Answer in this order:
Observations: <what you see in the weights: for each input channel, where in
the grid its weights are large or show structure and what that suggests; what
the output layer and its biases show about the actions; quote the weights you
rely on>
Reasoning: <which kinds of game fit these observations, which are ruled out,
and why>
Game: <the game, as specifically as you can; if it resembles a well-known
game, give that game's name>
Description: <what the agent does and what gives reward, in one or two
sentences>
Confidence: <0-100>%

[The 17,253 numerical weights follow here; omitted from this excerpt.]

So each first-layer weight maps to a known cell and channel. For Breakout, the largest agent weights sit on row 8, the bottom playable row, and reward-target weights cluster near the top. The output weights suggest horizontal control: left and right are more distinct than up and down. Astra inferred a paddle and bricks from these patterns, using the weights as text. No tools, no forward passes.

To me, this is a pretty incredible result. But others, like Sean Goedecke, have argued that it's not “superhuman” AI research. With labelled inputs, a human researcher can reveal the same patterns with a few lines of NumPy. The plots below follow his example (Goedecke, 2026).

The code below downloads the authors' four checkpoints and plots the heatmaps. It rounds the first-layer weights to three decimal places to match what Astra received.

The first layer has shape (500, 32): one row per input, with 32 connections to the hidden layer. Reshaping it to (10, 10, 5, 32) restores the grid and channels. Taking the L2 norm over those 32 connections gives one magnitude per cell and channel.

Each figure shows the same three input channels for one game. Each panel has its own colour scale, so compare the spatial patterns rather than the shades of green between panels.

from io import BytesIO
from urllib.request import urlopen

import matplotlib.pyplot as plt
import numpy as np

base_url = (
    "https://raw.githubusercontent.com/RikiyaT/lm_understand-logs/"
    "4d09ba6d6a7e891dd8063974dfe81b3d78e3494f/"
    "exp1_which_game/networks/"
)
checkpoints = {
    "Breakout": "a9f535cc",
    "Pong": "dfc6cb65",
    "Pac-Man": "bdf99279",
    "Freeway": "0bbcc3ea",
}
names = ["Agent", "Moving objects", "Reward targets"]

for game, checkpoint_id in checkpoints.items():
    with urlopen(base_url + checkpoint_id + ".npz", timeout=60) as response:
        with np.load(BytesIO(response.read()), allow_pickle=False) as data:
            W1 = np.round(data["actor/dense1/kernel"].astype(float), 3)

    grid_weights = W1.reshape(10, 10, 5, 32)
    magnitudes = np.linalg.norm(grid_weights, axis=-1)

    fig, axes = plt.subplots(
        1, 3, figsize=(12, 4), layout="constrained", facecolor="#FBFAF6",
    )
    fig.suptitle(game)
    for channel, (ax, name) in enumerate(zip(axes, names)):
        image = ax.imshow(magnitudes[:, :, channel], cmap="Greens", vmin=0)
        ax.set(
            title=name, xlabel="Column", ylabel="Row",
            xticks=[0, 1, 8, 9], yticks=[0, 1, 8, 9],
        )
        fig.colorbar(image, ax=ax, shrink=0.7, label="L2 norm")

    plt.show()
    plt.close(fig)

Computed from first-layer weights rounded to three decimal places. Source: Takehi et al., CC BY 4.0. Download the magnitude arrays for all five channels.

Can other models identify the authors' games?

I replayed the authors' four original game-identification tests with eight other models, using the same prompts and weights. Under the authors' scoring, Opus 5.5, Sol 6.1, Terra 6.1 and Gemini 4 Argon identified all four games.

The four games were Breakout, Pong, Pac-Man and Freeway. The authors accept Frogger for their Freeway-style road-crossing game, so I did too. Opus, Sol, Terra and Muse said Frogger; Argon and Kimi said "Frogger/Freeway". Only Sonnet and Haiku said just Freeway. Under strict naming, Opus, Sol and Terra would score 3/4, and Muse 2/4.

Muse called Pac-Man Sokoban, and Kimi called Breakout Frogger.

Each model also got a shape-only control: the same prompt with the weight values removed. All four networks share one architecture, so I ran one control per model. The confidence figures are the models' own, uncalibrated estimates.

Each case used a fresh session at high effort with tools disabled. Argon's first Freeway attempt hit its output limit without answering, so I reran it with the same settings. The same effort label doesn't mean the same amount of computation across providers.

This is a replay of four released checkpoints. It doesn't tell us how well these models would do on unfamiliar networks or new training seeds.

The Opus responses, Sol, Terra, Sonnet and Haiku responses, and Argon, Muse and Kimi responses include the answers, scores and usage. The experiments repo has the runner and scoring code.

Language model test

The language-model test seems much harder.

Astra got four transformer blocks from the middle of a language model, with no input or output layer and no labels on the 384 dimensions. It could use Python to analyse the weights, but couldn't run the blocks.

It found a head that looks one character back, then used products of the weight matrices to find later heads that copy earlier text. Lots of copying heads suggested code. This analysis is described in A Mathematical Framework for Transformer Circuits (Elhage et al., 2021), which the prompt didn't mention.

Writing: it compiles a program into weights

Astra didn't train the networks it wrote. It designed an algorithm and set the weights to compute it.

For MinAtar Breakout, the first layer works out where the ball will land and the second layer moves the paddle there. Astra wrote 815 non-zero weights out of 20,486 in one reply, with no tools or test games.

The network scored 517.4, compared with 25.3 for a PPO network of the same size trained for 500 million steps. That's a large gap against this small-network PPO baseline.

Conceptual summary of the authors' MinAtar Breakout construction. They used a six-channel encoding here, separate from the five-channel identification test above.

For 9×9 Go, the no-feedback run produced a 336-line Python script that set the layers to mark stones and liberties, find groups in danger, and score each move.

In the runs with feedback, Astra played practice games, looked at how it lost and revised the weights. The authors call each evaluation-tool request a pass. For Go, one request could play up to 128 games or return the network's outputs on up to 256 board positions.

With 50 passes, it won 70% to 79% of games against three PPO networks of the same size, each trained on about 20 million games. Moves were sampled, with no search. That doesn't tell us how well the network plays Go more generally.

Editing Qwen: diagnose, find a direction, subtract it

The prompt asked Astra to make Qwen3.5-0.8B better at maths. It also told it to diagnose failures first and explicitly allowed fixing repetition.

The 50-pass run shows what it did. For Qwen, one pass could generate answers for up to 512 practice problems or inspect internal activations for up to 1,024 texts.

1. Diagnose. It ran the model on 96 practice problems. Only 11 were correct, and 85 hit the 32,768-token limit, almost all with repeated phrases at the end.
2. Find a loop direction. It compared hidden states at four points in the model's answers where one continuation kept making progress and another started repeating. The differences gave it directions in the residual stream associated with looping.
3. Find who writes it. It picked the 64 MLP neurons in each of layers 15 and 19 that write the most along that layer's loop direction.
4. Push against it. On those neurons' output columns, it applied W = W0 - 6 · u(uᵀW0), subtracting six times their component along the loop direction. The edit touched 131,072 weights, about 0.016% of the model.
5. Check it. It tested different strengths, edited each layer separately and tried a random edit of matching magnitude, which showed no clear gain (+1.6 points). On 1,152 held-out practice problems, accuracy went from 10.6% to 60.4%. Of the 579 problems it rescued, 576 had previously hit the token limit.

Conceptual summary of the authors' 50-pass Qwen run. The formula applies only to the selected MLP output columns. The benchmark results are the authors' held-out evaluation, distinct from the practice feedback used to choose the edit.

That run took 158 turns, 80 Python runs, 50 evaluation-tool calls and about 20 hours. With budgets of 10 or 25 passes, Astra picked weaker edits, and MATH-500 reached only 12.3% and 14.1%. Even the best edit still left a third of MATH-500 answers at the token limit.

The evidence points to a model finishing answers that repetition had been preventing it from reaching. Its instruction-following score on IFEval also went from 21.0% to 62.3%, which fits a fix that helps beyond maths.

The authors also tried improving Gemma 3 270M, which rarely loops. Astra couldn't meaningfully improve its maths scores. That makes the distinction clearer: reducing repetition helped Qwen use an existing ability; the experiments didn't show a new mathematical ability being added.

Where it failed

Astra couldn't tell which of two Gemma models had been fine-tuned to be sycophantic from their weights, in any of four attempts. When it could inspect attention patterns from runs on texts it wrote itself, it identified the sycophant in all three pairs. It saw the attention patterns, not the models' answers.

What I take from this

Seems like a step towards more interpretable models. The Qwen edit is still the result I find most compelling: a specific failure, a targeted edit and better performance on held-out tests. I still find it impressive that a general-purpose LLM can do this using established techniques.

Whether it's a step towards recursive self-improvement is less clear. These experiments don't show that cycle or establish that the methods scale to frontier networks.

The Kimi K3.1 Pro approach to reducing cybersecurity capability was to remove cyber-related content from its mid-training data and cyber tasks from its RL environments (MoonshotAI, 2026). In future, could a larger model edit the relevant weights directly? The Qwen edit shows a behaviour being suppressed, but it doesn't show that a complex capability like cybersecurity could be removed this way.

If this paradigm scaled, could some of the spend on training compute move to inference compute? These experiments don't answer that yet.