{"id":1146,"date":"2026-11-05T12:34:03","date_gmt":"2026-11-05T12:34:03","guid":{"rendered":"https:\/\/perit.ai\/blogs\/?p=1146"},"modified":"2026-11-05T12:34:03","modified_gmt":"2026-11-05T12:34:03","slug":"weight-space-programming-llm-edit-neural-network-weights","status":"publish","type":"post","link":"https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/","title":{"rendered":"Weight-Space Programming: The Model Knew the Answer. It Couldn&#8217;t Stop."},"content":{"rendered":"\n<div class=\"wsp-post\" style=\"max-width: 50rem;margin: 0 auto;font-family: Verdana, Geneva, Tahoma, sans-serif;font-size: 1rem;line-height: 1.72;color: #0b0b0b\">\n<div class=\"meta\" style=\"font-size: .92rem;color: #52514e;margin-bottom: 1.6rem;padding-bottom: .65rem;border-bottom: 1px solid #e6e5e0\">A small language model finds the right answer to a maths problem, then repeats one line until it runs out of room. Another model edits 0.016% of its weights and the loop stops. What weight-space programming is, what the first experiments showed, and what the famous 7.9% to 48.1% jump actually measured.<\/div>\n<div class=\"takeaways\" style=\"margin: 0 0 1.8rem;padding: .95rem 1.1rem .4rem;border: 1px solid #e6e5e0;border-left: 4px solid #2a78d6;border-radius: 0 10px 10px 0;background-color: #fcfcfb\"><p class=\"takeaways-head\" style=\"margin: 0 0 .45rem;color: #2a78d6;font-size: .78rem;font-weight: 700;letter-spacing: .08em;text-transform: uppercase\">In short<\/p><ul style=\"padding: 0;margin: 0 0 .6rem 1.2rem\"> <li style=\"margin: 0 0 .45rem;padding-left: .15rem;font-size: .95rem;line-height: 1.55\"><strong>Weight-space programming<\/strong> is a term coined on 2 November 2026 by four MIT CSAIL researchers. A general-purpose LLM reads, fixes or writes another neural network&#8217;s weights directly, with no gradient descent and no training loop.<\/li> <li style=\"margin: 0 0 .45rem;padding-left: .15rem;font-size: .95rem;line-height: 1.55\">On small networks, GPT-6 Astra named the game a policy was trained on from 17,253 raw numbers, and wrote a Breakout network from scratch that scored 517.4 against 25.3 for PPO trained for 500 million steps.<\/li> <li style=\"margin: 0 0 .45rem;padding-left: .15rem;font-size: .95rem;line-height: 1.55\">The headline edit changed 131,072 of Qwen3.5-0.8B&#8217;s weights and lifted MATH-500 from 7.9% to 48.1%. Independent analysis shows it removes a repetition loop. With thinking switched off, the unedited model already scores 43.6%.<\/li> <li style=\"margin: 0 0 .45rem;padding-left: .15rem;font-size: .95rem;line-height: 1.55\">The published edit reproduces at 50.6% and 51.0%. One fresh run of the editing process, with the same budget, reached 12.9%. An edit is only as good as the test that judges it.<\/li> <\/ul><\/div>\n<p class=\"lede\" style=\"margin: 0 0 1rem;font-size: 1.08rem\">The problem is a gentle one. Convert the point (0, 3) from rectangular to polar coordinates. A small language model, Qwen3.5-0.8B, starts thinking out loud.<\/p>\n<p style=\"margin: 0 0 1rem\">By token 1,423 it has the angle: \u03b8 = \u03c0\/2. By token 2,553 it has written the full answer. A student would put the pen down here. The model keeps checking.<\/p>\n<p style=\"margin: 0 0 1rem\">At token 7,200 it writes a sentence. Then it writes it again.<\/p>\n<div class=\"trace\" style=\"margin: 1.2rem 0 1.3rem;padding: .85rem 1rem;border: 1px solid #e6e5e0;border-radius: 10px;background-color: #fcfcfb;font-family: Menlo, Consolas, &apos;Courier New&apos;, monospace;font-size: .86rem;line-height: 1.6;color: #52514e;overflow: auto\">Wait, tan(3\u03c0\/2) = tan(\u03c0 + \u03c0\/2) = tan(\u03c0\/2) = \u221e.<br \/>Wait, tan(3\u03c0\/2) = tan(\u03c0 + \u03c0\/2) = tan(\u03c0\/2) = \u221e.<br \/>Wait, tan(3\u03c0\/2) = tan(\u03c0 + \u03c0\/2) = tan(\u03c0\/2) = \u221e.<br \/>\u2026 until token 32,768<\/div>\n<p style=\"margin: 0 0 1rem\">It keeps writing that line, with small variations, for more than 25,000 tokens, until it hits its limit and stops mid-thought. There is no final answer in the box. The grader marks it wrong. <a href=\"#ref-2\" style=\"color: #2a78d6;text-decoration: underline\">[2]<\/a><\/p>\n<p style=\"margin: 0 0 1rem\">Now give the same problem to the same model after a different AI has changed 131,072 of its 0.8 billion weights. Same prompt, same sampling settings, no new data, no training. It answers correctly in 10,097 tokens.<\/p>\n<p style=\"margin: 0 0 1rem\">What changed? The obvious answer is the one that went round the internet on 2 November. An AI made another AI better at maths by rewriting its brain, and MATH-500 went from 7.9% to 48.1%. That should be the whole story. It is not. The gap between that headline and what happened inside the network is the most useful thing to understand about weight-space programming.<\/p>\n<div class=\"thesis\" style=\"margin: 1.6rem 0;padding: 1rem 1.15rem;border-left: 4px solid #2a78d6;background-color: #fcfcfb;font-family: Georgia, &apos;Times New Roman&apos;, serif;font-size: 1.18rem;line-height: 1.55\">The model already knew the answer. It could not stop. So far, weight-space programming is very good at finding the one thing that blocks a network, and poor at adding something the network never had. The weights do not tell you which one you got. Only the test does.<\/div>\n<h2 id=\"what-is-weight-space-programming\" style=\"font-size: 1.28rem;line-height: 1.35;margin: 2.5rem 0 .7rem;font-weight: 700;color: #0b0b0b\">What is weight-space programming?<\/h2>\n<p style=\"margin: 0 0 1rem\">Weight-space programming means asking a general-purpose large language model to read, fix or write the weights of another neural network directly, from a plain-language goal. It uses no gradient descent and no training loop. The term comes from a post published on 2 November 2026 by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab at MIT CSAIL, \u201cSparks of Weight-Space Programming\u201d. It reports eight experiments run with OpenAI&#8217;s GPT-6 Astra. <a href=\"#ref-1\" style=\"color: #2a78d6;text-decoration: underline\">[1]<\/a><\/p>\n<p style=\"margin: 0 0 1rem\">Why does that sound so strange? Until now, neural network weights have come from three places. Almost all come from an optimizer: gradient descent, through Adam or Muon, nudging millions of numbers a little at a time until a loss goes down. A few come from networks trained to produce other networks&#8217; weights. A handful come from compilers such as Tracr, which turn a small formal program into transformer weights. <a href=\"#ref-3\" style=\"color: #2a78d6;text-decoration: underline\">[3]<\/a> None of these understands what an individual weight is for.<\/p>\n<p style=\"margin: 0 0 1rem\">Weight-space programming adds a fourth route. A model reasons about what each part of a network computes, or should compute, and sets the numbers to match. The authors&#8217; argument is simple. We already let LLMs optimise prompts, programs and training code, because people can edit those by hand and people cannot edit weights. Their claim is that this excuse is running out.<\/p>\n<h2 id=\"a-weight-is-a-very-small-line-of-code\" style=\"font-size: 1.28rem;line-height: 1.35;margin: 2.5rem 0 .7rem;font-weight: 700;color: #0b0b0b\">A weight is a very small line of code<\/h2>\n<p style=\"margin: 0 0 1rem\">The easiest way to see why this is not magic is to shrink the network to one neuron.<\/p>\n<p style=\"margin: 0 0 1rem\">Picture Breakout on a grid ten columns wide. The paddle moves along the bottom row and the ball is somewhere above it. You want a neuron that says \u201cmove right\u201d when the ball is to the right of the paddle. Give it one input for each column the ball could be in, and one for each column the paddle could be in. Set the weight on ball column <em>c<\/em> to <em>c<\/em>, and the weight on paddle column <em>c<\/em> to minus <em>c<\/em>. The neuron now computes:<\/p>\n<div class=\"eq\" style=\"margin: 1.2rem 0 1.3rem;padding: .85rem 1rem;border: 1px solid #e6e5e0;border-radius: 10px;background-color: #fcfcfb;font-family: Georgia, &apos;Times New Roman&apos;, serif;font-size: 1.08rem;text-align: center;overflow: auto\">move-right score = (ball column) \u2212 (paddle column)<\/div>\n<p style=\"margin: 0 0 1rem\">Positive means go right, negative means go left, zero means stay. That is twenty weights, and each one has a meaning you can say out loud. It is a program. It is written in multiplications and sums instead of if-statements, but it is still a program.<\/p>\n<p style=\"margin: 0 0 1rem\">A trained network is this idea scaled up and scrambled. Gradient descent does not care whether a weight has a meaning you can say, so the meanings smear across thousands of numbers. Reading a trained network means unsmearing them. Writing one by hand means doing the opposite: decide the meaning first, then choose the numbers. Weight-space programming is the bet that a frontier model can now do both.<\/p>\n<h2 id=\"reading-a-network-means-finding-where-it-looks\" style=\"font-size: 1.28rem;line-height: 1.35;margin: 2.5rem 0 .7rem;font-weight: 700;color: #0b0b0b\">Reading a network means finding where it looks<\/h2>\n<p style=\"margin: 0 0 1rem\">In the first experiment, the researchers trained four small policies with PPO, a standard reinforcement learning method. Each played the team&#8217;s own 10-by-10 grid version of Breakout, Pong, Pac-Man or Freeway. Every cell has five input channels, covering the agent, moving objects, rewards, obstacles and the previous frame. That gives 500 inputs, two hidden layers of 32 units and five actions: 17,253 weights. All four networks have exactly the same shape, so only the values can give the game away.<\/p>\n<p style=\"margin: 0 0 1rem\">The researchers pasted each network&#8217;s numbers into a prompt. They told Astra it had no tools, no code and no calculator, and asked which game the network was trained on, without listing any options. It named all four correctly. <a href=\"#ref-1\" style=\"color: #2a78d6;text-decoration: underline\">[1]<\/a><\/p>\n<p style=\"margin: 0 0 1rem\">How? The prompt says which input belongs to which cell and channel. Add up how strongly each cell connects to the first hidden layer and you get a heat map of where the network pays attention. In the Breakout network, the agent channel lights up along the bottom row, where the paddle lives. In Pac-Man, the obstacle channel traces a maze. Sean Goedecke, a software engineer, rebuilt those maps in three lines of numpy. He argued that anyone shown them could name the games. <a href=\"#ref-4\" style=\"color: #2a78d6;text-decoration: underline\">[4]<\/a><\/p>\n<p style=\"margin: 0 0 1rem\">He is right, and it does not make the result smaller. Astra did that arithmetic without running any code. Simon Willison then gave the same prompts to Qwen 3.8 27B, a 17 GB model on his laptop. After 52 minutes and 25 seconds of reasoning it answered \u201cBreakout\u201d, 88% confident. It also got Pac-Man and Pong right, and missed Freeway. <a href=\"#ref-5\" style=\"color: #2a78d6;text-decoration: underline\">[5]<\/a> Reading a small network is no longer a frontier-only trick. It mostly takes patience.<\/p>\n<p style=\"margin: 0 0 1rem\">The other reading experiments go further. Shown only the weights of three tiny diffusion models, Astra said which one drew a circle, a triangle or a square. It drew each shape as an outline, although the models produce filled shapes. That mistake is a clue: it was reading the geometry of the output layer, not simulating the network. Next it was given only the four middle blocks of three 7.2-million-parameter character-level models, with the input and output layers removed. From those it named each training language (Python, English or Japanese in the Latin alphabet) in both attempts. It also found every copying head, the attention heads that let a model repeat earlier text, and named no other head as one. It did this by multiplying weight matrices together, a method from Anthropic&#8217;s 2021 transformer-circuits work that nobody had mentioned to it. <a href=\"#ref-1\" style=\"color: #2a78d6;text-decoration: underline\">[1]<\/a> <a href=\"#ref-6\" style=\"color: #2a78d6;text-decoration: underline\">[6]<\/a><\/p>\n<h2 id=\"writing-a-network-means-choosing-the-meaning-first\" style=\"font-size: 1.28rem;line-height: 1.35;margin: 2.5rem 0 .7rem;font-weight: 700;color: #0b0b0b\">Writing a network means choosing the meaning first<\/h2>\n<p style=\"margin: 0 0 1rem\">Writing is the stranger direction. For three MinAtar games, small versions of Atari classics, Astra got the rules, the game&#8217;s source code and an empty network of the right shape. It had no data, no loss, no gradients, no tools and no chance to try its network in the game. It replied with the weights as text.<\/p>\n<div class=\"table-wrap\" style=\"overflow: auto;margin: 1.8rem 0\"><table style=\"width: 100%;border-collapse: collapse;font-size: .92rem;line-height: 1.5\"> <thead><tr><th style=\"text-align: left;padding: .65rem .6rem;border-bottom: 2px solid #0b0b0b\">Game<\/th><th class=\"num\" style=\"padding: .65rem .6rem;border-bottom: 2px solid #0b0b0b;text-align: right\">PPO, 100M steps<\/th><th class=\"num\" style=\"padding: .65rem .6rem;border-bottom: 2px solid #0b0b0b;text-align: right\">PPO, 500M steps<\/th><th class=\"num\" style=\"padding: .65rem .6rem;border-bottom: 2px solid #0b0b0b;text-align: right\">Astra, one shot, no feedback<\/th><\/tr><\/thead> <tbody> <tr><td style=\"padding: .65rem .6rem;border-bottom: 1px solid #e6e5e0;vertical-align: top\">Breakout<\/td><td class=\"num\" style=\"padding: .65rem .6rem;border-bottom: 1px solid #e6e5e0;vertical-align: top;text-align: right;white-space: nowrap\">15.8<\/td><td class=\"num\" style=\"padding: .65rem .6rem;border-bottom: 1px solid #e6e5e0;vertical-align: top;text-align: right;white-space: nowrap\">25.3<\/td><td class=\"num\" style=\"padding: .65rem .6rem;border-bottom: 1px solid #e6e5e0;vertical-align: top;text-align: right;white-space: nowrap\"><strong>517.4<\/strong><\/td><\/tr> <tr><td style=\"padding: .65rem .6rem;border-bottom: 1px solid #e6e5e0;vertical-align: top\">Seaquest<\/td><td class=\"num\" style=\"padding: .65rem .6rem;border-bottom: 1px solid #e6e5e0;vertical-align: top;text-align: right;white-space: nowrap\">5.6<\/td><td class=\"num\" style=\"padding: .65rem .6rem;border-bottom: 1px solid #e6e5e0;vertical-align: top;text-align: right;white-space: nowrap\"><strong>88.9<\/strong><\/td><td class=\"num\" style=\"padding: .65rem .6rem;border-bottom: 1px solid #e6e5e0;vertical-align: top;text-align: right;white-space: nowrap\">53.1<\/td><\/tr> <tr><td style=\"padding: .65rem .6rem;border-bottom: 1px solid #e6e5e0;vertical-align: top\">Asterix<\/td><td class=\"num\" style=\"padding: .65rem .6rem;border-bottom: 1px solid #e6e5e0;vertical-align: top;text-align: right;white-space: nowrap\">25.8<\/td><td class=\"num\" style=\"padding: .65rem .6rem;border-bottom: 1px solid #e6e5e0;vertical-align: top;text-align: right;white-space: nowrap\"><strong>55.2<\/strong><\/td><td class=\"num\" style=\"padding: .65rem .6rem;border-bottom: 1px solid #e6e5e0;vertical-align: top;text-align: right;white-space: nowrap\">53.2<\/td><\/tr> <\/tbody><\/table> <p class=\"table-note\" style=\"margin: 0 0 1rem;font-size: .88rem;color: #52514e;margin-top: .5rem\">Game scores reported by Takehi et al. (2026), Experiment 5. Higher is better.<\/p><\/div>\n<p style=\"margin: 0 0 1rem\">The Breakout network is the toy neuron above, grown up. Astra wrote a first layer that works out where the ball will land and a second layer that moves the paddle there. That is a better program than PPO had found after 500 million steps, which is why a network written in one go scored twenty times higher. On Seaquest and Asterix, where the right strategy is harder to say in a sentence, it beat the 100-million-step policy and lost to the 500-million-step one. <a href=\"#ref-1\" style=\"color: #2a78d6;text-decoration: underline\">[1]<\/a><\/p>\n<p style=\"margin: 0 0 1rem\">Repairs work the same way. In a physics sandbox called Kinetix, a car has to back up to get a run at a ramp. PPO&#8217;s policy had settled on driving straight at the ramp and getting stuck. Astra got the weights, a description of the scene and one recording of the failure. It added a small side path through a few units per layer that reverses the car until it is past a red strip, then switches off and lets the trained network drive. In one try with no tools, changing 3.5% of the weights, the car succeeded in all 1,024 episodes. <a href=\"#ref-1\" style=\"color: #2a78d6;text-decoration: underline\">[1]<\/a> One caution: OpenWSP, an independent replication project, reports that the same one-shot repair failed when it reran the experiment with Astra. <a href=\"#ref-7\" style=\"color: #2a78d6;text-decoration: underline\">[7]<\/a><\/p>\n<h2 id=\"the-rules-changed-as-the-networks-grew\" style=\"font-size: 1.28rem;line-height: 1.35;margin: 2.5rem 0 .7rem;font-weight: 700;color: #0b0b0b\">The rules changed as the networks grew<\/h2>\n<p style=\"margin: 0 0 1rem\">From here the experiments change shape, and the headline does not make this obvious. A 9-by-9 Go network of about 1.5 million weights is too big to paste into a prompt. So Astra got a Python sandbox, and later a budget of forward passes: chances to run the network and see what it does. It wrote the network as a Go program in 336 lines. Its early layers mark stones and their empty neighbouring points, the middle layers trace connected groups in danger, and the last layer scores each move. With 50 batches of practice games, it beat an equally sized PPO network trained on 20 million games. <a href=\"#ref-1\" style=\"color: #2a78d6;text-decoration: underline\">[1]<\/a><\/p>\n<p style=\"margin: 0 0 1rem\">Matthew Honnibal, the creator of spaCy, sorted every experiment by what Astra was allowed to do. He called the post&#8217;s framing misleading. <a href=\"#ref-8\" style=\"color: #2a78d6;text-decoration: underline\">[8]<\/a> His grouping, redrawn:<\/p>\n<figure class=\"viz\" style=\"margin: 2.2rem 0\"><div class=\"viz-card\" style=\"border: 1px solid #e6e5e0;border-radius: 10px;background-color: #fcfcfb;padding: 14px 14px 8px;overflow: auto\"><a href=\"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-1.webp\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" class=\"wp-image-1142\" src=\"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-1.webp\" alt=\"Experiments grouped by network size and tool access: raw weights without tools cover small networks; experiments on language models use Python, sometimes with forward passes.\" width=\"1520\" height=\"1040\" loading=\"lazy\" style=\"display:block;width:100%;height:auto\" srcset=\"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-1.webp 1520w, https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-1-300x205.webp 300w, https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-1-1024x701.webp 1024w, https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-1-768x525.webp 768w\" sizes=\"auto, (max-width: 1520px) 100vw, 1520px\" \/><\/a><\/div><figcaption class=\"viz-caption\" style=\"margin-top: .65rem;font-size: .9rem;line-height: 1.55;color: #52514e\"><strong>Figure 1.<\/strong> Every experiment in the weight-space programming post, by network size and by what the editor was allowed to do. Raw weights in a prompt with no tools covered only the networks under 200,000 weights. Every experiment on a real language model used Python, and most also ran the network.<\/figcaption><\/figure>\n<p style=\"margin: 0 0 1rem\">The pure setting, with raw numbers in a prompt, no tools and no running the network, covers only networks of 17,000 to 172,000 weights. Every experiment on a real language model used Python, and the Qwen fix also ran the model on 50 batches of practice problems. The abstract opens with \u201cjust by looking at the raw weights\u201d and then describes the Qwen result. Willison made the same point more gently: these are two different questions that deserve separate answers. <a href=\"#ref-5\" style=\"color: #2a78d6;text-decoration: underline\">[5]<\/a><\/p>\n<p style=\"margin: 0 0 1rem\">Honnibal also granted the part that matters: \u201cFiguring out how to do this productively is not nothing, and AI models of a year or two ago would have failed at all these tasks.\u201d <a href=\"#ref-8\" style=\"color: #2a78d6;text-decoration: underline\">[8]<\/a><\/p>\n<h2 id=\"the-qwen-edit-removed-one-direction\" style=\"font-size: 1.28rem;line-height: 1.35;margin: 2.5rem 0 .7rem;font-weight: 700;color: #0b0b0b\">The Qwen edit removed one direction<\/h2>\n<p style=\"margin: 0 0 1rem\">Experiment 8 is the one that spread. Qwen&#8217;s own model card warns that Qwen3.5-0.8B is prone to thinking loops. In the researchers&#8217; tests, 93% of its MATH-500 answers ran all the way to the 32,768-token limit. Astra got a Python session with the weights loaded, practice problems from the GSM8K and MATH training sets, and a budget of forward passes, each a batch of about 64 answers. It was not allowed gradients or training.<\/p>\n<p style=\"margin: 0 0 1rem\">Astra traced the loops to one direction that the MLP blocks of layers 15 and 19 write into the model&#8217;s running internal state. It subtracted that direction from 64 output columns in each of the two layers. That is 131,072 weights, or 0.016% of the model. An independent analysis by Hedwig AI compared the published weights with the originals and confirmed the shape of the edit. Only two matrices change, each change is the simplest kind possible (rank one), and the two layers&#8217; directions point almost the same way. <a href=\"#ref-1\" style=\"color: #2a78d6;text-decoration: underline\">[1]<\/a> <a href=\"#ref-2\" style=\"color: #2a78d6;text-decoration: underline\">[2]<\/a><\/p>\n<p style=\"margin: 0 0 1rem\">The move has a history. In 2024, Arditi and colleagues showed that refusal in chat models is controlled by a single direction, and that deleting it from the weights removes the behaviour. <a href=\"#ref-9\" style=\"color: #2a78d6;text-decoration: underline\">[9]<\/a> Astra found a different direction behind a different behaviour, on its own and on a budget.<\/p>\n<figure class=\"viz\" style=\"margin: 2.2rem 0\"><div class=\"viz-card\" style=\"border: 1px solid #e6e5e0;border-radius: 10px;background-color: #fcfcfb;padding: 14px 14px 8px;overflow: auto\"><a href=\"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-2.webp\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" class=\"wp-image-1143\" src=\"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-2.webp\" alt=\"Qwen editing results for zero, 10, 25 and 50 forward passes: MATH-500 rises from 7.9% to 48.1%, while answers hitting the token limit fall from 93.0% to 33.1%.\" width=\"1520\" height=\"860\" loading=\"lazy\" style=\"display:block;width:100%;height:auto\" srcset=\"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-2.webp 1520w, https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-2-300x170.webp 300w, https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-2-1024x579.webp 1024w, https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-2-768x435.webp 768w\" sizes=\"auto, (max-width: 1520px) 100vw, 1520px\" \/><\/a><\/div><figcaption class=\"viz-caption\" style=\"margin-top: .65rem;font-size: .9rem;line-height: 1.55;color: #52514e\"><strong>Figure 2.<\/strong> Qwen3.5-0.8B after Astra&#8217;s edits with three forward-pass budgets. With 10 or 25 passes the loops barely moved; with 50, answers hitting the 32,768-token limit fell from 93.0% to 33.1%. GSM8K, not shown, rose from 13.8% to 67.1%.<\/figcaption><\/figure>\n<p style=\"margin: 0 0 1rem\">The budget mattered more than anything else. With 10 or 25 forward passes, the edits barely dented the loops: more than 86% of answers still hit the limit, and MATH-500 stayed between 12% and 14%. With 50 passes, the share hitting the limit fell to 33.1% and MATH-500 jumped to 48.1%. GSM8K rose from 13.8% to 67.1%. IFEval, a test of following instructions, rose from 21.0% to 62.3%. <a href=\"#ref-1\" style=\"color: #2a78d6;text-decoration: underline\">[1]<\/a><\/p>\n<h2 id=\"the-tempting-explanation-the-edit-taught-it-maths\" style=\"font-size: 1.28rem;line-height: 1.35;margin: 2.5rem 0 .7rem;font-weight: 700;color: #0b0b0b\">The tempting explanation: the edit taught it maths<\/h2>\n<p style=\"margin: 0 0 1rem\">Read those numbers quickly and a story writes itself. One AI opened another&#8217;s brain, found the bad wiring and made it six times better at mathematics. It is a satisfying story, and it is mostly wrong.<\/p>\n<p style=\"margin: 0 0 1rem\">Go back to the polar-coordinates trace. The model had the right answer by token 2,553. Whatever the edit did, it did not teach it the angle of (0, 3).<\/p>\n<p style=\"margin: 0 0 1rem\">Hedwig measured when the 64 changed neurons are active. In layer 19, the edit removes 4.5 times as much at a repeated token as at a new one, and in 92 of 100 random pairs it removes more at the repeat. The same neurons fire on loops that have nothing to do with maths. Asked for a short story about a lighthouse keeper, the unedited model wrote \u201cOkay, let&#8217;s write.\u201d over and over and never wrote the story. Across eight non-maths prompts, unedited answers hit the limit 22 times out of 32. Edited answers hit it 7 times. <a href=\"#ref-2\" style=\"color: #2a78d6;text-decoration: underline\">[2]<\/a><\/p>\n<p style=\"margin: 0 0 1rem\">So the edit is real, and it is specific. A random edit of the same size, on the same neurons, left 80 of 100 answers stuck and got 19 right. Astra&#8217;s edit left 34 stuck and got 51 right. The direction matters, and Astra found the right one. But it is a brake on repetition, not a lesson in algebra.<\/p>\n<p style=\"margin: 0 0 1rem\">The deciding number is a baseline. The model card says Qwen3.5-0.8B runs without thinking by default. Run MATH-500 that way and the unedited model scores 43.6%. The edited model scores 45.2%. <a href=\"#ref-2\" style=\"color: #2a78d6;text-decoration: underline\">[2]<\/a><\/p>\n<figure class=\"viz\" style=\"margin: 2.2rem 0\"><div class=\"viz-card\" style=\"border: 1px solid #e6e5e0;border-radius: 10px;background-color: #fcfcfb;padding: 14px 14px 8px;overflow: auto\"><a href=\"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-3.webp\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" class=\"wp-image-1144\" src=\"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-3.webp\" alt=\"MATH-500 comparison: unedited thinking on 7.9%, thinking cut 34.3%, thinking off 43.6%; edited thinking off 45.2% and thinking on 48.1%. Settings differ across evaluations.\" width=\"1520\" height=\"864\" loading=\"lazy\" style=\"display:block;width:100%;height:auto\" srcset=\"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-3.webp 1520w, https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-3-300x171.webp 300w, https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-3-1024x582.webp 1024w, https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-3-768x437.webp 768w\" sizes=\"auto, (max-width: 1520px) 100vw, 1520px\" \/><\/a><\/div><figcaption class=\"viz-caption\" style=\"margin-top: .65rem;font-size: .9rem;line-height: 1.55;color: #52514e\"><strong>Figure 3.<\/strong> MATH-500 scores for the unedited and edited model under different thinking settings. The two thinking-on scores are the authors&#8217;; the others are Hedwig AI&#8217;s runs, which use the model card&#8217;s sampling settings and an 8,192-token limit, so the rows are not a controlled comparison. Let the unedited model stop thinking and it already scores 43.6%.<\/figcaption><\/figure>\n<p style=\"margin: 0 0 1rem\">The authors said much the same, in a control the headline left behind. They pointed Astra at Gemma 3 270M, a model that rarely loops, and asked for the same improvement. Astra read Gemma&#8217;s failures and found ordinary arithmetic and algebra mistakes, with no single component behind them. Its small, targeted edits never moved MATH-500 by more than about a point. The authors&#8217; conclusion is the most honest sentence in the post: editing weights \u201ccould remove an obstacle that hid an ability, but it could not add an ability the model lacked.\u201d <a href=\"#ref-1\" style=\"color: #2a78d6;text-decoration: underline\">[1]<\/a><\/p>\n<p style=\"margin: 0 0 1rem\">Goedecke put it more bluntly. Write some looping and non-looping texts, find the difference, trim the weights that cause it and test again: that is a standard steering playbook that a competent researcher would follow. He compared editing weights by hand to patching a compiled binary. It is an impressive trick, but the normal way to change a program is still to edit the source and recompile. <a href=\"#ref-4\" style=\"color: #2a78d6;text-decoration: underline\">[4]<\/a><\/p>\n<h2 id=\"every-weight-edit-needs-a-judge\" style=\"font-size: 1.28rem;line-height: 1.35;margin: 2.5rem 0 .7rem;font-weight: 700;color: #0b0b0b\">Every weight edit needs a judge<\/h2>\n<p style=\"margin: 0 0 1rem\">The next part is easy to miss because it looks like plumbing. Look again at Figure 2. The three edits differ in one thing only: how many times Astra could run the model and look at what came out. The reasoning model was the same. The starting weights were the same. What separated a 14% edit from a 48% edit was how much evaluation was built into the loop.<\/p>\n<p style=\"margin: 0 0 1rem\">Gradient descent has a loss function. Weight-space programming swaps the optimizer for a reasoner, but it does not remove the need for a loss. Something still has to decide whether an edit helped, on which problems and against which baseline. In Experiment 8 that judge was a set of practice problems with known answers, a token limit, and a grader that reads the last boxed answer. Change any one of them and you change what \u201cbetter\u201d means.<\/p>\n<p style=\"margin: 0 0 1rem\">The sycophancy experiment shows the same thing from the other side. Two copies of Gemma 3 270M were fine-tuned in the same way, one to answer correctly and one to agree with whatever answer the user suggested. From the weights alone, across four attempts, Astra could not tell which was which. It was then allowed to see attention patterns, but not outputs, on texts it wrote itself. With that, it picked the sycophant in all three pairs. In one middle-layer head, the sycophant puts 85.7% of its attention on the user&#8217;s suggested answer, against 0.75% in the honest model. <a href=\"#ref-1\" style=\"color: #2a78d6;text-decoration: underline\">[1]<\/a> Behaviour shows up in how a network responds to inputs. Someone has to write the inputs that make it show.<\/p>\n<p style=\"margin: 0 0 1rem\">Then there is variance. The OpenWSP project, which has no link to the authors, gave GPT-6 Astra the authors&#8217; task, rules and 50-pass budget and let it make its own edit. That edit scored 12.9% on MATH-500, and 84% of its answers still hit the limit. The same team re-scored the authors&#8217; released edit and got 50.6%. Hugging Face&#8217;s Ahsen Khaliq rebuilt it from the published logs and got 51.0%. <a href=\"#ref-7\" style=\"color: #2a78d6;text-decoration: underline\">[7]<\/a> <a href=\"#ref-1\" style=\"color: #2a78d6;text-decoration: underline\">[1]<\/a><\/p>\n<figure class=\"viz\" style=\"margin: 2.2rem 0\"><div class=\"viz-card\" style=\"border: 1px solid #e6e5e0;border-radius: 10px;background-color: #fcfcfb;padding: 14px 14px 8px;overflow: auto\"><a href=\"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-4.webp\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" class=\"wp-image-1145\" src=\"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-4.webp\" alt=\"The released Qwen edit scores 48.1%, 50.6% and 51.0% in reported evaluations; one independent fresh editing run with 50 passes scores 12.9%.\" width=\"1520\" height=\"868\" loading=\"lazy\" style=\"display:block;width:100%;height:auto\" srcset=\"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-4.webp 1520w, https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-4-300x171.webp 300w, https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-4-1024x585.webp 1024w, https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-4-768x439.webp 768w, https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-figure-4-1400x800.webp 1400w\" sizes=\"auto, (max-width: 1520px) 100vw, 1520px\" \/><\/a><\/div><figcaption class=\"viz-caption\" style=\"margin-top: .65rem;font-size: .9rem;line-height: 1.55;color: #52514e\"><strong>Figure 4.<\/strong> Independent checks of Experiment 8. Re-scoring the released edit gives 50.6% and 51.0%. Asking GPT-6 Astra to make a new edit under the same rules and 50-pass budget gave 12.9%. OpenWSP&#8217;s live run averages four samples per problem; its re-score uses one.<\/figcaption><\/figure>\n<p style=\"margin: 0 0 1rem\">The artifact reproduces. The process, tried once more, did not. Agent builders know this gap as the difference between pass@1 and <a href=\"https:\/\/perit.ai\/blogs\/pass-k-ai-agent-reliability\/\" rel=\"noopener\" style=\"color: #2a78d6;text-decoration: underline\" target=\"_blank\">pass^k<\/a>: one successful run shows what can happen, not what will happen. An editing agent is still an agent, and it needs the same evaluation discipline as any other.<\/p>\n<p style=\"margin: 0 0 1rem\">That discipline is mostly unglamorous. You need practice problems with verified answers, and a baseline taken at the model&#8217;s best settings rather than its worst. You need held-out behaviour the editor never saw, such as instruction following. And you need probes written by people who know what failure looks like in their field, the same logic behind <a href=\"https:\/\/perit.ai\/blogs\/the-model-got-8-10-for-what\/\" rel=\"noopener\" style=\"color: #2a78d6;text-decoration: underline\" target=\"_blank\">a rubric that says what an 8\/10 was for<\/a>. Weight-space programming makes edits cheap, which makes judging them the expensive part. That judging layer is what Perit builds, and teams who need one can start from <a href=\"https:\/\/perit.ai\/rl-environments\" rel=\"noopener\" style=\"color: #2a78d6;text-decoration: underline\" target=\"_blank\">Perit&#8217;s RL environments<\/a>.<\/p>\n<h2 id=\"what-this-does-not-say\" style=\"font-size: 1.28rem;line-height: 1.35;margin: 2.5rem 0 .7rem;font-weight: 700;color: #0b0b0b\">What this does not say<\/h2>\n<p style=\"margin: 0 0 1rem\">It does not say weight-space programming is a parlour trick. A general-purpose model read circuits out of raw matrices, wrote working networks from a game&#8217;s rules, and found a repetition brake that a random edit of the same size could not match. Those are real abilities, and they arrived quickly.<\/p>\n<p style=\"margin: 0 0 1rem\">It does not say the Qwen edit is useless. In thinking mode, which the model card itself warns about, it is a large and cheap fix that touches two tensors and leaves every other weight alone.<\/p>\n<p style=\"margin: 0 0 1rem\">It does not say editing can never add a capability. One small model that rarely loops is one data point, and the authors say they are building a benchmark to measure this properly.<\/p>\n<p style=\"margin: 0 0 1rem\">And it says nothing yet about frontier-scale models. The largest network in the study has 0.8 billion weights, every experiment used one editor model, and the work is a blog post, not a peer-reviewed paper. It also does not show recursive self-improvement. A large model repaired a small one; it did not improve itself. <a href=\"#ref-1\" style=\"color: #2a78d6;text-decoration: underline\">[1]<\/a><\/p>\n<h2 id=\"back-to-tan3\u03c02\" style=\"font-size: 1.28rem;line-height: 1.35;margin: 2.5rem 0 .7rem;font-weight: 700;color: #0b0b0b\">Back to tan(3\u03c0\/2)<\/h2>\n<p style=\"margin: 0 0 1rem\">Open the trace once more. Token 1,423: \u03b8 = \u03c0\/2. Token 2,553: the full answer, correct. Token 7,200: \u201cWait, tan(3\u03c0\/2) = \u2026\u201d, and then the same line until the budget runs out. The model was never missing the mathematics. It was missing a way to stop.<\/p>\n<p style=\"margin: 0 0 1rem\">Astra found the place in the weights where repetition feeds on itself, and turned it down. The edited model reaches the same answer and, this time, writes it in the box. On the score sheet, that is a six-fold improvement in maths. Inside the network, it is a brake.<\/p>\n<p style=\"margin: 0 0 1rem\">Both descriptions are true. Only a well-chosen test tells you which one you paid for.<\/p>\n<p style=\"margin: 0 0 1rem\">Training teaches a model what to say. This edit taught one when to stop.<\/p>\n<p style=\"margin: 0 0 1rem\"><strong>The weights are now writable. The test still decides what got written.<\/strong><\/p>\n<h2 id=\"weight-space-programming-common-questions\" style=\"font-size: 1.28rem;line-height: 1.35;margin: 2.5rem 0 .7rem;font-weight: 700;color: #0b0b0b\">Weight-space programming: common questions<\/h2>\n<h3 style=\"font-size: 1.05rem;line-height: 1.4;margin: 1.6rem 0 .45rem;font-weight: 700;color: #0b0b0b\">Is weight-space programming the same as fine-tuning?<\/h3>\n<p style=\"margin: 0 0 1rem\">No. Fine-tuning computes gradients over a dataset and lets an optimizer adjust many weights by small amounts. In weight-space programming, an LLM forms a hypothesis about a mechanism, such as \u201cthis direction causes looping\u201d, and writes a targeted change itself. In the reported experiments that meant 3.5% of a car policy&#8217;s weights and 0.016% of Qwen3.5-0.8B&#8217;s.<\/p>\n<h3 style=\"font-size: 1.05rem;line-height: 1.4;margin: 1.6rem 0 .45rem;font-weight: 700;color: #0b0b0b\">How is it different from model editing methods such as ROME?<\/h3>\n<p style=\"margin: 0 0 1rem\">Methods such as ROME use a fixed algorithm to make one narrow change, like updating a single stored fact. <a href=\"#ref-10\" style=\"color: #2a78d6;text-decoration: underline\">[10]<\/a> In weight-space programming, a general-purpose model decides what to change and why, for goals ranging from naming a network&#8217;s task to writing a network from scratch.<\/p>\n<h3 style=\"font-size: 1.05rem;line-height: 1.4;margin: 1.6rem 0 .45rem;font-weight: 700;color: #0b0b0b\">Does weight-space programming use gradient descent?<\/h3>\n<p style=\"margin: 0 0 1rem\">No. Every experiment banned gradients and training loops. Some allowed a Python sandbox and a fixed budget of forward passes, so the editor could test an edit before submitting it.<\/p>\n<h3 style=\"font-size: 1.05rem;line-height: 1.4;margin: 1.6rem 0 .45rem;font-weight: 700;color: #0b0b0b\">Can I try weight-space programming myself?<\/h3>\n<p style=\"margin: 0 0 1rem\">Yes. The authors published every prompt, response and evaluation at github.com\/RikiyaT\/lm_understand-logs, and the edited Qwen model is on Hugging Face. Willison ran the game-naming prompt on a laptop. The edit only removes a loop, so a fair before-and-after test should include the model&#8217;s thinking-off mode.<\/p>\n<h2 id=\"sources\" style=\"font-size: 1.28rem;line-height: 1.35;margin: 2.5rem 0 .7rem;font-weight: 700;color: #0b0b0b\">Sources<\/h2>\n<ol class=\"refs\" style=\"font-size: .9rem;color: #52514e\"> <li id=\"ref-1\" style=\"margin: 0 0 .55rem\">Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, <a href=\"https:\/\/rikiyat.github.io\/blog\/weight-space-programming\/\" rel=\"noopener\" style=\"color: #2a78d6;text-decoration: underline\" target=\"_blank\">\u201cSparks of Weight-Space Programming: LLMs can natively read &amp; write neural network weights\u201d<\/a>, MIT CSAIL, 2 November 2026 (blog post, not peer-reviewed). Experiments 1 to 8 and the Gemma 3 270M control. Logs at <a href=\"https:\/\/github.com\/RikiyaT\/lm_understand-logs\" rel=\"noopener\" style=\"color: #2a78d6;text-decoration: underline\" target=\"_blank\">github.com\/RikiyaT\/lm_understand-logs<\/a>; edited model at <a href=\"https:\/\/huggingface.co\/RikiyaT\/Qwen3.5-0.8B-weight-space-programming\" rel=\"noopener\" style=\"color: #2a78d6;text-decoration: underline\" target=\"_blank\">huggingface.co\/RikiyaT\/Qwen3.5-0.8B-weight-space-programming<\/a>.<\/li> <li id=\"ref-2\" style=\"margin: 0 0 .55rem\">Hedwig AI, <a href=\"https:\/\/www.hedwigai.com\/blog\/weight-edit-math-or-loops\" rel=\"noopener\" style=\"color: #2a78d6;text-decoration: underline\" target=\"_blank\">\u201cDoes a weight edit teach Qwen3.5-0.8B math, or stop it looping?\u201d<\/a>, November 2026. Source of the polar-coordinates and lighthouse traces, the weight diff, repetition measurements, random-edit control (first 100 MATH-500 problems) and thinking-off and thinking-budget scores.<\/li> <li id=\"ref-3\" style=\"margin: 0 0 .55rem\">David Lindner, J\u00e1nos Kram\u00e1r, Sebastian Farquhar, Matthew Rahtz, Thomas McGrath and Vladimir Mikulik, <a href=\"https:\/\/arxiv.org\/abs\/2301.05062\" rel=\"noopener\" style=\"color: #2a78d6;text-decoration: underline\" target=\"_blank\">\u201cTracr: Compiled Transformers as a Laboratory for Interpretability\u201d<\/a>, NeurIPS 2023.<\/li> <li id=\"ref-4\" style=\"margin: 0 0 .55rem\">Sean Goedecke, <a href=\"https:\/\/www.seangoedecke.com\/llms-are-not-superhuman-at-reading-and-writing-neural-network-weights\/\" rel=\"noopener\" style=\"color: #2a78d6;text-decoration: underline\" target=\"_blank\">\u201cLLMs are not superhuman at reading and writing neural network weights\u201d<\/a>, 3 November 2026.<\/li> <li id=\"ref-5\" style=\"margin: 0 0 .55rem\">Simon Willison, <a href=\"https:\/\/simonwillison.net\/2026\/Nov\/3\/qwen-can-interpret-model-weights\/\" rel=\"noopener\" style=\"color: #2a78d6;text-decoration: underline\" target=\"_blank\">\u201cQwen 3.8 27B on my laptop can interpret model weights, too\u201d<\/a>, 3 November 2026.<\/li> <li id=\"ref-6\" style=\"margin: 0 0 .55rem\">Nelson Elhage, Neel Nanda, Catherine Olsson and colleagues, <a href=\"https:\/\/transformer-circuits.pub\/2021\/framework\/index.html\" rel=\"noopener\" style=\"color: #2a78d6;text-decoration: underline\" target=\"_blank\">\u201cA Mathematical Framework for Transformer Circuits\u201d<\/a>, Transformer Circuits Thread, 2021.<\/li> <li id=\"ref-7\" style=\"margin: 0 0 .55rem\">OpenWSP, <a href=\"https:\/\/github.com\/openwsp\/openwsp\" rel=\"noopener\" style=\"color: #2a78d6;text-decoration: underline\" target=\"_blank\">open tooling and an independent replication of weight-space programming<\/a>, README as of 5 November 2026. Experiment 8 live run (four samples per problem), calibration of the released edit (one sample per problem), the Ahsen Khaliq rebuild result and the raw-weight replications.<\/li> <li id=\"ref-8\" style=\"margin: 0 0 .55rem\">Matthew Honnibal, <a href=\"https:\/\/honnibal.dev\/blog\/astra-read-write-misleading\" rel=\"noopener\" style=\"color: #2a78d6;text-decoration: underline\" target=\"_blank\">\u201cThe \u2018Astra can read and write weights to improve Qwen\u2019 post is misleading\u201d<\/a>, 3 November 2026.<\/li> <li id=\"ref-9\" style=\"margin: 0 0 .55rem\">Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee and Neel Nanda, <a href=\"https:\/\/arxiv.org\/abs\/2406.11717\" rel=\"noopener\" style=\"color: #2a78d6;text-decoration: underline\" target=\"_blank\">\u201cRefusal in Language Models Is Mediated by a Single Direction\u201d<\/a>, NeurIPS 2024.<\/li> <li id=\"ref-10\" style=\"margin: 0 0 .55rem\">Kevin Meng, David Bau, Alex Andonian and Yonatan Belinkov, <a href=\"https:\/\/arxiv.org\/abs\/2202.05262\" rel=\"noopener\" style=\"color: #2a78d6;text-decoration: underline\" target=\"_blank\">\u201cLocating and Editing Factual Associations in GPT\u201d<\/a>, NeurIPS 2022.<\/li> <\/ol>\n<hr \/>\n<p class=\"refs\" style=\"margin: 0 0 1rem;font-size: .9rem;color: #52514e\"><em>Research checked 5 November 2026. Experiment results are as reported by the authors unless they are credited to Hedwig AI, OpenWSP, Ahsen Khaliq or Simon Willison. Those independent runs use their own sampling settings, sample counts and hardware, which the figure captions note. The token positions and repeated lines in the opening trace come from Hedwig AI&#8217;s re-reading of the authors&#8217; published outputs. Perit did not reproduce any of these results, and no Perit measurements are reported in this post.<\/em><\/p>\n<\/div>\n\n","protected":false},"excerpt":{"rendered":"<p>Weight-space programming lets an LLM edit a neural network&#8217;s weights without training. What GPT-6 Astra&#8217;s 0.016% Qwen edit fixed, and what it did not.<\/p>\n","protected":false},"author":2,"featured_media":1141,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[22,19],"tags":[],"class_list":["post-1146","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-artificial-intelligence","category-research"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.6 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Weight-Space Programming: The Model Knew the Answer. It Couldn&#039;t Stop. - Perit Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Weight-Space Programming: The Model Knew the Answer. It Couldn&#039;t Stop. - Perit Blog\" \/>\n<meta property=\"og:description\" content=\"Weight-space programming lets an LLM edit a neural network&#039;s weights without training. What GPT-6 Astra&#039;s 0.016% Qwen edit fixed, and what it did not.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/\" \/>\n<meta property=\"og:site_name\" content=\"Perit Blog\" \/>\n<meta property=\"article:published_time\" content=\"2026-11-05T12:34:03+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-llm-edit-neural-network-weights.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"2400\" \/>\n\t<meta property=\"og:image:height\" content=\"1260\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"author\" content=\"Pratyaksh@Perit\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@baldwin_IVth\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Pratyaksh@Perit\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"20 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/weight-space-programming-llm-edit-neural-network-weights\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/weight-space-programming-llm-edit-neural-network-weights\\\/\"},\"author\":{\"name\":\"Pratyaksh@Perit\",\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/#\\\/schema\\\/person\\\/6b951f8ca802e00473088655d8a2a1bc\"},\"headline\":\"Weight-Space Programming: The Model Knew the Answer. It Couldn&#8217;t Stop.\",\"datePublished\":\"2026-11-05T12:34:03+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/weight-space-programming-llm-edit-neural-network-weights\\\/\"},\"wordCount\":4058,\"commentCount\":0,\"image\":{\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/weight-space-programming-llm-edit-neural-network-weights\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/weight-space-programming-llm-edit-neural-network-weights.webp\",\"articleSection\":[\"Artificial Intelligence\",\"Research\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/perit.ai\\\/blogs\\\/weight-space-programming-llm-edit-neural-network-weights\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/weight-space-programming-llm-edit-neural-network-weights\\\/\",\"url\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/weight-space-programming-llm-edit-neural-network-weights\\\/\",\"name\":\"Weight-Space Programming: The Model Knew the Answer. It Couldn't Stop. - Perit Blog\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/weight-space-programming-llm-edit-neural-network-weights\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/weight-space-programming-llm-edit-neural-network-weights\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/weight-space-programming-llm-edit-neural-network-weights.webp\",\"datePublished\":\"2026-11-05T12:34:03+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/#\\\/schema\\\/person\\\/6b951f8ca802e00473088655d8a2a1bc\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/weight-space-programming-llm-edit-neural-network-weights\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/perit.ai\\\/blogs\\\/weight-space-programming-llm-edit-neural-network-weights\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/weight-space-programming-llm-edit-neural-network-weights\\\/#primaryimage\",\"url\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/weight-space-programming-llm-edit-neural-network-weights.webp\",\"contentUrl\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/weight-space-programming-llm-edit-neural-network-weights.webp\",\"width\":2400,\"height\":1260,\"caption\":\"Weight-space programming: Qwen finds the correct angle, then repeats a line until its token limit; an edit changes 131,072 weights, or 0.016%, and removes the loop.\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/weight-space-programming-llm-edit-neural-network-weights\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Weight-Space Programming: The Model Knew the Answer. It Couldn&#8217;t Stop.\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/#website\",\"url\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/\",\"name\":\"Perit Blog\",\"description\":\"Frontier training data, collected and graded by people who do the work.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/#\\\/schema\\\/person\\\/6b951f8ca802e00473088655d8a2a1bc\",\"name\":\"Pratyaksh@Perit\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/pfp-150x150.webp\",\"url\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/pfp-150x150.webp\",\"contentUrl\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/pfp-150x150.webp\",\"caption\":\"Pratyaksh@Perit\"},\"sameAs\":[\"http:\\\/\\\/perit.ai\\\/blogs\",\"https:\\\/\\\/x.com\\\/baldwin_IVth\"],\"url\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/author\\\/pratyakshpatel\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Weight-Space Programming: The Model Knew the Answer. It Couldn't Stop. - Perit Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/","og_locale":"en_US","og_type":"article","og_title":"Weight-Space Programming: The Model Knew the Answer. It Couldn't Stop. - Perit Blog","og_description":"Weight-space programming lets an LLM edit a neural network's weights without training. What GPT-6 Astra's 0.016% Qwen edit fixed, and what it did not.","og_url":"https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/","og_site_name":"Perit Blog","article_published_time":"2026-11-05T12:34:03+00:00","og_image":[{"width":2400,"height":1260,"url":"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-llm-edit-neural-network-weights.webp","type":"image\/webp"}],"author":"Pratyaksh@Perit","twitter_card":"summary_large_image","twitter_creator":"@baldwin_IVth","twitter_misc":{"Written by":"Pratyaksh@Perit","Est. reading time":"20 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/#article","isPartOf":{"@id":"https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/"},"author":{"name":"Pratyaksh@Perit","@id":"https:\/\/perit.ai\/blogs\/#\/schema\/person\/6b951f8ca802e00473088655d8a2a1bc"},"headline":"Weight-Space Programming: The Model Knew the Answer. It Couldn&#8217;t Stop.","datePublished":"2026-11-05T12:34:03+00:00","mainEntityOfPage":{"@id":"https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/"},"wordCount":4058,"commentCount":0,"image":{"@id":"https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/#primaryimage"},"thumbnailUrl":"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-llm-edit-neural-network-weights.webp","articleSection":["Artificial Intelligence","Research"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/","url":"https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/","name":"Weight-Space Programming: The Model Knew the Answer. It Couldn't Stop. - Perit Blog","isPartOf":{"@id":"https:\/\/perit.ai\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/#primaryimage"},"image":{"@id":"https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/#primaryimage"},"thumbnailUrl":"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-llm-edit-neural-network-weights.webp","datePublished":"2026-11-05T12:34:03+00:00","author":{"@id":"https:\/\/perit.ai\/blogs\/#\/schema\/person\/6b951f8ca802e00473088655d8a2a1bc"},"breadcrumb":{"@id":"https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/#primaryimage","url":"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-llm-edit-neural-network-weights.webp","contentUrl":"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/11\/weight-space-programming-llm-edit-neural-network-weights.webp","width":2400,"height":1260,"caption":"Weight-space programming: Qwen finds the correct angle, then repeats a line until its token limit; an edit changes 131,072 weights, or 0.016%, and removes the loop."},{"@type":"BreadcrumbList","@id":"https:\/\/perit.ai\/blogs\/weight-space-programming-llm-edit-neural-network-weights\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/perit.ai\/blogs\/"},{"@type":"ListItem","position":2,"name":"Weight-Space Programming: The Model Knew the Answer. It Couldn&#8217;t Stop."}]},{"@type":"WebSite","@id":"https:\/\/perit.ai\/blogs\/#website","url":"https:\/\/perit.ai\/blogs\/","name":"Perit Blog","description":"Frontier training data, collected and graded by people who do the work.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/perit.ai\/blogs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/perit.ai\/blogs\/#\/schema\/person\/6b951f8ca802e00473088655d8a2a1bc","name":"Pratyaksh@Perit","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/09\/pfp-150x150.webp","url":"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/09\/pfp-150x150.webp","contentUrl":"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/09\/pfp-150x150.webp","caption":"Pratyaksh@Perit"},"sameAs":["http:\/\/perit.ai\/blogs","https:\/\/x.com\/baldwin_IVth"],"url":"https:\/\/perit.ai\/blogs\/author\/pratyakshpatel\/"}]}},"_links":{"self":[{"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/posts\/1146","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/comments?post=1146"}],"version-history":[{"count":1,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/posts\/1146\/revisions"}],"predecessor-version":[{"id":1147,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/posts\/1146\/revisions\/1147"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/media\/1141"}],"wp:attachment":[{"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/media?parent=1146"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/categories?post=1146"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/tags?post=1146"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}