The 'Astra can read and write weights to improve Qwen' post is misleading
Matthew Honnibal, co-founder of Explosion and creator of spaCy, published a critique on his blog at honnibal.dev on November 3, 2026, arguing that a widely shared post about GPT-6 Astra overstates what the underlying research shows. The research report, titled Sparks of Weight-Space Programming, came from a team led by Rikiya Takehi, affiliated with MIT CSAIL, and was published on November 2, 2026. Honnibal's reading is that only three of the ten experiments in the post read raw weights directly.
Those raw-weight experiments cover networks of 17K to 172K weights, far smaller than the Qwen model. The headline result is a math fix for Qwen3.5-0.8B. Astra located a reasoning loop, which Qwen's model card warns about, and edited 0.016% of the weights, or 131,072 values. MATH-500 accuracy rose from 7.9% to 48.1%. Honnibal says that run depended on Python code and repeated model runs, up to 50 batches of practice problems, so it was not pure inspection of raw numbers.
Honnibal's central objection is that the raw-weight tests and the Qwen test use different techniques and should not sit under one claim of general ability. He does not dismiss the Qwen gain. He describes it as work a human machine learning engineer could do, the kind models a year or two older would have failed at, and he notes that engineers routinely write code that analyzes or edits weights directly.
Why it matters
For founders and operators, the dispute is a reminder that a capability claim is only as strong as its method description. A system that writes code to inspect and edit weights is a different product claim from one that reads parameters unaided, and buyers, partners and reviewers will test which one they are getting. Teams building model-tuning or interpretability tooling should note that the reported gain came from a scaffold of code and repeated test runs rather than a new primitive.
For investors, the work suggests frontier models can be pointed at smaller public models to remove specific failure modes. The control result matters here: on Gemma 3 270M, which rarely loops, the same approach did not improve math scores, so the method appears to remove obstacles rather than add new skills. Diligence should ask for the exact tooling, the number of runs and the benchmark sets before treating a demo as a general capability, and should not assume a 0.8B model result carries over to production systems.
Key points
- 1Only three of the post's ten experiments read raw weights directly, on networks of 17K to 172K weights.
- 2The Qwen3.5-0.8B math fix edited 0.016% of the model's weights, which the paper puts at 131,072 values.
- 3MATH-500 accuracy on Qwen3.5-0.8B rose from 7.9% to 48.1% after the weight edits.
- 4Honnibal says the Qwen run relied on Python code and up to 50 batches of practice problems.
- 5
