Nobody reads the assembly
Works · July 2026
1957
The standard analogy for AI code generation is the 1957 Fortran compiler. The argument is that just like developers eventually stopped reading assembly, we will eventually stop reading AI output.
But the parallel is broken. A compiler is deterministic. You certify the tool once, and trust the output. An LLM generates a different program every run, so there is nothing static to certify.
Because you can't certify the model, I wanted to see if the model could at least learn from its own mistakes in real time.
So I ran it
I ran a loop on a local 30B model over a few tasks with hidden tests. I assumed showing the agent its past failures would help. The first run said no, but the run was flawed. The tasks were too easy, so there was rarely a failure to remember. So I fixed it and made the tasks harder.
What got hard
Harder didn't mean more complex. The tasks that broke the model weren't the algorithmically hard ones, they were the ones that went against its defaults. That's from one model, so I'd want to see it hold on others before calling it the pattern.
Technical
Pure algorithmic and edge-case tasks stayed one-shot even with traps added, like a mismatched bracket (] or an invalid token. The tasks it struggled with each broke a default assumption: banker's rounding on ties (round, solved 47% of the time), conflicting title-case rules (titlecase, 20%), and a size-zero input that should stop the loop but instead hangs it (chunk, 53%). Iterations on those rose several times over.
Memory
Now there were real failures to remember. It still didn't help. On the hardest task the model got stuck, repeating the same wrong fix over and over, and it never used the tool I gave it to look up its own past failures.
Technical
Splitting the runs by how much of its own history the model saw:
| context | success |
|---|---|
| the full log | 0.63 |
| last observation | 0.70 |
| a truncated slice | 0.77 |
More context did slightly worse, at five runs each. It never looked up its own record in any run that had the tool. On the title-case task the model oscillated, around nine case-flips a run, and stopped reliably fixing even one failing case per turn.
Where I am
The one thing that held across every run is that the agent never reaches for its own record, even when it has the tool. I think this information has use, but unsure how to get the model to use it.
agent-trace-outcomes
Technical
Agent Trace, the RFC from Cursor and Cognition, already tracks which model and conversation wrote each line. It stops at attribution. It never says whether the change passed, whether anyone looked at it, or whether the same idea failed last week.
For every change it drops one JSON file into the repo, pinned to the commit:
| field | what it holds |
|---|---|
intent | what the change was trying to do |
checks[] | each check that ran, with its name and pass/fail |
verdict | derived from the checks, overridable. verified only if every non-skipped check passed and at least one ran |
lesson | optional. a short note on what was learned, scoped to path globs |
recordOutcome() writes it, or a CI action does on merge. There's no server, so the file just sits next to the code. Then I read it back. atrace-outcomes log src/auth/ shows everything that's been tried under a path and how it went. verdict <sha> only exits 0 if the commit actually verified, so I can put it straight in front of a merge. And inside a loop, queryLessons() pulls up the recorded lessons for the files it's about to touch, each with its verdict.