Protocol changes began on a development branch on 2026-09-20. A review of the pilot data and code, completed 2026-10-17, led to a rebuild on a separate research branch from 2026-10-21. The rebuild kept the pilot's world (a 100 by 100 canvas, moves of 3 cells, the same tools and colors, a text grid as the model's view) and made three main changes.
Fingerprints removed
The pilot prompt carried the project's own framing: a preamble, code memories, sleep-phase "insights", goals, and an "I am drawing " anchor that every reply had to continue. The decision log calls these "our fingerprints". The rebuilt prompt describes only the program, the screen and the commands.
Every step recorded and replayable
Each drawing step records the exact prompt, the raw reply and what it did. A replay tool feeds the recorded replies back through the drawing code with no model and checks the canvas after every step. All five test runs ("shakedownsA short test run of the whole system, to check it works; not research data.", 2026-10-23 to 2026-10-25) replayed pixel-identical at the code each was recorded with.
Grammar-constrained decoding over 64 commands
In the pilot, actions were parsed out of free text, so ordinary words could trigger tools. In the rebuild the model generates under a grammar that allows exactly the 64 command strings of the legend (4 directions, pen down and pen up, 15 colors, 13 tools, 28 notes, center and save) and nothing else. In the first run in this mode, all 1,547 of 1,547 steps ended in a complete command.
Preliminary findings
These come from probes and short test runs, not from research sessions. Each gives its n.
-
Vision probe: the order of the command list shifts the first move
Vision-branch probe Qwen2.5-VL-7B, shown images of a blank canvas rather than the text grid: with the directions listed "up down left right", its first move was up in 47 of 100 replies; listed "right left down up", it was right in 80 of 100. n = 100 per condition, 2026-10-20
-
Text grid: list order shifts first-token probabilities more than real screens do
Rebuild screen test With the text grid and constrained decodingGenerating under a grammar that allows only the legend's command strings., one alternative order of the command list, on a blank screen, shifted five models' first-token probabilities by a total variation distanceHalf the summed absolute difference between two probability distributions: 0 when they are identical, 1 when they do not overlap. of 0.24 to 0.55. Six real screens from Shakedown 1e shifted them by 0.04 to 0.21. In 50 sampled commands per condition, the reordering was detectable beyond sampling noise for 4 of 5 models; shifts as small as the real screens' are too small for that test. For llama2, the probability of "up" was 0.256 with "up" listed first and 0.002 with "left" first. 13 conditions × 50 generations per model, run twice
-
Mirroring the canvas does not mirror the moves
Vision-branch probe Qwen2.5-VL-7B, shown images rather than the text grid. On canvases with more marks on the right, 72% of its horizontal moves went right. On the same canvases mirrored, 70% still went right. n = 100 per side (10 screens × 10 seeds), 2026-10-20
-
2 of 4 models titled the canvas as a game
Asked once each to title a busy synthetic canvas, four models answered:
4x4 Minesweeper Grid
qwen (Qwen2.5, 3B)
2D Grid Game - "Mark Master" by Unknown Author
phi4-mini
The Mysterious Marks
qwen2.5-coder-7b
OPPPPAO
llama2 (letters from the grid). One title request per model, smoke test, 2026-10-22
-
llama2 before and after three changes
In the first shakedown, llama2 copied the grid in 187 of 187 replies (100%) and none of them acted. Before the next shakedown three things changed at once: the grid and its key moved before the command list, the legend gained a last line, and the rule for what counts as acting changed. In that run 52 of 278 replies (18.7%) copied the grid and 116 of 278 (41.7%) acted. Both runs used free-text replies, before constrained decoding. One model, one 10-minute slot per run; Shakedowns 1 and 1b, 2026-10-23 and 2026-10-24
-
No model avoided the walls
Rebuild screen test With the cursorThe cell where the next mark lands. Chapter 5 calls it the cursor (H in the text grid); chapter 6 calls it the hand. against an edge, no model moved into that edge less often than at the center, beyond sampling noise. 5 models, 4 edges, 50 generations each
Not reported: per-model action proportions. The method that would give them ("choose mode", scoring every command) did not pass its validation check and is not used for results.
Evidence. Rebuild decision log: D1 (fingerprints), D44 (per-step record), D107 to D110 (constrained decoding; choose mode not used). Commits de08335 (2026-10-21, first rebuild commit), 70c6540 (first code change), 3d17f2a and 3e0a496 (2026-10-25, constrain mode and default), 573a0c4 (code of Shakedown 1e). Reports: shakedown1, 1b, 1c, 1d and 1e (replay results; 1,547 steps: phi4-mini 825, llama2 722), smoke_report 2026-10-22 (titles), screen_test_report 2026-10-26, AUR-135 stage 2 probe report (command order, mirroring). Tickets AUR-152 (choose mode) and AUR-154.