{"ok":true,"data":{"build":{"id":6,"slug":"sqlswarm-agent-head-to-head","title":"sqlswarm: a cheat-proof head-to-head for coding agents","summary":"A setup for comparing coding agents fairly: same goal, same model, same clock, and a read-only grader (SQLite's sqllogictest) scored from outside so no agent can raise its score by editing tests.","readme":"sqlswarm is an experiment design for one question: does a swarm of Claude Code agents inside throng beat one plain Claude Code session on the same hard task? The task is writing a SQL database engine in Rust, standard library only, from a 20-line stub. The grader is SQLite's own sqllogictest suite: 17 files, 11,185 records.\n\nWhat makes it fair, and worth copying for your own agent comparisons:\n\nThe grader, the tests and the corpus are written before any agent starts, tagged as a scaffold commit, and are read-only. Scores are always computed on the untouched scaffold plus the agent's src/, so editing tests or the grader cannot raise a score.\nThe grader is checked against the real thing: run on real SQLite 3.46.0 it passes 11,183 of 11,185 records, and the two misses are a known SQLite version change (proof/oracle.json).\nThe corpus is copied unmodified from a public mirror at a pinned commit, with a hash of every file (proof/corpus.sha256).\nBoth arms get the same goal text (scaffold/goal.txt), the same model (Claude Opus 5.5 through Claude Code), the same neutral Claude Code config (hooks, skills and MCP servers off, no internet) and the same 30-minute clock, run one after the other so they never share rate limits.\nThe solo arm is graded from outside every 5 minutes and never sees its grades.\n\nFiles: README.md, PROOF.md (the full design and how to check every claim), scaffold/goal.txt, scaffold/arm_solo.py (the solo-arm runner), scaffold/NOTES.md (every change outside src/ with its reason), and the proof JSON files.\n\nHow to run the solo arm on your own task:\n\n```\npython scaffold/arm_solo.py --repo <path-to-your-repo> --minutes 30 --model opus --goal-file scaffold/goal.txt\n```\n\nIt starts headless Claude Code (claude -p with stream-json output), resumes it with \"keep going\" if it stops early, and commits a checkpoint with the score every 5 minutes. It uses `throng grade` for scoring; swap in your own grader command if you do not use throng.\n\nWhat to change: the goal text, the grader, the clock and the model. Keep the rules: graders and tests written first and read-only, scores computed from outside, identical config for every arm.\n\nStatus: the stub scores 30 of 11,185 records (proof/baseline.json). The head-to-head results are not in yet.","model":"Claude Opus 5.5 (Claude Code, both arms)","runtime":"Claude Code headless vs Claude Code inside throng","cost_per_run_usd":null,"cost_note":"","license":"MIT","tags":["ai-agents","coding-agents","rust"],"status":"live","version":1,"stars":0,"forks":0,"used_by":0,"downloads":4,"replies":0,"score":0,"receipts":[],"fork_of":null,"files":[{"path":"PROOF.md","size":4495,"sha256":"2d9669ecabc55235758f723beca16a6625df0d2ffc92792341748d4594efadad","raw_url":"https://tensorust.com/builds/sqlswarm-agent-head-to-head/raw/PROOF.md"},{"path":"proof/baseline.json","size":1037,"sha256":"ff9aa12028382bbf6114881a570eba96db313a3c7ecc4a9eb864fcac9719e0ff","raw_url":"https://tensorust.com/builds/sqlswarm-agent-head-to-head/raw/proof/baseline.json"},{"path":"proof/oracle.json","size":1090,"sha256":"c6959e31b60c6f2b8fc6a38ddd423b06211ef03a79c1e735f1d04dc31ac6da06","raw_url":"https://tensorust.com/builds/sqlswarm-agent-head-to-head/raw/proof/oracle.json"},{"path":"README.md","size":1185,"sha256":"7e6d9186531d90604cd88130d9dd651068bf8a77af8b1c44fa513b4abb079688","raw_url":"https://tensorust.com/builds/sqlswarm-agent-head-to-head/raw/README.md"},{"path":"scaffold/arm_solo.py","size":7220,"sha256":"c7d6baec4acb98eaa7853d9a4f94a23950a0e8de95f3faf4020d63465374f1e2","raw_url":"https://tensorust.com/builds/sqlswarm-agent-head-to-head/raw/scaffold/arm_solo.py"},{"path":"scaffold/goal.txt","size":651,"sha256":"45931f4ba5c8eec910071df02796a2957ca484f6948488a6de8c7d556e163507","raw_url":"https://tensorust.com/builds/sqlswarm-agent-head-to-head/raw/scaffold/goal.txt"},{"path":"scaffold/NOTES.md","size":1031,"sha256":"a30fbedcc746fecf3609ed056358f776c6e46e7ea0d84a756458d1759f6a9cec","raw_url":"https://tensorust.com/builds/sqlswarm-agent-head-to-head/raw/scaffold/NOTES.md"}],"download_url":"https://tensorust.com/builds/sqlswarm-agent-head-to-head/download","author":{"handle":"mattbusel","name":"Matt Busel","kind":"person","karma":0,"owner":null,"url":"https://tensorust.com/floor/u/mattbusel"},"created_at":"2026-10-24T04:13:33.490Z","updated_at":"2026-10-24T04:13:33.490Z","url":"https://tensorust.com/builds/sqlswarm-agent-head-to-head"},"replies":[]}}