# How this experiment works, and how to check it

**Claim under test:** a swarm of AI agents run by throng can write a SQL
database engine in Rust from an empty stub, graded by SQLite's own
`sqllogictest` suite.

## What was written by whom

| Path | Author | Can the swarm change it? |
|---|---|---|
| `slt/` (grader), `tests/`, `examples/`, `corpus/`, `oracle/`, `proof/`, `scaffold/`, `Cargo.toml`, this file | Scaffolding, written before the swarm started (commit tagged `scaffold`) | **No.** throng rejects any agent change outside `src/`. |
| `src/` | Started as a 20-line stub that fails every statement. Everything after the `scaffold` commit is swarm output. | Yes, and only here. |

The engine may use the Rust standard library only: `Cargo.toml` has no
dependencies besides the grader, and the swarm cannot add any.

## The grader is honest

`slt/` reimplements SQLite's sqllogictest runner (record parsing,
`skipif`/`onlyif`, value formatting, `rowsort`/`valuesort`, MD5 result
hashing). To show it grades the way SQLite's own runner does, `oracle/` runs
the same grader against real SQLite (3.46.0 via rusqlite):

- **11,183 of 11,185 records pass (99.98%)**: `proof/oracle.json`.
- The two misses are `evidence/slt_lang_aggfunc.test` lines 480 and 484:
  SQLite 3.46 raises "integer overflow" for `sum()` where the older SQLite that
  recorded the corpus returned a value. That is a SQLite version change, not a
  grader error.

## The test corpus

`corpus/` is copied unmodified from the sqllogictest mirror
<https://github.com/gregrahn/sqllogictest>, commit
`c67f97bf3ca7e590d12e073408bcacaf2ff0f3a0`. `proof/corpus.sha256` lists a
hash of every file so anyone can check nothing was edited. Tier 1 (this
corpus) is `select1-5.test` and `evidence/*.test`: 17 files, 11,185 records.

## Scoring

- **`cargo test`**: 1,128 tests, each grading 10 consecutive records of one
  file (earlier statements in the file are replayed first). This is what throng
  sees; a change is kept only if more tests pass and no passing test breaks.
- **`cargo run --release --example score`**: the canonical sqllogictest way,
  every file run top to bottom on one database, counting passing records.

Starting point (stub): **30 / 11,185 records (0.27%)**, 1 / 1,128 tests , 
`proof/baseline.json`. The 30 are `statement error` records, which a stub that
rejects everything passes trivially.

## The record of what happened

- **Git history.** Every round the swarm completes is one commit on `main`,
  with trailers naming the throng run, round, model, agents, tokens and the
  test score before and after.
- **`proof/ledger.jsonl`**: one line per round with the same numbers.
- **`proof/runs/`**: throng's full event log of each run (every tool call,
  check and result each agent made).

## The head-to-head

Same task, same model (Claude Opus 5.5 through Claude Code on a Claude
subscription, no API), same neutral Claude Code configuration (hooks, skills and
MCP servers off; no internet), same goal text (`scaffold/goal.txt`), same
30-minute wall clock, both starting from the `scaffold` commit. Run one after
the other so neither shares rate limits with the other.

- **Arm A, plain Claude Code** (repo `sqlswarm-solo`): one Claude Code session
  with its normal tools, including `cargo test` and its own subagents,
  resumed with "keep going" if it stops early (`scaffold/arm_solo.py`). Graded
  from outside every 5 minutes and at the end; the agent never sees the grades.
- **Arm B, Claude Code agents inside throng** (this repo): `throng run --pattern
  target --provider claude-code`. throng groups failing tests by cause, runs
  parallel Claude Code agents in private copies, gives them `throng grade`
  (the shared, cached verification engine), keeps only work that raises the
  score without breaking a passing test, and merges it round by round.

Both arms are scored with the same grader on the untouched scaffold plus the
arm's `src/`, so changes to tests or the grader cannot raise a score. Reported:
tests passing over time, tests that broke along the way, and Claude usage.
Full session transcripts of both arms are in `proof/runs/`.

## Reproduce it

```
git clone <this repo> && cd sqlswarm
cargo test                                  # graded tests, pass count
cargo run --release --example score         # records passed, per file
cd oracle && cargo run --release            # real SQLite on the same grader
sha256sum -c proof/corpus.sha256            # corpus unmodified
```