Build a database in Rust: AI passed 866K tests, then failed ROLLBACK

TL;DR: One headless Claude Code session (Opus 5.5) wrote a SQL database engine in Rust, standard library only, in 12.5 minutes: 4,191 lines that pass all 11,185 records of the SQLite test files it was graded on. Today I ran it on 866,331 records from 95 test files it had never seen. It passed every one. Then I gave it 79 ordinary SQL statements the test suite never asks for. It matched real SQLite on 55. ROLLBACK errors out and the row it should have undone stays in the table, inserts get 4x slower every time the table doubles, and triggers are accepted but never run.

Short answer for the search. If you want to build a database in Rust, the shape that works is five small layers: a lexer, a parser, a planner, an executor and an in-memory store, with SQLite's own sqllogictest files as the grader and real SQLite as the reference answer. That gets a query engine passing hundreds of thousands of SQLite's test records in a few thousand lines. It does not get you a database. Transactions, indexes that actually index, and everything the test files do not exercise are separate work, and a test score will not tell you which of them you are missing.

This is the third piece in a row where the interesting number came from testing what the demo did not. In the LLM proxy benchmark it was a 30 second slowdown hiding behind a same-prompt demo; in stop coding agents editing tests it was agents quietly changing the grader. Here the grader was locked, the agent did nothing wrong, and the score was still the wrong question.

The setup: a stub, a locked grader and SQLite's own tests

sqllogictest is how SQLite checks itself against other databases. Each test file is a list of records: a statement that must succeed or fail, or a query with its expected answer. SQLite's testing page says the full suite runs 7.2 million queries in 1.12 GB of test data. A record looks like this (real, from select1.test):

query I nosort
SELECT CASE WHEN c>(SELECT avg(c) FROM t1) THEN a*2 ELSE b*10 END
  FROM t1
 ORDER BY 1
----
30 values hashing to 3c13dee48d9356ae19af2515e05e6b54

Big answers are stored as an MD5 hash of the formatted values, so the engine has to get every value right, including how SQLite prints integers versus reals. There is no partial credit.

The repo was built before the agent started and the agent could only write to src/. It had three parts. A grader in Rust (slt/, no dependencies) that parses the record format and does the hashing. A corpus of 17 files copied from the gregrahn/sqllogictest mirror (select1 to select5 plus 12 "evidence" files, 11,185 records, every file hashed). And an oracle: the same grader run against real SQLite 3.46 through rusqlite, which scores 11,183 of 11,185 (the two misses are a sum() overflow that newer SQLite turned into an error). The engine's whole contract is one trait:

// slt/src/lib.rs (the grader, written before the run)
pub trait Engine: Send + 'static {
    /// A new, empty, in-memory database.
    fn open() -> Self
    where
        Self: Sized;
    /// Execute one SQL statement.
    fn execute(&mut self, sql: &str) -> Result<Output, String>;
}

The stub that started the run failed every statement and scored 30 of 11,185 (statements that are supposed to fail, which a stub passes for free). The goal text said: standard library only, behave like SQLite on type affinity, NULLs, integer versus real arithmetic and output format. No crates, no internet.

12.5 minutes, 4,179 lines, then the first compile

The session ran on September 29, 2026 at 1:40 AM Eastern as claude -p in headless mode, Claude Code 2.1.284, model Opus 5.5, with hooks, skills, MCP servers, web search and web fetch all switched off. Its full transcript is in the repo, so every minute below is from the log, not from memory:

Minute	What happened
0.0 to 2.3	Read the grader, the test harness and the corpus. Counted which SQL features the corpus uses with grep | sort | uniq -c.
2.9 to 10.7	Wrote 9 files, 4,179 lines, in this order: values, lexer, AST, parser, storage, planner, functions, executor, the Database itself. No compile in between.
10.7	First cargo build. No errors printed.
10.8	First test run: 1,126 of 1,128 tests pass.
11.6	One more edit (the parser now splits ;-separated statements): 1,128 of 1,128, and 11,185 of 11,185 records on the full-file score.
12.5	Session ends: 24 turns, 13 shell commands, 9 file writes, 86,847 output tokens (22,357 of them thinking).

Claude Code's own cost field for the session reads $3.05. That is the API-equivalent figure; it ran on a subscription, so nothing was billed per token. The outside grader that checked the repo every 5 minutes saw 1 of 1,128 at minute 5 and minute 10, because there was nothing compiled yet. Then the score went from 1 to 1,128 in about a minute.

The line that explains everything that follows is in the transcript at minute 2.3, right after the feature count:

"Scope is clear: comma joins only, few functions (abs, coalesce, count/avg/sum/total/min/max/group_concat), heavy CASE/BETWEEN/EXISTS/compound selects. Writing the engine now."

It read the exam first and built exactly what the exam asks for. That is a good engineering move. It is also the reason the rest of this article exists.

How the engine is built

Nothing exotic, and that is the useful part if you are building your own. The lexer turns text into tokens. The parser is hand-written recursive descent with SQLite's operator precedence (1,071 lines, the biggest file). The planner binds every column name once, including names from outer queries in correlated subqueries, pulls out aggregates and splits WHERE into separate AND conditions so the executor can apply each one as early as possible. The executor does nested-loop joins with a greedy join order, which is what keeps select5.test's 8-table joins from exploding. value.rs carries SQLite's type rules: '10' + 5 is 15, 7/2 is 3, integer overflow turns into a real.

The top of it is short enough to quote whole:

// src/lib.rs (written by the agent)
impl Engine for Database {
    fn open() -> Self {
        Database { cat: Catalog::default() }
    }

    fn execute(&mut self, sql: &str) -> Result<Output, String> {
        let stmts = parser::parse(sql)?;
        let mut out = Output::Done;
        for stmt in stmts {
            out = self.exec_stmt(stmt)?;
        }
        Ok(out)
    }
}

The storage layer is 73 lines. A table is a Vec<Vec<Value>> kept sorted by its integer primary key. CREATE INDEX is recorded in the catalog and never used to find a row. Hold that thought.

The fresh test: 866,331 records it never saw

A 100% score on the files you were graded on proves less than it sounds, especially when the agent read those files first. So today (October 18, 2026, 1:12 to 1:15 PM Eastern) I pulled a hold-out set from the same mirror at the same commit: the index/ and random/ trees, which the run never touched. Three smallest files plus five random ones (seed 20261018) from each of the 12 categories, 96 files, 171 MB. Every download was checked against the git blob hash in the repo's tree, 0 mismatches. One file is empty upstream, which leaves 95.

The harness is the scaffold's own slt::check, wrapped to keep every failure and the wall time, and run once with the agent's engine and once with SQLite, one process per file:

// holdout_common.rs (my harness for this test, trimmed)
pub fn run_files<E: slt::Engine>(engine: &str, files: &[String]) {
    for path in files {
        let recs = slt::records(path);
        // engine runs on its own thread; any record over 10 s ends the file
        let mut db = E::open();
        for r in recs.iter() {
            let res = slt::check(&mut db, r);   // the same check the scaffold grades with
            // ... count passes, keep every failure with its line and SQL, time it
        }
    }
}

The result: 866,331 of 866,331 records pass. Every category, every file, zero failures, zero timeouts. SQLite also passed all of them on the same grader, which confirms the files and the harness. That is 77 times more records than the run was graded on, in shapes it never saw: BETWEEN on indexed columns, IN on 1,000-row tables, views, DELETE on 10,000-row tables, random expressions with nested CASE and CAST.

So the engine did not memorize its test files. Inside the slice of SQL that sqllogictest covers, it generalizes. On the same i7-13700KF it is a median 1.6 times slower than SQLite per file (63 files over 50 ms), and 3.8 times slower in total (57.1 s against 15.1 s), which for a 12-minute first draft against one of the most tuned C codebases on earth is honestly impressive.

The total hides one row, though. index/delete runs 17.6 times slower, and one file in it, delete/10000/slt_good_0.test, took 13.0 seconds against SQLite's 0.2. That is a 65x gap, and it has a cause.

Where it breaks 1: inserts that get 4x slower

To isolate it I inserted N rows, one statement each, into a table with an integer primary key and a UNIQUE text column, and timed both engines with the same harness:

10,000 rows: 4.0 s. 20,000: 15.1 s. 40,000: 59.6 s. Every doubling multiplies the time by about 4, the signature of an O(n²) loop. SQLite does 40,000 in 2.3 s through the same harness, and most of that is my harness spawning a thread per statement. Here is the loop:

// src/lib.rs, insert_row (written by the agent)
// Uniqueness.
let mut conflicts: Vec<usize> = Vec::new();
for u in &t.uniques {
    if u.iter().any(|&c| row[c] == Value::Null) {
        continue;
    }
    for (ri, r) in t.rows.iter().enumerate() {          // every existing row, every insert
        if u.iter().all(|&c| compare(&r[c], &row[c]).is_eq()) {
            match conflict {
                Conflict::Abort => return Err(format!("UNIQUE constraint failed: {}", unique_desc(t, u))),
                Conflict::Ignore => return Ok(()),
                Conflict::Replace => conflicts.push(ri),
            }
        }
    }
}

Each insert scans every row once per unique constraint, and the primary key counts as one, so the table above pays two full scans per row. Rows inserted without an id pay one more, a max() over the table to pick the next number. The 17 graded files never noticed: the biggest inserts 1,000 rows in total, spread over 9 tables, and the per-record timeout is 10 seconds. A real app with a 100,000-row table would. The fix is the index that CREATE INDEX already pretends to build: a BTreeMap from key to row position per unique constraint turns each scan into a lookup.

Where it breaks 2: the SQL the tests never ask for

Speed is fixable. The bigger question is what else is missing that no test file asks about. So I wrote 79 statements the way an app would actually use a database (probes.sql: a users table, an orders table, then joins, string functions, CTEs, upserts, transactions, triggers) and ran them on both engines, comparing answers line by line:

55 of 79 match SQLite. The good news first: everything that looks like the corpus is solid. LEFT JOIN, LIKE, GLOB, COLLATE NOCASE, substr with negative offsets, CAST('42abc' AS INTEGER), integer overflow to real, the right constraint errors on a duplicate key. 17 statements fail loudly with an error, which is the honest kind of failure: CTEs including WITH RECURSIVE, every window function, all three date functions, upsert, RETURNING, ALTER TABLE, printf and sqlite_master.

The 7 that worry me are the ones that fail silently, with no error and a wrong answer:

Statement	SQLite	Agent's engine
ROLLBACK after BEGIN + INSERT, then count the row	0 (undone)	error on ROLLBACK, row still there (1)
Insert into a table with an AFTER INSERT trigger, count the audit rows	1	0 (trigger never ran)
ORDER BY age NULLS LAST	Cy, Ada, dee, bob	bob, Cy, Ada, dee (NULL first)
last_insert_rowid() after an insert	5	0
changes() after updating 2 rows	2	0
PRAGMA table_info(orders)	4 rows of column info	OK, no rows
Read back the row the failed upsert should have changed	upserted	first (old value)

The transaction one is the scary one, and the cause is five lines in the parser:

// src/parser.rs (written by the agent)
if self.eat_kw("BEGIN") || self.eat_kw("COMMIT") || self.eat_kw("END") || self.eat_kw("ANALYZE") || self.eat_kw("VACUUM") {
    while self.peek() != &Tok::Eof {
        self.next();
    }
    return Ok(Stmt::Nop);
}

BEGIN and COMMIT are swallowed as no-ops, because the corpus uses them and never checks that they mean anything. ROLLBACK is not on the list, because the corpus never sends one. An app that wraps a payment in a transaction and rolls back on failure would get an error from the rollback and keep the half-written payment. To be fair to the agent: its final message said, unprompted, "Triggers are only checked and stored; their bodies never run." It disclosed the trigger gap. It just never mentioned transactions, because nothing it was measured on touched them.

Why a 100% score hid all of this

None of this is the agent cheating. The grader was locked, the corpus was hashed, the agent wrote only src/, and it generalized to 77 times more test records than it was shown. By the rules of the task, it won.

The problem is that a test suite is two things at once: a measurement and a spec. sqllogictest was designed to compare query answers across engines, so it is very good at SELECT and says almost nothing about durability, transactions, performance at size or the 100 functions apps call every day. When the agent counted features at minute 2 and wrote "Scope is clear", it turned the measurement into the spec. Everything outside that scope came out as a no-op, an error, or a linear scan.

My read, and I think it holds well beyond databases: AI coding agents are now good enough that the test suite is the product spec, whether you meant it to be or not. What you do not test does not exist, and sometimes it exists as a quiet Ok(Stmt::Nop), which is worse than not existing. The fix is not a better agent. It is a hold-out set the agent never sees and a probe file written from the user's side, which together took me about ten minutes today.

Build a database in Rust yourself: the order that works

If you want to build one, by hand or with an agent, this is the path the transcript followed, plus the parts it skipped:

1. Grader first. Use the sqllogictest crate (sqllogictest-rs) or write the 600 lines yourself; the record format is small. Pin the corpus to a commit and hash the files.
2. An oracle next to it. Run the same grader on SQLite through rusqlite. If SQLite does not score about 100%, your grader is wrong, not your engine.
3. Values before parsing. The agent wrote value.rs first: affinity, comparison across types, integer versus real. Most wrong answers in SQL engines are type rules, not query plans.
4. Lexer, parser, planner, executor, in that order, each a plain module. Recursive descent is enough. Bind names in a planner, not during execution.
5. Split your tests three ways. The files the builder sees, a hold-out it never sees, and a probe file written from the user's side (transactions, upserts, RETURNING, dates, window functions, a 100,000-row insert). Grade on the first, report on all three.
6. Make unsupported statements fail loudly. Grep your parser for anything that returns a no-op. A Nop for VACUUM is fine; a Nop for COMMIT is a promise you are not keeping.
7. Time a big insert early. One loop of 10,000, 20,000 and 40,000 rows tells you whether you have indexes or a scan.

If you are running agents on code like this, the setup for running 20 coding agents on one PC is where this scaffold came from, and the locked-grader pattern is in stop coding agents editing tests.

The data, and how to rerun it

The engine is commit 5a6342f of my local sqlswarm-solo repo: the scaffold, the agent's src/, the ledger and the full session transcript (proof/runs/solo/session-part1.jsonl, every tool call with its timestamp). The hold-out run, the probe file and the insert test are in one folder with their raw output: holdout_common.rs and probe_common.rs (the two harnesses, built as an example in the engine crate and a binary in the oracle crate), run.py, the file lists (sel2.txt, sel3.txt), holdout.jsonl and holdout2.jsonl (one line per file and engine), probes.sql with both engines' answers, and scale.json. Built with rustc 1.91.0 in release mode on an i7-13700KF with 32 GB of RAM, run October 18, 2026 between 1:12 and 1:18 PM Eastern. The test files are at gregrahn/sqllogictest c67f97b; pick your own hold-out and you should see the same 100% and the same 4x curve.

What I would like to know from anyone who has built one: which of the missing pieces is hardest to add after the fact? My bet is transactions, because every write path has to learn to undo itself, and that is exactly the kind of change a test score will not push an agent toward.

If you built your app with an AI agent and it passes its own tests, the open question is the same one as here: what does it do with the inputs nobody wrote a test for? That is what the $49 Launch Check is for: a pass over your live app from the user's side, looking for the silent failures before strangers find them.