pgrust: ClickHouse called it a cheat, then let it beat them 3x

Short answer: pgrust is Postgres rewritten in Rust with AI, and as of today (Oct 25 2026) it is back on ClickBench, the analytics leaderboard ClickHouse runs. ClickHouse's own maintainer threw it out on Sep 22 for "cheating", then merged the new open-source build this afternoon. In the official result files we pulled, pgrust finishes all 43 queries cold in 32.0 seconds; ClickHouse takes 97.0 on the same machine. The slop police lost this round.

TL;DR: On a c6a.4xlarge, pgrust v0.4-beta runs ClickBench's 43 queries cold in 32.0 s vs ClickHouse 97.0 s, DuckDB 115.2 s and stock PostgreSQL 12,404 s, which is 3.0x faster than the referee and 387x faster than the database it copies. The catch is real but narrower than the word "cheat": on repeat runs its caches make Q20 come back 1,254x faster than the first run, versus 16x for ClickHouse, and ClickHouse accepted that cache this time because it is disclosed and switchable. Meanwhile the same AI-assisted project filed 14 real Postgres bug reports in 17 days, and five of them got fixed by Tom Lane, David Rowley, Michael Paquier, Fujii Masao and John Naylor.

Key numbers

What	Number	Source, checked Oct 25 2026
pgrust v0.4-beta, ClickBench cold total (43 queries, c6a.4xlarge)	32.0 s	ClickBench results
ClickHouse, same machine, same day	97.0 s (3.0x slower)	ClickBench results
Stock PostgreSQL, same machine	12,404 s (387x slower)	ClickBench results
192-core Graviton (c8g.metal-48xl): pgrust vs ClickHouse cold	29.2 s vs 78.3 s	PR #2751 result files
Q20 repeat-run speedup, pgrust v0.4-beta vs ClickHouse	1,254x vs 16x	Our math on the result files
Removal PR opened to merged, by the same maintainer	29 seconds (Sep 22)	PR #2165
Days off the leaderboard	33	PR #2751, merged Oct 25
Postgres bug reports from pgrust testing	14 (Jul 29 to Aug 14 2026), 5 fixes committed	postgresql.org archive search
What pgrust cost to build, per its author	$100K, 4 attempts, 1.8M lines of Rust	malisper.me

The referee owns the leaderboard

ClickBench is ClickHouse's benchmark. It lives in the ClickHouse GitHub org, ClickHouse wrote the rules, and ClickHouse's Alexey Milovidov merges the entries. Every database company quotes it, and ClickHouse is the reference everyone else is measured against.

So when an open-source Postgres clone built mostly by AI agents showed up claiming to be faster than ClickHouse, the referee was also the team that had the most to lose. On Sep 22 2026 at 4:25 AM UTC, Milovidov opened PR #2165, "Remove pgrust: the entry is cheating". He merged it himself 29 seconds later. No review, no reply window, 1,258 lines of pgrust scripts and results deleted.

Nobody noticed for a month. Then the PR hit Hacker News on Oct 23 (193 points, 160 comments) right as pgrust shipped its source, and two days later Dan Luu's essay "We should be more ambitious with the software we build" held pgrust up as the example of what AI-built software can now be (740 points, 688 comments). The fight was on.

What ClickHouse actually alleged

Credit where due: Milovidov's write-up is a serious piece of forensics, not a drive-by. He ran the published v0.4-preview binary on a 96-core Graviton and found three things.

• Per-query code. The binary embeds files named after ClickBench query numbers (q00.rs, q01.rs through q36.rs) and error messages like "clickbench group count: filter is not k <> 0". Some queries were answered from per-part statistics without scanning.
• Hot runs that replay. His Q20 test went 10.8 s cold, then 11 ms, then 5 ms. He called the repeat runs "a lookup table of the benchmark, not a database".
• A narrow engine. 20 of 67 near-benchmark probe queries were refused with "statement over columnar relation hits is not served".

He also wrote, in the same PR, that the answers to all 43 queries were correct. That line matters. Nobody faked a result. The question was whether the engine was general or shaped to the test.

The defense, and why it mostly held

Michael Malis, who builds pgrust with Jason Seibel, answered on HN: the q00.rs "kernels" were IR probes that never ran in the benchmark, the "result cache" was a dictionary filter cache of the kind ClickBench allows (ClickHouse ships its own condition cache), and the narrow SQL coverage was a real mistake. "I acknowledge that we submitted it prematurely," he wrote.

Then he did the one thing that ends this kind of argument. He published the code. pgrust v0.4-beta went up as AGPL source on Oct 22, and PR #2751 listed every cache with its switch: condition cache, dictionary filter cache, zone map cache, all in memory, all emptied by the restart before each cold run.

ClickHouse's bot ran it on six machine types. Milovidov merged it on Oct 25 at 2:32 PM ET. Same referee, same rules, 33 days later. If the first build was a cheat, the second one is the receipt that the engine is real.

By the numbers: 32 seconds vs 97

We pulled the raw result JSON for every system on the standard c6a.4xlarge (16 vCPU) and summed the first, cold run of each query. That is the run with every cache empty, so it is the fairest number in the file.

• pgrust v0.4-beta: 32.0 s. Load 115 s, 7.4 GB on disk.
• The build ClickHouse removed: 39.6 s. The reinstated one is 19% faster than the "cheat".
• ClickHouse: 97.0 s, 9.4 GB.
• DuckDB: 115.2 s, 20.5 GB.
• pgrust v0.2 (Aug 1): 134.2 s. Four times faster in under three months.
• Stock PostgreSQL: 12,404 s, 106.5 GB, 1,004 s to load.

It is not one lucky box. On the 192-core Graviton c8g.metal-48xl the cold totals are 29.2 s for pgrust and 78.3 s for ClickHouse (2.7x); on c6a.metal, 31.3 s vs 76.3 s. Using ClickBench's own scoring (each query's time against the best, geometric mean) pgrust is the best of the four on both cold and hot runs.

The pgrust team's launch post says "as much as 3x faster than ClickHouse". On our sum of cold runs it is 3.03x. For once a vendor benchmark claim is exactly what the files say.

Where the cheat smell comes from: 1,254x on a repeat

Here is the part the pgrust fans skip. Q20 is URL LIKE '%google%' over the full hits table. pgrust's first run takes 2.508 s. The repeat takes 0.002 s. That is 1,254x. ClickHouse goes 6.151 s to 0.378 s (16x), DuckDB 15x, Postgres 1x.

The cache that does it is the same one ClickHouse flagged in September, now on by default and documented: pgrust remembers which rows matched a filter, with match counts, keyed by the filter and its constants. ClickHouse's own condition cache keeps only which granules had no matches and rescans the rest. pgrust's keeps the answer to the count.

That is legal under the rules ClickHouse just applied when it merged the entry. It is also why the hot column of ClickBench now measures cache design more than scan speed, for every system that plays the game. Our read: the hot leaderboard is broken, the cold leaderboard is the one that counts, and pgrust wins the cold one by 3x anyway.

Who wins, who loses

Follow the money. ClickHouse is a company whose sales pitch is "fastest analytics database", and ClickBench is the chart in the pitch. An AGPL Postgres clone that speaks plain Postgres SQL and beats ClickHouse on its own chart is a direct hit on that pitch. The 29-second self-merge was the reflex of a company protecting a moat, whatever the technical merits of the PR.

A Hacker News commenter, remywang, put the stakes plainly after reading the beta source: 4 of the 5 systems ahead of ClickHouse on ClickBench (Umbra, elosdb, Hyper, CedarDB) are closed-source. pgrust is the open one. If you care about open software, this is the entry to root for.

The losers are the gatekeepers of "real engineering". pgrust's own author says it took four attempts, $100K and up to 40 concurrent subagents to produce 1.8 million lines of Rust. That is years of a database team's output for $100K, and the incumbents know it.

The vibeslop project that fixed Postgres

The best evidence pgrust is not slop is not a benchmark at all. It is the Postgres bug tracker. pgrust uses real Postgres as its oracle and diffs every behavior against it, and the diffs turned up bugs in Postgres itself.

We searched the postgresql.org archive for reports copied to the pgrust author. There are 14, filed between Jul 29 and Aug 14 2026: macaddr input that accepts 9-digit octets, a money division overflow, a jsonpath format cache serving the wrong parse, three memory-safety defects in the ispell dictionary loader, a reachable "impossible" case in SP-GiST, a pg_waldump range bug, a wrong COUNT from a pushed-down WHERE, and more.

Five have fix commits in the archive: Tom Lane on the dictionary loader (Aug 2), David Rowley on money overflow (Aug 4), Michael Paquier on tid parsing (Aug 6), Fujii Masao on LSN parsing (Aug 12), and John Naylor on SP-GiST (Oct 14). Those are some of the most senior names in the project, fixing bugs an AI-driven rewrite found. That is a gift to every Postgres user, including the ones calling it slop.

HN argued about AI, not about the benchmark

We counted keywords across both threads through the HN Algolia API. In the ClickBench removal thread, 42 of 156 comments (27%) mention AI, LLMs, Claude or agents, and 31 (20%) talk about cheating or caches. In Dan Luu's thread, 261 of 653 (40%) are about AI and 54 (8%) say "slop" or "vibe".

The top comment on the removal thread opens "Sounds kind of like Claude was cheating". Another commenter, whinvik, called it: "Its like a lot of people really want pgrust to fail." The pattern says the verdict on pgrust was written before anyone read the PR, because of how it was built.

Dan Luu's point is the one that survives. As he puts it, people are "implicitly carrying around a pre-agentic cost model". His friend Yossi Kreinin argues the opposite in the same essay: most people should be less ambitious, because agent loops fail without clear pass/fail criteria. pgrust is the case where both are right. It had the clearest pass/fail criteria in software (Postgres's own regression suite and a public benchmark), and that is exactly why it worked.

Our own receipt: the grader is the whole game

We ran the small version of this experiment ourselves. An AI agent built us a SQL engine in Rust against SQLite's test files, and it passed 866,331 records it had never seen, then failed ROLLBACK, because the tests never asked for it. Same lesson as pgrust's September build: a score only tells you what the test covers.

pgrust's answer was to widen the oracle (all of Postgres's regression and isolation tests, plus differential testing that found real Postgres bugs) and then open the source. That is the template. It is the same one behind the AI-written Goose language that beat Rust on a third-party benchmark, and the opposite of the flood that got AI projects banned from r/opensource.

Our bet

Nobody has proven it yet, but the pattern says this ends one way. Within a year, an AI-built open-source engine sits at number one on ClickBench's cold ranking, and ClickBench adds a rule that randomizes query constants or bans filter-result caches on hot runs, because that is the only way the hot column means anything again.

And the next time a benchmark owner calls an AI-built challenger a cheat, ask for two things before you believe it: the source, and the cold run.

What this means for you

• If you pick databases: read ClickBench's cold column, not the hot one, and test with your own constants. Our LLM and Rust hubs track the tools we actually measure.
• If you build with agents: steal pgrust's method. Pick an oracle first (an existing implementation, a test suite, a spec), diff against it constantly, and publish the source before someone calls you a cheat.
• If you maintain a benchmark: the AI-built entries are coming. Write the cache rules before you need them, not 29 seconds after.

The fastest way to run the pgrust method on your own project is the $29 AI Builder Kit: five offline apps that pin your agent to a spec and a grader so it builds what you meant. If it does not do what it promises, we keep fixing it with you at no extra cost until it does. More AI-built software stories live under coding agents and Rust.

Sources

• ClickHouse/ClickBench PR #2165, "Remove pgrust: the entry is cheating", Alexey Milovidov, Sep 22 2026
• ClickHouse/ClickBench PR #2751, "pgrust: add v0.4-beta", Michael Malis, opened Oct 22, merged Oct 25 2026
• ClickBench result files for pgrust, ClickHouse, DuckDB and PostgreSQL, pulled Oct 25 2026
• Michael Malis's response on Hacker News, Oct 23 2026
• Hacker News thread on the removal PR, Oct 23 2026
• Dan Luu, "We should be more ambitious with the software we build", Oct 2026 (HN thread)
• pgrust, "pgrust v0.4-beta", Michael Malis, Oct 22 2026
• malisper.me, "Postgres in Rust: three dead ends before we passed 100% of the regression suite", Jul 2026
• malisper/pgrust on GitHub (5,492 stars, AGPL-3.0), checked Oct 25 2026
• PostgreSQL BUG #19595, dictionary loader memory safety, Aug 2 2026, and the other 13 reports in our CSV
• postgresql.org mailing list archive search, Oct 25 2026