Skip to content

ADR-0013 — Postgres + DuckDB dual engine, and what makes the benchmark fair

  • Status: proposed (accept in CP13)
  • Date: 2026-07-02
  • Domain: storage engines · benchmark methodology

Context

Comparison 3 asks the row-store vs column-store question on real accumulated data: a rolling hourly edit-rate aggregate per wiki, over weeks of Wikimedia EventStreams accumulation (ADR-0015 — chosen over the Hacker News side for volume; HN's snapshot cadence accumulates far more slowly). It is deliberately a written decision with a number, not a live toggle (per the project spine: engine tuning is a weaker architect signal than the freshness and coupling exhibits, so it gets the smaller budget). A benchmark that isn't demonstrably fair is worse than no benchmark — it invites exactly the scrutiny a portfolio should welcome, and must survive it.

Decision

  • Postgres remains the system of record and runs the query on the native agg_wiki_edit_rate_hourly table with realistic indexes.
  • DuckDB runs the identical query over a Parquet snapshot of the same model, exported to MinIO after dbt runs. Same rows, same query semantics — different physical layout, which is precisely the variable under test. The row-store pays its layout; the column-store gets its native format. That asymmetry is the comparison, and the ADR says so rather than burying it.
  • Fairness protocol, published with the numbers: identical logical query; warm runs (discard first); N ≥ 5 repetitions; report the median; record row count, snapshot timestamp, and both engine versions. Re-runs happen live via a globally rate-limited endpoint (1/10 min); results are cached and timestamped.

Consequences

  • The benchmark is reproducible and criticizable — anyone can object to the method because the method is written down. That is the point.
  • Snapshot freshness lags the fact table (export after dbt runs); displayed accumulating_since and row count make the data volume honest.
  • MinIO (CP6) gets a second real consumer; DuckDB reads with a read-only scoped key.

Alternatives considered

  • DuckDB attached directly to Postgres (postgres_scanner) — rejected: measures Postgres's wire protocol more than DuckDB's engine; muddies what's being compared.
  • Both engines on Parquet — rejected: erases the row-store's real-world position (it serves the live workload from its native table; benchmarking it elsewhere would be unfair to it).
  • A live interactive toggle — rejected in the project spine: two exhibits built well beat three built thin.