Benchmark, phase 1 · run p1-20261114 · Linux x86-64

pgrustj against PostgreSQL 18.6 + pgvector 0.8.7

On one x86-64 node, with each server alone in a matched 6-core, 24 GiB container and the same settings wherever both have the knob, pgrustj’s pgvector port gives the same recall as C PostgreSQL 18.6 + pgvector 0.8.7 at every search setting (within 0.005) and builds indexes of the same size. It is slower at almost every vector operation: 0.72–0.74× C’s HNSW search throughput at matched settings, and 1.90–2.13× C’s time for a serial HNSW build. As a plain PostgreSQL it is close to C (pgbench -S 1.10×, TPC-B-like 0.94×), and an hour-long mixed soak finished with zero failed transactions on both servers.

Recall at every setting Same within 0.0035 on HNSW, 0.0048 on IVFFlat
HNSW search, matched ef_search 100, 8 clients 0.72–0.74× C’s queries per second on vector; 0.61× on halfvec
Serial HNSW build time 1.90–2.13× C’s time on vector; IVFFlat 3.65–4.84×
Plain SQL, pgbench -S 1.10× C’s transactions per second, possibly client-bound

Every ratio on this page is pgrustj ÷ C. Above 1 is better for pgrustj in queries, rows and transactions per second, and worse in time, CPU and memory.

What was compared

C
PGDG PostgreSQL 18.6 with the PGDG pgvector 0.8.7 package, both written in C.
pgrustj
The roadius2/pgrust fork at a9c5743a3c, built from source in its container (6 min 19 s, release profile). It reports itself as PostgreSQL 18.6 (pgrust 0.4-beta).
Host
borg2: Intel i9-11900 (8 cores, 16 threads, AVX-512, F16C), 61 GiB, Ubuntu 25.04, Incus 6.0.6 on ZFS. The host also ran other instances.
Containers
One database container at a time, six whole cores and 24 GiB each. The load generator ran in its own container on four other threads.
Settings
One file for both: shared_buffers 8GB, maintenance_work_mem 8GB, work_mem 64MB, max_parallel_workers_per_gather 2, max_wal_size 8GB. pgrustj also needs io_method=sync; C keeps its default worker.
Datasets
From ann-benchmarks: glove-100-angular (1,183,514 × 100, cosine), as vector and as halfvec; sift-128-euclidean (1,000,000 × 128, L2), as vector. Each has 10,000 held-out queries with their true top 100.
Indexes
HNSW with m = 16, ef_construction = 64; IVFFlat with 1184 lists on glove and 1000 on sift. Searches return the top 10 at 1, 8 and 16 clients; the client computes recall@10 from the ground truth.

Results

pgrustj C: PostgreSQL 18.6 + pgvector 0.8.7

Search: recall against throughput

Each point is one search setting (ef_search for HNSW, probes for IVFFlat). Across: recall@10 from the 1-client pass over 10,000 queries. Up: queries per second at 8 clients. The curves sit at the same recall and separate in throughput: pgrustj’s points are the lower ones.

HNSW (one scale for all three)

IVFFlat (one scale for both)

Show the numbers

Index builds: wall time, median of 3

Serial builds and builds with 4 parallel workers, maintenance_work_mem 8GB. Shorter is better. Both sides’ indexes come out the same size on every row.

HNSW (seconds)

IVFFlat (seconds, its own scale)

Show the numbers

The one-hour soak: transactions per second, by 5-minute window

8 clients mixing HNSW searches (ef_search 100, weight 9) and inserts (weight 1), with a VACUUM every 10 minutes. Zero failed transactions on both servers; both stay flat. pgrustj served 0.79× C’s transactions over the hour.

Show the numbers

Benefits and negatives

Where pgrustj is equal or ahead

  • Recall: the same at every search setting, within 0.0035 on HNSW and 0.0048 on IVFFlat, and in the exact scans.
  • Index sizes: equal to the MiB on all 30 builds.
  • Plain SQL: pgbench -S at 1.10× C, using 0.91× C’s CPU per transaction. Only 4.8 of 6 cores were busy on both sides, so the client may have been the limit. pgbench -i takes the same time.
  • Exact vector scan at 8 clients: 1.06–1.11× C’s throughput, with both servers using all six cores. Per CPU-second pgrustj looks ahead here, a lead confounded by C’s parallel-worker start-up and possible JIT.
  • Memory of a 4-worker HNSW build: 0.35–0.40× C’s container peak (3.45–4.19 against 9.98–10.60 GiB). C’s shared memory reserves the whole 8 GB graph area up front, most likely.
  • Loading: 1.01× C on glove and 0.94× on sift.
  • Stability: zero failed transactions in the hour, flat 5-minute windows, no memory growth after the first five minutes, and no error rows in 175 measurements a side.
  • Portability: it builds from source and runs on Linux x86-64; the measured binary took 6 min 19 s to build.

Where pgrustj is behind

  • Search throughput at matched settings, 8 clients: 0.72–0.74× on HNSW vector, 0.61× on HNSW halfvec, 0.76–0.84× on IVFFlat. p99 latency is 1.19–1.89× C’s.
  • HNSW builds: 1.90–2.13× C’s time serially and 1.58–1.66× with 4 workers on vector; 2.74× and 2.19× on halfvec.
  • IVFFlat builds, the largest gap: 3.65–4.84× C’s time serially and 2.75–2.97× with 4 workers.
  • Exact scan at 1 client: 0.47–0.51× on vector and 0.24× on halfvec. C runs it with two parallel workers; pgrustj’s planner builds no parallel plans.
  • CPU: 1.39–1.64× C’s per HNSW search query at ef_search 100, and 1.38× per TPC-B-like transaction.
  • Inserts under an HNSW index: 0.85–0.95× C’s rate. The DELETE + VACUUM after them takes 1.34–2.84× C’s time.
  • Memory held: higher in every step except the 4-worker builds: 1.5–2.4× C’s container peak during searches and 1.74× at the soak’s peak.
  • Soak throughput: 0.79× C’s; TPC-B-like 0.94×.

Why: causes and leads

A cause follows from the measurements. A lead is a likely explanation that nothing in this run measured on x86-64.

Headline tables

The report’s three headline blocks, with the matched-setting rows moved to the top. A matched setting runs both engines at the same ef_search or probes, with each one’s 1-client recall in brackets; where the recalls agree, it is the like-for-like comparison. A “recall ≥ R” row takes the first setting whose recall reaches R, or the highest setting if none does (“not reached”).

† The cell’s setting, or the one before it, has a recall within 0.005 of the target. Where both engines’ curves agree that closely, which setting first clears the bar can turn on a noise-level recall difference and swing the ratio up to 2×. Read the matched-setting rows instead.

¹ Container peak is the database container’s memory high-water mark during the step, including whatever it already held; the file cache (ZFS’s ARC) is outside it. The build increment is rep 1’s peak minus the previous step’s peak, a lower bound on what the build added.

Exact scans, inserts and plain SQL

Exact scan, before any index

Inserts under the HNSW index, and the DELETE + VACUUM after them

Plain SQL, 8 clients × 60 s

Limits of this measurement

Reproduce and evidence

The scripts are in scripts/bench/; its README lists the site constants and prerequisites. The full report, with every table and its derivation, is docs/superpowers/reports/bench-borg-phase1-2026-11-15.md. The run used orchestrator a2e6b06f3b and pgrust a9c5743a3ca35502680d990125eed590b176d50a.

scripts/bench/sync.sh
ssh -n borg2 bash ~/bench/scripts/borg.sh create-db pgrust-db-c      # once; also create-db pgrust-db-rust and create-client
ssh -n borg2 bash ~/bench/scripts/install-pg-c.sh
ssh -n borg2 'PGRUST_REF=a9c5743a3ca35502680d990125eed590b176d50a bash ~/bench/scripts/install-pgrust.sh'
ssh -n borg2 bash ~/bench/scripts/client/setup.sh
ssh -n borg2 bash ~/bench/scripts/client/datasets.sh glove-100-angular sift-128-euclidean
ssh -n borg2 'bash ~/bench/scripts/run-phase1.sh pg-c RUN_ID'     # 5.5 h
ssh -n borg2 'bash ~/bench/scripts/run-phase1.sh pgrust RUN_ID'   # 8.3 h, never overlapping
scp -r borg2:bench/results/RUN_ID ~/.cache/pgrust/bench-borg/
python3 scripts/bench/report.py ~/.cache/pgrust/bench-borg/RUN_ID