tools/loadtest/published/2026-10-26/. The
package holds every request of every run (runs/<run>/requests.ndjson.gz), each run’s statistics and
configuration, and at the top summary.json and config.json. tools/loadtest/README.md documents the
driver, the datasets and the cache states. tools/loadtest/package.sh is the script that produced the
package.
All figures are milliseconds. Every latency is measured from when its request was due (open loop) or sent
(closed loop), over every request sent, so a failure counts at the time it took to fail and a timeout at
30 s.
Stack
Each run’s
host.json records the listings by state before and after it: every listing of the catalog was
published throughout. The host is shared, so the package was run with LOAD_WAIT_BELOW=2: each run started
only once the host’s 1-minute load average was below 2 (it was between 0.25 and 1.98 when the runs started).
host.json records each run’s wait (at most 240 s) and the load average before and after it.
Cached and uncached search
Open loop, 120 s after a 30 s warm-up.warm: every query’s embedding is cached. uncached: every query is
new, so each runs the embedding model, or is answered lexical-only with partialResults: true when the
embedder is at its cap (saturated). cold-index: a listing is added every 10 s, and the first answer at
each new catalog revision includes the index rebuild.
The
warm (search) and warm (list page) rows are one run: 80 % searches and 20 % /discovery/resources
pages at 100 requests a second. With uncached at 100/s on catalog A every query ran the model; at 300/s
the embedder was at its cap for 18,530 of 36,000 queries, which were answered lexical-only. On catalog B,
partialResults also marks warm answers whose semantic ranking was cut at its depth (about a third), and
at 100/s the embedder was at its cap for 3,225 of the 12,000 uncached queries.
A /health probe at 5/s ran beside the warm and uncached runs: p50/p95/p99 1.9/4.9/9.1 beside warm
50/s, 1.6/7.9/9.9 beside uncached 100/s and 4.8/8.0/9.2 beside uncached 300/s on catalog A;
6.7/10.4/11.5, 10.9/17.3/24.8 and 57.5/153.1/219.5 on catalog B. Every probe answered 200.
Over capacity
uncached at 300/s on catalog B is above what the service completes (see the curve below). The service
sheds discovery load (see Load limits): at most 32 discovery requests run at
once and 64 wait, each at most 1 s, and every other request is refused at once with 503
discovery_overloaded. Discovery reads the catalog through a database pool of its own, and a page reuses
listings read within the last second at the same catalog revision.
The service answered 33,802 searches (282 a second, about the 285 a second of the 64-client closed loop) and
refused 2,198 with
503 (median 29 ms to refuse), with no 500 and no timeout. 30,991 of the answered
searches were lexical-only (saturated, partialResults: true). Every /health probe succeeded. The driver
records statuses, not error codes; the service’s log shows no error during the runs (one warning per stack:
the corpus’s testnet assets are absent from the private network).
Concurrency curve
Closed loop, 60 s after a 10 s warm-up: each client sends its next request when its previous one answers. Throughput is completed requests per second; latency is service time at that throughput.
No closed-loop request failed. Past 4 clients, uncached throughput rises only because the embedder’s cap
answers most queries lexical-only (
Saturated).
Settlement under a search flood
Realexact payments settled through POST /settle of the same process, 0.9 a second for 120 s, alone and
beside an uncached flood, on each catalog. Every settlement succeeded.
Settle latency on the private network has two modes, about one and about two ledger intervals, and the
share of each differs from run to run, which moves p50 between them. Within each mode the flood’s cost
shows: on catalog A the fast mode’s median was 1,206 alone, 1,281 beside 100/s and 1,469 beside 300/s, the
slow mode’s 2,214, 2,295 and 2,484. On catalog B, where each uncached search takes more of the service’s
two CPUs, the flood costs more: 47 of 108 settlements fell in the slow mode alone (median 2,217), 97 beside
100/s (median 2,579) and 106 beside 300/s (median 2,928). The private network closes ledgers at its own
pace, so the absolute figures are not those of a public network.
Reproduce
tools/loadtest/published/<date>/.