Skip to content

Durability

This is the page to read before you design anything on top of this database.

A 200 on a write means the bytes covering that write are in object storage. Not in a buffer, not on one machine’s disk, not queued for replication. If every machine we run were destroyed the instant after you received the response, that write would still be there when a replacement came up.

The response says so explicitly:

{ "durability": "durable", "epoch": 3, "txid": 412, "waitedMs": 118 }
Field
durability "durable" for a write that reached object storage. "read" for a request that wrote nothing.
txid The durable high-water mark at acknowledgement. null for a read.
epoch Which ownership generation served the request. Changes when a database moves to a different worker.
waitedMs How long the request spent waiting for object storage. Effectively 0 for reads.

The same three facts come back as headers, so a client that only reads headers does not have to parse the body: X-Durability, X-Durable-TXID, and X-Database-Epoch.

Measured on the deployed stack, not estimated:

Measurement p50 p95
Durable write, one transaction per batch 119 ms 157 ms
Read served by the owning worker < 1 ms < 1 ms
A raw 4 KB PUT to the same bucket 32 ms 38 ms

Note the third row. Object storage answers a single PUT in about 32 ms, and yet a commit takes 119. The gap is not overhead — a commit is two sequential round trips:

  1. Write the segment. An immutable object holding the transactions in this batch.
  2. Advance the pointer. A conditional write publishing the new durable high-water mark.

Your acknowledgement waits on the second one. It has to: a worker that took over later and read a stale high-water mark would be entitled to acknowledge less than this worker already had, which is exactly how an acknowledged write gets lost. So the pointer moves before you are told the write is durable.

Two things follow that you can act on:

  • Batch aggressively. The cost is per batch, not per statement. Ten inserts in one batch cost about 120 ms; ten separate requests cost about 1.2 seconds.
  • Concurrent writers amortize it for free. Requests arriving together are committed into one segment and share both round trips. 119 ms is the worst case — one lonely transaction — rather than the typical one under load.

Everything above describes success. There is exactly one result in this API that means we do not know:

{
"type": "https://errors.sqlited.dev/durability-uncertain",
"title": "Durability uncertain",
"status": 503,
"detail": "the write may or may not have become durable; retry an idempotent request, or read back to check",
"retryable": true
}

It happens when the segment reached object storage but ownership could no longer be proven at the moment of acknowledgement. The data may be there. It may not be. Nobody, including us, can tell you which from outside.

What to do: retry if the request is idempotent, or read back and check. That is the complete advice.

Why it is not hidden: reporting it as success would be a lie, and reporting it as a 4xx would tell you the write definitely failed — which is the one thing we do not know. Every system that stores data has this state; most of them return 500 and let you guess.

Design for it early. A write with a client-generated unique id is safely retryable, and that single habit turns this error from a hard problem into a retry:

INSERT INTO events (id, kind) VALUES (?, ?) ON CONFLICT (id) DO NOTHING

Exactly one worker owns a database at a time. That is what makes single-writer SQLite safe here, and it is enforced by a compare-and-swap on a pointer object rather than by a lock anyone has to trust.

When ownership moves — a deploy, a restart, a host failure — the new owner takes a new epoch and writes under its own prefix. You will observe this as:

  • X-Database-Epoch increasing, which is normal and needs no action;
  • possibly one fenced error (503, retryable), which means the worker you reached no longer owns the database. Retry it; the next request opens a fresh handle.

An unclean restart costs about 40 seconds before the database serves again: the new owner waits out the previous lease, then replays the epoch’s segments. A clean restart is about 300 ms, because a graceful shutdown leaves a marker that lets the local copy be reused. Nothing is lost either way — that is P1 and P4, and it is verified by a crash gate that ran 10,000 crash injections against 105,268 acknowledged writes and recovered every one.

  • Your own mistakes. DELETE FROM users with no WHERE is durable too. There is no undo and no point-in-time restore yet.
  • Reads. A read can observe a write that was never acknowledged to anyone. See Consistency — it is a short page and this is the surprising part of it.