SQLStyle

Documentation

Database core and capture

Understand how the Bedrock database core feeds capture for WAL2 retention, page encoding, ring publication, and DBH handoff.

SQLStyle ships with the Bedrock build, which uses patched SQLite with WAL2. The database core is fixed by the artifact you install, not selected per database connection.

The Bedrock adapter defines how SQLStyle observes a commit, pins the preceding database view, and controls checkpoint or WAL2 reuse. It exposes one boundary to the capture ring, DBH format, replication protocol, and rsync implementation.

Commits publish work; capture performs it

While holding the native writer lock, the capture adapter maps the database core’s WAL position to a SQLStyle transaction, publishes the requested capture frontier, and wakes the capture worker. This path does not compress pages, write DBH files, or wait for a network operation.

Capture reads the WAL2 frames described by that frontier. Each published record carries the source page, SQLStyle transaction, commit boundary, database size at commit, and the encoding used for the page body. A transaction becomes available downstream without requiring the database core to implement replication or backup separately.

WAL reuse waits for capture completion

SQLStyle maintains requested and completed capture frontiers for each WAL slot. A requested frontier identifies source frames that capture still needs. The matching completed frontier advances only after those frames are represented by release-published records in the capture ring.

Checkpoint, WAL restart, truncation, generation changes, and slot reuse must observe that completion before destroying the covered bytes. This is the fence that closes the interval between SQLite publishing a commit in its WAL index and SQLStyle finishing physical capture.

If capture-host ownership changes during that interval, the successor validates the WAL identities, ring tail, and published frontiers before repairing or continuing progress. It fences the database when those sources do not describe one continuous history.

Page differences require an exact predecessor

For a transaction T, capture attempts to read each changed page from the database view at T - 1. It compares that page with the new WAL frame and emits a compact sequence of changed byte ranges when the result is expected to be smaller than a complete LZ4 page.

The predecessor is a correctness dependency. If the required view is busy, missing, belongs to a reused WAL generation, or otherwise cannot be proven, capture uses a self-contained LZ4 page for the transaction. It does not wait or retry in the capture path, and it never constructs a difference against an uncertain page.

Both encodings carry the same transaction metadata. Replication can send them without re-encoding, and DBH can persist them with the page’s encoding recorded in the file.

The ring has one shared publication edge

Encoded records enter a fixed-size circular buffer in the database’s shared memory. The producer’s reservation and write positions remain private to the capture host. A release-published ring edge is the shared statement that a complete record, including its source trailer, is available to readers.

Scratch bytes written before that publication are irrelevant after a crash. A same-boot successor starts from the published edge, validates the last complete record, and walks backward to the retained DBH boundary to reconstruct the WAL and transaction positions it inherits.

DBH generation and attached live replicas consume published records for different purposes. DBH provides the required local history consumer; replication cursors are revocable and are covered in replication and catch-up.

DBH makes the ring suffix recoverable

Two DBH workers alternate at transaction boundaries. One appends records from the ring to the active segment while the other finalizes the preceding segment. The active worker flushes the visible bytes and publishes a checksummed DBH prefix before retiring the corresponding ring records.

The ring edge and DBH prefix are exact same-boot handoff evidence. A replacement capture host resumes from those boundaries instead of inferring a transaction from the newest database page. Finalized DBH segments then enter the durable catalog and backup path described in backup and recovery history.