IndexBus architecture overview (v1, non-normative)
This document gives a big-picture view of IndexBus as implemented in this repository. It is non-normative: for exact behavioral contracts, see the v1 spec.
Rustdoc entry points
For the API-level surface behind this overview, start with:
Reading order
- Start here for topology and roles.
- Then read the v1 spec:
- For production posture and blessed paths:
- For ops workflow:
Mental model
IndexBus is a bounded, shared-memory message transport built around fixed-capacity regions. It favors:
- explicit backpressure (
Full), - at-most-once delivery,
- predictable memory usage,
- minimal synchronization (Acquire/Release atomics, no global locks).
Core entities
- Region: a memory-mapped layout defined in
indexbus-abi. - Slot pool: fixed number of payload slots. Publishing consumes a slot; receiving frees it.
- Queues: bounded rings that move slot indices (not payload bytes).
- Participants:
- Producer publishes events into a queue.
- Consumer receives from a queue.
- Router is a dedicated step that moves indices from a producer→router queue into one or more consumer queues.
Common topologies
Events (direct queues)
- SPSC: one producer → one consumer.
- MPSC: many producers → one consumer.
Use when you want the lowest overhead and don’t need fanout.
Fanout (router-mediated)
Producer(s) publish into a producer→router queue. A router loop routes each message to:
- Broadcast: attempt delivery to each consumer queue (partial delivery allowed).
- WorkQueue: deliver to at most one consumer (selection is implementation-defined; v1 uses round-robin cursor).
Router-enforced credits optionally bound per-consumer backlog (and can drop/park/detach depending on policy).
State stream (overwrite-latest)
A single writer publishes the latest snapshot; readers sample it.
This is designed for “current state” distribution (configs, health, latest book, etc.), not event history.
Sequencer + gating sequences (control plane)
A sequencer region coordinates strict ordering and stage barriers:
- producer publishes cursor,
- consumers update gating sequences,
- wrap-prevention bounds the producer.
Payload storage is typically caller-owned (ring indexed by seq % capacity).
Process / thread placement (typical)
- Producers: application threads.
- Consumers: application threads.
- Router: dedicated thread/process (often one router per region; optionally per NUMA node).
What IndexBus does not do (by design)
- no durability,
- no retries/acks,
- no exactly-once,
- no fairness guarantees,
- no cross-host reliability.
Those are intentionally pushed to higher layers.