#Architecture

ByteOr is a four-layer stack where each layer has a clear contract with the layers around it. This page explains how the layers compose and where the boundaries sit.

#Stack Overview

┌─────────────────────────────────────────────┐
│               ByteOr Cloud                  │
│   Control plane: versions, bundles,         │
│   deployments, agents, artifacts, replay    │
├─────────────────────────────────────────────┤
│            ByteOr Enterprise                │
│   Runtime: governance, approvals, replay,   │
│   DataGuard, adapters, operator CLI         │
├─────────────────────────────────────────────┤
│              ByteOr OSS                     │
│   Spec model, validation, compilation,      │
│   deterministic execution, WASM lowering    │
├─────────────────────────────────────────────┤
│               IndexBus                      │
│   Bounded SHM transport, deterministic      │
│   lane semantics, zero-copy IPC             │
└─────────────────────────────────────────────┘

#IndexBus — Transport Substrate

IndexBus is the bounded shared-memory transport layer. It provides:

• Deterministic lane semantics — Events, Journal, SequencedSlots, and FanoutBroadcast lane kinds with well-defined capacity and backpressure behavior.
• Zero-copy IPC — Processes communicate through shared-memory regions with repr(C) layouts and no serialization overhead.
• Router and merge infrastructure — Fanout and fan-in patterns with bounded memory and deterministic ordering guarantees.

IndexBus is a standalone open-source project. It can be used independently of ByteOr for any application that needs bounded, deterministic inter-process communication.

#ByteOr OSS — Spec and Execution

ByteOr OSS defines the pipeline model:

• Spec format — Pipelines are defined in a structured KV format that is deterministic, canonicalizable, and diff-friendly.
• Validation — Specs are validated against structural and semantic rules before compilation.
• Compilation — Validated specs are lowered through a compiler-backed path into canonical execution plans.
• Execution — The kernel executes plans using IndexBus as the transport substrate, with backing implementations for shared memory and in-memory operation.

The OSS layer provides both SingleRing and LaneGraph execution models through a unified source authoring contract.

#ByteOr Enterprise — Governance and Operations

Enterprise adds the operator surface:

• Runtime CLI — validate, describe, dot, diff, run, doctor, snapshot, incident-bundle, replay, capabilities, compatibility.
• Policy and approvals — Environment-scoped approval requirements with cryptographic credential verification.
• Incident bundles and replay — Captured state for triage, dry-run investigation, and governed re-execution.
• DataGuard — Schema-aware data governance with registry-based validation, redaction, and audit reporting.
• Adapters — Transport-facing integrations for Kafka, NATS, Redis Streams, RabbitMQ, gRPC, HTTP, WebSocket, S3, and PostgreSQL.
• Product binaries — EdgePlane (adapter loops), ActionGraph (side-effect stages), DataGuard (protected data flows).

#ByteOr Cloud — Control Plane

Cloud provides the hosted SaaS experience:

• Organizations, projects, and environments — Multi-tenant resource hierarchy.
• Pipeline editor — Draft authoring with execution target selection, plan-based compilation, and version creation.
• Config bundles — Environment-specific runtime configuration with guided tuning profiles.
• Deployments — Version + bundle + environment targeting with approval gates.
• Agents — Environment-scoped agent enrollment, heartbeats, and runtime key management.
• Artifacts and incidents — Artifact storage, incident bundle management, and replay launch paths.
• Audit trails — Recorded history for approval-sensitive actions, deployments, and membership changes.

#Deployment Modes

ByteOr supports multiple deployment modes:

In all modes, the runtime executes on customer infrastructure. The control plane manages versions, bundles, and deployments but never touches the data path.

#Crate Layering

The ByteOr pipeline workspace is organized as a layered set of crates, each with a clear responsibility boundary:

┌─────────────────────────────────────────────────┐
│                byteor-pipeline                  │  ← Facade / start-here crate
│   (re-exports: plan, lower, spec, kernel, exec) │
└─────────────┬───────────────────────────────────┘
              │
    ┌─────────┴──────────┐
    │                    │
┌───▼────────────┐  ┌────▼───────────────┐
│ byteor-pipeline│  │ byteor-pipeline    │
│ -plan          │  │ -macros            │  ← Optional proc-macro DSL
│ (PipelinePlan  │  └────────────────────┘
│  V2 + validate)│
└───┬────────────┘
    │
┌───▼────────────────┐
│ byteor-pipeline    │
│ -lower             │  ← Explicit lowering: PipelinePlanV2 → PipelineSpecV1
│ (lower_v2)         │
└───┬────────────────┘
    │
┌───▼────────────────┐
│ byteor-pipeline    │
│ -spec              │  ← Stable contract: types, kv codec, validation, canonical hash
│ (PipelineSpecV1)   │
└───┬────────────────┘
    │
┌───▼────────────────┐
│ byteor-pipeline    │
│ -kernel            │  ← Hot-loop traits: LaneTx/LaneRx, step primitives, SeqBarrier
│ (IO-free, thread-  │     (no product policy)
│  free, log-free)   │
└───┬───────┬────────┘
    │       │
┌───▼───┐ ┌─▼──────────────────┐
│backings│ │ backings           │
│-mem    │ │ -shm               │  ← Attach-only SHM lane implementations
│(tests, │ │ (events, fanout,   │
│ replay)│ │  journal, slots)   │
└───┬────┘ └───┬────────────────┘
    │          │
    │    ┌─────▼─────────────┐
    │    │ byteor-transport  │
    │    │ -shm              │  ← OS-facing SHM open/create/validate
    │    │ (untrusted input) │
    │    └───────────────────┘
    │
┌───▼────────────────┐
│ byteor-pipeline    │
│ -exec              │  ← Executors: SingleRing SHM, LaneGraph SP/MP, stage resolvers
│ (runs loops)       │
└────────────────────┘

┌────────────────────┐
│ byteor-abi         │  ← repr(C) layouts, generated spec/byteor_v1.h
└────────────────────┘

┌────────────────────┐
│ byteor-pipeline    │
│ -wasm              │  ← Browser bindings: validate, lower, encode, decode, describe, diff
└────────────────────┘

#Data Flow

The recommended production path uses PipelinePlanV2 rather than constructing specs directly. The flow from author intent to running executor:

Author intent
    │
    ▼
PipelinePlanV2  ← byteor-pipeline-plan (validate_plan_v2)
    │
    ▼
lower_v2(plan, target, opts)  ← byteor-pipeline-lower
    │
    ▼
PipelineSpecV1::SingleRing    ← byteor-pipeline-spec (validate_v1)
  or ::LaneGraph
    │
    ▼
Attach backings               ← byteor-pipeline-backings-{mem,shm}
    │                            byteor-transport-shm
    ▼
Run executor                  ← byteor-pipeline-exec
    │
    ▼
Observe via snapshots/counters