Development Documentation (main branch) - For stable release docs, see docs.rs/eidetica

Module sync

eidetica

Module sync 

Source
Expand description

Synchronization module for Eidetica database.

§Quick Start

ⓘ
// Enable sync
instance.enable_sync().await?;
let sync = instance.sync().unwrap();

// Register transports with their configurations
sync.register_transport("http", HttpTransport::builder()
    .bind("127.0.0.1:8080")
).await?;
sync.register_transport("p2p", IrohTransport::builder()).await?;

// Start accepting incoming connections
sync.accept_connections().await?;

// Outbound sync works via the registered transports
sync.sync_with_peer(&Address::http("peer:8080"), Some(&tree_id)).await?;

§Architecture

The sync system uses a Background Sync architecture with command-pattern communication:

  • Sync: Thread-safe frontend using Arc<Sync> with interior mutability (OnceLock). Provides the public API and sends commands to the background.
  • background::BackgroundSync: Single background thread handling all sync operations via a command loop. Owns the transport and retry queue.
  • Write Callbacks: Automatically trigger sync when entries are committed via database write callbacks.

§Connection Model

The sync system separates outbound and inbound connection handling:

This design provides security by default. Nodes don’t accept incoming connections unless explicitly opted in.

§Bootstrap Protocol

The sync protocol detects whether a client needs bootstrap or incremental sync:

  • Empty tips → Full bootstrap (complete database transfer)
  • Has tips → Incremental sync (only missing entries)

Use Sync::sync_with_peer which handles both cases automatically.

§Duplicate Prevention

Uses Merkle-DAG tip comparison instead of tracking individual sent entries:

  1. Exchange tips with peer
  2. Compare DAGs to find missing entries
  3. Send only what peer doesn’t have
  4. Receive only what we’re missing

This approach requires no extra storage apart from tracking relevant Merkle-DAG tips.

§Transport Layer

Two transport implementations are available:

Transports are registered via Sync::register_transport with their builders. State (like Iroh node identity) is automatically persisted per named instance. Both implement the transports::SyncTransport trait.

§Peer and State Management

Peers and sync relationships are stored in a dedicated sync database (_sync):

  • peer_manager::PeerManager: Handles peer registration and relationships
  • state::SyncStateManager: Tracks sync cursors, metadata, and history

§Connection Behavior

  • Lazy connections: Established on-demand, not at peer registration
  • Periodic sync: Configurable interval (default 5 minutes)
  • Retry queue: Failed sends retried with exponential backoff

Re-exports§

pub use error::SyncError;
pub use peer_types::Address;
pub use peer_types::ConnectionState;
pub use peer_types::PeerId;
pub use peer_types::PeerInfo;
pub use peer_types::PeerStatus;
pub use ticket::DatabaseTicket;

Modules§

background
Background sync engine implementation.
error
Error types for the synchronization module.
handler
Sync request handler trait and implementation.
peer_manager
Internal peer management for the sync module.
peer_types
Peer management types for the sync module.
protocol
Protocol definitions for sync communication.
state
Sync state tracking for managing synchronization progress and metadata.
ticket
URL-based shareable database link.
transports
Transport abstractions for sync communication.
utils
Sync-specific utility functions.

Structs§

AuthParams
Authentication parameters for sync operations.
BootstrapRequest
A bootstrap request awaiting manual approval
Sync
Synchronization manager for the database.
SyncHandle
Handle for tracking sync status with a specific peer.
SyncPeerInfo
Information needed to register a peer for syncing.
SyncStatus
Current sync status for a tree/peer pair.

Enums§

RequestStatus
Status of a bootstrap request