Eidetica Documentation

Welcome to the official documentation for Eidetica - a decentralized database built on Merkle-CRDT principles with built-in peer-to-peer synchronization.

Key Features

• Decentralized Architecture: No central server required - peers connect directly
• Conflict-Free Replication: Automatic merge resolution using CRDT principles
• Content-Addressable Storage: Immutable, hash-identified data entries
• Real-time Synchronization: Background sync with configurable batching and timing
• Multiple Transport Protocols: HTTP and Iroh P2P with NAT traversal
• Local Daemon Mode: Share an Instance across processes via Unix socket RPC
• Async-First API: Built on Tokio for efficient, non-blocking operations
• Authentication & Security: Ed25519 signatures for all operations
• Flexible Data Models: Support for documents, key-value, and structured data

Project Structure

Eidetica is organized as a Cargo workspace:

• Library (crates/lib/): The core Eidetica library crate
• CLI Binary (crates/bin/): Command-line interface using the library
• Examples (examples/): Standalone applications demonstrating usage

Quick Links

Documentation Sections

• User Guide: Learn how to use the Eidetica library
• Getting Started: Set up your first Eidetica database
• Synchronization Guide: Enable peer-to-peer sync
• Database Sharing: Manage user intent and create point-in-time locators
• Internal Documentation: Understand the internal design and contribute
• Design Documents: Architectural documents used for development

Examples

• Chat Application - Complete multi-user chat with P2P sync
• Todo Application - Task management example

User Guide

Welcome to the Eidetica User Guide. This guide will help you understand and use Eidetica effectively in your applications.

What is Eidetica?

Eidetica is a Rust library for managing structured data with built-in history tracking. It combines concepts from distributed systems, Merkle-CRDTs, and traditional databases to provide a unique approach to data management:

• Efficient data storage with customizable Databases
• History tracking for all changes via immutable Entries forming a DAG
• Structured data types via named, typed Stores within logical Databases
• Atomic changes across multiple data structures using Transactions

How to Use This Guide

This user guide is structured to guide you from basic setup to advanced concepts:

1. Getting Started: Installation, basic setup, and your first steps.
2. Basic Usage Pattern: A quick look at the typical workflow.
3. Core Concepts: Understand the fundamental building blocks:
   • Entries & Databases: The core DAG structure.
   • Databases: How data is stored.
   • Stores: Where structured data lives (DocStore, Table, YDoc).
   • Transactions: How atomic changes are made.
4. Service (Daemon) Mode: Share an Instance across processes via a local daemon.
5. Database Sharing: Save user settings, then create a point-in-time locator separately.
6. Tutorial: Todo App: A step-by-step walkthrough using a simple application.
7. Code Examples: Focused code snippets for common tasks.

Quick Overview: The Core Flow

Eidetica revolves around a few key components working together:

1. Database: You start by choosing or creating a storage Database (e.g., SQLite).
2. Instance: You create a Instance instance, providing it the Database. This is your main database handle.
3. Database: Using the Instance, you create or load a Database, which acts as a logical container for related data and tracks its history.
4. Transaction: To read or write data, you start a Transaction from the Database. This ensures atomicity and consistent views.
5. Store: Within a Transaction, you get handles to named Stores (like DocStore or Table<YourData>). These provide methods (set, get, insert, remove, etc.) to interact with your structured data.
6. Commit: Changes made via Store handles within the Transaction are staged. Calling commit() on the Transaction finalizes these changes atomically, creating a new historical Entry in the Database.

Basic Usage Pattern

Here’s a quick example showing creating a user, database, and writing new data.

extern crate eidetica;
extern crate tokio;
extern crate serde;
use eidetica::{Instance, NewAccount, crdt::Doc, store::{DocStore, Table}};
use serde::{Serialize, Deserialize};

#[derive(Serialize, Deserialize, Clone, Debug)]
struct MyData {
    name: String,
}

#[tokio::main]
async fn main() -> eidetica::Result<()> {
// Open an Instance by URL — `memory://` is ephemeral; use `sqlite://./app.db`
// for persistent storage. See the backends concepts page for the URL grammar.
let (instance, maybe_user) =
    Instance::connect_or_create("memory://", NewAccount::passwordless("alice")).await?;
let mut user = maybe_user.expect("memory:// is always fresh");


// Create a database
let mut settings = Doc::new();
settings.set("name", "my_database");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;

// --- Writing Data ---
// Start a Transaction
let txn = database.new_transaction().await?;
let inserted_id = { // Scope for store handles
    // Get Store handles
    let config = txn.get_store::<DocStore>("config").await?;
    let items = txn.get_store::<Table<MyData>>("items").await?;

    // Use Store methods
    config.set("version", "1.0").await?;
    items.insert(MyData { name: "example".to_string() }).await?
}; // Handles drop, changes are staged in txn
// Commit changes
let new_entry_id = txn.commit().await?;
println!("Committed changes, new entry ID: {}", new_entry_id);

// --- Reading Data ---
// Use Database::get_store_viewer for a read-only view
let items_viewer = database.get_store_viewer::<Table<MyData>>("items").await?;
if let Ok(item) = items_viewer.get(&inserted_id).await {
   println!("Read item: {:?}", item);
}
Ok(())
}

See Transactions and Code Examples for more details.

Project Status

Eidetica is currently under active development. The core functionality is working, but APIs are considered experimental and may change in future releases. It is suitable for evaluation and prototyping, but not yet recommended for production systems requiring long-term API stability.

Getting Started

This guide will walk you through the basics of using Eidetica in your Rust applications. We’ll cover the essential steps to set up and interact with the database.

For contributing to Eidetica itself, see the Contributing guide.

Installation

Add Eidetica to your project dependencies:

[dependencies]
eidetica = "0.1.0"  # Update version as appropriate
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }
# Or if using from a local workspace:
# eidetica = { path = "path/to/eidetica/crates/lib" }

Eidetica uses an async-first API built on Tokio. All database operations are async and must be called within a Tokio runtime.

Setting up the Database

To start using Eidetica, you need to:

1. Choose and initialize a Backend (storage mechanism)
2. Create an Instance (the infrastructure manager)
3. Create an Account and log in (authentication and session)
4. Create or access a Database through the AccountSession (logical container for data)

Here’s a simple example:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, NewAccount, crdt::Doc};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
    // Open or initialise an Instance from a URL. `memory://` is an
    // ephemeral in-process backend — great for tests and embedded use.
    // The first user created on a fresh instance is automatically granted
    // Admin on the system databases.
    let (_instance, maybe_user) = Instance::connect_or_create(
        "memory://",
        NewAccount::passwordless("alice"),
    ).await?;
    let mut user = maybe_user.expect("memory:// is always fresh");

    // Create a database in the user's context
    let mut settings = Doc::new();
    settings.set("name", "my_database");

    // Get the default key (earliest created key)
    let default_key = user.get_default_key()?;
    let _database = user.create_database(settings, &default_key).await?;

    Ok(())
}

Note: NewAccount::passwordless("alice") is sugar for NewAccount::with_password("alice", ""). All credentials use the encrypted bundle format, but an empty password provides no meaningful confidentiality against a bundle holder. Use a nonempty password when confidentiality is needed.

The URL passed to Instance::connect_or_create selects the backend. Some other common forms:

• memory:// — ephemeral in-process backend (above).
• memory:///abs/path/snap.json — in-process backend with a JSON snapshot file (load-on-start, writes via Instance::flush).
• sqlite://./my_data.db — embedded SQLite database, persisted automatically. The URL is passed through to sqlx, so any sqlx-accepted form works (?journal_mode=WAL, ?busy_timeout=5000, etc.).
• postgres://user:pwd@host/db — embedded PostgreSQL backend.
• unix:///run/eidetica/service.sock — connect to a running daemon.

For persistent storage, swap the URL:

use eidetica::{Instance, NewAccount, crdt::Doc};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
    // First run: bootstrap a fresh persistent instance.
    // Subsequent runs: load the existing instance (Option<AccountSession> is None).
    let (instance, maybe_user) = Instance::connect_or_create(
        "sqlite://./my_data.db",
        NewAccount::passwordless("alice"),
    ).await?;

    let mut user = match maybe_user {
        Some(u) => u,
        None => instance.login_account("alice", "").await?,
    };
    // ... all changes are automatically persisted to my_data.db

    Ok(())
}

When you only want to load an already-initialised database (strict — errors if missing), use Instance::connect:

use eidetica::Instance;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
    // Strict load — errors if the database isn't initialised yet.
    let instance = Instance::connect("sqlite://./my_data.db").await?;
    let user = instance.login_account("alice", "").await?;
    // ... data persists across restarts

    Ok(())
}

Session-Centric Architecture

Eidetica uses a user-centric architecture:

• Instance: Manages infrastructure (user accounts, backend, system databases)
• AccountSession: Handles all contextual operations (database creation, key management)

All database and key operations happen through an AccountSession after login. This provides:

• Clear separation: Infrastructure management vs. contextual operations
• Strong isolation: Each user has separate keys and preferences
• Uniform credential storage: Every password string, including empty, uses the same authenticated encrypted bundle

Passwordless Users (embedded/single-user apps):

let admin = instance.login_account("admin", "").await?;
admin.admin().await?.create_account(eidetica::NewAccount::passwordless("alice")).await?;
let user = instance.login_account("alice", "").await?;

Password-Protected Users (multi-user apps):

let admin = instance.login_account("admin", "").await?;
admin.admin().await?.create_account(eidetica::NewAccount::with_password("bob", "password123")).await?;
let user = instance.login_account("bob", "password123").await?;

Every login, including an empty-password login, derives the wrapping key with Argon2id and decrypts the credential bundle. Signing uses the already unlocked keys, without repeating password entry or derivation.

Working with Data

Eidetica uses Stores to organize data within a database. One common store type is Table, which maintains a collection of items with unique IDs.

Defining Your Data

Any data you store must be serializable with serde:

Basic Operations

All operations in Eidetica happen within an atomic Transaction:

Inserting Data:

extern crate eidetica;
extern crate tokio;
extern crate serde;
use eidetica::{backend::database::Sqlite, Instance, crdt::Doc, store::Table, Database};
use serde::{Serialize, Deserialize};

#[derive(Clone, Debug, Serialize, Deserialize)]
struct Person {
    name: String,
    age: u32,
}

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;

// Start an authenticated transaction
let txn = database.new_transaction().await?;

// Get or create a Table store
let people = txn.get_store::<Table<Person>>("people").await?;

// Insert a person and get their ID
let person = Person { name: "Alice".to_string(), age: 30 };
let _id = people.insert(person).await?;

// Commit the changes (automatically signed with the user's key)
txn.commit().await?;
Ok(())
}

Reading Data:

extern crate eidetica;
extern crate tokio;
extern crate serde;
use eidetica::{backend::database::Sqlite, Instance, crdt::Doc, store::Table, Database};
use serde::{Serialize, Deserialize};

#[derive(Clone, Debug, Serialize, Deserialize)]
struct Person {
    name: String,
    age: u32,
}

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
// Insert some test data
let txn = database.new_transaction().await?;
let people = txn.get_store::<Table<Person>>("people").await?;
let test_id = people.insert(Person { name: "Alice".to_string(), age: 30 }).await?;
txn.commit().await?;
let id = &test_id;

let txn = database.new_transaction().await?;
let people = txn.get_store::<Table<Person>>("people").await?;

// Get a single person by ID
if let Ok(person) = people.get(id).await {
    println!("Found: {} ({})", person.name, person.age);
}

// Search for all people (using a predicate that always returns true)
let all_people = people.search(|_| true).await?;
for (id, person) in all_people {
    println!("ID: {}, Name: {}, Age: {}", id, person.name, person.age);
}
Ok(())
}

Updating Data:

extern crate eidetica;
extern crate tokio;
extern crate serde;
use eidetica::{backend::database::Sqlite, Instance, crdt::Doc, store::Table, Database};
use serde::{Serialize, Deserialize};

#[derive(Clone, Debug, Serialize, Deserialize)]
struct Person {
    name: String,
    age: u32,
}

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
// Insert some test data
let txn_setup = database.new_transaction().await?;
let people_setup = txn_setup.get_store::<Table<Person>>("people").await?;
let test_id = people_setup.insert(Person { name: "Alice".to_string(), age: 30 }).await?;
txn_setup.commit().await?;
let id = &test_id;

let txn = database.new_transaction().await?;
let people = txn.get_store::<Table<Person>>("people").await?;

// Get, modify, and update
if let Ok(mut person) = people.get(id).await {
    person.age += 1;
    people.set(id, person).await?;
}

txn.commit().await?;
Ok(())
}

Deleting Data:

extern crate eidetica;
extern crate tokio;
extern crate serde;
use eidetica::{backend::database::Sqlite, Instance, crdt::Doc, store::Table, Database};
use serde::{Serialize, Deserialize};

#[derive(Clone, Debug, Serialize, Deserialize)]
struct Person {
    name: String,
    age: u32,
}

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
let _id = "test_id";

let txn = database.new_transaction().await?;
let people = txn.get_store::<Table<Person>>("people").await?;

// FIXME: Table doesn't currently support deletion
// You can overwrite with a "deleted" marker or use other approaches

txn.commit().await?;
Ok(())
}

Complete Examples

For complete working examples, see:

• Chat Example - Multi-user chat application demonstrating:

  • Accounts and authentication
  • Real-time synchronization with HTTP and Iroh transports
  • Bootstrap protocol for joining rooms
  • TUI interface with Ratatui

• Todo Example - Task management application

Next Steps

After getting familiar with the basics, you might want to explore:

• Core Concepts to understand Eidetica’s unique features
• Synchronization Guide to set up peer-to-peer data sync
• Authentication Guide for secure multi-user applications
• Service (Daemon) Mode to share an Instance across multiple processes
• Advanced operations like querying and filtering
• Using different store types for various data patterns

Core Concepts

Understanding the fundamental ideas behind Eidetica will help you use it effectively and appreciate its unique capabilities.

Architectural Foundation

Eidetica builds on several powerful concepts from distributed systems and database design:

1. Content-addressable storage: Data is identified by the hash of its content, similar to Git and IPFS
2. Directed acyclic graphs (DAGs): Changes form a graph structure rather than a linear history
3. Conflict-free replicated data types (CRDTs): Data structures that can merge concurrent changes automatically
4. Immutable data structures: Once created, data is never modified, only new versions are added

These foundations enable Eidetica’s key features: robust history tracking, efficient synchronization, and eventual consistency in distributed environments.

Merkle-CRDTs

Eidetica is inspired by the Merkle-CRDT concept from OrbitDB, which combines:

• Merkle DAGs: A data structure where each node contains a cryptographic hash of its children, creating a tamper-evident history
• CRDTs: Data types designed to resolve conflicts automatically when concurrent changes occur

In a Merkle-CRDT, each update creates a new node in the graph, containing:

1. References to parent nodes (previous versions)
2. The updated data
3. Metadata for conflict resolution

This approach allows for:

• Strong auditability of all changes
• Automatic conflict resolution
• Efficient synchronization between replicas

Data Model Layers

Eidetica organizes data in a layered architecture:

+-----------------------+
| User Application      |
+-----------------------+
| Instance              |
+-----------------------+
| Databases             |
+----------+------------+
| Stores | Operations   |
+----------+------------+
| Entries (DAG)         |
+-----------------------+
| Database Storage      |
| (local or daemon RPC) |
+-----------------------+

Each layer builds on the ones below, providing progressively higher-level abstractions:

1. Database Storage: Physical storage of data (SQLite, PostgreSQL, InMemory, or remote via daemon mode)
2. Entries: Immutable, content-addressed objects forming the database’s history
3. Databases & Stores: Logical organization and typed access to data
4. Operations: Atomic transactions across multiple stores
5. Instance: The top-level database container and API entry point

Entries and the DAG

At the core of Eidetica is a directed acyclic graph (DAG) of immutable Entry objects:

• Each Entry represents a point-in-time snapshot of data and has:

  • A unique CID derived from its content (making it content-addressable)
  • Links to parent entries via CID references (forming the graph structure)
  • Data payloads organized by store
  • Metadata for database and store relationships

• The DAG enables:

  • Full history tracking (nothing is ever deleted)
  • Efficient verification of data integrity
  • Conflict resolution when merging concurrent changes

IPLD Data Model

Eidetica uses the IPLD (InterPlanetary Linked Data) data model for content addressing and serialization:

• CIDs (Content Identifiers): Every entry is identified by a CID — a self-describing hash that encodes the hash algorithm, the content codec (DAG-CBOR), and the hash digest. CID strings look like bafyr4i... (base32lower encoding).
• DAG-CBOR: Entries are serialized using DAG-CBOR, a deterministic subset of CBOR that ensures identical content always produces identical bytes (and therefore identical CIDs).
• Multihash: The CID format carries the hash algorithm in a self-describing prefix via the multihash specification. BLAKE3 is the hash algorithm used throughout eidetica — a database must use a single hash algorithm end-to-end, because parent pointers inside entries reference CIDs that must match the algorithm used to key entries in storage. Cross-algorithm trees (e.g., bootstrapping a pre-BLAKE3 database from a peer) are not supported without explicit multi-CID-per-entry support in the backend.

This aligns Eidetica with the broader content-addressed ecosystem (IPFS, Filecoin, libp2p) and makes entries interoperable with IPLD-aware tools. BLAKE3 is also used as the content-addressing hash by iroh, the P2P transport Eidetica uses for sync.

Stores: A Core Innovation

Eidetica extends the Merkle-CRDT concept with Stores, which partition data within each Entry:

• Each store is a named, typed data structure within a Database
• Stores can use different data models and conflict resolution strategies
• Stores maintain their own history tracking within the larger Database

This enables:

• Type-safe, structure-specific APIs for data access
• Efficient partial synchronization (only needed stores)
• Modular features through pluggable stores
• Atomic operations across different data structures

Planned future stores include:

• Object Storage: Efficiently handling large objects with content-addressable hashing
• Backup: Archiving database history for space efficiency

Atomic Operations and Transactions

All changes in Eidetica happen through atomic Transactions:

1. A Transaction is created from a Database
2. Stores are accessed and modified through the Transaction
3. When committed, all changes across all stores become a single new Entry
4. If the Transaction fails, no changes are applied

This model ensures data consistency while allowing complex operations across multiple stores.

Settings as Stores

In Eidetica, even configuration is stored as a store:

• A Database’s settings are stored in a special “settings” Store internally that is hidden from regular usage
• This approach unifies the data model and allows settings to participate in history tracking

CRDT Properties and Eventual Consistency

Eidetica is designed with distributed systems in mind:

• All data structures have CRDT properties for automatic conflict resolution
• Different store types implement appropriate CRDT strategies:
  • DocStore uses structural merge by default: concurrent writes to the same key use last-writer-wins (LWW), while writes to different keys are combined. Docs can be marked as atomic to use full LWW replacement, where the entire document replaces its predecessor instead of merging field-by-field. This is used for data that must be treated as a complete unit.
  • Table preserves all items, with LWW for updates to the same item

These properties ensure that when Eidetica instances synchronize, they eventually reach a consistent state regardless of the order in which updates are received.

History Tracking and Time Travel

One of Eidetica’s most powerful features is comprehensive history tracking:

• All changes are preserved in the Entry DAG
• “Tips” represent the latest state of a Database or Store
• Historical states can be reconstructed by traversing the DAG

This design allows for future capabilities like:

• Point-in-time recovery
• Auditing and change tracking
• Historical queries and analysis
• Branching and versioning

Entry Verification and the Verified Frontier

Every entry carries a verification status that records whether this node has cryptographically validated it against the authentication settings it was signed under:

• Verified — the entry’s signature and permissions were checked and accepted by a local validation pass.
• Unverified — the entry is stored but not yet validated by this node. This is the state every entry arrives in over sync, and it is the only state the storage layer will store on put: a peer can never assert “this entry is verified” — verification is always a local decision.
• Failed — the entry was checked and definitively rejected (bad signature, unauthorized key). Failed entries are dropped from reads everywhere.

Entries you create locally are validated as part of the commit and become Verified immediately. Entries received from a peer start Unverified; Database::verify() re-validates them against the _settings they pin, and a normal read also triggers an opportunistic verification pass when a tip is still Unverified.

Verification is prefix-closed: an entry can only be Verified if its entire ancestor history is Verified. It is therefore impossible for a tip to be Verified while one of its ancestors is not, and a Failed ancestor taints its descendants (they become Failed too).

Because of this, a Database handle exposes only the Verified frontier by default: the largest ancestor-closed prefix of the DAG in which every entry is Verified. Anything reachable only through a still-Unverified entry is hidden, so a default read never reflects state this node could not authenticate. Call .allow_unverified() on the handle to opt into the looser view that also includes Unverified entries (Failed is always dropped).

Current Status and Roadmap

Eidetica is under active development, and some features mentioned in this documentation are still in planning or development stages. Here’s a summary of the current status:

Implemented Features

• Core Entry and Database structure
• SQLite and PostgreSQL persistent storage backends (InMemory available for testing)
• DocStore, Table, and YDoc (Y-CRDT) store implementations
• PasswordStore for transparent password-based encryption wrapping any store
• CRDT functionality:
  • Doc (hierarchical nested document structure with recursive merging and tombstone support)
• Atomic operations across stores
• Tombstone support for proper deletion handling in distributed environments
• Signature-based authentication with crypto-agile key types and granular permissions
• Per-entry verification status with prefix-closed verification and Verified-frontier reads
• Account identity and session management with encrypted key storage
• Peer-to-peer synchronization via Iroh and HTTP transports
• Local service (daemon) mode for multi-process shared access over Unix sockets

Planned Features

• Object Storage store for efficient handling of large objects
• Backup store for archiving database history
• IPFS-compatible addressing for distributed storage
• Point-in-time recovery
• Sparse and shallow checkouts

This roadmap is subject to change as development progresses. Check the project repository for the most up-to-date information on feature availability.

Entries & Databases

The basic units of data and organization in Eidetica.

Entries

Entries are the fundamental building blocks in Eidetica. An Entry represents an atomic unit of data with the following characteristics:

• Content-addressable: Each entry has a unique ID (a CID) derived from its content, similar to Git commits. The ID is computed by serializing the entry to DAG-CBOR and hashing.
• Immutable: Once created, entries cannot be modified.
• Parent references: Entries maintain references to their parent entries, forming a directed acyclic graph (DAG).
• Database association: Each entry belongs to a database and can reference parent entries within both the main database and stores.
• Store data: Entries can contain data for one or more stores, representing different aspects or types of data.

Entries function similar to commits in Git - they represent a point-in-time snapshot of data with links to previous states, enabling history tracking.

Databases

A Database in Eidetica is a logical container for related entries, conceptually similar to:

• A traditional database containing multiple tables
• A branch in a version control system
• A collection in a document database

Key characteristics of Databases:

• Root Entry: Each database has a root entry that serves as its starting point.
• Named Identity: Databases typically have a name stored in their settings store.
• History Tracking: Databases maintain the complete history of all changes as a linked graph of entries.
• Store Organization: Data within a database is organized into named stores, each potentially using different data structures.
• Atomic Operations: All changes to a database happen through transactions, which create new entries.

Database Transactions

You interact with Databases through Transactions:

extern crate eidetica;
extern crate tokio;
use eidetica::{backend::database::Sqlite, Instance, crdt::Doc, store::DocStore, Database};

use eidetica::Result;

async fn example(database: Database) -> Result<()> {
    // Create a new transaction
    let txn = database.new_transaction().await?;

    // Access stores and perform actions
    let settings = txn.get_store::<DocStore>("settings").await?;
    settings.set("version", "1.2.0").await?;

    // Commit the changes, creating a new Entry
    let new_entry_id = txn.commit().await?;

    Ok(())
}

#[tokio::main]
async fn main() -> Result<()> {
    let backend = Sqlite::in_memory().await?;
    let (instance, mut user) = eidetica::Instance::create_backend(
        Box::new(backend),
        eidetica::NewAccount::passwordless("alice"),
    ).await?;
    let mut settings = Doc::new();
    settings.set("name", "test");
    let default_key = user.get_default_key()?;
    let database = user.create_database(settings, &default_key).await?;
    example(database).await?;
    Ok(())
}

When you commit a transaction, Eidetica:

1. Creates a new Entry containing all changes
2. Links it to the appropriate parent entries
3. Adds it to the database’s history
4. Returns the ID of the new entry

Database Settings

Each Database maintains its settings as a key-value store in a special “settings” store:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::SettingsStore, Database};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
// Setup database for testing
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings_doc = Doc::new();
settings_doc.set("name", "example_database");
settings_doc.set("version", "1.0.0");
let default_key = user.get_default_key()?;
let database = user.create_database(settings_doc, &default_key).await?;
// Access database settings through a transaction
let transaction = database.new_transaction().await?;
let settings_store = transaction.get_settings()?;

// Access common settings
let name = settings_store.get_name().await?;
println!("Database name: {}", name);

// Access custom settings via the underlying DocStore
let doc_store = settings_store.as_doc_store();
if let Ok(version_value) = doc_store.get("version").await {
    println!("Database version available");
}

transaction.commit().await?;
Ok(())
}

Common settings include:

• name: The identifier for the database (used by AccountSession::find_database). This is the primary standard setting currently used.
• Other application-specific settings can be stored here.

Tips and History

Databases in Eidetica maintain a concept of “tips” - the latest entries in the database’s history. Tips represent the current state of the database and are managed automatically by the system.

When you create transactions and commit changes, Eidetica automatically:

• Updates the database tips to point to your new entries
• Maintains the complete history of all previous states
• Ensures efficient access to the current state through tip tracking

This historical information remains accessible, allowing you to:

• Track all changes to data over time
• Reconstruct the state at any point in history (requires manual traversal or specific backend support - see Backends)

Database vs. Store

While a Database is the logical container, the actual data is organized into Stores. This separation allows:

• Different types of data structures within a single Database
• Type-safe access to different parts of your data
• Fine-grained history tracking by store
• Efficient partial replication and synchronization

See Stores for more details on how data is structured within a Database.

Database Storage

Database storage implementations in Eidetica define how and where data is physically stored.

The Database Abstraction

The Database trait abstracts the underlying storage mechanism for Eidetica entries. This separation of concerns allows the core database logic to remain independent of the specific storage details.

Key responsibilities of a Database:

• Storing and retrieving entries by their unique IDs
• Tracking relationships between entries
• Calculating tips (latest entries) for databases and stores
• Managing the graph-like structure of entry history

Selecting a Backend via URL

The recommended way to open an Instance is by URL — Instance::connect(url) and Instance::connect_or_create(url, NewAccount::…) dispatch to the right backend based on the URL scheme:

The in-memory sqlite URL requires sqlx’s single-colon URI form (sqlite: rather than sqlite://) due to limitations in sqlx. The file: prefix + cache=shared are what make the database visible across the connection pool. Without them each pooled connection gets its own isolated database.

Backends whose Cargo feature isn’t compiled in return InstanceError::BackendUnavailable { scheme, missing_feature } at runtime — no link-time surprise.

For escape-hatch construction (custom sqlx pool config, custom clock, etc.) the per-backend constructors below remain pub; see Instance::open_backend, connect_or_create_backend, and create_backend for the URL-less entry points.

Available Backend Implementations

SQLite

SQLite is the default and recommended backend. It provides embedded persistent storage with excellent performance. Enabled with the sqlite feature.

A SqlxBackend exclusively owns its persistent storage namespace until the backend is dropped. For SQLite, that namespace is the canonical database file. For PostgreSQL, it is the database and active schema. A competing direct open fails immediately with BackendError::StorageAlreadyOwned, including another backend in the same process. Clients that need to share the same instance should connect through one Eidetica service daemon instead.

SQLite uses a cooperative advisory lock on a stable .eidetica-owner sidecar next to the canonical database path. The backend retains the nonblocking operating-system lock for its lifetime, and leaves the sidecar in place when released; the file’s existence alone does not mean the database is owned. Relative, absolute, and existing symlink aliases that resolve to the same canonical path therefore contend; a final-component symlink whose target does not exist is rejected. This protocol requires a local filesystem with reliable advisory locks. Applications must not use hard-link aliases, rename or replace a live database, or delete or replace its sidecar. Unrelated SQLite tools do not participate and can bypass it.

PostgreSQL stores an ownership token in the active schema: every pooled session holds a shared namespace lock and validates that token, while takeover requires the exclusive namespace lock. A surviving old session therefore blocks takeover, and a stale pool cannot reconnect after ownership changes. The operating system or database server releases the locks when their sessions exit or crash. Dropping a PostgreSQL backend closes its sessions without waiting for the server, so an immediate reopen of the same namespace can still briefly fail with BackendError::StorageAlreadyOwned.

Both SQL backends use an internal integer schema counter, independent of releases and the InMemory file counter. Mainline schema 0 migrates directly to current 4; current opens reuse persisted Verified tips and local-account records. The experimental 1 prefix format is unsupported. See the migration matrix. Both SQL backends reject a stored schema version newer than the supported version before creating schema tables or indexes or running migrations. Refusal leaves the version marker, Entries, instance metadata and Store-state records unchanged. The error names both versions. This is not a read-only open: SQLite may still set WAL mode and open its ownership sidecar, and PostgreSQL still performs ownership bookkeeping before checking the schema version.

use eidetica::{Instance, NewAccount};

// Recommended: by URL.
let (instance, _) = Instance::connect_or_create(
    "sqlite://./my_data.db",
    NewAccount::passwordless("alice"),
).await?;

// Escape-hatch: build the backend manually for custom configuration.
use eidetica::backend::database::Sqlite;
let backend = Sqlite::open("my_data.db").await?;
let instance = Instance::open_backend(Box::new(backend)).await?;

PostgreSQL

PostgreSQL provides production-grade persistent storage for larger deployments. Enabled with the postgres feature.

use eidetica::{Instance, NewAccount};

// By URL — passed through to sqlx unchanged, so any sqlx-accepted query
// string works (sslmode, application_name, etc.).
let (instance, _) = Instance::connect_or_create(
    "postgres://user:pass@localhost/mydb?sslmode=require",
    NewAccount::passwordless("alice"),
).await?;

InMemory

The in-process backend is intended for tests, examples, and embedded apps that don’t need durable storage. It can optionally serialize to and load from a JSON file via the memory:///abs/path URL form.

Prefer sqlite’s in-memory mode when the sqlite feature is enabled. The sqlite:file::memory:?cache=shared URL exercises the same backend used in production, so integration tests stay closer to deployed behaviour. Reach for memory:// only when you’ve built without the sqlite feature or specifically want the JSON-snapshot story.

Not for production. Prefer sqlite:// (file-backed) or postgres:// for any deployed workload. Specifically:

• JSON snapshots use internal integer format counter _v (currently 3), not a release version or stability promise. Mainline unversioned/0 files reconstruct Verified tips on load; version 2 loads without history repair, while experimental 1 and future formats are refused. See the supported migration matrix.
• The snapshot serializes the device signing key in plaintext. For memory:///path.json, that key ends up on disk in cleartext — fine for tests, unsafe for production.
• Every flush() rewrites the full backend state; there’s no incremental persistence, MVCC, or multi-process access.

The ephemeral sqlite form uses Instance::connect_or_create like every other URL:

use eidetica::{Instance, NewAccount};

let (instance, _) = Instance::connect_or_create(
    "sqlite:file::memory:?cache=shared",
    NewAccount::passwordless("alice"),
).await?;

The memory:// URL forms remain useful for tests and short-lived embedded scenarios:

use eidetica::{Instance, NewAccount};

// Ephemeral (state dies with the process).
let (instance, _) =
    Instance::connect_or_create("memory://", NewAccount::passwordless("alice")).await?;

// With a JSON snapshot — loaded on construction, written by flush() or
// (best-effort) by Drop. The snapshot path must be absolute.
let (instance, _) = Instance::connect_or_create(
    "memory:///var/lib/myapp/snap.json",
    NewAccount::passwordless("alice"),
).await?;
// ... do work ...
instance.flush()?; // checkpoint the snapshot to disk atomically; instance keeps running

Instance::flush() is the canonical way to checkpoint an in-memory snapshot. It’s reentrant, sync, and takes &self, so call it as often as you like — the snapshot path stays armed and the instance keeps working after every flush. The signature is sync because the write itself is sync (std::fs::write + atomic rename); from a tokio task it briefly blocks the worker, negligible for small snapshots. The Drop impl falls back to a best-effort save on the last handle (errors are logged via tracing::error!, not surfaced) — apps that care about snapshot durability should call flush() at checkpoints and inspect the Result.

Remote (Service Daemon)

A unix:// URL connects to an Eidetica service daemon. All storage operations are forwarded as RPCs to the daemon, which holds the actual backend. This enables multiple client processes to share the same storage.

use eidetica::Instance;

let instance = Instance::connect("unix:///run/eidetica/service.sock").await?;
// Or, with env / default resolution:
let instance = Instance::connect(eidetica::service::default_socket_url()).await?;

The service feature must be enabled (included in the default full feature set). connect_or_create against a unix:// URL degrades to connect — the daemon owns its own initialisation. See Service (Daemon) Mode for setup details.

Database Trait Responsibilities

The Database trait (eidetica::backend::Database) defines the core interface required for storage. Beyond simple get and put for entries, it includes methods crucial for navigating the database’s history and structure:

• snapshot(tree_id): Finds the latest entries in a specific Database.
• store_snapshot(tree_id, subtree_name): Finds the latest entries for a specific Store within a Database.
• all_roots(): Finds all top-level Database roots stored in the database.
• get_tree(tree_id) / get_subtree(...): Retrieve all entries for a database/store, typically sorted topologically (required for some history operations, potentially expensive).

Implementing these methods efficiently often requires the database to understand the DAG structure, making the database more than just a simple key-value store.

Database Performance Considerations

The Database implementation significantly impacts database performance:

• Entry Retrieval: How quickly entries can be accessed by ID
• Graph Traversal: Efficiency of history traversal and tip calculation
• Memory Usage: How entries are stored and whether they’re kept in memory
• Concurrency: How concurrent operations are handled

Stores

Stores provide structured, type-safe access to different kinds of data within a Database.

The Store Concept

In Eidetica, Stores extend the Merkle-CRDT concept by explicitly partitioning data within each Entry. A Store:

• Represents a specific type of data structure (like a key-value store or a collection of records)
• Has a unique name within its parent Database
• Maintains its own history tracking
• Is strongly typed (via Rust generics)

Stores are what make Eidetica practical for real applications, as they provide high-level, data-structure-aware interfaces on top of the core Entry and Database concepts.

Why Stores?

Stores offer several advantages:

• Type Safety: Each store implementation provides appropriate methods for its data type
• Isolation: Changes to different stores can be tracked separately
• Composition: Multiple data structures can exist within a single Database
• Efficiency: Only relevant stores need to be loaded or synchronized
• Atomic Operations: Changes across multiple stores can be committed atomically

Available Store Types

Eidetica provides several store types, each optimized for different data patterns:

DocStore (Document-Oriented Storage)

The DocStore store provides a document-oriented interface for storing and retrieving structured data. It wraps the crdt::Doc type to provide ergonomic access patterns with both simple key-value operations and path-based operations for nested data structures.

Basic Usage

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::DocStore, path};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let backend = Box::new(Sqlite::in_memory().await?);
let (instance, mut user) = eidetica::Instance::create_backend(
    backend,
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
// Get a DocStore store
let txn = database.new_transaction().await?;
let store = txn.get_store::<DocStore>("app_data").await?;

// Set simple values
store.set("version", "1.0.0").await?;
store.set("author", "Alice").await?;

// Path-based operations for nested structures
// This creates nested maps: {"database": {"host": "localhost", "port": "5432"}}
store.set_path(path!("database.host"), "localhost").await?;
store.set_path(path!("database.port"), "5432").await?;

// Retrieve values
let version = store.get("version").await?; // Returns a Value
let host = store.get_path(path!("database.host")).await?; // Returns Value

txn.commit().await?;
Ok(())
}

Important: Path Operations Create Nested Structures

When using set_path("a.b.c", value), DocStore creates nested maps, not flat keys with dots:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::DocStore, path};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let backend = Box::new(Sqlite::in_memory().await?);
let (instance, mut user) = eidetica::Instance::create_backend(
    backend,
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
let txn = database.new_transaction().await?;
let store = txn.get_store::<DocStore>("app_data").await?;
// This code:
store.set_path(path!("user.profile.name"), "Bob").await?;

// Creates this structure:
// {
//   "user": {
//     "profile": {
//       "name": "Bob"
//     }
//   }
// }

// NOT: { "user.profile.name": "Bob" } ❌
txn.commit().await?;
Ok(())
}

Use cases for DocStore:

• Application configuration
• Metadata storage
• Structured documents
• Settings management
• Any data requiring path-based access

Table

The Table<T> store manages collections of serializable items, similar to a table in a database:

extern crate eidetica;
extern crate tokio;
extern crate serde;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::Table};
use serde::{Serialize, Deserialize};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let backend = Box::new(Sqlite::in_memory().await?);
let (instance, mut user) = eidetica::Instance::create_backend(
    backend,
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
// Define a struct for your data
#[derive(Serialize, Deserialize, Clone)]
struct User {
    name: String,
    email: String,
    active: bool,
}

// Get a Table store
let txn = database.new_transaction().await?;
let users = txn.get_store::<Table<User>>("users").await?;

// Insert items (returns a generated UUID)
let user = User {
    name: "Alice".to_string(),
    email: "alice@example.com".to_string(),
    active: true,
};
let id = users.insert(user).await?;

// Get an item by ID
if let Ok(user) = users.get(&id).await {
    println!("Found user: {}", user.name);
}

// Update an item
if let Ok(mut user) = users.get(&id).await {
    user.active = false;
    users.set(&id, user).await?;
}

// Delete an item
let was_deleted = users.delete(&id).await?;
if was_deleted {
    println!("User deleted successfully");
}

// Search for items matching a condition
let active_users = users.search(|user| user.active).await?;
for (id, user) in active_users {
    println!("Active user: {} (ID: {})", user.name, id);
}
txn.commit().await?;
Ok(())
}

Use cases for Table:

• Collections of structured objects
• Record storage (users, products, todos, etc.)
• Any data where individual items need unique IDs
• When you need to search across records with custom predicates

SettingsStore (Database Settings Management)

The SettingsStore provides a specialized, type-safe interface for managing database settings and authentication configuration. It wraps the internal _settings subtree to provide convenient methods for common settings operations.

Basic Usage

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::SettingsStore};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let backend = Box::new(Sqlite::in_memory().await?);
let (instance, mut user) = eidetica::Instance::create_backend(
    backend,
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
// Get a SettingsStore for the current transaction
let transaction = database.new_transaction().await?;
let settings_store = transaction.get_settings()?;

// Set database name
settings_store.set_name("My Application Database").await?;

// Get database name
let name = settings_store.get_name().await?;
println!("Database name: {}", name);

transaction.commit().await?;
Ok(())
}

Authentication Management

SettingsStore provides convenient methods for managing authentication keys:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::SettingsStore};
use eidetica::auth::{AuthKey, Permission};
use eidetica::auth::crypto::generate_keypair;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
// Setup database for testing
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "stores_auth_example");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
// Generate a keypair for the new user
let (_alice_signing_key, alice_public_key) = generate_keypair();
let transaction = database.new_transaction().await?;
let settings_store = transaction.get_settings()?;

// Add a new authentication key (name is optional metadata)
let auth_key = AuthKey::active(Some("alice_laptop"), Permission::Write(10));
// The pubkey is used as the key identifier when storing
settings_store.set_auth_key(&alice_public_key, auth_key).await?;

// Get an authentication key by pubkey
let key = settings_store.get_auth_key(&alice_public_key).await?;
println!("Alice's key name: {:?}", key.name());

// Revoke a key
settings_store.revoke_auth_key(&alice_public_key).await?;

transaction.commit().await?;
Ok(())
}

Multiple Key Operations

Multiple auth operations within a transaction are accumulated in a single entry:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::SettingsStore};
use eidetica::auth::{AuthKey, Permission};
use eidetica::auth::crypto::generate_keypair;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
// Setup database for testing
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "complex_auth_example");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
// Generate keypairs for multiple users
let (_bob_signing_key, bob_public_key) = generate_keypair();
let bob_key = AuthKey::active(Some("bob_device"), Permission::Write(20));
let (_charlie_signing_key, charlie_public_key) = generate_keypair();
let charlie_key = AuthKey::active(Some("charlie_device"), Permission::Admin(15));
let (_old_user_signing_key, old_user_public_key) = generate_keypair();
let old_user_key = AuthKey::active(Some("old_device"), Permission::Write(30));
// Add old_user first so we can revoke it
let setup_txn = database.new_transaction().await?;
let setup_store = setup_txn.get_settings()?;
setup_store.set_auth_key(&old_user_public_key, old_user_key).await?;
setup_txn.commit().await?;
let transaction = database.new_transaction().await?;
let settings_store = transaction.get_settings()?;

// Set multiple keys directly (each write is incremental)
settings_store.set_auth_key(&bob_public_key, bob_key).await?;
settings_store.set_auth_key(&charlie_public_key, charlie_key).await?;

// Revoke an old key
settings_store.revoke_auth_key(&old_user_public_key).await?;

transaction.commit().await?;
Ok(())
}

Advanced Usage

For operations not covered by the convenience methods, access the underlying DocStore:

let transaction = database.new_transaction().await?;
let settings_store = transaction.get_settings()?;

// Access underlying DocStore for advanced operations
let doc_store = settings_store.as_doc_store();
doc_store.set_path(path!("custom.config.option"), "value").await?;

transaction.commit().await?;

Use cases for SettingsStore:

• Database configuration and metadata
• Authentication key management
• User permission management
• Bootstrap and sync policies
• Any settings that need type-safe, validated access

YDoc (Y-CRDT Integration)

The YDoc store provides integration with Y-CRDT (Yjs) for real-time collaborative editing. This requires the “y-crdt” feature:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::YDoc};
use eidetica::y_crdt::{Map, Text, Transact};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
// Setup database for testing
let backend = Sqlite::in_memory().await?;
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(backend),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "y_crdt_stores");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;

// Get a YDoc store
let txn = database.new_transaction().await?;
let doc_store = txn.get_store::<YDoc>("document").await?;

// Work with Y-CRDT structures
doc_store.with_doc_mut(|doc| {
    let text = doc.get_or_insert_text("content");
    let metadata = doc.get_or_insert_map("meta");

    let mut txn = doc.transact_mut();

    // Collaborative text editing
    text.insert(&mut txn, 0, "Hello, collaborative world!");

    // Set metadata
    metadata.insert(&mut txn, "title", "My Document");
    metadata.insert(&mut txn, "author", "Alice");

    Ok(())
}).await?;

txn.commit().await?;
Ok(())
}

Use cases for YDoc:

• Real-time collaborative text editing
• Shared documents with multiple editors
• Conflict-free data synchronization
• Applications requiring sophisticated merge algorithms

PasswordStore<S> (Encrypted Wrapper)

PasswordStore<S> wraps any store type S with transparent password-based encryption. Entry payloads and derived cached state are encrypted with AES-256-GCM using keys derived from a Store password with Argon2id. The type parameter S specifies the wrapped store (e.g., PasswordStore<DocStore>, PasswordStore<Table<T>>). Table caches hide logical row keys behind stable keyed hashes and order scans by those physical keys; other Store types use an encrypted opaque state cache.

For usage, recovery limits, and observable metadata, see the Encryption Guide.

Subtree Index

Eidetica automatically maintains an index of all user-created subtrees in a special _index subtree. This index stores metadata about each subtree, including its Store type and configuration.

What is the Subtree Index?

The _index subtree tracks:

• Subtree names: Which subtrees exist in the database
• Store types: What type of Store manages each subtree (e.g., “docstore:v0”, “table:v0”)
• Configuration: Store-specific settings for each subtree
• Subtree settings: Common settings like height strategy overrides

The index is maintained automatically when you access stores via get_store() and is useful for:

• Discovery: Finding what subtrees exist in a database
• Type information: Understanding what Store type manages each subtree
• Tooling: Building generic database browsers and inspectors

The index is accessed via Transaction::get_index(), which returns a Registry - a general-purpose type for managing name → {type, config} mappings.

Automatic Registration

When you first access a Store using Transaction::get_store(), it’s automatically registered in the _index with its Store type and default configuration:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::DocStore};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let backend = Box::new(Sqlite::in_memory().await?);
let (instance, mut user) = eidetica::Instance::create_backend(
    backend,
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
// First access to "app_config" - automatically registered in _index
let txn = database.new_transaction().await?;
let config: DocStore = txn.get_store("app_config").await?;
config.set("version", "1.0.0").await?;
txn.commit().await?;

// The 'app_config' Store is now registered with type "docstore:v0"
Ok(())
}

Registration happens immediately when get_store() is called for a new subtree.

System Subtrees: The special system subtrees (_settings, _index, _root) are excluded from the index to avoid circular dependencies.

Querying the Index

Use get_index() to query information about registered subtrees:

extern crate eidetica;
extern crate tokio;
extern crate serde;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::{DocStore, Table}};
use serde::{Serialize, Deserialize};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let backend = Box::new(Sqlite::in_memory().await?);
let (instance, mut user) = eidetica::Instance::create_backend(
    backend,
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
// Create some subtrees first
#[derive(Serialize, Deserialize, Clone)]
struct User { name: String }
let setup_txn = database.new_transaction().await?;
let _config: DocStore = setup_txn.get_store("config").await?;
let _users: Table<User> = setup_txn.get_store("users").await?;
setup_txn.commit().await?;
// Query the index to discover subtrees
let txn = database.new_transaction().await?;
let index = txn.get_index().await?;

// List all registered subtrees
let subtrees = index.list().await?;
for name in subtrees {
    println!("Found subtree: {}", name);
}

// Check if a specific subtree exists
if index.contains("config").await {
    // Get metadata about the subtree
    let info = index.get_entry("config").await?;
    println!("Type: {}", info.type_id);  // e.g., "docstore:v0"
    println!("Config: {}", info.config);  // Store-specific configuration
}
Ok(())
}

Manual Registration

You can manually register or update subtree metadata using set_entry() on the index. This is useful for pre-registering subtrees with custom configuration:

let txn = database.new_transaction()?;
let index = txn.get_index()?;

// Pre-register a subtree with custom configuration
index.set_entry(
    "documents",
    "ydoc:v0",
    r#"{"compression":"zstd","cache_size":1024}"#
)?;

txn.commit()?;

// Future accesses will use the registered configuration

When to Use the Subtree Index

Many applications don’t need to interact with the subtree index directly and can let auto-registration handle everything automatically. Use get_index() when you need to:

• List subtrees: Build a database browser or inspector
• Query metadata: Check Store types or configurations
• Pre-configure: Set custom configuration before first use
• Build tooling: Create generic tools that work with any database structure

For more information on how the index system works internally, see the Subtree Index Design Document.

Store Implementation Details

Each Store implementation in Eidetica:

1. Implements the Store trait
2. Provides methods appropriate for its data structure
3. Handles serialization/deserialization of data
4. Manages the store’s history within the Database

The Store trait defines the minimal interface:

pub trait Store: Sized {
    fn new(txn: &Transaction, store_name: &str) -> Result<Self>;
    fn name(&self) -> &str;
}

Store implementations add their own methods on top of this minimal interface.

Store Height Strategies

Each store can configure its own height calculation strategy, independent of the database-level strategy. When branches diverge and later merge (e.g., after a network split), heights determine the order entries are processed. Lower heights come first; ties are broken by entry hash.

Available Methods

All Store types have these height strategy methods available:

• get_height_strategy(): Get the current strategy (None = inherit from database)
• set_height_strategy(Option<HeightStrategy>): Set or clear an override

Use Cases

For detailed usage examples, see Height Strategies in the Transactions guide.

Store History and Merging (CRDT Aspects)

Custom Store data implements both Codec (persistence encoding) and CRDT (merging), plus Default for history reconstruction. Neither trait requires Serde or cloning. See Encoding and the Rust API for the migration from the old Data marker and the unchanged built-in formats.

While Eidetica uses Merkle-DAGs for overall history, the way data within a Store is combined when branches merge relies on Conflict-free Replicated Data Type (CRDT) principles. This ensures that even if different replicas of the database have diverged and made concurrent changes, they can be merged back together automatically without conflicts (though the merge result depends on the CRDT strategy).

Each Store type implements its own merge logic, typically triggered implicitly when an Transaction reads the current state of the store (which involves finding and merging the tips of that store’s history):

• DocStore: Uses the internal Doc type with structural merge by default. When merging concurrent writes to the same key or path, the write associated with the later Entry “wins” (LWW), and its value is kept. Writes to different keys are simply combined. Deleted keys (via delete()) are tracked with tombstones to ensure deletions propagate properly. Docs marked as atomic (via Doc::atomic()) use full Last-Writer-Wins replacement — the entire Doc replaces its predecessor rather than merging field-by-field. This is used for data that should be treated as a complete unit.

• Table<T>: Also uses LWW for updates to the same row ID. If two concurrent operations modify the same row, the later write wins. Inserts of different rows are combined (all inserted rows are kept). Deletions generally take precedence over concurrent updates (though precise semantics might evolve).

Note: The CRDT merge logic happens internally when a Transaction loads the initial state of a Store or when a store viewer is created. You typically don’t invoke merge logic directly.

Future Store Types

Eidetica’s architecture allows for adding new Store implementations. Potential future types include:

• ObjectStore: For storing large binary blobs.

These are not yet implemented. Development is currently focused on the core API and the existing DocStore and Table types.

Transactions: Atomic Changes

In Eidetica, all modifications to the data stored within a Database’s Stores happen through an Transaction. This is a fundamental concept ensuring atomicity and providing a consistent mechanism for interacting with your data.

Authentication Note: All transactions in Eidetica are authenticated by default. Every transaction uses the database’s default signing key to ensure that all changes are cryptographically verified and can be traced to their source.

A Transaction bundles multiple Store operations (which affect individual subtrees) into a single atomic Entry that gets committed to the database.

Why Transactions?

Transactions provide several key benefits:

• Atomicity: Changes made to multiple Stores within a single Transaction are encoded together in one signed Entry. A failure before persistence writes nothing, but storage, verification, callbacks and acknowledgement are separate steps: an error or lost acknowledgement can follow persistence. Read back verified state before retrying; a failed commit() does not imply rollback.
• Consistency: A Transaction captures a snapshot of the Database’s state (specifically, the tips of the relevant Stores) when it’s created or when a Store is first accessed within it. All reads and writes within that Transaction occur relative to this consistent state.
• Change Staging: Modifications made via Store handles are staged within the Transaction object itself, not written directly to the database until commit() is called.
• Authentication: All transactions are automatically authenticated using the database’s default signing key, ensuring data integrity and access control.
• History Creation: A successful commit() results in the creation of a new Entry in the Database, containing the staged changes and linked to the previous state (the tips the Transaction was based on). This is how history is built.

The Transaction Lifecycle

Using a Transaction follows a distinct lifecycle:

1. Creation: Start an authenticated transaction from a Database instance.

   extern crate eidetica;
   extern crate tokio;
   use eidetica::{backend::database::Sqlite, Instance, crdt::Doc};
   
   #[tokio::main]
   async fn main() -> eidetica::Result<()> {
   // Setup database
   let backend = Sqlite::in_memory().await?;
   let (instance, mut user) = Instance::create_backend(
       Box::new(backend),
       eidetica::NewAccount::passwordless("alice"),
   ).await?;
   let mut settings = Doc::new();
   settings.set("name", "test");
   let default_key = user.get_default_key()?;
   let database = user.create_database(settings, &default_key).await?;
   
   let _txn = database.new_transaction().await?; // Automatically uses the database's default signing key
   Ok(())
   }

2. Store Access: Get handles to the specific Stores you want to interact with. This implicitly loads the current state (tips) of that store into the transaction if accessed for the first time.

   extern crate eidetica;
   extern crate tokio;
   extern crate serde;
   use eidetica::{backend::database::Sqlite, Instance, crdt::Doc, store::{Table, DocStore, SettingsStore}, Database};
   use serde::{Serialize, Deserialize};
   
   #[derive(Clone, Debug, Serialize, Deserialize)]
   struct User {
       name: String,
   }
   
   #[tokio::main]
   async fn main() -> eidetica::Result<()> {
   // Setup database and transaction
   let backend = Sqlite::in_memory().await?;
   let (instance, mut user) = Instance::create_backend(
       Box::new(backend),
       eidetica::NewAccount::passwordless("alice"),
   ).await?;
   let mut settings = Doc::new();
   settings.set("name", "test");
   let default_key = user.get_default_key()?;
   let database = user.create_database(settings, &default_key).await?;
   let txn = database.new_transaction().await?;
   
   // Get handles within a scope or manage their lifetime
   let _users_store = txn.get_store::<Table<User>>("users").await?;
   let _config_store = txn.get_store::<DocStore>("config").await?;
   let _settings_store = txn.get_settings()?;  // For database settings
   
   txn.commit().await?;
   Ok(())
   }

3. Staging Changes: Use the methods provided by the Store handles (set, insert, get, remove, etc.). These methods interact with the data staged within the Transaction.

   extern crate eidetica;
   extern crate tokio;
   extern crate serde;
   use eidetica::{backend::database::Sqlite, Instance, crdt::Doc, store::{Table, DocStore, SettingsStore}};
   use serde::{Serialize, Deserialize};
   
   #[derive(Clone, Debug, Serialize, Deserialize)]
   struct User {
       name: String,
   }
   
   #[tokio::main]
   async fn main() -> eidetica::Result<()> {
   // Setup database and transaction
   let backend = Sqlite::in_memory().await?;
   let (instance, mut user) = Instance::create_backend(
       Box::new(backend),
       eidetica::NewAccount::passwordless("alice"),
   ).await?;
   let mut settings = Doc::new();
   settings.set("name", "test");
   let default_key = user.get_default_key()?;
   let database = user.create_database(settings, &default_key).await?;
   let txn = database.new_transaction().await?;
   let users_store = txn.get_store::<Table<User>>("users").await?;
   let config_store = txn.get_store::<DocStore>("config").await?;
   let settings_store = txn.get_settings()?;
   
   // Insert a new user and get their ID
   let user_id = users_store.insert(User { name: "Alice".to_string() }).await?;
   let _current_user = users_store.get(&user_id).await?;
   config_store.set("last_updated", "2024-01-15T10:30:00Z").await?;
   settings_store.set_name("Updated Database Name").await?;  // Manage database settings
   
   txn.commit().await?;
   Ok(())
   }

   Note: get methods within a transaction read from the staged state, reflecting any changes already made within the same transaction.

4. Commit: Finalize the changes. This consumes the Transaction object, calculates the final Entry content based on staged changes, cryptographically signs the entry, writes the new Entry to the Database, and returns the ID of the newly created Entry.

   extern crate eidetica;
   extern crate tokio;
   use eidetica::{backend::database::Sqlite, Instance, crdt::Doc};
   
   #[tokio::main]
   async fn main() -> eidetica::Result<()> {
   // Setup database
   let backend = Sqlite::in_memory().await?;
   let (instance, mut user) = Instance::create_backend(
       Box::new(backend),
       eidetica::NewAccount::passwordless("alice"),
   ).await?;
   let mut settings = Doc::new();
   settings.set("name", "test");
   let default_key = user.get_default_key()?;
   let database = user.create_database(settings, &default_key).await?;
   
   // Create transaction and commit
   let txn = database.new_transaction().await?;
   let new_entry_id = txn.commit().await?;
   println!("Changes committed. New state represented by Entry: {}", new_entry_id);
   Ok(())
   }

   After commit(), the txn variable is no longer valid.

Convenience: with_transaction

For the common pattern of “create a transaction, do some work, commit,” Database::with_transaction handles the lifecycle for you:

extern crate eidetica;
extern crate tokio;
extern crate serde;
use eidetica::{backend::database::Sqlite, Instance, crdt::Doc, store::Table};
use serde::{Serialize, Deserialize};

#[derive(Clone, Debug, Serialize, Deserialize)]
struct User {
    name: String,
}

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let backend = Sqlite::in_memory().await?;
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(backend),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;

// Insert a record and get its generated key
let user_id = database.with_transaction(|txn| async move {
    let users = txn.get_store::<Table<User>>("users").await?;
    users.insert(User { name: "Alice".into() }).await
}).await?;

// Multiple operations in one atomic transaction
database.with_transaction(|txn| async move {
    let users = txn.get_store::<Table<User>>("users").await?;
    users.insert(User { name: "Bob".into() }).await?;
    users.insert(User { name: "Charlie".into() }).await?;
    Ok(())
}).await?;
Ok(())
}

The closure receives the Transaction, and with_transaction commits it after the closure returns Ok. If the closure returns Err, the transaction is dropped without committing — no partial writes.

Use with_transaction when you don’t need the commit ID. If you need the ID of the committed entry (e.g., for tracking DAG structure), use new_transaction() and commit() directly.

Managing Database Settings

Within transactions, you can manage database settings using SettingsStore. This provides type-safe access to database configuration and authentication settings:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::SettingsStore};
use eidetica::auth::{AuthKey, Permission};
use eidetica::auth::crypto::generate_keypair;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
// Setup database for testing
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "settings_example");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
// Generate keypairs for old user and add it first so we can revoke it
let (_old_user_signing_key, old_user_public_key) = generate_keypair();
let old_user_key = AuthKey::active(Some("old_device"), Permission::Write(15));
let setup_txn = database.new_transaction().await?;
let setup_store = setup_txn.get_settings()?;
setup_store.set_auth_key(&old_user_public_key, old_user_key).await?;
setup_txn.commit().await?;
let transaction = database.new_transaction().await?;
let settings_store = transaction.get_settings()?;

// Update database name
settings_store.set_name("Production Database").await?;

// Generate keypairs for new users (hidden in production code)
let (_new_account_signing_key, new_account_public_key) = generate_keypair();
let (_alice_signing_key, alice_public_key) = generate_keypair();

// Add authentication keys (indexed by pubkey, name is optional metadata)
let new_account_key = AuthKey::active(Some("new_device"), Permission::Write(10));
settings_store.set_auth_key(&new_account_public_key, new_account_key).await?;

// Set auth keys directly
let alice_key = AuthKey::active(Some("alice_laptop"), Permission::Write(5));
settings_store.set_auth_key(&alice_public_key, alice_key).await?;
settings_store.revoke_auth_key(&old_user_public_key).await?;

transaction.commit().await?;
Ok(())
}

This ensures that settings changes are atomic and properly authenticated alongside other database modifications.

Height Strategies

Each Entry in Eidetica has a height value. When branches diverge and later merge (e.g., after a network split), heights determine the order entries are processed. Lower heights come first; ties are broken by entry hash. The height strategy determines how this value is calculated during commit.

Available Strategies

Database-Level Strategy

Configure the height strategy for the entire database via SettingsStore:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, HeightStrategy};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "height_example");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
let txn = database.new_transaction().await?;
let settings = txn.get_settings()?;

// Set timestamp-based heights for time-series data
settings.set_height_strategy(HeightStrategy::Timestamp).await?;

txn.commit().await?;
Ok(())
}

Per-Subtree Height Strategy

Individual stores can override the database strategy for independent height tracking:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, HeightStrategy, Store, store::DocStore};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "per_subtree_example");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
let txn = database.new_transaction().await?;

// This store uses its own incremental counter
let audit_log = txn.get_store::<DocStore>("audit_log").await?;
audit_log.set_height_strategy(Some(HeightStrategy::Incremental)).await?;
audit_log.set("event", "user_login").await?;

// This store inherits the database strategy
let messages = txn.get_store::<DocStore>("messages").await?;
messages.set("content", "Hello").await?;

txn.commit().await?;
Ok(())
}

How Height Inheritance Works

Subtrees with no explicit height strategy inherit from the database-level strategy:

• No explicit height (None): The subtree inherits from the tree (database) height
• Explicit height (Some(h)): The subtree has an independent height value

When querying Entry.subtree_height(), the returned value reflects this inheritance transparently.

Read-Only Access

While Transactions are essential for writes, you can perform reads without an explicit Transaction using Database::get_store_viewer:

extern crate eidetica;
extern crate tokio;
extern crate serde;
use eidetica::{backend::database::Sqlite, Instance, crdt::Doc, store::Table, Database};
use serde::{Serialize, Deserialize};

#[derive(Clone, Debug, Serialize, Deserialize)]
struct User {
    name: String,
}

#[tokio::main]
async fn main() -> eidetica::Result<()> {
// Setup database with some data
let backend = Sqlite::in_memory().await?;
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(backend),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
// Insert test data
let txn = database.new_transaction().await?;
let users_store = txn.get_store::<Table<User>>("users").await?;
let user_id = users_store.insert(User { name: "Alice".to_string() }).await?;
txn.commit().await?;

let users_viewer = database.get_store_viewer::<Table<User>>("users").await?;
if let Ok(_user) = users_viewer.get(&user_id).await {
    // Read data based on the current tips of the 'users' store
}
Ok(())
}

A SubtreeViewer provides read-only access based on the latest committed state (tips) of that specific store at the time the viewer is created. It does not allow modifications and does not require a commit().

Choose with_transaction for the common case of writing to one or more stores and committing. Use new_transaction() directly when you need the commit ID or complex control flow. Choose get_store_viewer for simple, read-only access to the latest state.

Write Callbacks

Database::on_write registers a callback that fires every time entries are written to that database — both local commits and entries received via sync. Useful for “rebuild a derived view when this DB changes,” “wake up a UI,” or any other change-driven workflow.

#![allow(unused)]
fn main() {
extern crate eidetica;
extern crate tokio;
use eidetica::{Database, Result};
async fn example(database: &Database) -> Result<()> {
let cb = database.on_write(|event, db| {
    let source = event.source();
    let prev = event.previous_tips().clone();
    let post = event.post_tips().clone();
    let db = db.clone();
    async move {
        // Expand the cursor advance into the concrete set of new entry IDs.
        // Skip this for "any write" wake-ups — the brackets alone suffice.
        let new_ids = db.ids_added(&prev, &post).await?;
        println!("{} entries written to {} ({source:?})", new_ids.len(), db.root_id());
        Ok(())
    }
}).await?;

// Drop `cb` to unregister, or keep the callback registered for the life
// of the Instance with `cb.detach()`.
Ok(())
}
}

WriteEvent carries cursor brackets only — previous_tips (the callback’s own frontier before this fire) and post_tips (the frontier after). Callbacks that just need a “something changed” ping can act directly on the event; callbacks that need to enumerate the new entries call Database::ids_added with those brackets.

The returned WriteCallback handle controls the registration’s lifetime: drop it to unregister, or call .detach() to keep the callback live until the Instance itself drops.

Callback timing — local vs. connected

The timing contract depends on whether the Instance is local (one backend in this process) or connected (built via Instance::connect to a daemon over a Unix socket):

The asynchronous fire on a connected Instance is intentional. By deferring to the daemon’s ordering, every subscriber sees writes in the same sequence — your own callback, another client’s callback for your write, and a sync-peer entry that arrived between them are all interleaved by the daemon and pushed in that order to everyone. The trade-off is that commit().await returning no longer means your own callback has run; if you need synchronous “callback fired before commit returned” semantics for your own writes, use a local Instance.

On a connected Instance, the first on_write registration for a given database lazily subscribes that client to push notifications for it. Subsequent registrations on the same database reuse that subscription. Subscriptions are torn down implicitly when the client disconnects.

Authentication Guide

How to use Eidetica’s authentication system for securing your data.

Quick Start

Every Eidetica database requires authentication. Here’s the minimal setup:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite};
use eidetica::crdt::Doc;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let backend = Sqlite::in_memory().await?;
let (instance, _) = Instance::connect_or_create_backend(
    Box::new(backend),
    eidetica::NewAccount::passwordless("admin"),
).await?;

// Create and login a passwordless user via the bootstrapped admin
// (the new user gets an Ed25519 keypair automatically)
let admin = instance.login_account("admin", "").await?;
admin.admin().await?.create_account(eidetica::NewAccount::passwordless("alice")).await?;
let mut user = instance.login_account("alice", "").await?;

// Create a database using the user's default key
let mut settings = Doc::new();
settings.set("name", "my_database");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;

// All operations are now authenticated
let txn = database.new_transaction().await?;
// ... make changes ...
txn.commit().await?;  // Automatically signed
Ok(())
}

Key Concepts

Mandatory Authentication: Every entry must be signed - no exceptions.

First Account Becomes Admin: Whoever is named in the NewAccount passed to Instance::connect_or_create (URL-based) or Instance::create_backend (pre-built backend) is the instance’s first user and is automatically granted Admin(0) on the system databases (_users, _databases). There is no hidden default account: production deployments run eidetica daemon init --username <NAME> [--password PASS | --passwordless] with explicit credentials. Subsequent users are created by an existing admin via admin.admin().await?.create_account(NewAccount::…) and land as non-admins until promoted.

Permission Levels:

• Admin: Can modify settings and manage keys
• Write: Can read and write data
• Read: Can only read data

Key Storage: Public keys belong in database Auth settings; private-key custody is separate. The login convenience path stores authenticated encrypted credential bundles in native installation-local records, not replicated Account Entries. Passwords are required string values; every string is accepted, including empty, without length/complexity restrictions or an insecure-use acknowledgement. NewAccount::passwordless(name) is sugar for NewAccount::with_password(name, ""); login uses login_account(name, ""). Empty-string wrapping uses the same Argon2id/AES-256-GCM format, but provides no meaningful confidentiality against a bundle holder. Unlocked signers remain usable without re-entering the password; rewrapping a password is not signing-key revocation. There is no automatic migration or old plaintext-format import. For explicit caller-held keys and daemon-independent signing, see credential bundles and signers.

Common Tasks

Adding Users

Give other users access to your database:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::SettingsStore};
use eidetica::auth::{AuthKey, Permission};
use eidetica::auth::crypto::generate_keypair;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
// Setup database for testing
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "auth_example");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
let transaction = database.new_transaction().await?;
// Generate a keypair for the new user
let (_alice_signing_key, alice_verifying_key) = generate_keypair();
let settings_store = transaction.get_settings()?;

// Add a user with write access (indexed by pubkey, name is optional metadata)
let user_key = AuthKey::active(Some("alice_laptop"), Permission::Write(10));
settings_store.set_auth_key(&alice_verifying_key, user_key).await?;

transaction.commit().await?;
Ok(())
}

Making Data Public (Read-Only)

Allow anyone to read your database:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite};
use eidetica::store::SettingsStore;
use eidetica::auth::{AuthKey, Permission};
use eidetica::crdt::Doc;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
let transaction = database.new_transaction().await?;
let settings_store = transaction.get_settings()?;

// Global permission for public read access
// Any valid signature is accepted
let public_key = AuthKey::active(None, Permission::Read);
settings_store.set_global_auth_key(public_key).await?;

transaction.commit().await?;
Ok(())
}

Collaborative Databases (Read-Write)

Create a collaborative database where anyone can read and write without individual key management:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite};
use eidetica::store::SettingsStore;
use eidetica::auth::{AuthKey, Permission};
use eidetica::crdt::Doc;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "collaborative_notes");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;

// Set up global write permissions
let transaction = database.new_transaction().await?;
let settings_store = transaction.get_settings()?;

// Global permission allows any device to read and write
let collaborative_key = AuthKey::active(None, Permission::Write(10));
settings_store.set_global_auth_key(collaborative_key).await?;

transaction.commit().await?;
Ok(())
}

How it works:

1. Any device can bootstrap without approval (global permission grants access)
2. Devices discover available SigKeys using Database::find_sigkeys()
3. Select a SigKey from the available options (will include "*" for global permissions)
4. Open the database with the selected SigKey
5. All transactions automatically use the configured permissions
6. No individual keys are added to the database’s auth settings

Example of opening a collaborative database:

use eidetica::{Instance, backend::database::Sqlite};

let instance = Instance::open_backend(Box::new(Sqlite::in_memory().await?)).await?;

// Log in as an existing user
let mut user = instance.login_account("bob", "").await?;

let database_root_id = /* ID from existing collaborative database */;

// Open the database — the AccountSession resolves the signing key automatically
let database = user.open_database(&database_root_id).await?;

// Create transactions as usual — automatically signed with the user's key
let txn = database.new_transaction().await?;
// ... make changes ...
txn.commit().await?;

This is ideal for:

• Team collaboration spaces
• Shared notes and documents
• Public wikis
• Development/testing environments

Security note: Use appropriate permission levels. Write(10) allows Write and Read operations but not Admin operations (managing keys and settings).

Revoking Access

Remove a user’s access:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite};
use eidetica::store::SettingsStore;
use eidetica::auth::{AuthKey, Permission};
use eidetica::auth::crypto::generate_keypair;
use eidetica::crdt::Doc;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
// Generate a keypair for alice
let (_alice_signing_key, alice_verifying_key) = generate_keypair();
// First add alice key so we can revoke it
let transaction_setup = database.new_transaction().await?;
let settings_setup = transaction_setup.get_settings()?;
settings_setup.set_auth_key(&alice_verifying_key, AuthKey::active(Some("alice"), Permission::Write(10))).await?;
transaction_setup.commit().await?;
let transaction = database.new_transaction().await?;

let settings_store = transaction.get_settings()?;

// Revoke the key by its pubkey identifier
settings_store.revoke_auth_key(&alice_verifying_key).await?;

transaction.commit().await?;
Ok(())
}

Note: Historical entries created by revoked keys remain valid.

Historical Signatures and Operating Keys

Database::verify_entry_signature(id) checks historical authorization and the signature, not current authority or application identity. Service clients run the same check on the daemon; the handle’s acting identity needs current Read on the target and every claimed delegated database, not Write or Admin. Missing evidence or an unsupported query is not success.

To identify an operating key, fetch the Entry through an authorized handle, require historical verification to succeed, then call auth::crypto::verify_entry_signature(&entry, &public_key) with concrete authorized keys. Read the identity database through its own authorized handle, and test each candidate if key names collide; names, hints and record labels do not identify a signer. Check current membership and application policy separately: later revocation does not invalidate historical signatures.

Multi-Account Setup Example

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::SettingsStore};
use eidetica::auth::{AuthKey, Permission};
use eidetica::auth::crypto::generate_keypair;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
// Setup database for testing
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "multi_user_example");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
let transaction = database.new_transaction().await?;
let settings_store = transaction.get_settings()?;

// Generate keypairs for different users
let (_super_admin_signing_key, super_admin_verifying_key) = generate_keypair();
let (_dept_admin_signing_key, dept_admin_verifying_key) = generate_keypair();
let (_user1_signing_key, user1_verifying_key) = generate_keypair();

// Set auth keys directly
// Super admin (priority 0 - highest)
settings_store.set_auth_key(&super_admin_verifying_key, AuthKey::active(
    Some("super_admin"),
    Permission::Admin(0),
)).await?;

// Department admin (priority 10)
settings_store.set_auth_key(&dept_admin_verifying_key, AuthKey::active(
    Some("dept_admin"),
    Permission::Admin(10),
)).await?;

// Regular users (priority 100)
settings_store.set_auth_key(&user1_verifying_key, AuthKey::active(
    Some("user1"),
    Permission::Write(100),
)).await?;

transaction.commit().await?;
Ok(())
}

Advanced: Cross-Database Authentication (Delegation)

Delegation allows databases to reference other databases as sources of authentication keys. This enables powerful patterns like:

• Users manage their own keys in personal databases
• Multiple projects share authentication across databases
• Hierarchical access control without granting admin privileges

How Delegation Works

When you delegate to another database:

1. The delegating database references another database in its _settings.auth
2. The delegated database maintains its own keys in its _settings.auth
3. Permission clamping ensures delegated keys can’t exceed specified bounds
4. Delegation paths reference databases by their root entry ID and resolve the final key in the delegated database

Basic Delegation Setup

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc};
use eidetica::auth::{DelegatedTreeRef, Permission, PermissionBounds, TreeReference};
use eidetica::store::SettingsStore;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let default_key = user.get_default_key()?;

// Create user's personal database
let alice_database = user.create_database(Doc::new(), &default_key).await?;

// Create main project database
let project_database = user.create_database(Doc::new(), &default_key).await?;
// Get the user's database root and current tips
let user_root = alice_database.root_id().clone();
let user_tips = alice_database.snapshot().await?.into_tips();

// Add delegation reference to project database
let transaction = project_database.new_transaction().await?;
let settings = transaction.get_settings()?;

settings.add_delegated_tree(DelegatedTreeRef {
    permission_bounds: PermissionBounds {
        max: Permission::Write(15),
        min: Some(Permission::Read),
    },
    tree: TreeReference {
        root: user_root,
        tips: user_tips,
    },
}).await?;

transaction.commit().await?;
Ok(())
}

Now any key in Alice’s personal database can access the project database, with permissions clamped to the specified bounds.

Understanding Delegation Paths

Critical concept: A delegation path traverses through databases using two different types of identifiers:

1. Root tree IDs - Identify delegated databases (DelegatedTreeRef stored by root ID)
2. Key hints - Identify signing keys (by pubkey or name) in the final database

Delegated Tree References

Delegations are stored in the delegating database’s auth settings by the delegated tree’s root entry ID:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc};
use eidetica::auth::{DelegatedTreeRef, Permission, PermissionBounds, TreeReference};
use eidetica::store::SettingsStore;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let default_key = user.get_default_key()?;
let alice_db = user.create_database(Doc::new(), &default_key).await?;
let alice_root = alice_db.root_id().clone();
let alice_tips = alice_db.snapshot().await?.into_tips();
let project_db = user.create_database(Doc::new(), &default_key).await?;
let transaction = project_db.new_transaction().await?;
let settings = transaction.get_settings()?;
// In project database: delegation stored by Alice's database root ID
settings.add_delegated_tree(DelegatedTreeRef {
    tree: TreeReference {
        root: alice_root,  // <- Root ID used as storage key
        tips: alice_tips,
    },
    permission_bounds: PermissionBounds {
        max: Permission::Write(15),
        min: Some(Permission::Read),
    },
}).await?;
transaction.commit().await?;
Ok(())
}

This creates an entry in the project database’s auth settings:

• Key: The root entry ID of Alice’s database (a CID string, e.g., bafyr4i...)
• Value: DelegatedTreeRef with permission bounds and tree reference

Signing Key Names

These are names in the delegated database’s auth settings that point to public keys:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc};
use eidetica::auth::{AuthKey, Permission};
use eidetica::store::SettingsStore;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let default_key_id = user.get_default_key()?;
let alice_db = user.create_database(Doc::new(), &default_key_id).await?;
let transaction = alice_db.new_transaction().await?;
let settings = transaction.get_settings()?;
// In Alice's database: keys are indexed by pubkey
// (The default key was added automatically during bootstrap)
// We can update it with a descriptive name
settings.set_auth_key(
    &default_key_id,
    AuthKey::active(
        Some("alice_work"),  // Optional name metadata
        Permission::Write(10),
    )
).await?;
transaction.commit().await?;
Ok(())
}

This updates the entry in Alice’s database auth settings:

• Key: Indexed by the pubkey string (e.g., "ed25519:ABC...")
• Name: "alice_work" (optional human-readable metadata)
• Permissions: The access level for this key

Using Delegated Keys

A delegation path names each delegated root and the snapshot the signer observed. Claims must cover the configured pointer and observations inherited from all parents; complete locally Verified dependency history is required. Admins may rewind a configured pointer, but that alone does not erase inherited floors. Effective removal resets only the direct component, not nested observations. See the canonical causal rules and examples. If proof is incomplete, verify dependencies first and retry; automatic dependency acquisition/retry is deferred to PR #126.

A delegation path is a sequence of steps that traverses from the delegating database to the signing key:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc};
use eidetica::auth::{SigKey, DelegationStep};
use eidetica::store::DocStore;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let default_key = user.get_default_key()?;
let project_db = user.create_database(Doc::new(), &default_key).await?;
let user_db = user.create_database(Doc::new(), &default_key).await?;
let user_tips = user_db.snapshot().await?.into_tips();
// Create a delegation path:
// - path: list of delegated trees to traverse (by root ID)
// - hint: identifies the final signer in the last delegated tree
let delegation_path = SigKey::Delegation {
    path: vec![
        // Step: Traverse to Alice's database (using its root ID)
        DelegationStep {
            tree: user_db.root_id().clone(),  // Root ID of delegated database
            tips: user_tips,  // Tips for Alice's database
        },
    ],
    // Final signer hint - resolved in Alice's database auth settings
    hint: eidetica::auth::KeyHint::from_name("alice_laptop"),
};

// Use the delegation path to create an authenticated transaction
// Note: This requires the actual signing key to be available
// project_database.new_transaction_with_sig_key(delegation_path)?;
Ok(())
}

Path traversal:

1. Start in project database auth settings
2. Look up root ID in path → finds DelegatedTreeRef (stored by that root ID) → jumps to Alice’s database
3. Use hint to find the signer: look up "alice_laptop" (by name) → finds AuthKey → gets Ed25519 public key
4. Use that public key to verify the entry signature

Permission Clamping

Permissions from delegated databases are automatically clamped:

Account DB key: Admin(5)  →  Project DB clamps to: Write(15)  (max bound)
Account DB key: Write(10) →  Project DB keeps:      Write(10) (within bounds)
Account DB key: Read      →  Project DB keeps:      Read      (above min bound)

Rules:

• If delegated permission > max bound: lowered to max
• If delegated permission < min bound: raised to min (if specified)
• Permissions within bounds are preserved
• Admin permissions only apply within the delegated database

This makes it convenient to reuse the same validation rules across both databases. Only an Admin can grant permissions to a database by modifying the Auth Settings, but we can grant lower access to an Account, and allow it to use any key they want, by granting access to an Account-controlled database and giving that the desired permissions. The Account can then manage its own keys using its own Admin keys, under exactly the same rules.

Multi-Level Delegation

Delegated databases can themselves delegate to other databases, creating chains:

// Entry signed through a delegation chain:
{
  "auth": {
    "sig": "...",
    "key": [
      {
        "key": "team@example.com",      // Step 1: Delegation ref in Main DB → Team DB
        "tips": ["team_db_tip"]
      },
      {
        "key": "alice@example.com",     // Step 2: Delegation ref in Team DB → Alice's DB
        "tips": ["alice_db_tip"]
      },
      {
        "key": "alice_laptop",          // Step 3: Signing key in Alice's DB → pubkey
        // No tips - this is a pubkey, not a tree
      }
    ]
  }
}

Path traversal:

1. Look up "team@example.com" in Main DB → finds DelegatedTreeRef → jump to Team DB
2. Look up "alice@example.com" in Team DB → finds DelegatedTreeRef → jump to Alice’s DB
3. Look up "alice_laptop" in Alice’s DB → finds AuthKey → get Ed25519 public key
4. Use that public key to verify the signature

Each level applies its own permission clamping, with the final effective permission being the minimum across all levels.

Common Delegation Patterns

Account-Managed Access:

Project DB → delegates to → Alice's Personal DB
                              ↓
                         Alice manages her own keys

Team Hierarchy:

Main DB → delegates to → Team DB → delegates to → Account DB
          (max: Admin)            (max: Write)

Cross-Project Authentication:

Project A ───┐
             ├→ delegates to → Shared Auth DB
Project B ───┘

Key Aliasing

Auth settings can contain multiple names for the same public key with different permissions:

{
  "_settings": {
    "auth": {
      "Ed25519:abc123...": {
        "pubkey": "Ed25519:abc123...",
        "permissions": "admin:0",
        "status": "active"
      },
      "alice_work": {
        "pubkey": "Ed25519:abc123...",
        "permissions": "write:10",
        "status": "active"
      },
      "alice_readonly": {
        "pubkey": "Ed25519:abc123...",
        "permissions": "read",
        "status": "active"
      }
    }
  }
}

This allows:

• The same key to have different permission contexts
• Readable key names for human-friendly lookups
• Fine-grained access control based on how the key is referenced

Upgrade prerequisite

Before trusting an existing database under these rules, follow the offline trust-reset and re-verification procedure. There is no automatic migration: skipping it may trust legacy Verified labels. Clearing only the derived cache is not a substitute.

Best Practices

1. Use descriptive key names: "alice_laptop", "deploy_bot" for keys that will be looked up by name
2. Set appropriate permission bounds: Don’t grant more access than needed
3. Manage delegation tips deliberately: A pointer rewind alone does not reset inherited floors. See the rewind, effective-removal and nested examples before using removal as a recovery tool
4. Track delegated database root IDs: Delegation paths use root IDs, so document which IDs correspond to which databases
5. Document delegation chains: Complex hierarchies can be hard to debug

See Also

• Delegation Design - Technical details
• Permission System - How permissions work

Troubleshooting

“Authentication failed”: Check that:

• The key exists in database settings
• The key status is Active (not Revoked)
• The key has sufficient permissions for the operation

“Key name conflict”: When using set_auth_key() with different public key:

• set_auth_key() provides upsert behavior for same public key
• Returns KeyNameConflict error if key name exists with different public key
• Use get_auth_key() to check existing key before deciding action

“Cannot modify key”: Admin operations require:

• Admin-level permissions
• Equal or higher priority than the target key

Multi-device conflicts: During bootstrap sync between devices:

• If same key name with same public key: Operation succeeds (safe)
• If same key name with different public key: Operation fails (prevents conflicts)
• Consider using device-specific key names like “alice_laptop”, “alice_phone”

Network partitions: Authentication changes merge automatically using Last-Write-Wins. The most recent change takes precedence.

Bootstrap Security Policy

When new devices join existing databases through bootstrap synchronization, Eidetica provides two approval methods to balance security and convenience.

Bootstrap Approval Methods

Eidetica supports two bootstrap approval approaches, checked in this order:

1. Global Wildcard Permissions - Databases with global ‘*’ permissions automatically approve bootstrap requests if the requested permission is satisfied
2. Manual Approval - Default secure behavior requiring admin approval for each device

Default Security Behavior

By default, bootstrap requests are rejected for security:

use eidetica::{sync::DatabaseTicket, account::types::SyncSettings};

// Bootstrap will fail without explicit policy configuration
user.request_database_access(
    &sync,
    &ticket,
    &key_id,  // Account's key ID
    Permission::Write(100),
    SyncSettings::disabled(),
    None,
).await; // Returns PermissionDenied error

Global Wildcard Permissions (Recommended for Collaboration)

The simplest approach for collaborative databases is to use global wildcard permissions:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::SettingsStore};
use eidetica::auth::{AuthKey, Permission};
use eidetica::auth::crypto::generate_keypair;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "wildcard_example");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
let (_admin_signing_key, admin_verifying_key) = generate_keypair();
let transaction = database.new_transaction().await?;
let settings_store = transaction.get_settings()?;

// Add admin key
settings_store.set_auth_key(&admin_verifying_key, AuthKey::active(
    Some("admin"),
    Permission::Admin(1),
)).await?;

// Add global wildcard permission for automatic bootstrap
settings_store.set_global_auth_key(AuthKey::active(
    None,
    Permission::Write(10),  // Allows Read and Write(11+) requests
)).await?;

transaction.commit().await?;
Ok(())
}

Benefits:

• No per-device key management required
• Immediate bootstrap approval
• Simple configuration - one permission setting controls all devices
• See Bootstrap Guide for details

Manual Approval Process

For controlled access scenarios, use manual approval to review each bootstrap request:

Security Recommendations:

• Development/Testing: Use global wildcard permissions for convenience
• Production: Use manual approval for controlled access
• Team Collaboration: Use global wildcard permissions with appropriate permission levels
• Public Databases: Use global wildcard permissions for open access, or manual approval for controlled access

Bootstrap Flow

1. Client Request: Device requests access with public key and permission level
2. Global Permission Check: Server checks if global ‘*’ permission satisfies request
3. Global Permission Approval: If global permission exists and satisfies request, access is granted immediately
4. Manual Approval Queue: If no global permission, request is queued for admin review
5. Admin Decision: Admin explicitly approves or rejects the request
6. Database Access: Approved devices can read/write according to granted permissions

See Also

• Core Concepts - Understanding Databases and Entries
• Getting Started - Basic database setup
• Authentication Reference - Technical reference

Explicit credential bundles and signers

An Account locator, private credentials and database authorization are different things. A locator identifies the Account; it does not grant access or select an Identity. An installation should hold its own independently authorized keys, not another installation’s personal bundle. An app receives a separate scoped bundle with its own Account locator and key(s). Its scope comes from database Auth, not a bundle label.

Available interfaces

CredentialBundle encrypts exactly the supplied locator and keys with one password. AccountSigner binds a locator, an actual signing key and an explicit Auth identity immutably. These interfaces work without registering a daemon login; they do not discover, enroll or import keys from Account references.

The ordinary Instance::login_account convenience path stores login credentials, additional keys, signer hints and sync preferences in native installation-local records. These records are not Entries and do not replicate with AccountDB. Adding an Account reference neither imports credentials nor activates its target. Existing Account histories are not made secret-free by removing current fields; this change does not delete or migrate live data.

All convenience logins use the same authenticated CredentialBundle format as explicit caller-held credentials. AccountSession::from_credentials consumes an unlocked bundle and an explicit Account key selection; from_signer accepts one AccountSigner, including an explicit delegated identity. Neither constructor needs a registered daemon login. There is no pluggable client-owned credential-store backend, device-enrollment workflow or recovery/export feature in this API.

Convenience login, logout and returning login

One local password unlocks the explicitly stored keys for that login. Do not register an owner’s personal credentials as an app login: provision an independently authorized app key instead. Passwords are required strings; any string is allowed, including empty, without length/complexity restrictions or an insecure-use acknowledgement. NewAccount::passwordless(name) is just NewAccount::with_password(name, ""); use login_account(name, "") to return. All persisted login and added-key credentials use authenticated encryption with the same KDF, salt and nonce handling. Empty-string wrapping provides no meaningful confidentiality against a bundle holder, who can also unlock and reseal it. Signing uses already unlocked keys and does not prompt for a password on each operation.

use eidetica::{Instance, NewAccount, backend::database::InMemory};
#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut session) = Instance::create_backend(
    Box::new(InMemory::new()), NewAccount::with_password("local", "local password"),
).await?;
let key_id = session.add_private_key(Some("this installation")).await?;
let retained = session.acting_as(&key_id, None)?;
session.logout()?;
assert!(instance.login_account("local", "wrong password").await.is_err());
let returning = instance.login_account("local", "local password").await?;
assert_eq!(returning.get_signing_key(&key_id)?.public_key(), retained.key_id());
Ok(())
}

Connected logins use the same login_account call on an Instance::connect("unix:///path/to/eidetica.sock") result. Password derivation, key decryption and signing run on the client; additional local key records are returned only within that authenticated login’s scope. The socket is still filesystem-trusted, including the pre-auth encrypted-login lookup. Logout drops that session’s key owners; it does not revoke Auth or erase independent signer/database handles.

Set up, unlock and retain an explicit signer

Supply only keys the caller is intended to hold. The locator below is an example value; real callers use the existing Account’s root ID and obtain authorization separately.

#![allow(unused)]
fn main() {
use eidetica::{ID, account::{AccountSigner, CredentialBundle}, auth::crypto::generate_keypair};

let account_id = ID::from_bytes("example-account-locator");
let (installation_key, key_id) = generate_keypair();
let bundle = CredentialBundle::seal(&account_id, &[installation_key], "local password")?;
let saved_bytes = serde_json::to_vec(&bundle)?; // Write to local storage, never an Account Entry.

// Read the encrypted bytes later and unlock on the client.
let saved: CredentialBundle = serde_json::from_slice(&saved_bytes)?;
let unlocked = saved.unlock_for(&account_id, "local password")?;
let acting = AccountSigner::new(account_id.clone(), unlocked.signer(&key_id)?);
assert_eq!(acting.key_id(), key_id);
assert!(saved.unlock("wrong password").is_err());
assert!(saved.unlock_for(&ID::from_bytes("another-account"), "local password").is_err());

// Ending this unlock does not revoke independently retained signer handles.
drop(unlocked);
let relogin = saved.unlock_for(&account_id, "local password")?;
assert_eq!(relogin.key_ids(), vec![key_id]);
assert_eq!(acting.account_id(), &account_id);
Ok::<(), Box<dyn std::error::Error>>(())
}

AccountSession::import_private_key(key, label) adds an explicitly supplied key to that session’s local bundle using the same observed-bundle update as key generation. It rejects duplicate keys without replacing their metadata and does not enroll Identity membership, grant database Auth or activate a target.

UnlockedCredentials::add_key adds only an explicitly supplied key. Call seal() and explicitly persist the result to save that addition; a failure to save does not update the old encrypted bundle. signer(key_id) fails for an absent key rather than falling back to a personal or default signer. rewrap(new_password) produces a new encrypted bundle with the same keys and a new salt. It is not key rotation or revocation: old copies remain decryptable with their old password, and retained handles can still sign. Losing every usable key/bundle/password has no automatic recovery guarantee.

Full sessions without daemon login storage

Use a caller-unlocked bundle when the application, rather than the daemon login directory, owns credential storage. The session checks Account Auth with the selected actual signer before returning. This example uses an existing authorized key; provisioning a separate app Account/key and device enrollment are separate operations.

use eidetica::{Instance, NewAccount, account::{AccountSession, CredentialBundle}, backend::database::InMemory, crdt::Doc};
#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, owner) = Instance::create_backend(Box::new(InMemory::new()), NewAccount::with_password("example", "example password")).await?;
let account_id = owner.account().id().clone();
let key_id = owner.get_default_key()?;
let saved = CredentialBundle::seal(&account_id, &[owner.get_signing_key(&key_id)?], "client password")?;
owner.logout()?;
let unlocked = saved.unlock_for(&account_id, "client password")?;
let mut session = AccountSession::from_credentials(&instance, unlocked, &key_id).await?;
assert!(session.account_info().is_none()); // No daemon login registration was used.
let target = session.create_database(Doc::new(), &key_id).await?;
session.set_reference("example", "root", target.root_id()).await?;
let retained = session.acting_as(&key_id, None)?;
session.logout()?;
let reopened = AccountSession::from_signer(&instance, retained).await?;
assert_eq!(reopened.reference("example", "root").await?, Some(target.root_id().clone()));
assert!(reopened.databases().await?.is_empty()); // Activation was session-local.
Ok(())
}

For a service, create a fresh proof-authenticated connection first with AccountSigner::connect, then pass it to the full-session constructor. The constructor does not reauthenticate an existing connection or fall back to a daemon login.

#![allow(unused)]
fn main() {
use eidetica::{ID, Result, account::{AccountSession, AccountSigner, CredentialBundle}, auth::crypto::PublicKey};

async fn connect_session(url: &str, account_id: &ID, bundle: &CredentialBundle, password: &str, key_id: &PublicKey) -> Result<AccountSession> {
    let unlocked = bundle.unlock_for(account_id, password)?;
    let signer = AccountSigner::new(account_id.clone(), unlocked.signer(key_id)?);
    let connected = signer.connect(url).await?;
    AccountSession::from_credentials(&connected, unlocked, key_id).await
}
}

Caller-held sessions keep key hints, tracking and activation preferences only in session memory. They do not register logins, write native daemon credential records or enable background synchronization. login_id() and username() are empty and account_info() is None for these sessions. A new session can read portable references but does not inherit the previous session’s local target mappings. Use track_database with a held key or an explicit AccountSigner to open an authorized target. A reference never supplies a missing signer or membership.

Keys added through a caller-held session belong only to that session until the caller explicitly saves them in its own credential store. They do not update the original encrypted bundle. To persist such a key, supply session.get_signing_key(&new_id)? to the caller’s unlocked bundle with add_key, then seal and save it explicitly. This is low-level storage control, not a recovery/export service.

Embedded and service use

acting.open_database(&instance, &root_id).await opens the exact target under the selected signer and Auth identity. Target Auth and verified proof availability still apply. A direct key needs no Identity database. For delegation, construct the handle with an explicit DatabaseKey/SigKey representing the chosen Identity path, or use with_identity to obtain a new handle. An invalid path fails even if the same key has a separate direct grant. Never turn an organizational Identity reference or lookup hint into an authorization path automatically.

AccountSession::acting_as(&key_id, None) selects a direct signer from an already unlocked session. Passing Some(identity) selects that exact Auth identity. Both return a new AccountSigner; existing handles retain their original keys and identities. logout() consumes the session, not independent handles or other connections.

For a service, use acting.connect("unix:///path/to/eidetica.sock").await to create a fresh connection and prove possession without fetching a login record. Then use open_database on that instance. The daemon receives public keys and challenge signatures, not the caller’s password or private keys. A new selection does not reauthenticate or silently rebind existing connections. No GUI split or untrusted-app sandbox is provided.

The existing socket is for filesystem-trusted local callers, not hostile clients. Such callers can still request existing encrypted login records; processes sharing backend-file access are not isolated from each other. The optional dashboard retains its existing daemon-process signing sessions during this transition. See local storage and session ownership and the service threat model.

Encryption Guide

PasswordStore<S> adds password-based encryption to a Store. It protects Store configuration, Entry payloads, and derived cached state while preserving the wrapped Store’s API.

This is separate from password protection for a user’s signing key. Logging in unlocks a signing key; it does not unlock a PasswordStore. Each encrypted Store has its own password and salt and must be opened in each transaction that uses it.

Quick start

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::{PasswordStore, DocStore}};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let backend = Box::new(Sqlite::in_memory().await?);
let (instance, mut user) = eidetica::Instance::create_backend(
    backend,
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "secrets_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
let tx = database.new_transaction().await?;
let mut encrypted = tx.get_store::<PasswordStore<DocStore>>("secrets").await?;
encrypted.initialize("my_password", Doc::new()).await?;

let docstore = encrypted.inner().await?;
docstore.set("api_key", "sk-secret-12345").await?;
tx.commit().await?;
Ok(())
}

initialize is for a new Store. It records the public encryption parameters and an encrypted copy of the wrapped Store’s type and configuration. The transaction is then unlocked and can return the inner Store.

Opening an existing Store

Open the wrapper before asking for the inner Store:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::{PasswordStore, DocStore}};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let backend = Box::new(Sqlite::in_memory().await?);
let (instance, mut user) = eidetica::Instance::create_backend(
    backend,
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "secrets_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
{
    let tx = database.new_transaction().await?;
    let mut encrypted = tx.get_store::<PasswordStore<DocStore>>("secrets").await?;
    encrypted.initialize("my_password", Doc::new()).await?;
    let docstore = encrypted.inner().await?;
    docstore.set("secret", "value").await?;
    tx.commit().await?;
}
let tx = database.new_transaction().await?;
let mut encrypted = tx.get_store::<PasswordStore<DocStore>>("secrets").await?;
encrypted.open("my_password")?;

let docstore = encrypted.inner().await?;
let _secret = docstore.get("secret").await?;
tx.commit().await?;
Ok(())
}

A wrong password cannot decrypt the wrapped Store metadata, so open fails before the inner Store is available. There is no recovery key or password-reset operation. Losing the password loses access to the Store; back up the password separately from the database.

The password and derived key remain in the client process until the last clone of the opened transaction drops. Their owning buffers, and PasswordStore’s temporary keys and plaintext JSON buffers, are cleared when dropped, including after errors. Applications should not assume that their own copies of the password, plaintext values returned to them, or parser, cryptographic or allocator copies are erased from memory. Errors about malformed decrypted data do not include the decrypted content. Keep transactions short and avoid logging passwords or decrypted values.

Choosing a wrapped Store

PasswordStore<S> can wrap any Store type. Its persistent layout depends on the wrapped Store’s state model.

DocStore and other opaque Stores

Opaque Stores cache one encrypted whole-state value. Reads still return the normal DocStore values, but a cold read may decrypt and merge the Store’s history before publishing that encrypted cache value.

caller                         backend
"api_key" -> "secret"         reserved cache key 0x00 -> nonce || ciphertext

The backend does not receive the plaintext document through the Store-state cache.

Table<T> row caches

Table<T> supports individually addressable cached rows. A password-wrapped Table transforms both parts of each cached row:

logical row                         derived cache record
"account-42" -> JSON row     =>    keyed_hash("account-42") -> nonce || ciphertext

The ciphertext contains both the logical key and the row bytes. This lets the client verify that a record was returned from the physical key where it belongs before returning plaintext to the Table.

Point reads compute the 32-byte physical key and fetch one encrypted record. set, insert, update, and delete operate on logical keys inside the transaction; commit writes the normal encrypted Entry delta. They do not edit a published cache generation. A later read at the new tips uses another derived generation, built from immutable Entries.

Scans and search use physical-key order. This order is deterministic for one encrypted Store but is not lexicographic primary-key order. TablePage::next is an exclusive cursor in that physical order. Treat it as opaque: pass it unchanged to the next scan_page call for the same Store and password. The cached path and the history fallback use the same physical order, so a continuation can cross between them without changing cursor domains.

Other Store types

A wrapped Store that does not define a row projection uses the opaque whole-state format. For example, PasswordStore<YDoc> does not expose YDoc fields as cached rows. Wrapping a type does not invent a record projection for it.

Service mode and synchronization

With a connected Instance, password derivation, decryption, CRDT merging for encrypted content, and row-envelope verification run in the client. Entry payloads and encrypted cache records cross the service socket as ciphertext. The daemon can serve a warm encrypted Table point read with point-record requests rather than a row scan or Store-history reconstruction.

Encrypted cache materializations uploaded by a client are scoped to that authenticated user. The daemon cannot validate their decrypted contents. The signed, content-addressed Entry DAG remains the source of truth, and peer synchronization still exchanges Entries rather than treating the cache as authoritative data.

What encryption hides and reveals

PasswordStore uses Argon2id and AES-256-GCM. It prevents a backend or relay without the password from reading wrapped configuration, Entry payloads, opaque cached state, Table logical keys, or Table row values.

It does not hide all metadata. A backend can observe:

• the database and Store receiving an operation;
• Entry graph structure, timing, and ciphertext sizes;
• cached record counts, access patterns, and page sizes;
• equality of repeated Table logical keys within and across cache generations for the same Store, because their keyed physical keys are stable;
• the public encryption algorithm, KDF parameters, salt, and format version in _index.

Authenticated encryption detects modification or relocation of a cached row between physical keys or Stores. It does not prove that a cache is fresh or complete, or that the backend returned every row. Derived cached state is disposable; when its correctness is in doubt, clear it and rebuild it from verified Entries.

Example: encrypted Table

extern crate eidetica;
extern crate tokio;
extern crate serde;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::{PasswordStore, Table}};
use serde::{Serialize, Deserialize};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let backend = Box::new(Sqlite::in_memory().await?);
let (instance, mut user) = eidetica::Instance::create_backend(
    backend,
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "creds_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
#[derive(Serialize, Deserialize, Clone)]
struct Credential {
    service: String,
    password: String,
}

let tx = database.new_transaction().await?;
let mut encrypted = tx.get_store::<PasswordStore<Table<Credential>>>("credentials").await?;
encrypted.initialize("vault_password", Doc::new()).await?;

let table = encrypted.inner().await?;
table.insert(Credential {
    service: "github.com".to_string(),
    password: "secret_token".to_string(),
}).await?;
tx.commit().await?;
Ok(())
}

See also

• PasswordStore API — API reference
• Stores — Store types and transaction usage
• Service mode — local daemon trust boundary
• Encryption internals — formats, derivation, and security boundaries

Synchronization Guide

Eidetica’s sync system enables real-time data synchronization between distributed peers.

Quick Start

1. Enable Sync

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let backend = Box::new(Sqlite::in_memory().await?);
let (instance, mut user) = Instance::create_backend(
    backend,
    eidetica::NewAccount::passwordless("alice"),
).await?;
instance.enable_sync().await?;

// Create and login a user via the bootstrapped admin
Ok(())
}

2. Start a Server

use eidetica::sync::transports::http::HttpTransport;

let sync = instance.sync().unwrap();

// Register transport with bind address and start accepting connections
sync.register_transport("http", HttpTransport::builder().bind("127.0.0.1:8080")).await?;
sync.accept_connections().await?;

3. Connect and Sync

use eidetica::sync::{Address, DatabaseTicket};

// Using a typed Address
sync.sync_with_peer(&Address::http("127.0.0.1:8080"), Some(&tree_id)).await?;

// Or using a ticket (contains both address and database ID)
let ticket: DatabaseTicket = "eidetica:?db=bafyr4i...&pr=http:127.0.0.1:8080".parse()?;
sync.sync_with_ticket(&ticket).await?;

The system automatically detects whether you need full bootstrap or incremental sync. Tickets embed the database ID, so sync_with_ticket requires no separate tree ID.

4. Share a Database

Sharing has two caller-visible operations: save this user’s durable signed preference, and separately create a point-in-time locator. The daemon’s combined configuration remains internal:

database.share().await?;

assert!(database.is_shared().await?);

let ticket = database.ticket().await?;
println!("Send this locator to your peer: {ticket}");

A successful write acknowledges durable intent, not daemon reconciliation or network readiness. If the connection fails while committing, the write may have succeeded; reading the user-scoped settings or retrying is safe. A ticket is a point-in-time locator and may contain no addresses when the owner is not currently advertising one. See Database Sharing for settings, ticket, and migration details.

A peer who receives the ticket calls sync.sync_with_ticket(&ticket) as shown in step 3. The ticket identifies the database and address hints; it does not grant access, so protected databases still require authorization or the bootstrap approval flow.

Connection Architecture

The sync system separates outbound and inbound connection handling:

• Outbound (sync_with_peer()): Works immediately after register_transport(). No server needed.
• Inbound (accept_connections()): Must be called to accept incoming connections.

register_transport()
    │ - Transport ready for outbound requests
    ▼
[OUTBOUND READY]
    │ - sync_with_peer() works
    │ - Push hooks work
    │ - NO incoming connections
    ▼
accept_connections()
    │ - Starts server on registered transports
    ▼
[FULL P2P]
    │ - Outbound works
    │ - Inbound works

This separation provides:

• Security by default: Nodes don’t accept incoming connections unless explicitly enabled
• Zero-config outbound: Applications can sync data immediately without server setup
• Flexible deployment: Support client-only, server-only, or full peer-to-peer modes

Transport Options

Eidetica supports multiple transports simultaneously, allowing peers to be reachable via different networks.

HTTP

Simple REST-based sync. Good for development and fixed-IP deployments.

use eidetica::sync::transports::http::HttpTransport;

sync.register_transport("http", HttpTransport::builder().bind("127.0.0.1:8080")).await?;
sync.accept_connections().await?;

Iroh P2P (Recommended)

QUIC-based with NAT traversal. Works through firewalls.

use eidetica::sync::transports::iroh::IrohTransport;

sync.register_transport("iroh", IrohTransport::builder()).await?;
sync.accept_connections().await?;
let my_address = sync.get_server_address_async().await?;  // Share this with peers

Eidetica uses iroh’s native default features and N0 networking settings, including relay selection and router port mapping. Unset relay settings also honor iroh’s IROH_FORCE_STAGING_RELAYS environment switch; explicit relay choices take precedence. By default, Iroh advertises and resolves addresses on the local network using mDNS, and publishes to and resolves from n0’s DNS service (via DNS and HTTPS). Known peers can be reached by endpoint ID even when their ticket’s direct addresses are missing or stale. This address lookup does not discover databases or grant access to them.

For same-host or same-LAN sync without public lookup or relay services:

use iroh::{RelayMode, endpoint::PortmapperConfig};
use eidetica::sync::transports::iroh::IrohTransport;

sync.register_transport(
    "iroh",
    IrohTransport::builder()
        .relay_mode(RelayMode::Disabled)
        .portmapper_config(PortmapperConfig::Disabled)
        .n0_dns(false),
).await?;
sync.accept_connections().await?;

Use .mdns(false) to disable local address advertisement and lookup where multicast is unavailable or unwanted. If mDNS cannot initialize, the transport logs a warning and still permits direct connections and any enabled n0 services. These lookup and portmapper selections are runtime builder options; the transport’s persisted node identity is unchanged. Supply the overrides again when constructing or registering the transport after a restart.

Multiple Transports

Enable multiple transports for maximum connectivity:

use eidetica::sync::transports::http::HttpTransport;
use eidetica::sync::transports::iroh::IrohTransport;

// Register both HTTP (for local network) and Iroh (for P2P)
sync.register_transport("http", HttpTransport::builder().bind("127.0.0.1:0")).await?;
sync.register_transport("iroh", IrohTransport::builder()).await?;

// Start servers on all transports
sync.accept_connections().await?;

// Get all server addresses
let addresses = sync.get_all_server_addresses_async().await?;
for (transport_name, addr) in addresses {
    println!("{}: {}", transport_name, addr);
}

Requests are automatically routed to the appropriate transport based on address type.

Declarative Sync API

For persistent sync relationships:

extern crate eidetica;
extern crate tokio;
use eidetica::sync::{SyncPeerInfo, Address};
use eidetica::auth::PublicKey;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let backend = Box::new(Sqlite::in_memory().await?);
let (instance, _) = Instance::connect_or_create_backend(
    backend,
    eidetica::NewAccount::passwordless("admin"),
).await?;
instance.enable_sync().await?;
let admin = instance.login_account("admin", "").await?;
admin.admin().await?.create_account(eidetica::NewAccount::passwordless("alice")).await?;
let mut user = instance.login_account("alice", "").await?;
let default_key = user.get_default_key()?;
let db = user.create_database(Doc::new(), &default_key).await?;
let tree_id = db.root_id().clone();
let sync = instance.sync().expect("Sync enabled");
let peer_pubkey = PublicKey::random();
// Register a peer for automatic background sync
let handle = sync.register_sync_peer(SyncPeerInfo {
    peer_pubkey,
    tree_id,
    addresses: vec![Address {
        transport_type: "http".to_string(),
        address: "http://peer.example.com:8080".to_string(),
    }],
    auth: None,
    display_name: Some("Peer Device".to_string()),
}).await?;
Ok(())
}

Background sync happens automatically. Check status with handle.status()?.

Sync Settings

Configure per-database sync behavior:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc};
use eidetica::account::types::SyncSettings;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let backend = Box::new(Sqlite::in_memory().await?);
let (instance, _) = Instance::connect_or_create_backend(
    backend,
    eidetica::NewAccount::passwordless("admin"),
).await?;
instance.enable_sync().await?;
let admin = instance.login_account("admin", "").await?;
admin.admin().await?.create_account(eidetica::NewAccount::passwordless("alice")).await?;
let mut user = instance.login_account("alice", "").await?;
let key = user.get_default_key()?;
let db = user.create_database(Doc::new(), &key).await?;
let db_id = db.root_id().clone();
// Track this database with sync on every commit + periodic sync every 60s
user.track_database(
    db_id,
    &user.get_default_key()?,
    SyncSettings::on_commit().with_interval(60),
).await?;
Ok(())
}

Authenticated Bootstrap

For joining databases that require authentication:

use eidetica::{sync::DatabaseTicket, account::types::SyncSettings};

// Request database access through AccountSession API
user.request_database_access(
    &sync,
    &ticket,
    &key_id,  // Account's key ID from user.add_private_key()
    eidetica::auth::Permission::Write,
    SyncSettings::on_commit(),
    None,
).await?;

See Bootstrap Guide for approval workflows.

Automatic Behavior

Once configured, the sync system handles:

• Immediate sync on commit (if sync_on_commit: true)
• Periodic sync at configured intervals
• Retry with exponential backoff for failed sends
• Bidirectional transfer in each sync operation

Troubleshooting

Example

See the Chat Example for a complete working application demonstrating multi-transport sync, bootstrap, and real-time updates.

Database Sharing

A Database opened or created through an AccountSession carries that caller’s private sharing capability. Its sharing methods work in embedded and service mode while keeping two kinds of state separate:

1. Database settings are replicated database-owned state, accessed through Database::get_settings().
2. SyncSettings are this caller’s durable installation-local preferences; they are not signed Account Entries.

Plain Database handles remain valid for ordinary reads and transactions, but cannot manage a user’s private sharing preference. Their sharing methods return an explicit missing-capability error.

Share a Database

AccountSession::create_database returns a capability-backed handle. For an existing tracked database, use AccountSession::open_database.

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, NewAccount, crdt::Doc};
#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, user) = Instance::connect_or_create(
    "memory://",
    NewAccount::passwordless("alice"),
).await?;
instance.enable_sync().await?;
let mut user = user.expect("memory backend is new");
let key = user.get_default_key()?;
let database = user.create_database(Doc::new(), &key).await?;

database.share().await?;

assert!(database.is_shared().await?);
println!("share {}", database.ticket().await?);
Ok(())
}

A successful call acknowledges durable native preference state. It does not say that the daemon has reconciled its internal configuration or is advertising an address. If the connection fails while committing, the write may have reached the owner. Safely retry it or read sync_settings().

Settings and Tickets

sync_settings() reads only this caller’s private preference. It does not expose the daemon’s configuration, another user’s preferences, peer telemetry, or runtime state. is_shared() returns its sync_enabled field.

ticket() is a separate point-in-time query and never changes settings. It requires this caller’s sharing setting to be enabled, even if another user keeps the daemon serving the database. When enabled, it returns a DatabaseTicket with the database ID and currently available owner addresses. It does not inspect daemon configuration or promise serving, reconciliation, or future reachability.

A ticket is a locator with address hints, not an access grant. The receiver still needs database authorization or the bootstrap and approval flow.

Inspect a Tracking Preference

Use AccountSession::tracked_database(&database_id).await to inspect one tracking row without opening or tracking the target database. Ok(Some(row)) contains this account’s preferences, including row.sync_settings.sync_enabled; Ok(None) means the row is absent. Authorization, storage read and decoding failures remain errors, not an absent row or a disabled preference.

This read uses native installation-local login state in embedded and service mode, or session-local state for caller-held credentials. It decodes the local record, never a replicated tracking Table or Store-state cache. It does not change keys or settings or grant access to the target database. Connected reads require the exact authenticated login scope; failures use the existing service error representation.

#![allow(unused)]
fn main() {
use eidetica::{AccountSession, ID};
async fn local_sync_preference(
    session: &AccountSession,
    database_id: &ID,
) -> eidetica::Result<Option<bool>> {
    Ok(session.tracked_database(database_id).await?
        .map(|row| row.sync_settings.sync_enabled))
}
}

Stop Sharing

database.stop_sharing() withdraws only this caller’s preference and preserves their other sync settings. Another user on the same instance can keep the daemon serving the database. The combined daemon state is intentionally not exposed.

Database-Centric Sharing

Sharing controls live on the Database handle returned by AccountSession. Use share, stop_sharing, sync_settings, and ticket on that handle; AccountSession does not expose ID-based sharing helpers.

Treat the returned ticket as a point-in-time locator, not proof that the preference was applied or that a peer can connect.

Bootstrapping

Overview

The Bootstrap system provides secure key management for Eidetica databases by controlling how new devices gain access to synchronized databases. It supports two approval methods:

1. Global Wildcard Permissions - Databases with global ‘*’ permissions automatically approve bootstrap requests without adding new keys
2. Manual Approval - Bootstrap requests are queued for administrator review and explicit approval

Global Permission Bootstrap

Global ‘*’ permissions provide the simplest and most flexible approach for collaborative or public databases:

How It Works

When a database has global permissions configured (e.g., {"*": {"pubkey": "*", "permissions": "Write: 10"}}), bootstrap requests are automatically approved if the requested permission level is satisfied by the global permission. No new keys are added to the database.

Devices use the global permission for both bootstrap approval and subsequent operations (transactions, reads, writes). The system automatically resolves to the global “*” key when a device’s specific key is not present in the database’s auth settings.

Advantages

• No key management: Devices don’t need individual keys added to database
• Immediate access: Bootstrap approval happens instantly
• Simple configuration: One permission setting controls all devices
• Flexible permissions: Set exactly the permission level you want to allow

Configuration Example

Configure a database with global write permission:

use eidetica::crdt::Doc;
use eidetica::auth::{AuthSettings, AuthKey, Permission};

// Create database settings with global permission
let mut settings = Doc::new();
settings.set("name", "shared_database");

// Configure auth settings
let mut auth_settings = AuthSettings::new();

// Add global permission for automatic bootstrap approval
auth_settings.add_key(
    "*",  // Wildcard matches any key
    AuthKey::active("*", Permission::Write(10))?,  // Allows Read and Write(11+) requests
)?;

settings.set("auth", auth_settings.to_doc());

// Create database through AccountSession API (specified key gets Admin)
let key_id = user.get_default_key()?;
let database = user.create_database(settings, &key_id).await?;

Permission Levels

Eidetica uses lower numbers = higher permissions:

• Global Write(10) allows: Read, Write(11), Write(15), etc.
• Global Write(10) denies: Write(5), Admin(*)

Choose your global permission level carefully based on your security requirements.

Client Workflow

From the client’s perspective, the bootstrap process follows these steps:

1. Initial Bootstrap Attempt

The client initiates a bootstrap request when it needs access to a synchronized database:

// Request database access through AccountSession API
user.request_database_access(
    &sync,
    &ticket,
    &key_id,                 // Client's key ID (from user.add_private_key())
    Permission::Write(5),    // Requested permission level
    SyncSettings::on_commit(),
    None,
).await

AccountSession::join takes the same arguments without the Sync handle and returns the opened database. It uses the instance’s own sync engine when embedded and the daemon’s when connected to a service daemon, where the user’s private key stays in the client process:

let database = user
    .join(&ticket, &key_id, Permission::Write(5), SyncSettings::on_commit(), None)
    .await?;

join has the same interface in embedded and connected modes and waits for the peer’s answer. A ticket is a locator, not an access grant. Success returns a Database after recording the selected user key’s SigKey mapping and the caller’s SyncSettings, so user.open_database() can open it again. SyncError::BootstrapPending changes no Account mappings or preferences: explicitly retry join with the same arguments after approval. SyncError::BootstrapRejected is terminal for that request and records no new mapping or preferences.

Success means the finite supplied response and its ancestry have verified locally and the selected key currently satisfies the requested permission; it is not global catch-up. Identical completed Account settings and established routing are not written again. Newer preferences observed after the network wait are preserved, and legacy provisional mappings are not deleted on Pending or rejection. An error, dropped future or disconnected caller can leave a peer request, replicated entries, routing or a persisted local account record without its acknowledgement. Reconnect and retry explicitly; a failed call does not imply rollback. There is no cancellation API, automatic restart-safe completion, or device enrollment. The client holds and signs with the user’s private keys; the daemon owns networking, without a second client sync engine. Later daemon replication uses its device key, which needs its own authorization or a global grant. User-key access alone does not promise ongoing private background replication.

Current recovery limits

Acquisition of an already-held database that the requesting key cannot read, including an orphan replica, remains refused. Join advertises empty tips and never sends local entries back, but this does not resolve native competing grant/revocation semantics. Missing delegated signing history leaves the affected entries Unverified and the transfer incomplete, not successful or terminally Failed. Native verification supports supplying and verifying the missing history before manually retrying on the same replica; join does not acquire that history automatically or bypass the held-tree Read gate. Complete held/unreadable, orphan and second-user recovery remains unsupported. The basic API uses existing native authority semantics; it does not fix competing grant/revocation resolution or guarantee that stale grants cannot restore access after ingestion. See the verification model for those native limitations.

2. Response Handling

The client must handle different response scenarios:

• Global Wildcard Permission Approved:

  • Request succeeds immediately
  • Client gains access via global permission
  • No individual key added to database
  • Can proceed with normal operations

• Manual Approval Required (default):

  • Request fails with an error
  • Error indicates request is “pending”
  • Bootstrap request is queued for admin review

3. Waiting for Approval

While the request is pending, the client has several options:

• Polling Strategy: Periodically retry sync operations
• Event-Based: Wait for notification from server (future enhancement)
• User-Triggered: Let user manually retry when they expect approval

4. After Admin Decision

If Approved:

• Retry user.request_database_access() with the same SyncSettings
• The retry records the selected settings while it synchronizes the database
• Once it succeeds, client can load and use the database normally

If Rejected:

• All sync attempts continue to fail
• Client remains unable to access the database
• May submit a new request with different parameters if appropriate

5. Retry Logic Example

use eidetica::{sync::DatabaseTicket, account::types::SyncSettings};

async fn bootstrap_with_retry(
    user: &mut AccountSession,
    sync: &Sync,
    ticket: &DatabaseTicket,
    key_id: &PublicKey,
) -> Result<()> {
    // Initial bootstrap request
    if let Err(_) = user.request_database_access(
        sync, ticket, key_id, Permission::Write(5), SyncSettings::on_commit(), None
    ).await {
        println!("Bootstrap request pending approval...");

        // Poll for approval (with backoff)
        for attempt in 0..10 {
            tokio::time::sleep(Duration::from_secs(30 * (attempt + 1))).await;

            // Retry with the same explicit preferences after potential approval.
            if user.request_database_access(
                sync, ticket, key_id, Permission::Write(5), SyncSettings::on_commit(), None
            ).await.is_ok() {
                println!("Access granted!");
                return Ok(());
            }
        }

        return Err("Bootstrap request timeout or rejected".into());
    }

    Ok(()) // Auto-approved
}

Usage Examples

Manual Approval Workflow

For administrators managing bootstrap requests:

// 1. List pending requests
let pending = sync.pending_bootstrap_requests()?;
for (request_id, request) in pending {
    println!("Request {}: {} wants {} access to tree {}",
        request_id,
        request.requesting_key_name,
        request.requested_permission,
        request.tree_id
    );
}

// 2. Approve a request (using AccountSession API)
user.approve_bootstrap_request(
    &sync,
    "bootstrap_a1b2c3d4...",
    &admin_key_id  // Your admin key ID (must have Admin permission)
).await?;

// 3. Or reject a request
user.reject_bootstrap_request(
    &sync,
    "bootstrap_e5f6g7h8...",
    &admin_key_id
).await?;

Complete Client Bootstrap Example

// Step 1: Initial bootstrap attempt with authentication
let bootstrap_result = user.request_database_access(
    &sync,
    &ticket,
    &key_id,  // Account's key ID from user.add_private_key()
    Permission::Write(5),
    SyncSettings::on_commit(),
    None,
).await;

// Step 2: Handle the response based on approval method
match bootstrap_result {
    Ok(_) => {
        // Global wildcard permission granted immediate access
        println!("Bootstrap approved via global permission! Access granted immediately.");
    },
    Err(e) => {
        // Manual approval required
        // The error indicates the request is pending
        println!("Bootstrap request submitted, awaiting admin approval...");

        // Step 3: Wait for admin to review and approve
        // Options:
        // a) Poll periodically
        // b) Wait for out-of-band notification
        // c) User-triggered retry

        // Step 4: After admin approval, retry with the same sync preferences.
        tokio::time::sleep(Duration::from_secs(30)).await;

        match user.request_database_access(
            &sync,
            &ticket,
            &key_id,
            Permission::Write(5),
            SyncSettings::on_commit(),
            None,
        ).await {
            Ok(_) => {
                println!("Access granted! Database synchronized.");
                // Client can now load and use the database
                let db = user.open_database(ticket.database_id()).await?;
            },
            Err(_) => {
                println!("Still pending or rejected. Check with admin.");
            }
        }
    }
}

// Handling rejection scenario
// If the request was rejected, all sync attempts will continue to fail
// The client will need to submit a new bootstrap request if appropriate

Security Considerations

Trust Model

• Global Wildcard Permissions: Trusts any device that can reach the sync endpoint

  • Suitable for: Development, collaborative projects, public databases
  • Risk: Any device can gain the configured global permissions
  • Benefit: Simple, immediate access for authorized scenarios

• Manual Approval: Requires explicit admin action for each device

  • Suitable for: Production, sensitive data, controlled access scenarios
  • Benefit: Complete control over which devices gain access
  • Risk: Administrative overhead for each new device

Troubleshooting

Common Issues

1. “Authentication required but not configured”

   • Cause: Sync handler cannot authenticate with target database
   • Solution: Ensure proper key configuration for database operations

2. “Invalid request state”

   • Cause: Attempting to approve/reject non-pending request
   • Solution: Check request status before operation

Performance Considerations

• Sync database grows linearly with request count
• Request queries are indexed by ID

Sync Quick Reference

A concise reference for Eidetica’s synchronization API with common usage patterns and code snippets.

Setup and Initialization

Basic Sync Setup

use eidetica::{Instance, backend::database::Sqlite};
use eidetica::sync::transports::http::HttpTransport;

// Create database with sync enabled
let backend = Box::new(Sqlite::in_memory().await?);
let (instance, mut user) = Instance::create_backend(
    backend,
    eidetica::NewAccount::passwordless("alice"),
).await?;
instance.enable_sync().await?;


// Register transport with bind address
let sync = instance.sync().unwrap();
sync.register_transport("http", HttpTransport::builder().bind("127.0.0.1:8080")).await?;
sync.accept_connections().await?;

Understanding BackgroundSync

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite};
use eidetica::sync::transports::http::HttpTransport;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
// Setup database instance with sync capability
let backend = Box::new(Sqlite::in_memory().await?);
let (db, _) = Instance::connect_or_create_backend(
    backend,
    eidetica::NewAccount::passwordless("admin"),
).await?;
db.enable_sync().await?;

// The BackgroundSync engine starts automatically with transport registration
let sync = db.sync().unwrap();
sync.register_transport("http", HttpTransport::builder().bind("127.0.0.1:0")).await?;

// Background thread configuration and behavior:
// - Command processing (immediate response to commits)
// - Periodic sync operations (5 minute intervals)
// - Retry queue processing (30 second intervals)
// - Connection health checks (60 second intervals)

// All sync operations are automatic - no manual queue management needed
println!("BackgroundSync configured with automatic operation timers");
Ok(())
}

Declarative Sync API (Recommended)

Register Sync Peer

Declare sync intent with automatic background synchronization:

extern crate eidetica;
extern crate tokio;
use eidetica::sync::{SyncPeerInfo, Address};
use eidetica::auth::PublicKey;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let backend = Box::new(Sqlite::in_memory().await?);
let (instance, _) = Instance::connect_or_create_backend(
    backend,
    eidetica::NewAccount::passwordless("admin"),
).await?;
instance.enable_sync().await?;
let admin = instance.login_account("admin", "").await?;
admin.admin().await?.create_account(eidetica::NewAccount::passwordless("alice")).await?;
let mut user = instance.login_account("alice", "").await?;
let default_key = user.get_default_key()?;
let db = user.create_database(Doc::new(), &default_key).await?;
let tree_id = db.root_id().clone();
let sync = instance.sync().expect("Sync enabled");
let peer_pubkey = PublicKey::random();
// Register a peer for persistent sync
let handle = sync.register_sync_peer(SyncPeerInfo {
    peer_pubkey,
    tree_id,
    addresses: vec![Address {
        transport_type: "http".to_string(),
        address: "http://peer.example.com:8080".to_string(),
    }],
    auth: None,
    display_name: Some("Peer Device".to_string()),
}).await?;

// Background sync engine now handles synchronization automatically
Ok(())
}

Monitor Sync Status

// Check current status
let status = handle.status()?;
println!("Has local data: {}", status.has_local_data);

// Wait for initial bootstrap
handle.wait_for_initial_sync().await?;

// Add more address hints
handle.add_address(Address {
    transport_type: "iroh".to_string(),
    address: "iroh://node_id".to_string(),
})?;

Legacy Sync API

Authenticated Bootstrap (Recommended for New Databases)

use eidetica::{sync::DatabaseTicket, account::types::SyncSettings};

// For new devices joining existing databases with authentication.
user.request_database_access(
    &sync,
    &ticket,
    &key_id,                         // Account's key ID from user.add_private_key()
    eidetica::auth::Permission::Write, // Requested permission level
    SyncSettings::on_commit(),
    None,
).await?;

// This automatically:
// 1. Connects to peer and performs handshake
// 2. Requests database access with specified permission level
// 3. Receives auto-approved access (or manual approval in production)
// 4. Downloads complete database state
// 5. Grants authenticated write access

Simplified Sync (Legacy/Existing Databases)

use eidetica::sync::{Address, DatabaseTicket};

// Using a typed Address
sync.sync_with_peer(&Address::http("peer.example.com:8080"), Some(&tree_id)).await?;

// Or using a ticket (contains address + database ID)
let ticket: DatabaseTicket = "eidetica:?db=bafyr4i...&pr=http:peer.example.com:8080".parse()?;
sync.sync_with_ticket(&ticket).await?;

// This automatically:
// 1. Connects to peer and performs handshake
// 2. Bootstraps database if you don't have it locally
// 3. Syncs incrementally if you already have the database
// 4. Handles peer registration internally

Database Discovery

use eidetica::sync::Address;

// Discover available databases on a peer
let peer_addr = Address::http("peer.example.com:8080");
let available_trees = sync.discover_peer_trees(&peer_addr).await?;
for tree in available_trees {
    println!("Available: {} ({} entries)", tree.tree_id, tree.entry_count);
}

// Bootstrap from discovered database
if let Some(tree) = available_trees.first() {
    sync.sync_with_peer(&peer_addr, Some(&tree.tree_id)).await?;
}

Manual Peer Registration (Advanced)

// Register peer manually (for advanced use cases)
let peer_key = "ed25519:abc123...";
sync.register_peer(peer_key, Some("Alice's Device"))?;

// Add addresses
sync.add_peer_address(peer_key, Address::http("192.168.1.100:8080")?)?;
sync.add_peer_address(peer_key, Address::iroh("iroh://peer_id")?)?;

// Use low-level sync with registered peer
sync.sync_tree_with_peer(&peer_key, &tree_id).await?;

// Note: Manual registration is usually unnecessary
// The sync_with_peer() method handles registration automatically

Peer Status Management

// List all peers
let peers = db.sync()?.list_peers()?;
for peer in peers {
    println!("{}: {} ({})",
        peer.pubkey,
        peer.display_name.unwrap_or("Unknown".to_string()),
        peer.status
    );
}

// Get specific peer info
if let Some(peer) = db.sync()?.get_peer_info(&peer_key)? {
    println!("Status: {:?}", peer.status);
    println!("Addresses: {:?}", peer.addresses);
}

// Update peer status
db.sync()?.update_peer_status(&peer_key, PeerStatus::Inactive)?;

Database Synchronization

Create and Share Database

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::DocStore};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
use eidetica::sync::transports::http::HttpTransport;
let backend = Box::new(Sqlite::in_memory().await?);
let (instance, _) = Instance::connect_or_create_backend(
    backend,
    eidetica::NewAccount::passwordless("admin"),
).await?;
instance.enable_sync().await?;
let sync = instance.sync().expect("Sync enabled");
sync.register_transport("http", HttpTransport::builder().bind("127.0.0.1:0")).await?;
sync.accept_connections().await?;
let admin = instance.login_account("admin", "").await?;
admin.admin().await?.create_account(eidetica::NewAccount::passwordless("alice")).await?;
let mut user = instance.login_account("alice", "").await?;
// Create a database to share
let mut settings = Doc::new();
settings.set("name", "My Chat Room");
settings.set("description", "A room for team discussions");

let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;

// Add some initial data
let txn = database.new_transaction().await?;
let store = txn.get_store::<DocStore>("messages").await?;
store.set("welcome", "Welcome to the room!").await?;
txn.commit().await?;

// Save durable sharing intent, then build a point-in-time locator.
database.share().await?;
let ticket = database.ticket().await?;
println!("Share this locator with peers: {ticket}");
Ok(())
}

For failure handling and migration from the removed ID-based AccountSession helpers, see Database Sharing.

Bootstrap from Shared Database

use eidetica::sync::{Address, DatabaseTicket};

// Join someone else's database using a ticket
let ticket: DatabaseTicket = "eidetica:?db=bafyr4i...&pr=http:peer.example.com:8080".parse()?;
sync.sync_with_ticket(&ticket).await?;

// Or use a typed Address with an explicit tree ID
sync.sync_with_peer(&Address::http("peer.example.com:8080"), Some(&room_id)).await?;

// You now have the full database locally
let database = db.load_database(&room_id).await?;
let txn = database.new_transaction().await?;
let store = txn.get_store::<DocStore>("messages").await?;
println!("Welcome message: {}", store.get_string("welcome").await?);

Ongoing Synchronization

// All changes automatically sync after bootstrap
let txn = database.new_transaction().await?;
let store = txn.get_store::<DocStore>("messages").await?;
store.set("my_message", "Hello everyone!").await?;
txn.commit().await?; // Automatically syncs to all connected peers

// Manually sync to get latest changes
sync.sync_with_peer(&Address::http("peer.example.com:8080"), Some(&tree_id)).await?;

Verifying Synced Entries

Entries received from a peer are stored Unverified — a peer cannot assert that data is trusted. By default a Database read exposes only the Verified frontier (the maximal all-Verified prefix of the history), so freshly synced, not-yet-verified data is invisible until this node validates it. A read triggers an opportunistic verification pass automatically, or you can drive it explicitly.

// Explicitly verify everything received so far. Validates each Unverified
// entry against the _settings it pins; prefix-closed (an entry verifies
// only once its whole history does). Already-Verified entries are skipped.
let report = database.verify().await?;
println!(
    "verified={}, failed={}, still_unverified={}",
    report.verified, report.failed, report.still_unverified
);

// Default read: only the Verified frontier is visible.
let tips = database.snapshot().await?;

// Opt into the looser view to also see Unverified tips (e.g. to inspect
// data whose pinned _settings this node has not received yet). Failed
// entries are always dropped, in both views.
let pending = database.clone().allow_unverified().snapshot().await?;

still_unverified counts entries left pending because their pinned _settings (or an ancestor) has not arrived yet — re-run verify() after more sync completes and they promote. failed counts entries that were checked and definitively rejected; those are dropped from reads permanently.

Advanced: Manual Sync Relationships

// For fine-grained control (usually not needed)
sync.add_tree_sync(&peer_key, &tree_id)?;

// List synced databases for peer
let databases = sync.get_peer_trees(&peer_key)?;

// List peers syncing a database
let peers = sync.get_tree_peers(&tree_id)?;

// Remove sync relationship
sync.remove_tree_sync(&peer_key, &tree_id)?;

Data Operations (Auto-Sync)

Basic Data Changes

use eidetica::store::DocStore;

// Any database operation automatically triggers sync
let txn = database.new_transaction().await?;
let store = txn.get_store::<DocStore>("data").await?;

store.set("message", "Hello World").await?;
store.set_path("user.name", "Alice").await?;
store.set_path("user.age", 30).await?;

// Commit triggers sync callbacks automatically
txn.commit().await?; // Entries queued for sync to all configured peers

Bulk Operations

// Multiple operations in single commit
let txn = database.new_transaction().await?;
let store = txn.get_store::<DocStore>("data").await?;

for i in 0..100 {
    store.set(&format!("item_{}", i), &format!("value_{}", i)).await?;
}

// Single commit, single sync entry
txn.commit().await?;

Monitoring and Diagnostics

Server Control

use eidetica::sync::transports::http::HttpTransport;

// Register transport and start accepting connections
let sync = db.sync()?;
sync.register_transport("http", HttpTransport::builder().bind("127.0.0.1:8080")).await?;
sync.accept_connections().await?;

// Check server status
if sync.is_server_running() {
    let addr = sync.get_server_address_async().await?;
    println!("Server running at: {}", addr);
}

// Stop server
sync.stop_server_async().await?;

Error Handling

Common Error Patterns

use eidetica::sync::SyncError;
use eidetica::sync::transports::http::HttpTransport;

// Connection errors
match sync.connect_to_peer(&addr).await {
    Ok(peer_key) => println!("Connected: {}", peer_key),
    Err(e) if e.is_sync_error() => {
        match e.sync_error().unwrap() {
            SyncError::HandshakeFailed(msg) => {
                eprintln!("Handshake failed: {}", msg);
                // Retry with different address or check credentials
            },
            SyncError::NoTransportEnabled => {
                eprintln!("Register transport first");
                sync.register_transport("http", HttpTransport::builder().bind("127.0.0.1:0")).await?;
            },
            SyncError::PeerNotFound(key) => {
                eprintln!("Peer {} not registered", key);
                // Register peer first
            },
            _ => eprintln!("Other sync error: {}", e),
        }
    },
    Err(e) => eprintln!("Non-sync error: {}", e),
}

Monitoring Sync Health

// Check server status
if !sync.is_server_running() {
    eprintln!("Warning: Sync server not running");
}

// Monitor peer connectivity
let peers = sync.list_peers()?;
for peer in peers {
    if peer.status != PeerStatus::Active {
        eprintln!("Warning: Peer {} is {}", peer.pubkey, peer.status);
    }
}

Configuration Examples

Development Setup

use eidetica::sync::transports::http::HttpTransport;

// Fast, responsive sync for development
// Register HTTP transport for easy debugging
db.sync()?.register_transport("http", HttpTransport::builder().bind("127.0.0.1:8080")).await?;
db.sync()?.accept_connections().await?;

// Connect to local test peer
let addr = Address::http("127.0.0.1:8081")?;
let peer = db.sync()?.connect_to_peer(&addr).await?;

Production Setup

use iroh::RelayMode;
use eidetica::sync::transports::iroh::IrohTransport;

// --- OPTION 1: Default relays (recommended for most deployments) ---
db.sync()?.register_transport("iroh", IrohTransport::builder()).await?;
db.sync()?.accept_connections().await?;

// --- OPTION 2: Custom relay server (enterprise deployment) ---
// Use this INSTEAD of Option 1 if you need a private relay
let relay_url: iroh::RelayUrl = "https://relay.example.com".parse()?;
let relay_node = iroh::RelayNode {
    url: relay_url,
    quic: Some(Default::default()),
};
db.sync()?.register_transport("iroh", IrohTransport::builder()
    .relay_mode(RelayMode::Custom(iroh::RelayMap::from_iter([relay_node])))
).await?;
db.sync()?.accept_connections().await?;

// --- After either option, connect to peers ---
let addr = Address::iroh(peer_node_id)?;
let peer = db.sync()?.connect_to_peer(&addr).await?;

// Sync happens automatically:
// - Immediate on commit
// - Retry with exponential backoff
// - Periodic sync every 5 minutes

Multi-Database Setup

use eidetica::sync::transports::http::HttpTransport;

// Run multiple sync-enabled databases
let db1 = Instance::open_backend(Box::new(Sqlite::in_memory().await?)).await?;
db1.enable_sync().await?;
db1.sync()?.register_transport("http", HttpTransport::builder().bind("127.0.0.1:8080")).await?;
db1.sync()?.accept_connections().await?;

let db2 = Instance::open_backend(Box::new(Sqlite::in_memory().await?)).await?;
db2.enable_sync().await?;
db2.sync()?.register_transport("http", HttpTransport::builder().bind("127.0.0.1:8081")).await?;
db2.sync()?.accept_connections().await?;

// Connect them together
let addr = Address::http("127.0.0.1:8080")?;
let peer = db2.sync()?.connect_to_peer(&addr).await?;

Testing Patterns

Testing with Iroh (No Relays)

#[tokio::test]
async fn test_iroh_sync_local() -> Result<()> {
    use iroh::RelayMode;
    use eidetica::sync::transports::iroh::IrohTransport;

    // Setup databases with local Iroh transport (no relay servers)
    let db1 = Instance::open_backend(Box::new(Sqlite::in_memory().await?)).await?;
    db1.enable_sync().await?;
    db1.sync()?.register_transport("iroh", IrohTransport::builder()
        .relay_mode(RelayMode::Disabled)
    ).await?;
    db1.sync()?.accept_connections().await?;

    let db2 = Instance::open_backend(Box::new(Sqlite::in_memory().await?)).await?;
    db2.enable_sync().await?;
    db2.sync()?.register_transport("iroh", IrohTransport::builder()
        .relay_mode(RelayMode::Disabled)
    ).await?;
    db2.sync()?.accept_connections().await?;

    // Get the serialized NodeAddr (includes direct addresses)
    let addr1 = db1.sync()?.get_server_address_async().await?;
    let addr2 = db2.sync()?.get_server_address_async().await?;

    // Connect peers using full NodeAddr info
    let peer1 = db2.sync()?.connect_to_peer(&Address::iroh(&addr1)).await?;
    let peer2 = db1.sync()?.connect_to_peer(&Address::iroh(&addr2)).await?;

    // Now they can sync directly via P2P
    Ok(())
}

Mock Peer Setup (HTTP)

use eidetica::sync::transports::http::HttpTransport;

#[tokio::test]
async fn test_sync_between_peers() -> Result<()> {
    // Setup first peer (Instance::create_backend bootstraps the first user as admin)
    let (instance1, mut user1) = Instance::create_backend(
        Box::new(Sqlite::in_memory().await?),
        eidetica::NewAccount::passwordless("peer1"),
    ).await?;
    instance1.enable_sync().await?;
    instance1.sync()?.register_transport("http", HttpTransport::builder().bind("127.0.0.1:0")).await?;
    instance1.sync()?.accept_connections().await?;

    let addr1 = instance1.sync()?.get_server_address_async().await?;

    // Setup second peer
    let (instance2, mut user2) = Instance::create_backend(
        Box::new(Sqlite::in_memory().await?),
        eidetica::NewAccount::passwordless("peer2"),
    ).await?;
    instance2.enable_sync().await?;
    instance2.sync()?.register_transport("http", HttpTransport::builder().bind("127.0.0.1:0")).await?;
    instance2.sync()?.register_transport("http", HttpTransport::builder().bind("127.0.0.1:0")).await?;

    // Connect peers
    let addr = Address::http(&addr1)?;
    let peer1_key = instance2.sync()?.connect_to_peer(&addr).await?;
    instance2.sync()?.update_peer_status(&peer1_key, PeerStatus::Active)?;

    // Setup sync relationship
    let key1 = user1.get_default_key()?;
    let tree1 = user1.create_database(Doc::new(), &key1).await?;
    let key2 = user2.get_default_key()?;
    let tree2 = user2.create_database(Doc::new(), &key2).await?;

    instance2.sync()?.add_tree_sync(&peer1_key, &tree1.root_id().to_string())?;

    // Test sync
    let txn1 = tree1.new_transaction().await?;
    let store1 = txn1.get_store::<DocStore>("data").await?;
    store1.set("test", "value").await?;
    txn1.commit().await?;

    // Wait for sync
    tokio::time::sleep(Duration::from_secs(2)).await;

    // Verify sync occurred
    // ... verification logic

    Ok(())
}

Best Practices Summary

✅ Do

• Use register_sync_peer() for persistent sync relationships (declarative API)
• Use sync_with_peer() for one-off sync operations (legacy API)
• Enable sync before creating databases you want to synchronize
• Use Iroh transport for production deployments (better NAT traversal)
• Monitor sync status via SyncHandle for declarative sync
• Share tree IDs to allow others to bootstrap from your databases
• Handle network failures gracefully (sync system auto-retries)
• Let BackgroundSync handle retry logic automatically
• Leverage automatic peer registration when peers connect to your server

❌ Don’t

• Manually manage peers unless you need fine control (use declarative API instead)
• Remove peer relationships for databases you want to synchronize
• Manually manage sync queues (BackgroundSync handles this)
• Ignore sync errors in production code
• Use HTTP transport for high-volume production (prefer Iroh)
• Assume sync is instantaneous (it’s eventually consistent)
• Mix APIs unnecessarily (pick declarative or legacy based on use case)

🚀 Sync Features

• Declarative sync API: Register intent, let background engine handle sync
• Automatic peer registration: Incoming connections register automatically
• Status tracking: Monitor sync progress with SyncHandle
• Zero-state joining: Join rooms/databases without any local setup
• Automatic protocol detection: Bootstrap vs incremental sync handled automatically
• Database discovery: Find available databases on peers
• Bidirectional sync: Both devices can share and receive databases
• Tree/peer relationship tracking: Automatic relationship management

🔧 Troubleshooting Checklist

1. Sync not working?

   • Check transport is enabled and server started
   • Verify peer status is Active
   • Confirm database sync relationships configured
   • Check network connectivity

2. Performance issues?

   • Consider using Iroh transport
   • Check for network bottlenecks
   • Verify retry queue isn’t growing unbounded
   • Monitor peer connectivity status

3. Memory usage high?

   • Check for dead/unresponsive peers
   • Verify retry queue is processing correctly
   • Consider restarting sync to clear state

4. Sync delays?

   • Remember sync is immediate on commit
   • Check if entries are in retry queue
   • Verify network is stable
   • Check peer responsiveness

Logging

Eidetica uses the tracing crate for structured logging throughout the library. This provides flexible, performant logging that library users can configure to their needs.

Quick Start

Eidetica uses the tracing crate, which means you can attach any subscriber to capture and format logs. The simplest approach is using tracing-subscriber:

[dependencies]
eidetica = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }

use tracing_subscriber::EnvFilter;

fn main() -> eidetica::Result<()> {
    // Initialize tracing subscriber to see Eidetica logs
    tracing_subscriber::fmt()
        .with_env_filter(EnvFilter::from_default_env())
        .init();

    // Now all Eidetica operations will log according to RUST_LOG
    // ...
}

You can customize formatting, filtering, and output destinations. See the tracing-subscriber documentation for advanced configuration options.

Configuring Log Levels

Control logging verbosity using the RUST_LOG environment variable:

# Show only errors
RUST_LOG=eidetica=error cargo run

# Show info messages and above
RUST_LOG=eidetica=info cargo run

# Show debug messages for sync module
RUST_LOG=eidetica::sync=debug cargo run

# Show all trace messages (very verbose)
RUST_LOG=eidetica=trace cargo run

Log Levels in Eidetica

Eidetica uses the following log levels:

• ERROR: Critical errors that prevent operations from completing
  • Failed database operations
  • Network errors during sync
  • Authentication failures
• WARN: Important warnings that don’t prevent operation
  • Retry operations after failures
  • Invalid configuration detected
  • Deprecated feature usage
• INFO: High-level operational messages
  • Sync server started/stopped
  • Successful synchronization with peers
  • Database loaded/saved
• DEBUG: Detailed operational information
  • Sync protocol details
  • Database synchronization progress
  • Hook execution
• TRACE: Very detailed trace information
  • Individual entry processing
  • Detailed CRDT operations
  • Network protocol messages

Using Eidetica with Logging

Once you’ve initialized a tracing subscriber, all Eidetica operations will automatically emit structured logs that you can capture and filter:

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let backend = Box::new(Sqlite::in_memory().await?);
// Initialise with alice as the first/admin user — this logs at INFO level.
let (instance, mut user) = Instance::create_backend(
    backend,
    eidetica::NewAccount::passwordless("alice"),
).await?;

// Create a database - this will log at INFO level
let mut settings = Doc::new();
settings.set("name", "my_database");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;

// Operations will emit logs at appropriate levels
// Use RUST_LOG to control what you see
Ok(())
}

Following a Sync Operation

With RUST_LOG=eidetica::sync=debug, outbound sync logs include a sync_operation span with an operation_id and kind. The ID is a process-local sequence number, not a database ID or a wire-protocol field. Use the process identity as well when combining logs from several peers. Address races, engine-served handshakes and requests, and the subsequent sync_tree lifecycle retain the same operation span. Incoming requests belong to the remote peer’s operation and do not receive its ID.

Each completed outbound operation emits one INFO-level Sync route outcome event. A peer round can exchange several trees under one ID; a new round or retry gets a new ID. Ticket-route preparation for a remote signer is a separate operation from submitting the signed request.

The summary reports:

• routes and attempted: address hints considered and handshake tasks submitted. Background selection skips hints with no matching transport (unsupported).
• route_failures, route_timeouts, and wrong_peer: results consumed before selection or total failure. Timeouts are included in failures.
• unobserved: submitted results not consumed when selection ended, including cancelled dials and queued results. These are not counted as failures.
• selected: the route used for the exchange, if any.
• trees_ok and trees_failed: completed tree exchanges. outcome is partial when a peer round has both successes and failures, and failed when all attempted trees fail, even if the background round itself returns Ok.
• error or last_tree_error: the operation failure or last failed tree. timeout_phase and peer_reachable retain the transport’s timeout distinction. An address-race deadline covers the whole handshake attempt, including engine queuing and peer registration, so it counts as a timeout without claiming a transport phase or reachability. Silence is not an authorization refusal.

Transport errors remain typed at the public sync API: SyncError::Timeout keeps its phase, while a rejected bootstrap is SyncError::BootstrapRejected. Applications should classify the returned error rather than describing every failure as a refusal. The summary adds diagnostics; it does not change routing, retry policy, authentication, or the on-disk database.

Performance Considerations

The tracing crate is designed to have minimal overhead when logging is disabled. Log statements that aren’t enabled at the current level are optimized away at compile time.

For performance-critical code paths, Eidetica uses appropriate log levels:

• Hot paths use trace! level to avoid overhead in production
• Synchronization operations use debug! for detailed tracking
• Only important events use info! and above

Integration with Observability Tools

The tracing ecosystem supports various backends for production observability:

• Console output: Default, human-readable format
• JSON output: For structured logging systems
• OpenTelemetry: For distributed tracing
• Jaeger/Zipkin: For trace visualization

See the tracing documentation for more advanced integration options.

Service (Daemon) Mode

Eidetica can run as a local daemon that serves an Instance to multiple client processes over a Unix domain socket. This allows multiple CLI tools and applications to operate on the same Eidetica data without each process opening its own backend.

When to Use Daemon Mode

Daemon mode is useful when:

• Multiple processes need concurrent access to the same Eidetica data
• CLI tools want fast startup without opening a storage backend each time
• Background sync should persist across short-lived client sessions
• A long-running process manages storage while lightweight clients connect on demand

For single-process applications, Instance::open_backend() with a local backend is simpler and has no IPC overhead.

Starting the Daemon

CLI

The eidetica daemon command starts a service server:

# Start with default socket path and SQLite backend
eidetica daemon

# Specify a custom socket path
eidetica daemon --socket /tmp/my-eidetica.sock

# Use a specific backend
eidetica daemon --backend postgres --postgres-url "postgresql://user:pass@host/db"

The daemon prints its socket path on startup and runs until interrupted (SIGINT/SIGTERM).

Optional web dashboard

eidetica daemon --dashboard also hosts the existing login/dashboard, health, and stats pages. It listens on exactly the address set by --dashboard-host and --dashboard-port (default 127.0.0.1:5942). The default daemon remains socket-only. Both listeners and the Iroh sync listener use the same Instance/backend, whether SQLite, PostgreSQL, or in-memory; no separate serve process is needed. The dashboard does not expose the trusted Unix service RPC or /api/v0 HTTP peer-sync endpoint. Peer sync continues over Iroh. If either listener stops unexpectedly, the daemon stops both, flushes sync, and exits with an error. Shutdown removes its socket.

The dashboard speaks plain HTTP. For remote access, terminate HTTPS at a reverse proxy (for example, Traefik) and forward to the dashboard’s bind address, preserving the browser’s Host and Origin headers. Who can reach the HTTP listener directly is up to the bind address and firewall; with a proxy on the same host, bind to loopback and leave the port closed.

services.eidetica = {
  enable = true;
  daemon = true;
  dashboard = true;
  host = "127.0.0.1";
  port = 5942;
  initialPasswordFile = "/run/agenix/eidetica-admin-password";
};

Browser sessions persist in daemon memory until logout or restart; cookies are HttpOnly, SameSite=Strict and host-only, and Secure when the browser origin is HTTPS. Unlike socket-only mode, dashboard logins decrypt signing keys inside the daemon’s web session memory; do not enable the dashboard if the daemon must never hold plaintext keys. Every web form POST requires an Origin that matches the request’s Host; requests without one are rejected.

A passwordless account uses the ordinary encrypted bundle with an empty-string password; this provides no meaningful confidentiality against a bundle holder. It can be logged into by anyone who can reach the dashboard: other local users, other hosts if the bind allows it, and web pages open in a local browser through DNS rebinding. Use passwords for any account a dashboard can reach. The separately launched serve command remains available for legacy HTTP sync and web deployments.

NixOS and Home Manager modules preserve their serve default. Set services.eidetica.daemon = true; to switch to the socket daemon and services.eidetica.dashboard = true; to opt into its dashboard; host and port set its bind address and port. A dashboard requires daemon mode. The NixOS module opens the port when openFirewall = true and a web listener (legacy serve or the dashboard) is running. The NixOS daemon puts its socket at dataDir/service.sock (not in systemd’s private /tmp); client processes need owner-approved access to that directory.

Dashboard Account information

The dashboard distinguishes the stable Account Root ID from the instance-local Username and Login ID. Public keys are shown by ID and optional display name; private keys and login credentials are not displayed. An Account is a private root database, while an Identity declares keys and delegation under its own Auth policy; they need not correspond one-to-one.

Tracked-database list failures are reported separately from an empty list. A database that cannot be opened remains visible as an unavailable row. Failed name or Entry reads show an error rather than an empty name or zero count. The detail page shows locally stored state, not runtime sync health. It still loads the full Entry list before displaying the first 100 IDs.

Self-service registration is disabled. The login and registration pages explain that the operator creates the initial account with eidetica daemon init; further accounts require an administrator through the existing Account administration API. The web interface does not provide an account-creation form.

Dashboard sync preferences

Open a tracked database’s detail page to edit your account’s sync-enabled, sync-on-commit, and periodic-interval preferences. Blank interval means the background default; explicit intervals must be positive whole seconds representable by the server’s timer. Saving retains the recorded key mapping, additional sync properties, and unrelated account settings. Only an already tracked database whose recorded key/mapping still authorizes reads can be updated; this form cannot create a tracking record or repair access.

These are preferences, not global daemon switches or a report of network health. Native local persistence and daemon reconciliation combine the enabled accounts’ preferences: sync-on-commit uses OR and explicit intervals use the minimum. Another account can therefore keep requesting sync after you disable it. Disabling neither cancels in-flight work nor revokes access. The current background scheduler uses a shared timer, so this is not an exact per-database deadline.

Requests from one browser session hold its write lock through the complete read/modify/persist; reads in that session take the corresponding read lock. Separate logins and socket clients do not share that browser lock. Native observed-value compare/exchange preserves unrelated local changes; preferences do not replicate to other installations. This is not distributed compare-and-swap or stale-form detection. Saving submits all three displayed preferences, so an older form can replace a newer edit to those fields. Reload before editing to see current values. A persistence error or lost response is not proof of rollback: the error page links to reload the actual preferences before retrying.

Default Socket Path

If no --socket is specified, the daemon uses:

1. $XDG_RUNTIME_DIR/eidetica/service.sock (preferred on Linux)
2. /tmp/eidetica-$USER/service.sock (fallback)

The EIDETICA_SOCKET environment variable can also set the socket path.

Programmatic

To start a daemon from Rust code:

use eidetica::Instance;
use eidetica::service::ServiceServer;
use tokio::sync::watch;

// `Instance::connect` accepts a URL describing the backend; here the
// daemon serves a sqlite file. See the rustdoc for the full URL grammar.
let instance = Instance::connect("sqlite://./my_data.db").await?;

let (shutdown_tx, shutdown_rx) = watch::channel(());
// Binding completes only after the socket is ready to accept clients.
let server = ServiceServer::bind(instance, eidetica::service::default_socket_path()).await?;

// Serve until shutdown_tx is dropped.
server.run(shutdown_rx).await?;

Connecting Clients

Clients reach the daemon by passing a unix:// URL to Instance::connect:

use eidetica::Instance;

// Connect to a running daemon
let instance = Instance::connect(eidetica::service::default_socket_url()).await?;

// Use it exactly like a local Instance. The daemon was initialised with
// an initial admin user via `eidetica daemon init --username ops`
// (see the CLI reference); log in as that user, then create application
// users via the admin path.
let admin = instance.login_account("ops", "").await?;
admin.admin().await?.create_account(eidetica::NewAccount::passwordless("alice")).await?;
let mut user = instance.login_account("alice", "").await?;

let default_key = user.get_default_key()?;
let db = user.create_database(eidetica::crdt::Doc::new(), &default_key).await?;

The returned Instance is fully transparent – all downstream code (Database, Transaction, Store, AccountSession) works identically whether the Instance is local or connected to a daemon.

Security Model

• Socket-client keys and passwords stay client-side. The socket daemon sees only encrypted key material and signed entries from those clients. Password verification and key derivation (Argon2id) happen in the client process. Optional dashboard sessions instead hold decrypted keys in daemon memory (see above).
• No plaintext secrets cross the socket. Authentication operations (user creation, login, key management) run locally in the client. PasswordStore decryption and encrypted-cache materialization are also client-side; a warm encrypted Table point read uses only encrypted point-record requests. Only storage operations (get, put, tips, etc.) are forwarded to the daemon.
• The socket is a local Unix domain socket. Access is controlled by filesystem permissions on the socket file. Only processes that can reach the socket path can connect.
• The socket directory defines who is trusted. Missing directories are created with mode 0700; the socket is mode 0660. An existing parent must be owned by the daemon user, must not be writable by group or others, and cannot be reached through a symlink. Its group, setgid bit, and traversal permissions may grant trusted Unix-group members access.

Any process that can connect gets the existing fully trusted service API. For cross-user clients, pre-provision an owner-controlled directory with the intended group and setgid/traversal bits; the daemon does not change its ownership or mode. Shared-writable, sticky, and symlinked layouts are rejected. An adjacent lock coordinates cooperating daemons, stale sockets are recovered, and graceful or dropped servers remove the socket they bound.

⚠️ The deployment bootstrap fails closed. Both the NixOS module and the published container image refuse to start on a fresh backend unless the operator supplies a credential source for the initial admin user. This avoids silently creating a passwordless admin that any reachable client could use.

NixOS module — set exactly one of:

• services.eidetica.initialPasswordFile = "/path/to/password-file"; (recommended; read via systemd LoadCredential so the service user never needs read access to the file itself), or
• services.eidetica.allowPasswordlessAdmin = true; (INSECURE; trusted or LAN deployments only — the module warns at rebuild time when this is combined with a web listener: legacy serve or the dashboard).

Container image — provide one of, in priority order:

1. A password file mounted at /run/secrets/admin_password (preferred; keeps the password off the process table and out of docker inspect).
2. EIDETICA_ADMIN_PASSWORD env.
3. EIDETICA_ALLOW_PASSWORDLESS_ADMIN=1 env (INSECURE; local/dev only).

Without any of the above the container entrypoint exits 1 with an actionable error. To bootstrap your own admin with a password manually, run eidetica daemon init --username <NAME> against the data directory before first start.

Multiple Clients

Multiple clients can connect to the same daemon simultaneously. Each client maintains its own connection and AccountSession:

// Client 1: an admin session creates the new user via the InstanceAdmin path.
let instance1 = Instance::connect(eidetica::service::default_socket_url()).await?;
let admin = instance1.login_account("ops", "").await?;
admin.admin().await?.create_account(eidetica::NewAccount::passwordless("alice")).await?;

// Client 2 (separate process or task): log in as the user that was just created.
let instance2 = Instance::connect(eidetica::service::default_socket_url()).await?;
let user = instance2.login_account("alice", "").await?;

All clients share the same underlying storage through the daemon’s backend.

Entry verification is owned by the daemon. Clients do not run their own verification pass — update_verification_status and the verification-status queries are not exposed over the socket. The daemon’s Instance stores synced entries as Unverified, runs Database::verify(), and serves reads from the resulting Verified frontier. Every connected client therefore sees the same verified view; there is no per-client allow_unverified() toggle over the wire. (See Core Concepts for the verification model.)

Configuration Reference

Limitations

• Sync management is server-side. The daemon always loads persisted sync state, starts its Iroh listener, and keeps syncing without connected service clients. Use --sync-ticket <TICKET> for owner-side bootstrap and peer setup; the resulting relationships persist across restart. A connected client can’t drive that lifecycle over the wire; enable_sync() on a remote Instance remains a no-op. To join a database from a ticket, a connected client calls AccountSession::join: the daemon performs the network exchange, while the client holds its private key and signs the access request. Success returns an opened database after recording the account’s key mapping and sync preferences; pending approval requires an explicit retry. Pending and rejection create no new Account state. Failed or interrupted calls can leave committed effects; retry explicitly rather than assuming rollback. There is no automatic restart-safe completion, cancellation API, or device enrollment. Held/unreadable acquisition, automatic delegated-history acquisition and stronger authority recovery remain unsupported; see current recovery limits. Ongoing daemon replication needs device-key authorization or a global grant; account-key access alone does not supply private background authority. Account tracking changes made through the socket are reconciled by the daemon’s Instance callbacks.
• Unix-only. The service module requires Unix domain sockets and is not available on Windows.
• Feature flag required. The service feature must be enabled (included in the default full feature set).

For the Database::on_write semantics on a connected Instance — including the deliberate trade that callbacks fire asynchronously to commit() in exchange for a single daemon-canonical event ordering — see Write Callbacks.

Dashboard access requests

An authenticated dashboard session can open Manage pending database access requests (/dashboard/requests). Only requests for databases where a held, mapped Account key currently has Admin permission are displayed; instance-admin status alone does not grant this authority. Each request shows its exact database ID, requester public key, requester-supplied key name, requested permission/priority and Pending lifecycle state. A key identifies a cryptographic signer, not an independently verified person.

Approve exact request and Reject request are explicit same-origin form submissions bound to both the request ID and database ID. Approval uses the requested permission and priority, subject to existing Auth rules; an admin cannot create a higher-priority key than its own. There is no automatic approval or new device authorization. Refresh before deciding a stale request: already-decided requests are no longer in the pending list, and opposite decisions cannot reverse a terminal result.

The requesting dashboard distinguishes pending approval, terminal rejection, network failure and completed access. Pending requests require an explicit Retry request after approval; they do not create new Account mappings or preferences. A successful transfer followed by a local bookkeeping error is reported separately, not as usable local access: local metadata may have persisted despite the error. Retry explicitly to repair or inspect the recorded state. The network wait does not hold the per-session lock, and bookkeeping preserves a tracked preference/key selection changed while that request was waiting. This observed-state check is not distributed compare-and-set or background completion.

CLI Reference

The eidetica binary provides a server and management commands for inspecting and operating on Eidetica instances.

Commands

serve (default)

Starts the Eidetica server with HTTP and sync endpoints.

Running eidetica with no subcommand is equivalent to eidetica serve. This default is likely to change in the future. Do not run serve alongside daemon on the same SQLite backend.

eidetica serve [OPTIONS]

health

Checks the health of a running Eidetica server by querying its /health endpoint.

eidetica health [URL] [OPTIONS]

Both http:// and https:// URLs are supported. If the URL doesn’t already end with /health, it is appended automatically.

Exits with code 0 on success, code 1 on failure.

info

Displays instance information: device ID, storage backend, user count, and database count.

eidetica info [OPTIONS]

Example output:

Device ID:   a1b2c3d4-...
Backend:     sqlite (./eidetica.db)
Users:       2
Databases:   5

daemon init

Initialises a fresh Eidetica instance on the chosen backend with an initial admin user. The first user created on an instance is automatically granted Admin on the system databases. Fails if the backend already has an instance on it.

eidetica daemon [BACKEND OPTIONS] init --username <NAME> [--password <PASS> | --passwordless]

--username has no default: operators must spell it out so no static credential ships by accident. --passwordless is shorthand for --password "", not a different storage mode or a required insecure-use acknowledgement. The explicit flag, password environment variable and interactive prompt all accept an empty string; leaving the option unset prompts rather than silently choosing a password. Empty and nonempty strings use the same encrypted bundle and KDF, but empty-string wrapping provides no meaningful confidentiality against a bundle holder.

Examples:

# Interactive password prompt:
eidetica daemon --data-dir /var/lib/eidetica init --username ops

# Non-interactive (e.g. CI provisioning):
EIDETICA_ADMIN_PASSWORD=… eidetica daemon --data-dir /var/lib/eidetica init --username ops

# Embedded / single-user dev workflow:
eidetica daemon --data-dir ~/.local/share/eidetica init --username me --passwordless

Backend options (--backend, --data-dir, --postgres-url, --cache-bytes, --cache-namespaces) go before the init subcommand and are shared with daemon (see below).

daemon

Runs the Eidetica service daemon against an already-initialised backend. Fails with a pointer at daemon init if the backend hasn’t been initialised yet. Multiple client processes can connect to the running daemon over the Unix socket to share the same backend storage.

eidetica daemon [OPTIONS]

All commands opening a backend also accept --cache-bytes (EIDETICA_CACHE_BYTES) and --cache-namespaces (EIDETICA_CACHE_NAMESPACES, default 1024). The byte defaults are 268435456 for InMemory and 4294967296 for file-backed SQL. These count live derived record key/value payload, not total memory or physical disk use. Zero disables retention; oversized derived materializations use history instead. The daemon installs these bounds before startup trim; remote clients cannot override them. See cache retention for reader-generation costs and legacy-client limits.

The daemon runs until interrupted with SIGINT or SIGTERM. Clients connect using Instance::connect("unix://..."). See Service (Daemon) Mode for full documentation.

db list

Lists all user-created databases with their root IDs and tip counts. System databases are excluded.

eidetica db list [OPTIONS]

Example output:

ROOT ID         TIPS
abc123def456    5
xyz789uvw012    2

Global Flags

The --json flag works with info and db list.

Storage Backends

Environment Variables

Command-line flags take precedence over environment variables.

Examples

# Start server with defaults (sqlite backend, port 5942)
eidetica

# Start with PostgreSQL backend on a custom port
eidetica serve --port 8080 --backend postgres \
  --postgres-url "postgresql://user:pass@host/db"

# Check health of a running server
eidetica health

# Show instance info as JSON
eidetica info --json

# List databases from a specific data directory
eidetica db list --data-dir /var/lib/eidetica

# Start a daemon for shared multi-process access
install -d -m 0700 "$XDG_RUNTIME_DIR/eidetica"
eidetica daemon --socket "$XDG_RUNTIME_DIR/eidetica/service.sock"

db reset-local-verification (offline trust reset)

Before upgrading an existing instance to new delegated-authorization verification rules, stop the daemon/server and all writers and readers, back up the database, then run the reset with the same backend configuration as the instance:

eidetica db reset-local-verification --backend sqlite --data-dir /var/lib/eidetica --confirm
# Or: --backend inmemory --data-dir <directory containing eidetica.json>
# Or: --backend postgres --postgres-url <instance connection URL>

The command does not run during startup or schema migration. Skipping it can leave old Verified labels trusted under the new rules. It resets all local statuses (Verified and Failed included) to Unverified, clears the retained Verified frontier in the same step, discards derived and incomplete Store-state namespaces, and keeps every immutable Entry and authoritative Store state. It does not verify entries itself. Start the new version only after the command succeeds, and follow the re-verification procedure below before relying on reads. Access/sync can attempt verification but does not promise to acquire or settle delegated dependencies automatically. Verification is prefix-closed: until ancestors verify, descendants remain Unverified. If any reset step fails, leave the service stopped, diagnose and retry the command; do not trust the old status labels. An in-memory persistence file must exist and parse successfully; a missing or corrupt file is never treated as an empty instance by this command.

SQLite and PostgreSQL commit status and cache changes in one transaction. The persisted in-memory backend writes a replacement JSON snapshot by atomic rename on POSIX; if writing fails before rename, the old file remains in place. Run the command with no other process or API user of the backend: the normal online clear_derived_store_state retains a generation for live readers, whereas the trust reset deliberately drops it. The in-memory JSON persistence path has no cross-process ownership lock. On platforms without atomic replacement rename, take an offline backup and verify the reopened file before starting the service.

db repair-verification (offline conservative repair)

Stop all users of the backend and make an offline backup first, just as for reset. Select an existing database and one tree explicitly:

eidetica db repair-verification --backend sqlite --data-dir /var/lib/eidetica --tree <ROOT_ID> --confirm
# Or: --backend inmemory --data-dir <directory containing eidetica.json>
# Or: --backend postgres --postgres-url <instance connection URL>

Repair preserves valid Verified history and existing Failed decisions, demotes only inconsistent Verified entries and their dependent region to Unverified, and rebuilds the tree’s Verified tips atomically. Missing or foreign parents cannot establish trust; malformed immutable data and storage errors abort without partial status/tip changes. The printed demotion count is not a count of successfully verified entries. Repeat repair is idempotent. Entries, authoritative Store records, and the independent materialization cache are not cleared. Use the SDK re-verification procedure below when missing ancestors arrive; repair never promotes entries.

BackendImpl::repair_verification exposes the same maintenance operation locally. Custom backends must implement atomic repair or return an explicit unsupported error. There is no service-wire repair endpoint. Unlike repair, reset clears all local trust decisions and disposable state. Neither command creates a missing persistence file or treats corrupt JSON as an empty instance.

Storage compatibility and older candidate data

SQL schema and InMemory persistence versions are independent internal integer format counters, not release versions or stability levels. SQL currently uses 4 and InMemory uses 3. Mainline SQL schema 0 migrates directly to 4; SQL 3 adds an empty native local-account table without rewriting keys or verification state. Unversioned or explicit _v: 0 InMemory files reconstruct retained state on load and write 3 on next save. Version 2 snapshots load without history repair and acquire empty local-account storage when saved. Migration reuses conservative repair, not cryptographic re-verification; SQL schema, version, and label/tip changes commit together. Clean current-format opens load persisted tips without a history scan. See the canonical supported migration matrix. The unreleased version-1 prefix-tracking format is unsupported and refused before SQL schema initialization; InMemory version-1 files are also refused. A trust reset does not convert that schema or file format.

For an old candidate instance, keep it stopped and retain an untouched offline backup (including SQLite WAL/SHM files when present, or a consistent PostgreSQL backup). Do not edit its version marker, drop tables, or overwrite the original. A supported recovery path is to restore a known mainline-format backup into a separate location and let the new version migrate that copy, then reverify and compare required Entries and authoritative records before changing application paths. Alternatively create a separate empty current-format instance and repopulate it through normal ingest/replication from a complete trusted source, followed by verification. That is recreation, not conversion: instance-local credentials and authoritative Store records are not guaranteed to replicate. If either is irreplaceable and no mainline backup exists, keep the old copy and stop for an explicit recovery plan. The migration fixtures verify mainline-copy repair and old-format refusal; no in-place version-1 conversion or automatic deletion is provided.

Upgrading a verification-only version-2 database

Existing tips-only version-2 SQL databases migrate atomically to version 3, adding size_bytes and last_access_tick when absent. Record sizes are backfilled once; existing valid durable ticks are preserved. This migration preserves Entries, statuses, retained Verified tips and authoritative records without verification repair. Only the supported v0 transition repairs verification closure. Ordinary v3 open does not backfill or repair. InMemory version-2 files are unchanged; derived records and recency remain ephemeral.

Stop all users of the backend and retain a consistent offline backup before upgrading. Snapshot digest keys miss old-key cache rows and rebuild from immutable Entries; old keys are never relabelled. Exclusive startup reclaims abandoned and retired derived generations, seeds recency and enforces the configured budget before exposing storage. This does not require a trust reset.

Pinned older version-2 constructors refuse the version-3 marker. Binary rollback requires restoring a compatible pre-upgrade backup in a separate location, not lowering the marker or opening the upgraded database with old software. Malformed, newer and unsupported version-1/prefix layouts are refused. No rollout or deployed client/server compatibility inventory is implied by these migration checks.

Re-verification after reset or incomplete proof

There is currently no db verify CLI subcommand and the reset command does not run verification. Use a local SDK maintenance process with the same backend, without a concurrent daemon or other backend owner. Database::verify() is a node-local operation, not a client RPC to a connected service instance.

1. After a successful reset, open the backend with the new version. Keep normal consumers stopped until you have inspected the rebuilt trust state.
2. Make the immutable main/settings histories and claimed/configured delegated snapshots available through ordinary replication or the application’s ingest API. Reset preserves any Entries already present; it does not fetch missing ones.
3. Explicitly verify delegated databases first, starting with their own dependencies, then retry the primary database. Repeat only after a known dependency arrives or verifies; running the primary pass alone cannot settle a present-but-Unverified delegated proof.
4. Inspect the returned VerifyReport and the default Verified-frontier reads. failed and still_unverified describe entries considered in that pass, not an inventory of all stored failures. A later empty report does not prove the whole history is healthy. Record unresolved dependency IDs and rejected branches rather than declaring recovery complete from a zero-error return.

For example, on an already opened local Instance, with the relevant roots ordered dependency-first:

#![allow(unused)]
fn main() {
use eidetica::{Database, Instance, ID};
async fn reverify(instance: &Instance, roots_dependency_first: &[ID]) -> eidetica::Result<()> {
for root in roots_dependency_first {
    let database = Database::open(instance, root).await?;
    let report = database.verify().await?;
    println!("{root}: {report:?}");
    println!("visible tips: {:?}", database.snapshot().await?);
}
Ok(())
}
}

A missing or present-but-Unverified dependency leaves the dependent entry Unverified and invisible to default reads. A proven invalid snapshot, signature or causal pin is Failed, not a dependency to fetch; ordinary retry does not clear that verdict. Operational storage errors should be diagnosed before retry. Do not manually set statuses to Verified or clear only the derived cache to force progress. An offline trust reset clears previous Failed decisions too, but the same invalid immutable entry will be rejected again under the new rules.

The supported manual sequence is exercised by test_delegated_entry_synced_unverified_then_verified, including reset and rebuild on the local backend matrix. Automatic dependency acquisition and retry ordering are deferred to PR #126. See the causal authorization contract for pointer rewinds, effective removal and the nested-floor recovery limitation.

Docker

Eidetica provides official container images for running the database server in containerized environments. Images are built using Nix, so they are minimal in size but lack any extra tools (curl, wget, etc).

There is a maintained Dockerfile in the repo but it is not published.

Container Registries

Official images are available from:

• GitHub Container Registry: ghcr.io/arcuru/eidetica
• Docker Hub: arcuru/eidetica

Both registries contain identical images with multi-architecture support (amd64 and arm64).

Available Tags

Configuration

Environment Variables

Data Storage

The container stores data in /config by default. When using Docker volumes, ensure the directory has proper ownership (UID 1000, the eidetica user in the container).

Health Checks

The container includes a built-in health check that verifies the server is responding. You can also run it manually:

docker exec <container> eidetica health

See CLI Reference for all available commands.

Quick Start

Pull and run the latest stable image:

docker run -p 5942:5942 ghcr.io/arcuru/eidetica:latest

Or from Docker Hub:

docker run -p 5942:5942 arcuru/eidetica:latest

Docker Compose

For production deployments, Docker Compose provides a convenient way to manage Eidetica alongside other services.

Basic Configuration

Create a docker-compose.yml file:

services:
  eidetica:
    image: ghcr.io/arcuru/eidetica:latest
    ports:
      - "5942:5942"
    volumes:
      - eidetica-data:/config
    restart: unless-stopped

volumes:
  eidetica-data:

Note: When using bind mounts instead of named volumes, ensure the host directory is owned by UID 1000:

sudo chown -R 1000:1000 /path/to/config

Start the service:

docker compose up -d

With PostgreSQL Backend

For production use with PostgreSQL as the storage backend:

services:
  eidetica:
    image: ghcr.io/arcuru/eidetica:latest
    ports:
      - "5942:5942"
    environment:
      EIDETICA_BACKEND: postgres
      EIDETICA_POSTGRES_URL: postgres://eidetica:secret@postgres:5432/eidetica
    depends_on:
      postgres:
        condition: service_healthy
    restart: unless-stopped

  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: eidetica
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: eidetica
    volumes:
      - postgres-data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U eidetica -d eidetica"]
      interval: 5s
      timeout: 5s
      retries: 5
    restart: unless-stopped

volumes:
  postgres-data:

Building Images Locally

The published image is built with Nix. The justfile contains scripts for building the image locally.

There is also a Dockerfile-based build available and maintained.

Build with:

just container nix
just container docker

Developer Walkthrough: Building with Eidetica

This guide walks through the Todo Example (examples/todo/src/main.rs) to explain Eidetica’s core concepts. The example is a simple command-line todo app that demonstrates databases, transactions, stores, and Y-CRDT integration.

Core Concepts

The Todo example demonstrates Eidetica’s key components working together in a real application.

1. The Database Backend (Instance)

The Instance is your main entry point. It wraps a storage backend and manages users and databases.

The Todo example loads or initialises an instance by URL. connect_or_create handles both the first-run and reopen cases:

async fn load_or_create_instance(path: &Path) -> Result<Instance> {
    // SQLite handles both creation and loading automatically.
    // The URL is passed through to sqlx — `mode=rwc` tells sqlx to
    // create the database file on first run if it doesn't exist.
    let url = format!("sqlite://{}?mode=rwc", path.display());
    let (instance, _maybe_user) =
        Instance::connect_or_create(&url, NewAccount::passwordless("todo-user")).await?;

    println!("✓ Instance initialized");
    Ok(instance)
}

Data is automatically saved to the SQLite file. Authentication is managed through the AccountSession system (see below).

2. Users (AccountSession)

Users provide authenticated access to databases. An AccountSession manages signing keys and database access. The Todo example creates a passwordless user for simplicity, bootstrapping it as the initial admin on a fresh data file via Instance::connect_or_create:

async fn load_or_create_instance_and_user(path: &Path) -> Result<(Instance, AccountSession)> {
    use eidetica::NewAccount;

    let username = "todo-user";
    // SQLite via the URL surface — `sqlite://<path>` is handed through to
    // sqlx, and `mode=rwc` tells sqlx to create the file on first run.
    let url = format!("sqlite://{}?mode=rwc", path.display());

    let (instance, maybe_user) =
        Instance::connect_or_create(&url, NewAccount::passwordless(username)).await?;

    let user = match maybe_user {
        Some(mut session) => {
            // Only the first creator initializes application roots. This embedded
            // SQLite example has exclusive instance storage ownership. Returning
            // logins never repair missing metadata by creating a replacement.
            let key = session.get_default_key()?;
            let mut settings = Doc::new();
            settings.set("name", "todo");
            let todos = session.create_database(settings, &key).await?;
            session.set_reference(TODO_APP, TODO_LIST, todos.root_id()).await?;
            println!("✓ Initialised new instance and bootstrapped {username} as admin");
            session
        }
        None => {
            let u = instance.login_account(username, "").await?;
            println!("✓ Logged in as passwordless user: {username}");
            u
        }
    };

    Ok((instance, user))
}

3. Databases (Database)

A Database is a primary organizational unit within an Instance. Think of it somewhat like a schema or a logical database within a larger instance. It acts as a container for related data, managed through Stores. Databases provide versioning and history tracking for the data they contain.

The Todo example keeps a reference to its todo database in its Account, independent of the target’s display name, and opens it through the session:

async fn open_todo_database(session: &AccountSession) -> Result<Database> {
    session
        .open_reference(TODO_APP, TODO_LIST)
        .await?
        .ok_or_else(|| {
            std::io::Error::new(
                std::io::ErrorKind::NotFound,
                "todo list reference is missing; this data file needs explicit repair",
            )
            .into()
        })
}

Setup runs only for a newly created embedded instance, under exclusive backend ownership. A returning login never creates a replacement on missing metadata or lookup failure. Partial setup requires explicit repair; ordinary CRDT writes are not distributed create-if-absent locks. See Accounts for the initialization and error contract.

4. Transactions and Stores

All data modifications happen within a Transaction. Transactions ensure atomicity and are automatically authenticated using the database’s default signing key.

Within a transaction, you access Stores - flexible containers for different types of data. The Todo example uses Table<Todo> to store todo items with unique IDs.

5. The Todo Data Structure

The example defines a Todo struct that must implement Serialize and Deserialize to work with Eidetica:

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Todo {
    pub title: String,
    pub completed: bool,
    pub created_at: DateTime<Utc>,
    pub completed_at: Option<DateTime<Utc>>,
}

impl Todo {
    pub fn new(title: String) -> Self {
        Self {
            title,
            completed: false,
            created_at: Utc::now(),
            completed_at: None,
        }
    }

    pub fn complete(&mut self) {
        self.completed = true;
        self.completed_at = Some(Utc::now());
    }
}

6. Adding a Todo

The add_todo() function shows how to insert data into a Table store:

async fn add_todo(database: &Database, title: String) -> Result<()> {
    // Start an atomic transaction (uses default auth key)
    let txn = database.new_transaction().await?;

    // Get a handle to the 'todos' Table store
    let todos_store = txn.get_store::<Table<Todo>>("todos").await?;

    // Create a new todo
    let todo = Todo::new(title);

    // Insert the todo into the Table
    // The Table will generate a unique ID for it
    let todo_id = todos_store.insert(todo).await?;

    // Commit the transaction
    txn.commit().await?;

    println!("Added todo with ID: {todo_id}");

    Ok(())
}

7. Updating a Todo

The complete_todo() function demonstrates reading and updating data:

async fn complete_todo(database: &Database, id: &str) -> Result<()> {
    // Start an atomic transaction (uses default auth key)
    let txn = database.new_transaction().await?;

    // Get a handle to the 'todos' Table store
    let todos_store = txn.get_store::<Table<Todo>>("todos").await?;

    // Get the todo from the Table
    let mut todo = todos_store.get(id).await?;

    // Mark the todo as complete
    todo.complete();

    // Update the todo in the Table
    todos_store.set(id, todo).await?;

    // Commit the transaction
    txn.commit().await?;

    Ok(())
}

These examples show the typical pattern: start a transaction, get a store handle, perform operations, and commit.

8. Y-CRDT Integration (YDoc)

The example also uses YDoc stores for user information and preferences. Y-CRDTs are designed for collaborative editing:

async fn set_user_info(
    database: &Database,
    name: Option<&String>,
    email: Option<&String>,
    bio: Option<&String>,
) -> Result<()> {
    // Start an atomic transaction (uses default auth key)
    let txn = database.new_transaction().await?;

    // Get a handle to the 'user_info' YDoc store
    let user_info_store = txn.get_store::<YDoc>("user_info").await?;

    // Update user information using the Y-CRDT document
    user_info_store.with_doc_mut(|doc| {
        let user_info_map = doc.get_or_insert_map("user_info");
        let mut txn = doc.transact_mut();

        if let Some(name) = name {
            user_info_map.insert(&mut txn, "name", name.clone());
        }
        if let Some(email) = email {
            user_info_map.insert(&mut txn, "email", email.clone());
        }
        if let Some(bio) = bio {
            user_info_map.insert(&mut txn, "bio", bio.clone());
        }

        Ok(())
    }).await?;

    // Commit the transaction
    txn.commit().await?;
    Ok(())
}

The example demonstrates using different store types in one database:

• “todos” (Table<Todo>): Stores todo items with automatic ID generation
• “user_info” (YDoc): Stores user profile using Y-CRDT Maps
• “user_prefs” (YDoc): Stores preferences using Y-CRDT Maps

This shows how you can choose the most appropriate data structure for each type of data.

Running the Todo Example

To see these concepts in action, you can run the Todo example:

# Navigate to the example directory
cd examples/todo

# Build the example
cargo build

# Run commands (this will create todo_db.json)
cargo run -- add "Learn Eidetica"
cargo run -- list
# Note the ID printed
cargo run -- complete <id_from_list>
cargo run -- list

Refer to the example’s README.md and test.sh for more usage details.

This walkthrough provides a starting point. Explore the Eidetica documentation and other examples to learn about more advanced features like different store types, history traversal, and distributed capabilities.

Code Examples

This page provides focused code snippets for common tasks in Eidetica.

Assumes basic setup like use eidetica::{Instance, Database, Error, ...}; and error handling (?) for brevity.

1. Initializing the Database (Instance)

Instance::connect_or_create dispatches by URL scheme. See the backends concepts page for the full URL grammar.

use eidetica::{Instance, NewAccount};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
    // Option A: ephemeral in-memory backend (for tests / short-lived processes)
    let (_instance, _) =
        Instance::connect_or_create("memory://", NewAccount::passwordless("alice")).await?;

    // Option B: persistent SQLite database — created on first run, loaded on
    // subsequent runs. The URL is passed through to sqlx, so any sqlx form
    // works.
    let (instance, _) = Instance::connect_or_create(
        "sqlite://./my_data.db",
        NewAccount::passwordless("alice"),
    ).await?;

    println!("Database opened at sqlite://./my_data.db");
    Ok(())
}

2. Creating or Loading a Database

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let tree_name = "my_app_data";

let database = match user.find_database(tree_name).await {
    Ok(mut databases) => {
        println!("Found existing database: {}", tree_name);
        databases.pop().unwrap() // Assume first one is correct
    }
    Err(e) if e.is_not_found() => {
        println!("Creating new database: {}", tree_name);
        let mut doc = Doc::new();
        doc.set("name", tree_name);
        let default_key = user.get_default_key()?;
        user.create_database(doc, &default_key).await?
    }
    Err(e) => return Err(e.into()), // Propagate other errors
};

println!("Using Database with root ID: {}", database.root_id());
Ok(())
}

3. Writing Data (DocStore Example)

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::DocStore};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;

// Start an authenticated transaction (automatically uses the database's default key)
let txn = database.new_transaction().await?;

{
    // Get the DocStore store handle (scoped)
    let config_store = txn.get_store::<DocStore>("configuration").await?;

    // Set some values
    config_store.set("api_key", "secret-key-123").await?;
    config_store.set("retry_count", "3").await?;

    // Overwrite a value
    config_store.set("api_key", "new-secret-456").await?;

    // Remove a value
    config_store.delete("old_setting").await?; // Ok if it doesn't exist
}

// Commit the changes atomically
let entry_id = txn.commit().await?;
println!("DocStore changes committed in entry: {}", entry_id);
Ok(())
}

4. Writing Data (Table Example)

extern crate eidetica;
extern crate tokio;
extern crate serde;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::Table};
use serde::{Serialize, Deserialize};

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
struct Task {
    description: String,
    completed: bool,
}

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;

// Start an authenticated transaction (automatically uses the database's default key)
let txn = database.new_transaction().await?;
let inserted_id;

{
    // Get the Table handle
    let tasks_store = txn.get_store::<Table<Task>>("tasks").await?;

    // Insert a new task
    let task1 = Task { description: "Buy milk".to_string(), completed: false };
    inserted_id = tasks_store.insert(task1).await?;
    println!("Inserted task with ID: {}", inserted_id);

    // Insert another task
    let task2 = Task { description: "Write docs".to_string(), completed: false };
    tasks_store.insert(task2).await?;

    // Update the first task (requires getting it first if you only have the ID)
    if let Ok(mut task_to_update) = tasks_store.get(&inserted_id).await {
        task_to_update.completed = true;
        tasks_store.set(&inserted_id, task_to_update).await?;
        println!("Updated task {}", inserted_id);
    } else {
        eprintln!("Task {} not found for update?", inserted_id);
    }

    // Delete a task (if you knew its ID)
    // tasks_store.delete(&some_other_id)?;
}

// Commit all inserts/updates/deletes
let entry_id = txn.commit().await?;
println!("Table changes committed in entry: {}", entry_id);
Ok(())
}

5. Reading Data (DocStore Viewer)

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::DocStore};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;

// Get a read-only viewer for the latest state
let config_viewer = database.get_store_viewer::<DocStore>("configuration").await?;

match config_viewer.get("api_key").await {
    Ok(api_key) => println!("Current API Key: {}", api_key),
    Err(e) if e.is_not_found() => println!("API Key not set."),
    Err(e) => return Err(e.into()),
}

match config_viewer.get("retry_count").await {
    Ok(count_str) => {
        // Note: DocStore values can be various types
        if let Some(text) = count_str.as_text() {
            if let Ok(count) = text.parse::<u32>() {
                println!("Retry Count: {}", count);
            }
        }
    }
    Err(_) => println!("Retry count not set or invalid."),
}
Ok(())
}

6. Reading Data (Table Viewer)

extern crate eidetica;
extern crate tokio;
extern crate serde;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::Table};
use serde::{Serialize, Deserialize};

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
struct Task {
    description: String,
    completed: bool,
}

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
let txn = database.new_transaction().await?;
let tasks_store = txn.get_store::<Table<Task>>("tasks").await?;
let id_to_find = tasks_store.insert(Task { description: "Test task".to_string(), completed: false }).await?;
txn.commit().await?;

// Get a read-only viewer
let tasks_viewer = database.get_store_viewer::<Table<Task>>("tasks").await?;

// Get a specific task by ID
match tasks_viewer.get(&id_to_find).await {
    Ok(task) => println!("Found task {}: {:?}", id_to_find, task),
    Err(e) if e.is_not_found() => println!("Task {} not found.", id_to_find),
    Err(e) => return Err(e.into()),
}

// Search for all tasks
println!("\nAll Tasks:");
match tasks_viewer.search(|_| true).await {
    Ok(tasks) => {
        for (id, task) in tasks {
            println!("  ID: {}, Task: {:?}", id, task);
        }
    }
    Err(e) => eprintln!("Error searching tasks: {}", e),
}
Ok(())
}

7. Working with Nested Data (Path-Based Operations)

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::DocStore, path, Database};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
// Setup database for testing
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "test_db");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;
// Start an authenticated transaction (automatically uses the database's default key)
let txn = database.new_transaction().await?;

// Get the DocStore store handle
let user_store = txn.get_store::<DocStore>("users").await?;

// Using path-based operations to create and modify nested structures
// Set profile information using paths - creates nested structure automatically
user_store.set_path(path!("user123.profile.name"), "Jane Doe").await?;
user_store.set_path(path!("user123.profile.email"), "jane@example.com").await?;

// Set preferences using paths
user_store.set_path(path!("user123.preferences.theme"), "dark").await?;
user_store.set_path(path!("user123.preferences.notifications"), "enabled").await?;
user_store.set_path(path!("user123.preferences.language"), "en").await?;

// Set additional nested configuration
user_store.set_path(path!("config.database.host"), "localhost").await?;
user_store.set_path(path!("config.database.port"), "5432").await?;

// Commit the changes
let entry_id = txn.commit().await?;
println!("Nested data changes committed in entry: {}", entry_id);

// Read back the nested data using path operations
let txn_viewer = database.new_transaction().await?;
let viewer_store = txn_viewer.get_store::<DocStore>("users").await?;

// Get individual values using path operations
let _name_value = viewer_store.get_path(path!("user123.profile.name")).await?;
let _email_value = viewer_store.get_path(path!("user123.profile.email")).await?;
let _theme_value = viewer_store.get_path(path!("user123.preferences.theme")).await?;
let _host_value = viewer_store.get_path(path!("config.database.host")).await?;

// Get the entire user object to verify nested structure was created
if let Ok(_user_data) = viewer_store.get("user123").await {
    println!("User profile and preferences created successfully");
}

// Get the entire config object to verify nested structure
if let Ok(_config_data) = viewer_store.get("config").await {
    println!("Configuration data created successfully");
}

println!("Path-based operations completed successfully");
Ok(())
}

8. Working with Y-CRDT Documents (YDoc)

The YDoc store provides access to Y-CRDT (Yrs) documents for collaborative data structures. This requires the “y-crdt” feature flag.

extern crate eidetica;
extern crate tokio;
use eidetica::{Instance, backend::database::Sqlite, crdt::Doc, store::YDoc, Database};
use eidetica::y_crdt::{Map as YMap, Transact};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
// Setup database for testing
let backend = Sqlite::in_memory().await?;
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(backend),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let mut settings = Doc::new();
settings.set("name", "y_crdt_example");
let default_key = user.get_default_key()?;
let database = user.create_database(settings, &default_key).await?;

// Start an authenticated transaction (automatically uses the database's default key)
let txn = database.new_transaction().await?;

// Get the YDoc store handle
let user_info_store = txn.get_store::<YDoc>("user_info").await?;

// Writing to Y-CRDT document
user_info_store.with_doc_mut(|doc| {
    let user_info_map = doc.get_or_insert_map("user_info");
    let mut txn = doc.transact_mut();

    user_info_map.insert(&mut txn, "name", "Alice Johnson");
    user_info_map.insert(&mut txn, "email", "alice@example.com");
    user_info_map.insert(&mut txn, "bio", "Software developer");

    Ok(())
}).await?;

// Commit the transaction
let entry_id = txn.commit().await?;
println!("YDoc changes committed in entry: {}", entry_id);

// Reading from Y-CRDT document
let read_op = database.new_transaction().await?;
let reader_store = read_op.get_store::<YDoc>("user_info").await?;

reader_store.with_doc(|doc| {
    let user_info_map = doc.get_or_insert_map("user_info");
    let txn = doc.transact();

    println!("User Information:");

    if let Some(name) = user_info_map.get(&txn, "name") {
        let name_str = name.to_string(&txn);
        println!("Name: {name_str}");
    }

    if let Some(email) = user_info_map.get(&txn, "email") {
        let email_str = email.to_string(&txn);
        println!("Email: {email_str}");
    }

    if let Some(bio) = user_info_map.get(&txn, "bio") {
        let bio_str = bio.to_string(&txn);
        println!("Bio: {bio_str}");
    }

    Ok(())
}).await?;

// Working with nested Y-CRDT maps
let prefs_op = database.new_transaction().await?;
let prefs_store = prefs_op.get_store::<YDoc>("user_prefs").await?;

prefs_store.with_doc_mut(|doc| {
    let prefs_map = doc.get_or_insert_map("preferences");
    let mut txn = doc.transact_mut();

    prefs_map.insert(&mut txn, "theme", "dark");
    prefs_map.insert(&mut txn, "notifications", "enabled");
    prefs_map.insert(&mut txn, "language", "en");

    Ok(())
}).await?;

prefs_op.commit().await?;

// Reading preferences
let prefs_read_op = database.new_transaction().await?;
let prefs_read_store = prefs_read_op.get_store::<YDoc>("user_prefs").await?;

prefs_read_store.with_doc(|doc| {
    let prefs_map = doc.get_or_insert_map("preferences");
    let txn = doc.transact();

    println!("User Preferences:");

    // Iterate over all preferences
    for (key, value) in prefs_map.iter(&txn) {
        let value_str = value.to_string(&txn);
        println!("{key}: {value_str}");
    }

    Ok(())
}).await?;
Ok(())
}

YDoc Features:

• Collaborative Editing: Y-CRDT documents provide conflict-free merging for concurrent modifications
• Rich Data Types: Support for Maps, Arrays, Text, and other Y-CRDT types
• Functional Interface: Access via with_doc() for reads and with_doc_mut() for writes
• Atomic Integration: Changes are staged within the Transaction and committed atomically

Use Cases for YDoc:

• User profiles and preferences (as shown in the todo example)
• Collaborative documents and shared state
• Real-time data synchronization
• Any scenario requiring conflict-free concurrent updates

9. Persistent Storage with SQLite

With SQLite, data is automatically persisted to disk when using a file path:

use eidetica::{backend::database::Sqlite, Instance, crdt::Doc};
use std::path::PathBuf;

#[tokio::main]
async fn main() -> eidetica::Result<()> {
    // Create or open a persistent SQLite database by URL. The URL is
    // passed through to sqlx, so any sqlx-accepted form works.
    let (instance, _maybe_user) = Instance::connect_or_create(
        "sqlite://./my_database.db",
        eidetica::NewAccount::passwordless("admin"),
    ).await?;

    // Subsequent runs land on the load arm (`_maybe_user` is `None`); the
    // operator creates and logs in additional users through the admin path.
    let admin = instance.login_account("admin", "").await?;
    admin.admin().await?.create_account(eidetica::NewAccount::passwordless("alice")).await?;
    let mut user = instance.login_account("alice", "").await?;
    let mut settings = Doc::new();
    settings.set("name", "persistent_example");
    let default_key = user.get_default_key()?;
    let _database = user.create_database(settings, &default_key).await?;

    println!("Database created/opened at sqlite://./my_database.db");
    Ok(())
}

Key Points:

• SQLite automatically handles persistence — no manual save needed.
• Use memory:// for ephemeral state (tests, throwaway processes).
• Use sqlite://./path.db (or any sqlx-accepted URL) for persistent storage.
• Use memory:///abs/path/snap.json if you want in-memory speed plus an opt-in JSON snapshot.

10. Verifying Synced Data

Entries authored locally are validated at commit time and are immediately Verified. Entries received from a peer arrive Unverified and are hidden from the default (Verified-frontier) view until this node validates them.

extern crate eidetica;
extern crate tokio;
use eidetica::{Database, Instance, backend::database::Sqlite, crdt::Doc};

#[tokio::main]
async fn main() -> eidetica::Result<()> {
let (instance, mut user) = eidetica::Instance::create_backend(
    Box::new(Sqlite::in_memory().await?),
    eidetica::NewAccount::passwordless("alice"),
).await?;
let default_key = user.get_default_key()?;
let database = user.create_database(Doc::new(), &default_key).await?;
// `database` here has received entries from a sync peer.

// Validate everything received so far (prefix-closed: an entry is
// promoted only once its whole ancestor history is Verified).
let report = database.verify().await?;
println!(
    "verified={} failed={} still_unverified={}",
    report.verified, report.failed, report.still_unverified
);

// Default read shows only the Verified frontier.
let visible = database.snapshot().await?;

// Inspect not-yet-verified data explicitly (Failed is still dropped).
let with_pending = database.clone().allow_unverified().snapshot().await?;
let _ = (visible, with_pending);
Ok(())
}

• report.still_unverified > 0 means some entries are waiting on _settings (or an ancestor) that has not synced yet — re-run verify() after more sync and they promote.
• report.failed > 0 means entries were checked and rejected; they are permanently excluded from reads.

Complete Example: Chat Application

For a full working example that demonstrates Eidetica in a real application, see the Chat Example in the repository.

The chat application showcases:

• Account Management: Automatic passwordless user creation with key management
• Multiple Databases: Each chat room is a separate database
• Table Store: Messages stored with auto-generated IDs
• Multi-Transport Sync: HTTP for local testing, Iroh for P2P with NAT traversal
• Bootstrap Protocol: Automatic access requests when joining rooms
• Real-time Updates: Periodic message refresh with automatic sync
• TUI Interface: Interactive terminal UI using Ratatui

Key Architectural Concepts

The chat example demonstrates several advanced patterns:

1. AccountSession API with Automatic Key Management

// Initialize instance with sync enabled
let (instance, _) =
    Instance::connect_or_create("memory://", NewAccount::passwordless("admin")).await?;
instance.enable_sync().await?;

// Create passwordless user (or use existing) via the bootstrapped admin
let username = "alice";
let admin = instance.login_account("admin", "").await?;
let _ = admin.admin().await?.create_account(NewAccount::passwordless(username)).await;

// Login to get AccountSession (handles key management automatically)
let user = instance.login_account(username, "").await?;

// AccountSession API automatically manages cryptographic keys for databases
let default_key = user.get_default_key()?;
println!("Account {} has key: {}", username, default_key);

2. Room Creation with Global Access

// Create a chat room (database) with settings
let mut settings = Doc::new();
settings.set("name", "Team Chat");

let key_id = user.get_default_key()?;
let database = user.create_database(settings, &key_id)?;

// Add global permission so anyone can join and write
let tx = database.new_transaction()?;
let settings_store = tx.get_settings()?;
let global_key = auth::AuthKey::active(None, auth::Permission::Write(10));
settings_store.set_global_auth_key(global_key)?;
tx.commit()?;

println!("Chat room created with ID: {}", database.root_id());

3. Message Storage with Table

use chrono::{DateTime, Utc};
use uuid::Uuid;

#[derive(Debug, Clone, Serialize, Deserialize)]
struct ChatMessage {
    id: String,
    author: String,
    content: String,
    timestamp: DateTime<Utc>,
}

impl ChatMessage {
    fn new(author: String, content: String) -> Self {
        Self {
            id: Uuid::new_v4().to_string(),
            author,
            content,
            timestamp: Utc::now(),
        }
    }
}

// Send a message to the chat room
let message = ChatMessage::new("alice".to_string(), "Hello, world!".to_string());

let txn = database.new_transaction()?;
let messages_store = txn.get_store::<Table<ChatMessage>>("messages")?;
messages_store.insert(message)?;
txn.commit()?;

// Read all messages
let txn_viewer = database.new_transaction()?;
let viewer_store = txn_viewer.get_store::<Table<ChatMessage>>("messages")?;
let all_messages = viewer_store.search(|_| true)?;

for (_, msg) in all_messages {
    println!("[{}] {}: {}", msg.timestamp.format("%H:%M:%S"), msg.author, msg.content);
}

4. Bootstrap Connection to Remote Room

use eidetica::{sync::{DatabaseTicket, transports::http::HttpTransport}, account::types::SyncSettings};

// Join an existing room using a ticket URL
let ticket: DatabaseTicket = "eidetica:?db=bafyr4i...&pr=http:127.0.0.1:8080".parse()?;
let room_id = ticket.database_id().clone();

// Register sync transport
if let Some(sync) = instance.sync() {
    sync.register_transport("http", HttpTransport::builder().bind("127.0.0.1:0")).await?;

    // Request access to the room (bootstrap protocol)
    let key_id = user.get_default_key()?;
    user.request_database_access(
        &sync,
        &ticket,
        &key_id,
        eidetica::auth::Permission::Write(10),
        SyncSettings::on_commit(),
        None,
    ).await?;

    // Open the synced database
    let database = user.open_database(&room_id)?;
    println!("Joined room successfully!");
}

5. Real-time Sync with Callbacks

// Automatic sync is configured via peer relationships
// When you add a peer for a database, commits automatically trigger sync
if let Some(sync) = instance.sync() {
    if let Ok(peers) = sync.list_peers() {
        if let Some(peer) = peers.first() {
            // Add tree sync relationship - this enables automatic sync on commit
            sync.add_tree_sync(&peer.pubkey, &database.root_id()).await?;

            println!("Automatic sync enabled for database");
        }
    }
}

// Manually trigger immediate sync for a specific database
sync.sync_with_peer(&Address::http("127.0.0.1:8080"), Some(&database.root_id())).await?;

Running the Chat Example

# From the repository root
cd examples/chat

# Create a new room (default uses Iroh P2P transport)
cargo run -- --username alice

# Or use HTTP transport for local testing
cargo run -- --username alice --transport http

# Connect to an existing room
cargo run -- <room_address> --username bob

Creating a new room: When you run without a room address, the app will:

1. Create a new room
2. Display the room address that others can use to join
3. Wait for you to press Enter before starting the chat interface

Example output:

🚀 Eidetica Chat Room Created!
📍 Room Address: abc123@127.0.0.1:54321
👤 Username: alice

Share this address with others to invite them to the chat.
Press Enter to start chatting...

Joining an existing room: When you provide a room address as the first argument, the app connects and starts the chat interface immediately.

Transport Options

HTTP Transport (--transport http):

• Simple client-server model for local networks
• Server binds to 127.0.0.1 with random port
• Address format: room_id@127.0.0.1:PORT
• Best for testing and same-machine demos

Iroh Transport (--transport iroh, default):

• Peer-to-peer with built-in NAT traversal
• Uses QUIC protocol with relay servers
• Address format: room_id@{node-info-json}
• Best for internet connections across networks

Architecture Highlights

The chat example demonstrates production-ready patterns:

• Multi-database architecture: Each room is isolated with independent sync state
• Account management: Automatic key discovery and database registration
• Bootstrap protocol: Seamless joining of rooms with access requests
• Dual transport support: Flexible networking for different environments
• CRDT-based messages: Eventual consistency with deterministic ordering
• Automatic sync: Background synchronization triggered by commits via callbacks

See the full chat example documentation for detailed usage instructions, complete workflow examples, troubleshooting tips, and implementation details.

Overview

This section contains documentation for developers and those who want to understand the full system and technology behind Eidetica. It covers architecture, internals, and development practices.

If you want to contribute, start with the Contributing guide for development environment setup and workflow.

Architecture Overview

Eidetica is a decentralized database designed to “Remember Everything.”

The system is built on a foundation of content-addressable entries organized in databases, with a pluggable backend system for storage. Entry objects are immutable and contain Tree/SubTree structures that form the Merkle-DAG, with integrated authentication using Ed25519 digital signatures to verify the integrity of the data and its history. Database and Store abstractions over these internal structures help to translate those concepts into something more familiar to developers.

See DAG Structure for details on the Merkle-DAG architecture.

Other key subsystems include:

• Sync: Merkle-CRDT based peer-to-peer synchronization
• Service (Daemon): Local daemon mode for multi-process access over Unix domain sockets
• Authentication: Ed25519 signature-based entry verification
• Encryption: PasswordStore formats, cache encryption, and security boundaries
• Errors: Modular error hierarchy with cross-process reconstruction

API Reference

For detailed API documentation, see the rustdoc API reference (development version) or docs.rs/eidetica (stable releases).

Contributing

This guide covers setting up a local development environment for contributing to Eidetica.

Prerequisites

Eidetica uses Nix for reproducible development environments. Install Nix with flakes enabled, or use the Determinate Systems installer which enables flakes by default. The Nix flake provides pinned versions of all development tools: Rust toolchain, cargo-nextest, mdbook, formatters, and more.

If you want to skip Nix, a standard Rust toolchain should be sufficient. The main project is structured as a Cargo workspace.

Command Runner

just provides convenient commands for common workflows. Commands wrap cargo, nix, and other tools as needed.

just   # See all available commands

Common Commands

Testing

Nix Commands

Direct Nix commands are available when needed:

Packages are organized into groups: test, doc, lint, coverage, sanitize. Each group supports .#<group>.default (fast), .#<group>.all (all), and .#<group>.<name> (specific). See CI/Build Infrastructure for details.

Binary caching via a binary cache speeds up builds by providing pre-built dependencies.

Benchmarks

Criterion benchmarks live in crates/lib/benches/. Run them with nix run .#bench, or a single target with cargo bench --bench backend_benchmarks. CI runs cargo bench --workspace weekly and uploads the results to Bencher.

Sample size is set per harness — 30 for backend_benchmarks, 50 for benchmarks — and --sample-size N overrides it:

cargo bench --bench backend_benchmarks -- --sample-size 60

Two groups build a large tree in every iteration’s setup, so they fall back to a reduced sample size of 10 when the flag is absent: large_tree_operations and get_tree_from_tips. Each announces that on stderr. Results are noisy at that size — pass --sample-size when a number needs to be trusted, and note that the tracked history for those two groups was collected at 10 samples, so it is not comparable with a larger run.

Development Workflow

1. Enter the dev shell: nix develop or use direnv.
2. Make changes and run just dev for SQLite tests and Clippy. Nextest builds the test binaries, and Clippy checks all targets, so a separate just build is not needed for routine validation.
3. During iteration, run a focused test with just test <filter>; use just test service for service-backend changes. These use Cargo’s incremental cache in the current worktree.
4. Run just ci for the full local check-only pipeline (format, lint, API docs, tests, doc tests and book links). Run just fix separately when fixes are needed; just ci does not edit the tree.
5. Use just ci nix for the reproducible push-CI graph before delivery: all checks and integration tests, including the real daemon smoke test. The release binary is built as an integration dependency; there is no separate binary-build pass.

CI Integration

The same checks that run locally also run in CI. See CI/Build Infrastructure for details on the CI systems.

CI/Build Infrastructure

Eidetica uses a CI/build system with GitHub Actions, Forgejo CI, and Nix flakes.

The philosophy for CI is that compute resources are cheap and developer resources (my time) are expensive. CI is used for comprehensive testing, status reporting, security checks, dependency updates, documentation generation, and releasing. In the current setup some of these are run multiple times on several platforms to ensure compatibility and reliability across different environments.

Fuzz / simulation testing are planned for the future.

CI Systems

GitHub Actions

The primary CI runs on GitHub with these workflows. Main CI and its dependent dev publisher share native workflow concurrency: the running pipeline finishes, and GitHub keeps one replaceable pending pipeline. PRs cancel only superseded runs of the same PR; other branches, manual release publishing and release events do not enter the main dev group:

• ci.yml: Main CI pipeline (lint, test, integration tests, docs)
• publish.yml: Publish pipeline (container images, release artifact caching, size tracking)
• sanitizers.yml: Memory and thread sanitizer checks
• benchmarks.yml: Performance benchmarks
• coverage.yml: Multi-backend code coverage tracking via Codecov
• deploy-docs.yml: Documentation deployment to GitHub Pages
• release-plz.yml: Automated releases and crates.io publishing
• mirror.yml: Forgejo mirror sync (main branch only)
• security-audit.yml: Daily advisory scanning
• cargo-update.yml: Monthly cargo dependency updates
• flake-update.yml: Monthly Nix flake input updates
• actions-update.yml: Monthly GitHub Actions version updates
• dependency-hold.yml: Merge gate for dependency PRs
• update-hold.yml: Automatic hold expiry

Ordinary pull requests run formatting (treefmt), lint (including Clippy), module evaluation, the debug binary build and the complete SQLite backend test suite (legacyPackages.aarch64-linux.ci-pr) as CI (aarch64-linux). SQLite is selected at runtime with TEST_BACKEND=sqlite; the test binaries still use all features. Standalone API documentation, doctests and minimal-feature tests are excluded from this lean PR target. Main pushes and manual CI runs retain all checks, including documentation, doctests, minimal-feature tests and the full backend suite, on both platforms, plus x86 integration tests. The full checks and legacyPackages.<system>.ci targets, individual backend/platform/integration targets and their dependency artifacts are unchanged. To test a PR’s branch with the full suite, select that branch in the CI workflow’s Run workflow branch selector. The manual run checks the selected branch head, not a synthetic merge with main.

Every CI run checks ARM on the persistent native ARM64 runner described below. On main pushes and manual main runs, the hosted x86 job starts only after the ARM job succeeds; it then checks x86 and integration and publishes the outputs it checked from that same store (see Cache Push Isolation). A failed, cancelled or skipped ARM job stops x86 checks, cache publication and the dev publisher. Manual runs of other branches check x86 on a hosted runner alongside ARM, without credentials or publication.

Each check job builds its whole target set in one nix build -L --keep-going graph with .github/scripts/ci-checks.sh, so fresh builds stream their logs. Afterwards it replays the stored log of every leaf check, named by platform and check (for example aarch64-linux test.sqlite), whether the check was just built, substituted or failed. A replayed log is a retained record, not a rerun: a substituted check’s log shows an earlier execution. Where no log is available it warns explicitly and infers no test result; log reporting never changes the build’s exit status.

Persistent ARM64 Runner

ARM checks run on a self-hosted runner registered to this repository only, with labels self-hosted, Linux, ARM64, zinc, nix. Its single listener runs one Eidetica job at a time; other repositories’ runners on the same host are independent and may build concurrently. The runner uses its own unprivileged account, work directory and registration, and is never shared with another repository. The host’s Nix daemon owns substituters, signature checking and build limits; the runner account is an ordinary untrusted client, not a trusted-user, so jobs cannot add substituters or keys. The store persists between jobs, so unchanged outputs are reused instead of rebuilt. Jobs receive no secrets, the runner holds no cache signing or upload credentials, and nothing it builds is signed or uploaded.

Public pull requests execute contributor code on this runner, so workflow runs from outside collaborators must require approval before they start (repository Settings → Actions → General → Approval for running fork pull request workflows from contributors → Require approval for all outside collaborators).

To provision or verify the runner:

1. Create a dedicated unprivileged user and work directory for Eidetica on the host, separate from any other runner.
2. Register a persistent (non-ephemeral) runner for arcuru/eidetica with a fresh registration token and the labels above; do not copy another runner’s credentials or configuration.
3. Confirm that nix config show as the runner user lists https://cache.eidetica.dev and its public key among the daemon’s substituters, and that the user is not a trusted user (nix store ping reports Trusted: 0).
4. Confirm the runner is idle in Settings → Actions → Runners, then run CI manually on a branch and check that CI (aarch64-linux) runs there, skips Nix installation and replays logs.

Forgejo CI

A dedicated Forgejo runner provides CI redundancy on code.jackson.dev. The Forgejo workflows mirror the testing in the GitHub Actions setup with minor adaptations for the Forgejo environment. The lint and test jobs each request their existing Nix targets in a single nix build invocation so shared dependencies can be scheduled together; job boundaries and checks are unchanged.

Nix Flake

The Nix flake defines reproducible builds and CI checks that run identically locally and in CI. just nix full builds the existing combined push-CI graph (checks and all integration tests) in one nix-fast-build invocation, sharing dependencies and overlapping integration with other checks. just nix check runs checks only; just nix integration runs all integration tests, including the real daemon smoke test. For one hermetic target, use just nix test sqlite, just nix lint clippy, just nix doc links, or just nix integration service. Bare just nix test retains the interactive archive runner; just nix test sqlite builds the cached Nix check.

• nix build - Build the default package
• nix flake check - Run all CI checks (audit, clippy, doc, test, etc.)

Backend test checks and interactive runners share one all-feature workspace nextest archive. Each backend check extracts only the runtime archive and remaps the workspace to its own writable source tree, rather than restoring the Cargo dependency and test build targets. The minimal-feature check still builds separately, and doctests remain in the documentation checks.

Binary caching via a nix binary cache speeds up builds by providing pre-built dependencies. It is located at:

• URL - https://cache.eidetica.dev
• Public Key - cache.eidetica.dev-1:eND5gRJlbnool3ZLCWT2H8kkygWS8JcsU76HYXbWPBI=

Nix Package Groups

The Nix flake organizes packages into groups with a consistent pattern:

• .#<group>.default — runs a sensible default subset
• .#<group>.all — runs ALL items in the group
• .#<group>.<name> — runs a specific item

Interactive Runners

Interactive runners execute commands with live output, accepting additional arguments:

Code Coverage

Code coverage runs against all storage backends to ensure comprehensive test coverage:

Local Coverage Commands

just coverage            # SQLite backend (default)
just coverage inmemory   # InMemory backend
just coverage postgres   # PostgreSQL (starts a container automatically)
just coverage all        # All backends, merged into coverage/lcov.info

The coverage all command runs all three backends sequentially and merges the LCOV reports using lcov. Individual reports are saved as coverage/lcov-{backend}.info.

CI Coverage

GitHub Actions runs coverage for each backend in parallel and uploads to Codecov with backend-specific flags. Codecov automatically merges the reports server-side, providing both combined coverage metrics and per-backend breakdowns.

For local development setup, see Contributing.

Dependency Management

Dependencies are managed through purpose-built GitHub Actions workflows rather than third-party tools. Three categories of dependencies are updated monthly, each with a mandatory hold period before merge. A daily security audit catches advisory database updates between dependency cycles.

Security Audits

The security-audit.yml workflow runs daily at 06:00 UTC. It executes cargo deny check advisories against the RustSec Advisory Database.

• On failure: creates or updates a GitHub issue with the security label containing the full advisory output
• On success: auto-closes any existing security advisory issue

The .config/deny.toml configuration contains an ignore list for known advisories from transitive dependencies (e.g., unmaintained crates pulled in by iroh). The CI lint target (nix build .#lint.deny) checks bans licenses sources but not advisories — advisories are excluded from regular CI because new advisories appear without code changes and would cause flaky builds. The dedicated daily workflow fills this gap.

Update Workflows

Three monthly update workflows run on the 1st of each month:

Each workflow:

1. Sets up the deps/ branch (creates new or rebases existing onto main)
2. Runs the update command
3. Checks if anything changed (exits cleanly if not)
4. Commits and pushes with --force-with-lease (preserving any existing user commits)
5. Creates a PR (or updates the existing one) with dependencies and on-hold labels
6. Embeds a <!-- hold-until: YYYY-MM-DD --> comment in the PR body; CI handles verification

Shared branch setup and commit/PR logic is extracted into composite actions at .github/actions/.

The flake update workflow generates an input diff summary with GitHub compare links for each changed flake input (nixpkgs, crane, fenix, etc.).

The actions update workflow parses uses: lines across all .github/workflows/*.yml and .forgejo/workflows/*.yml files, queries the GitHub API for the latest release of each action, and updates SHA pins in-place. It respects the Forgejo constraint: actions/checkout in .forgejo/ stays at v4.

All three workflows support workflow_dispatch for manual triggering with a configurable hold_days parameter (default: 7).

Hold Mechanism

Dependency update PRs are gated by a mandatory hold period to allow time for upstream issues to surface before merging.

The hold system has two components:

dependency-hold.yml — A PR status check that runs on every pull request to main. For branches that start with deps/, it queries the GitHub API for the PR’s current labels and fails if the on-hold label is present, blocking merge. For all other branches, it passes immediately. Live label queries (rather than event payload data) ensure manual re-runs always see current state. This job name (Dependency Hold) is configured as a required status check in branch protection.

update-hold.yml — A daily workflow (07:00 UTC) that scans open PRs with the on-hold label. It parses the <!-- hold-until: YYYY-MM-DD --> comment from each PR body. When today’s date meets or exceeds the hold date, it removes the on-hold label, creates a passing Dependency Hold check run on the PR’s HEAD commit, and posts a comment.

Day 0:  cargo-update.yml creates PR with on-hold label
        dependency-hold.yml → FAIL (on-hold label present, merge blocked)

Day 7:  update-hold.yml removes on-hold label + creates passing check run
        Dependency Hold check → PASS (merge allowed)

Required Setup

The hold mechanism requires repository configuration:

• An on-hold label in GitHub repo settings
• A dependencies label (for categorization)
• Dependency Hold added as a required status check in branch protection for main (displayed as Deps: Hold Gate / Dependency Hold in the checks UI)

Secret Management

Secrets are scoped to GitHub Environments with branch restrictions rather than stored as repo-level secrets. This limits the blast radius of a compromised workflow or action.

Environments

All environments are restricted to the main branch. Low-risk tokens that can only upload metrics or coverage data remain at the repo level.

Cache Push Isolation

Only the protected main x86 job in CI and the jobs in the publish workflow (publish.yml) receive the publish environment’s cache push credentials and signing key. Other CI check jobs, the ARM runner, coverage and sanitizer jobs remain secretless and read-only.

On trusted main push/manual runs, the main x86 job waits for the ARM checks, then builds the x86 checks and integration tests with the project cache as a substituter. It then lists, without building or fetching anything, the realized flake check outputs and the reusable test artifacts (legacyPackages.x86_64-linux.test.artifacts): the nextest archive, plus the Cargo targets when that job compiled the tests rather than substituting them. This happens before any secret enters its environment. Only its final step receives credentials; .github/scripts/publish-cache.sh refuses any other repository, event or ref, validates the listed store paths, signs them with nix store sign and copies them with nix copy. It evaluates and builds nothing, and publishes nothing from pull requests or the ARM runner. Integration test results and ARM check outputs are not published.

CI calls the reusable publisher only after that job succeeds. That calling job grants packages: write and names only the publisher’s required secrets. GitHub requires those names to be passed even for environment-scoped secrets; the protected callee environments supply their values and take precedence over caller-level values. The repo-level Bencher token is upload-only. The signing key stays in the protected publishing environment, not in CI check jobs. The publisher rebuilds from source rather than consuming PR artifacts.

Release Build Integrity

The publish workflow builds artifacts (binary and container image) from source using --option substituters 'https://cache.nixos.org', bypassing the project’s own binary cache entirely. This prevents a cache poisoning attack from affecting published artifacts. The from-source results are pushed to the cache afterward. The publisher does not rebuild check outputs: on dev runs, main CI has already published the x86 check outputs from the store that checked them. Its fresh publisher store is separate from that check store, so a cached check output never becomes a release artifact.

Releasing

Releases are automated via release-plz and configured in .config/release-plz.toml. The process uses unified workspace versioning — all crates share a single version and tags follow the v<major>.<minor>.<patch> format.

Publish Workflow

A single publish.yml workflow handles all artifact publishing. It is triggered in three ways:

1. A dependent job in main CI (workflow_call) — after the ARM checks and then the x86 checks and cache publication succeed, CI calls the dev publisher for the same commit. It pushes dev-tagged container images, caches the from-source release artifacts and tracks sizes. The native concurrency group covers CI, both publish architectures and the manifest, with no separate CI-completion queue or revision selector. A manual CI run on main uses the same path; a selected nonmain branch gets full CI without publishing.

2. After a release is created (workflow_dispatch) — triggered by the release-plz workflow after publishing to crates.io and creating a GitHub release. Accepts a tag name and pushes versioned container images (e.g. 1.2.3, 1.2, latest). Can also be triggered manually to retry a failed release.

3. Fallback (release: published) — kept as a fallback trigger in case the release is created with a token that fires events (e.g. a PAT or GitHub App token). Under normal operation, release-plz uses GITHUB_TOKEN, which does not fire the release event due to a GitHub Actions limitation, so the explicit workflow_dispatch from trigger 2 is the primary release path.

Release Flow

flowchart TD
    A["Trigger release PR
    (manual workflow_dispatch)"] --> B["Review and merge PR
    (human review)"]
    B --> C["CI passes on main"]
    B --> D["release-plz detects
    version bump"]
    C --> E["publish.yml
    (dependent reusable job)"]
    E --> F["Build from source +
    dev images +
    Bencher size tracking"]
    D --> G["Publish to crates.io +
    create GitHub release + tag"]
    G --> H["release-plz dispatches
    publish.yml
    (workflow_dispatch)"]
    H --> I["Build from source + push
    versioned container images"]

Step-by-Step

1. Trigger a release PR — Manually run the Release-plz workflow via GitHub Actions (Actions → Release-plz → Run workflow). This runs release-plz release-pr, which:

   • Analyzes commits since the last release to determine the version bump
   • Updates version numbers in Cargo.toml files
   • Generates the changelog using release-plz’s built-in changelog generator
   • Opens a PR titled chore: release v<version> with the release label

2. Review the release PR — The PR body includes a checklist:

   • Review the version bump (previous → next)
   • Review semver compatibility
   • Edit CHANGELOG.md to add custom commentary or highlights
   • Ensure CI passes

   Note: Subsequent pushes to main while the PR is open will cause release-plz to update the PR (re-analyze commits, adjust version if needed).

3. Merge the PR — Merging triggers two independent pipelines:

   • publish.yml: CI calls it after both architectures pass on main, within the same native concurrency group. It builds that CI run’s commit from source, caches those release artifacts, pushes dev-tagged container images and tracks sizes via Bencher.
   • release-plz.yml: Detects the version bump and runs release-plz release.

4. Automatic publishing — The release-plz release job:

   • Publishes the crate to crates.io via trusted publishing
   • Creates a GitHub release with tag v<version> and a release body containing the changelog, install instructions, and documentation links
   • Dispatches publish.yml via workflow_dispatch with the release tag name

5. Container image publishing — The dispatched publish.yml run:

   • Builds Nix binary and OCI container image for x86_64 and aarch64
   • Pushes versioned container images to GHCR and Docker Hub (e.g., 1.2.3, 1.2, latest)
   • Creates multi-arch manifests
   • Attaches SBOM and provenance attestations
   • Updates Docker Hub repository description

   If this workflow fails partway through, it can be manually retried via workflow_dispatch with the release tag name (e.g., v1.2.3). The retry checks out the tagged commit to ensure the build matches the release.

Container Image Tags

* latest is only pushed when the release is the highest non-prerelease version (determined via the GitHub Releases API). Backport releases for older branches do not overwrite latest.

Configuration

Notes

• The release-plz-pr job only updates an existing release PR on push to main — it does not create new ones automatically. New release cycles are initiated via workflow_dispatch.
• All publish runs build artifacts from source without the project’s own Nix cache substitution to prevent cache poisoning (see Release Build Integrity).
• The dev path exercises the same build and publish jobs as releases. Native concurrency coalesces main pipelines without cancelling a running publish. It does not search for a historically successful revision or promise commit-order scheduling; rerunning an old main workflow can publish its old commit. Dev and independent release runs use separate intermediate base-image tags.
• Pages deployment follows successful completion of the full CI/dev-publish pipeline, so documentation builds can use its published outputs. It does not follow independent release publishes.

Terminology

Eidetica uses two naming schemes:

Internal Data Structures

Trees and Subtrees. These align with the names used inside of an Entry:

• TreeNode: Main tree node within an Entry (root CID, parent references, metadata)
• SubTreeNode: Named subtree nodes within an Entry (name, parents, data payload)
• ID: A wrapper around a CID (Content Identifier). Entries are serialized to DAG-CBOR and hashed to produce their CID. The string representation uses multibase base32lower encoding (bafyr4i...).

Use these when discussing Entry internals, Merkle-DAG structure, or serialized data format.

User-Facing Abstractions

• Account: Private root database for a person, group, or application; identified by its root ID, not a login name. Exposed as the typed Account; its backing database is encapsulated, and applications use AccountSession reference operations for discovery.
• AccountSession: Unlocked instance-local access to an Account, returned by login; distinct from the Account and its local login record.
• Identity: Key-declaration/delegation database, distinct from the private Account root; links alone do not confer authority.
• Database: Collection of entries with shared authentication and history
• Store: Typed data access (DocStore, Table, YDoc) operating on named subtrees
• Projection: A Store-defined representation of current CRDT state. The default is one opaque whole-state record.
• Namespace: A backend-owned set of ordered opaque Store-state records for one database, Store, projection, source, and trust scope.
• Derived namespace: Immutable disposable state materialized from historical Entries.
• Authoritative namespace: Durable current Store state that cache clearing cannot select.
• Staging namespace: Unpublished records used while building a complete namespace; readers cannot resolve it.

Use these in public APIs, user documentation, and error messages.

A historical Database is an abstraction over a Tree, and Stores are an abstraction over the Subtrees within. Historical reads may resolve a derived Store-state projection that can be cleared and rebuilt from Entries.

DAG Structure

Eidetica organizes data in a layered Merkle-DAG called a Tree. A Tree consists of Entries that form the main DAG, and each Entry can contain data for multiple subtrees. Each subtree forms its own independent DAG across the Entries.

Each Entry is immutable and content-addressable. Its ID is a CID (Content Identifier) computed by serializing the entry to DAG-CBOR and creating a CID pointing to that deterministic representation. Parent references use these CIDs, forming the Merkle structure.

For simplicity, let’s walk through an example Tree with 4 Entries.

Entries Contain Subtrees

An Entry is the atomic unit. Each Entry can contain data for zero or more named subtrees:

graph LR
    subgraph E1[Entry 1]
        E1_t1[table_1]
        E1_t2[table_2]
    end

    subgraph E2[Entry 2]
        E2_t1[table_1]
    end

    subgraph E3[Entry 3]
        E3_t2[table_2]
    end

    subgraph E4[Entry 4]
        E4_t1[table_1]
        E4_t2[table_2]
    end

Entry 1 and Entry 4 contain data for both subtrees. Entry 2 only modifies table_1. Entry 3 only modifies table_2.

Main Tree DAG

The Tree DAG connects Entries through parent references (hashes of parent Entries). Entry 2 and Entry 3 are created in parallel (both reference Entry 1’s hash as their parent). Entry 4 merges the branches by listing both Entry 2 and Entry 3’s hashes as parents:

graph LR
    E1[Entry 1] --> E2[Entry 2]
    E1 --> E3[Entry 3]
    E2 --> E4[Entry 4]
    E3 --> E4

This shows the branching and merging capability of the DAG structure.

Subtree DAGs

Each subtree forms its own DAG by following subtree-specific parent references. These can skip Entries that didn’t modify that subtree.

table_1 DAG - Entry 3 is skipped (no table_1 data):

graph LR
    E1[Entry 1] --> E2[Entry 2] --> E4[Entry 4]

table_2 DAG - Entry 2 is skipped (no table_2 data):

graph LR
    E1[Entry 1] --> E3[Entry 3] --> E4[Entry 4]

The main tree branches and merges, but each subtree DAG remains linear because E2 and E3 modified different subtrees.

Atomic Cross-Subtree Edits

A Transaction creates a single Entry. This makes it the primitive for synchronized edits across multiple subtrees within a Tree.

In the example above, Entry 1 and Entry 4 modify both table_1 and table_2 in a single Entry. Because an Entry is atomic, you always see both edits or neither - there’s no state where only one subtree’s changes are visible. This enables reliable cross-subtree operations where related data must stay consistent.

Sparse Verified Checkouts

Because subtree DAGs are independent, you can sync and verify just one subtree without the full tree data.

To verify table_1:

1. Fetch only Entries that contain table_1 data (E1, E2, E4)
2. Follow table_1’s parent chain to verify the complete history
3. Entry 3 is not needed - it has no table_1 data

This enables efficient partial sync while maintaining full cryptographic verification of the synced data.

Two senses of “verify”. Here “verify” means cryptographically check the integrity and signatures of a synced history. That is distinct from an entry’s stored VerificationStatus (Unverified / Verified / Failed), which records whether this node has run that check and accepted the result. Synced entries arrive Unverified; Database::verify() (or the access-time hook) performs the check above against the _settings each entry pins and promotes accordingly, prefix-closed. See Core Concepts and the authentication design.

Settings Example

An example of how this is used effectively is the design of settings for the Tree.

The settings, including authentication, is stored in the _settings subtree. Each Entry in the Tree points to the latest tips of the _settings subtree.

What this means is that you can fully verify the authentication for any Entry only by syncing the _settings subtree, and without needing to download any other data from the Tree.

Subtrees

Each entry can contain multiple subtrees (e.g., “messages”, “_settings”). Subtrees maintain independent parent-child relationships within the DAG.

Subtree Root Entries

A subtree root is an entry that starts a named subtree:

• Contains a SubTreeNode for the subtree
• Has empty subtree parents ([])
• Still has normal main tree parents

Entry {
    tree: TreeNode {
        root: "tree_id",
        parents: ["main_parent_id"],  // Normal main tree parents
    },
    subtrees: [
        SubTreeNode {
            name: "messages",
            parents: [],  // Empty = subtree root
            data: "...",
        }
    ],
}

Subsequent entries reference previous subtree entries as parents:

SubTreeNode {
    name: "messages",
    parents: ["previous_messages_entry_id"],
    data: "...",
}

Automatic Parent Discovery

Transactions automatically determine subtree parents:

1. If using current database tips → get current subtree tips
2. If using custom parents → find subtree tips reachable from those parents
3. If first subtree entry → empty tips (creates subtree root)

Always use transactions for entry creation - they handle parent discovery automatically.

Subtree Heights

Each subtree can have its own height value. Heights provide deterministic ordering when merging DAG branches after network splits. Entries are processed in height order, with ties broken by entry hash:

• Height = None: Subtree inherits the tree (main database) height
• Height = Some(h): Subtree has an independent height value

The Entry.subtree_height() accessor handles inheritance transparently:

// If subtree.height is None, returns tree.height (inheritance)
// If subtree.height is Some(h), returns the independent value
let height = entry.subtree_height("messages")?;

Per-subtree height strategies are configured via the _index registry. System subtrees (_settings, _index, _root) always inherit from the tree.

See Height Strategy Design for implementation details.

Implementation

See src/entry/mod.rs and src/transaction/mod.rs for implementation.

CRDT Merging

Eidetica implements a Merkle-CRDT using content-addressable entries organized in a Merkle DAG structure. Entries store data and maintain parent references to form a distributed version history that supports deterministic merging.

Core Concepts

• Content-Addressable Entries: Immutable data units forming a directed acyclic graph
• CRDT Trait: Enables deterministic merging of concurrent changes
• Parent References: Maintain history and define DAG structure
• Tips Tracking: Identifies current heads for efficient synchronization

Encoding and the Rust API

Codec and CRDT are independent traits exported from eidetica::crdt:

• Codec: Sized provides encode(&self) -> Result<Vec<u8>> and decode(bytes: &[u8]) -> Result<Self> at persistence boundaries.
• CRDT: Sized provides only merge(&self, other: &Self) -> Result<Self>. Its existing merge semantics are unchanged.
• Store::Data requires CRDT + Codec + Default + 'static. History reconstruction needs a default state and encoding; merge-only code does not.
• Transaction::get_local_data<T> requires only T: Codec.

Neither trait requires Clone or Serde. There is no blanket Codec implementation: deriving Serde does not choose a persistence format.

Migrating implementers from Data

This is an intentional source-level Rust API change. Replace the old empty impl Data for MyData {} with an explicit Codec implementation. For an existing JSON-backed type, retain the same Serde representation: implement encode with Ok(serde_json::to_vec(self)?) and decode with Ok(serde_json::from_slice(bytes)?), using the existing serde_json dependency. The Codec API reference contains a complete implementation example.

The existing Doc, Value, List and YrsBinary types implement Codec; import it to call their persistence methods:

#![allow(unused)]
fn main() {
use eidetica::crdt::{Codec, Doc};

let mut value = Doc::atomic();
value.set("name", "example");
let bytes = value.encode()?;
assert_eq!(bytes, br#"{"_a":true,"children":{"name":{"Text":"example"}}}"#);
assert_eq!(Doc::decode(&bytes)?, value);
Ok::<(), eidetica::Error>(())
}

Keep the existing CRDT::merge implementation. Generic code that previously inherited serialization, cloning or a default value from CRDT must request only the additional traits it actually uses. Existing concrete types retain their Serde and Clone implementations; those capabilities are no longer requirements of either trait.

A codec must preserve tombstones and all other merge-relevant state. Decode one complete value and report malformed or trailing data as an error, not as a default state. Staged reads still treat zero bytes as absent; whitespace-only JSON remains a decode error. Custom codecs may use non-JSON bytes, but switching an existing Store’s durable format is a separate compatibility change.

No storage migration is needed for the built-in types. Doc, Value and List use their existing JSON representations, including atomic flags, tombstones and list positions. YrsBinary retains the JSON {"data":[...]} wrapper around Yjs v1 update bytes, rather than writing raw updates. Settings, Table rows, Entry hashing, encryption and service formats are unchanged. This does not introduce canonical JSON ordering or a new merge algebra.

Standalone Map and Lww Algebra

Map<K, V> and Lww<T> implement merge algebra independently of persistence. Neither provides a built-in Codec or Serde implementation, so neither selects a durable format or changes an existing Store. Their payloads need not implement Default or encoding traits.

Ordered register operations

Lww<T> has three operations: NoOp (the default and identity), Set(T) and Delete. Merging chooses the rightmost non-NoOp in the caller’s deterministic operation order. There are no timestamps or clocks, and merge is intentionally not commutative. The payload is replaced whole, not merged, and only needs Clone for CRDT::merge. Lww::from(value) or value.into() wraps a payload in Set without requiring Clone or other payload traits.

Delete is retained as a tombstone. A later Set replaces it at the value level; this is not a causal delete-wins rule or Table row resurrection policy.

#![allow(unused)]
fn main() {
use eidetica::crdt::{CRDT, Lww};

#[derive(Clone, Debug, PartialEq)]
struct Payload(&'static str); // No Default, CRDT, Codec or Serde.

let old = Lww::Set(Payload("before"));
let deleted = old.merge(&Lww::Delete)?;
assert_eq!(deleted, Lww::Delete);
assert_eq!(deleted.merge(&Lww::NoOp)?, Lww::Delete);
assert_eq!(deleted.merge(&Payload("after").into())?, Lww::Set(Payload("after")));
Ok::<(), eidetica::Error>(())
}

Per-key delta composition

Map<K, V> stores operations in a BTreeMap. The empty map is the identity. Merge preserves keys present on only one side and invokes V::merge exactly once for each overlapping key. It propagates value merge errors and leaves both inputs unchanged. Associativity follows from the value algebra; commutativity and idempotency are not required.

Keys are literal ordered keys, not document paths. get, iter and len include every stored operation, even Lww::NoOp and Lww::Delete. Map does not interpret or discard tombstones, and defines no row deletion/lifecycle policy. Values can be other Maps, Docs or application-defined CRDTs, not just Lww.

Use insert_delta for repeated same-key updates: it composes the new delta with the stored operation via V::merge. If merging fails, the stored operation is unchanged. The low-level insert instead replaces the stored operation verbatim and returns the previous value. It does not compose deltas.

#![allow(unused)]
fn main() {
use eidetica::{crdt::{CRDT, Map}, Result};

#[derive(Clone, Debug, PartialEq)]
struct Events(Vec<u8>); // An associative, non-idempotent application CRDT.

impl CRDT for Events {
    fn merge(&self, other: &Self) -> Result<Self> {
        Ok(Self([self.0.as_slice(), other.0.as_slice()].concat()))
    }
}

let mut updates = Map::new();
updates.insert_delta("events".to_owned(), Events(vec![1]))?;
updates.insert_delta("events".to_owned(), Events(vec![2]))?;
assert_eq!(updates.get("events"), Some(&Events(vec![1, 2])));
assert_eq!(updates.merge(&Map::new())?, updates);
Ok::<(), eidetica::Error>(())
}

Construction and iteration require no key or value traits. Lookup and literal insertion require ordered keys; delta insertion additionally requires V: CRDT, without cloning. Map merge requires K: Ord + Clone and V: CRDT + Clone to return an owned result while preserving one-sided operations. No V: Default is required.

These primitives do not integrate with Table, add a live-value convenience layer, or define lifecycle resets. Table behavior and persisted bytes are unchanged.

Fork and Merge

The system supports branching and merging through parent-child relationships:

• Forking: Multiple entries can share parents, creating divergent branches
• Merging: Entries with multiple parents merge separate branches
• Deterministic Ordering: Entries sorted by height then ID for consistent results

Merge Algorithm

Uses a recursive merge-base approach for computing CRDT states:

• Cache Check: Avoids redundant computation through automatic caching
• Merge Base Computation: Finds the common dominator for multi-parent entries (the lowest ancestor through which ALL paths must pass)
• Recursive Building: Computes ancestor states recursively
• Path Merging: Merges all entries from merge base to parents with proper ordering
• Local Integration: Applies current entry’s data to final state

Doc Merge Semantics

The Doc type supports two merge modes controlled by an atomic flag:

• Atomic merge (other.atomic == true): The incoming Doc replaces the existing one entirely (LWW). The result is a clone of other, including its atomic flag.

• Structural merge (default): Fields are merged recursively. For each key present in both sides, values are merged per-field using last-writer-wins. Keys unique to either side are preserved in the result. The result is non-atomic unless self was atomic (see below).

Contagious Atomic Flag

When self.atomic == true and other.atomic == false, a structural merge occurs (other’s fields are merged into self’s fields), but the result preserves the atomic flag from self. This “contagious” property ensures associativity:

Given entries E1, E2, E3(atomic), E4:

Left fold:   ((E1 ⊕ E2) ⊕ E3) ⊕ E4
Grouped:     (E1 ⊕ E2) ⊕ (E3 ⊕ E4)

E3 ⊕ E4 produces an atomic result (contagious from E3).
When merged with (E1 ⊕ E2), the atomic flag triggers LWW,
correctly overwriting all pre-E3 data.
Both forms produce identical results.

When to Use Atomic Docs

Atomic Docs are appropriate for configuration or typed data that should be treated as a complete unit — replacing the previous value entirely rather than merging individual fields. Store implementations use Doc::atomic() when writing config values (e.g., PasswordStoreConfig) to ensure the full config is replaced on each write.

Key Properties

• Correctness: Consistent state computation regardless of access patterns
• Performance: Caching eliminates redundant work
• Deterministic: Maintains ordering through proper merge base computation
• Immutable Caching: Entry immutability ensures cache validity

Store state

Stores define how their CRDT state is represented for reading. The default StoreStateModel caches state in one opaque record: it folds ordered Store deltas with the Store’s CRDT implementation and stores the serialized result under a reserved key. This default applies to any Store data type and does not assume Doc.

Backends persist each Store state as an opaque byte-keyed record set. Keys use unsigned lexicographic byte order, point reads address one key, and scans use half-open ranges with an exclusive continuation key. The backend does not parse record keys or values and does not know Store types.

Table uses the cached record format named eidetica/table/rows version 0. UTF-8 primary keys are record keys and each JSON row is one record value. A cold historical build fetches canonical Doc Entry history and accumulates its projection in a whole row map before staging it. Encrypted Tables also build a physical record map. Warm reads are point/page operations; cold construction is not incremental or bounded-memory streaming. Table mutations are converted back into the existing canonical Doc delta at historical commit, so Entry payload and wire semantics are unchanged.

PasswordStore preserves the wrapped Store’s state model but namespaces its descriptor. For a Table, the cached record key becomes a stable keyed hash of the logical key and the value becomes an authenticated encrypted envelope; scans and cursors use physical-key order. For DocStore, YDoc, and other opaque models, the complete serialized state remains one encrypted record at the reserved opaque key. See Encryption for the exact formats and trust boundary.

Table handles retain only their name and transaction. get deserializes one row, scan_page reads bounded deterministic pages, and search collects those pages only because its public return type is a Vec. Local and service-backed Tables use the same point and page operations. Custom backends without cached-record support, or declining optional retention, fold captured history. This fallback still materializes historical state; it is not a streaming-history API.

A record set has one lifecycle:

• Derived record sets materialize historical Entries at fixed Store tips, are immutable after publication, and can be cleared and rebuilt.
• Authoritative record sets contain durable current state and are not eligible for cache clearing.
• Staging builds are unpublished private state and are invisible to record readers.

Clearing derived state unlinks published record sets and reclaims the generation unlinked by the previous clear. An active reader keeps its view, while a new lookup misses and rebuilds. Authoritative record sets are never selected.

A builder creates private state, writes record chunks, then publishes atomically. A confirmed failed publication does not expose a partial record set. An uncertain commit remains an error and must not be retried as assumed nonpublication. Abandoned private builds are reclaimed on exclusive startup. Two builders can derive the same target concurrently. Publication resolves that race to one shared record set rather than failing the other builder. Format descriptors identify the Store-owned record format and version, so cached state from different formats or historical sources cannot collide.

Remote record operations use opaque server-issued views and staging tokens. Pages have exclusive continuation keys and encoded-byte bounds; one record that cannot fit fails with RecordTooLarge.

Historical Store-state cache

Historical current-state reads use derived record sets in the Store-state cache. Immutable Store tips identify the source, and the format descriptor identifies the representation.

The default opaque builder folds Store deltas in canonical historical order, serializes the Store’s CRDT type, and publishes one record at the reserved opaque key. A cold read builds and publishes the record set; a warm read resolves the published record set and reads that record. The runtime is generic over the Store CRDT type and does not assume Doc.

Cached-state construction writes into a private build first. A failed fold, serialization, private write, or publish leaves no partial published record set. Published derived record sets are immutable.

Clearing selects only the Derived lifecycle, so authoritative records remain byte-for-byte unchanged. Clearing is two-phase: it unlinks every published derived record set and reclaims the generation unlinked by the previous clear. An unlinked record set is no longer resolvable, so the next read rebuilds from immutable Entries, while a reader that resolved its view before the clear keeps reading the generation it is walking until the following clear reclaims it.

The legacy SQL crdt_cache_v2 table and the in-memory LRU are removed. Local and connected reads use Store-state records.

Historical Table state uses the eidetica/table/rows format, with one record per logical row keyed by its UTF-8 primary key. Opening a Table handle reads no rows. Point reads fetch one record, and ordered iteration uses bounded pages with exclusive continuation while transaction-local changes overlay the published record set.

PasswordStore<Table<T>> uses the namespaced eidetica/password/eidetica/table/rows descriptor. Its derived generations contain keyed 32-byte physical keys and authenticated encrypted row envelopes; ordering and exclusive cursors use those physical keys. The history fallback projects, overlays, and pages in the same order. Other wrapped Store types keep the encrypted opaque whole-state representation. See Encryption.

Connected instances use the same cached-state path over the service record protocol. The daemon binds each request to the authenticated session. It narrows a shared-scope request to the session user, refuses a foreign scope, and falls back to shared cached state on a user-scope miss. Encrypted materializations are client-computed and user-scoped; a warm encrypted Table point read uses point-record requests without scanning the generation or reconstructing history.

Retention and verification boundaries

Derived materializations use domain-separated BLAKE3-256 digests of complete canonical Snapshots, within separate database, Store, lifecycle, projection and trust-scope namespaces. Frontier width does not enlarge the SQL unique key. Old full-frontier keys are disposable cold rows, not relabelled digests. Verification status is not a key: the same immutable Snapshot has the same materialization before or after a local status transition. A cached value is never an authorization or verification proof.

The defaults are 1024 live derived namespaces, with 268435456 bytes (256 MiB) of live key/value payload for InMemory and in-memory SQLite, or 4294967296 bytes (4 GiB) for persistent SQL. Constructors accept a DerivedCachePolicy; CLI backend options are --cache-bytes and --cache-namespaces (EIDETICA_CACHE_BYTES, EIDETICA_CACHE_NAMESPACES). The owning daemon selects its policy before opening storage; clients cannot raise it. A zero bound disables retention. A candidate larger than the byte budget is not published and does not evict useful state. All derived state, including the current frontier, is evictable.

SQL publication, sizing, admission and eviction commit in one transaction. A successful commit has no later fallible maintenance step. A confirmed admission rollback can report StoreStateNotRetained; an uncertain commit or rollback remains an error, never a cache miss. Same-target builders adopt a winner only after validating identity under the publication locks.

Live access ticks are monotonic (saturating at the SQL signed-integer maximum). SQL flushes after 64 distinct dirty namespaces, or inside admission, in chunks of at most 256. Batches copy pending ticks and acknowledge only committed ticks; failed/cancelled batches and newer hits survive. Eviction compares the maximum of live and durable ticks, including in-flight batches. Optional recency-flush failure does not fail a successful data read. Authoritative reads do not enter derived recency. Crashes may lose recent unflushed order, not cached values.

Evicted namespaces stop resolving immediately but remain readable through existing views until the next eviction pass or clear. Clear-unlinked namespaces instead remain readable until the following clear. Exclusive SQL startup reclaims obsolete retired generations and abandoned builds, seeds the clock, and enforces a lowered policy before exposing the backend. An inability to establish the cap visibly refuses open rather than exposing an over-budget cache.

Row, opaque and authorization-floor callers fall back to captured history only for explicit optional-cache refusal, unsupported storage or invalidated views. After one stale-view retry, a row read uses history; a partial page is discarded and restarted from its original cursor. Opaque results already computed are returned without retention, and disabled admission skips cache encoding/upload. Authentication, proof, decryption, corruption and history/read errors still fail. No cache GC removes authoritative Entries or records.

These are logical live-payload limits, not RSS, physical disk or file-high-water quotas. Metadata, staging, two separately retained reader generations, database workspace and authoritative history have separate costs. This implementation still folds whole history and may build whole logical/physical row maps on a cold read or bypass. It is not incremental materialization, bounded-history streaming, or a whole-SDK memory bound. Existing service frame and single-record limits remain; old SDKs can surface quota refusal as an error rather than transparently bypassing it. No deployment/support-version promise follows from these local storage changes.

Authentication

Ed25519 signatures and causal settings determine whether an entry is authorized. The authentication design is the canonical explanation of delegated floors, trust boundaries and examples. The verification model owns status and visibility; the CLI guide owns the offline reset and dependency-first recovery procedure.

Authentication States

Normal transaction APIs require a signing key. A non-genesis write cannot use newly staged settings to authorize itself. Transactions prevent auth corruption before storage; remote verification also rejects corrupt historical auth.

Permission and Key Resolution

Direct keys live in _settings.auth.keys, indexed by public key; a name is a hint, not an identity. The wildcard supplies global permission for an actual signer. Delegations live in _settings.auth.delegations, indexed by database root ID. At each delegation step permission is clamped to its configured minimum/maximum; the final hint must resolve a concrete key, not the wildcard. Priority authorizes administrative actions; it does not change Doc merge ordering.

Historical signatures versus live access

Entry validation resolves delegation at the signed snapshots, subject to the configured and inherited causal floors. Later membership removal or downgrade does not by itself change that historical signature predicate.

Live service access, Database::current_permission, and sync’s Database::can_access instead use current accepted authority at every path step. The target’s current accepted declarations and bounds apply, even on a diagnostic allow_unverified handle or while verification uses raw history internally. Claimed tips remain bounded hints; they cannot select older authority or trigger identity fetching. The resolver reads the retained Verified frontier and proves its complete ancestry before folding settings at that immutable boundary. It does not recursively verify delegated databases while the caller holds a verification lock.

A public wildcard Read grant exposes identity metadata, not delegated membership. An active Read member can still be promoted by PermissionBounds.min. Independent direct target grants remain valid; sync’s pubkey-only discovery is still single-hop, while a supplied identity may name a bounded nested path.

This is observer-local authorization. A peer that has not accepted a removal can still use older state. There is no globally latest-state guarantee, wall-clock revocation bound during partitions, or new concurrent branch-validity policy. The live_delegation tests cover retained local and service identities, sync disclosure, reductions, bounds, path tampering and unaccepted identity state.

Validation Seams

• database/mod.rs: derive and match the signed causal pre-write settings pin; check main parents before remote promotion; retain undecidable entries.
• transaction/mod.rs: fixed main-parent settings/subtree reads and metadata; genuine-genesis-only staged-auth bootstrap; shared signature validator.
• auth/validation/floors.rs: join Verified parent projections, derive effective resulting settings, preserve separate direct/nested root observations, prove complete delegated ancestry, and rebuild cold derived state.
• auth/validation/delegation.rs: declaration lookup at historical snapshots, coverage of configured/inherited floors, bounds and key resolution.
• service/server.rs: store submitted entries Unverified and verify server-side; clients cannot assert a verification label. Historical signature queries also run the native validator server-side, gated by the handle’s acting identity and Read permission on the target and claimed delegated trees.
• backend/database/{in_memory/storage,sql/verified}.rs: atomic same-tree parent guards, status/Verified-tip publication, cascading demotion and conservative ancestry repair. These do not rerun signature or delegation validation; see the backend contract.
• backend/database/{in_memory,sql}/traversal.rs: anchored store boundaries; SQL reads both current tip sets in one statement, InMemory uses its lock.

A projection can be published after authentication succeeds but before the Entry is stored/promoted. Consumers must still check the parent’s local status before using it. Unsupported derived-state storage rebuilds from Entries; cache I/O failure is not an empty state. No recursive cross-tree verification belongs in these seams while a per-tree verification lock is held.

Invariant-to-Regression Map

Names below are runnable test filters, not a test-count target. floor_tests refers to crates/lib/src/auth/validation/floor_tests.rs; validation::tests to its sibling tests.rs. Integration paths are relative to crates/lib/tests/it/.

Floor unit tests use InMemory. Backend conformance tests and the delegated reset/rebuild receiver run on the local InMemory/SQLite/Postgres matrix; the latter uses InMemory when the service matrix is selected because verification is server-local. service.rs separately exercises the real Unix-socket boundary. Do not interpret a service matrix count as every raw backend test running over RPC.

For iteration, select these filters with cargo nextest run --all-features -E '...' inside the Nix dev shell. The completion gate is nix develop -c nix run .#fix followed by nix develop -c just nix full, including all backend matrices and integration VMs. A green suite is evidence for these invariants, not a proof of arbitrary security properties or automatic dependency acquisition.

Local credential and signer ownership

Login records, private keys, signer mappings, access timestamps and activation preferences use native installation-local records. Fresh Account histories contain portable references and public Auth, not key or unlock records. Existing histories are not rewritten or scrubbed. All convenience and caller-held credentials use the same authenticated CredentialBundle format. Full AccountSession values can also be constructed from caller-unlocked credentials or an explicit signer/Account locator without daemon login storage.

Storage boundaries

BackendImpl::local_account_record returns the exact stored bytes. compare_exchange_local_account_record(name, expected, replacement) inserts only when absent (expected = None) or replaces only the exact previously observed bytes. A conflicting write returns false; callers must reread/reconcile rather than overwrite a concurrent update. Keep the original observed bytes, not a newly serialized approximation of the expected value. This is installation-local atomicity, not distributed compare-and-set or a second request engine. There is no deletion or live credential migration in this substrate. Custom backends that do not implement it fail with LocalAccountStorageUnsupported, never fall back to a replicated store.

InMemory JSON revision 3 persists the map; it reads supported older revisions without inventing local records. SQL revision 4 adds an empty native table when opening supported earlier formats, retaining existing Entries, metadata/secrets, verification and Store-state data. Opening v3 does not redo the v2 cache backfill. Missing credential storage under a v4 marker is corruption and refuses open rather than silently recreating an empty directory. Older binaries refuse the new format counters instead of silently dropping credentials. These are tested synthetic storage transitions, not permission to open/migrate live installations. A binary rollback does not roll back data; see the format matrix.

Convenience-login records

login:<name> contains the installation login UUID and AccountInfo: local label, Account ID, root credential, creation time and status. Names are inserted atomically when absent; a duplicate registration is refused. account-state:<UUID> contains credentials (the complete current bundle), keys (public-key-indexed metadata), databases (ID-indexed TrackedDatabase values) and last_login. A key metadata record includes its public ID, display label and per-database SigKey hints, never a private key or per-key ciphertext. The directory’s initial bundle supplies the login root proof; after authentication, the scoped local-state read supplies the current complete bundle. Its authenticated Account locator must match the directory hint, and the login key must be present. Both bundles use the same wrapping key/salt and one password; they are not two different wrapping schemes. References remain ordinary Account Entries, never inputs to key import or activation. A fresh installation does not learn another installation’s keys or activation choices by replicating AccountDB.

All local edits reread and compare/exchange the observed native bytes. Mapping and preference edits are one atomic replacement, so concurrent sessions retain unrelated changes. After a lost acknowledgement, mapping caches are refreshed from persisted local state before the original error is reported. This is not a transaction with peer approval, database creation or login registration; those earlier effects can survive later bookkeeping failure.

For all convenience logins, KeyStorage::Encrypted contains a CredentialBundle as described below. Adding a key unlocks the observed complete bundle with the session wrapping key, explicitly adds the generated key and reseals with a fresh nonce in the same native compare/exchange as the metadata update. A stale session therefore preserves independent key additions rather than writing its old list over them. A changed salt/password, tampered ciphertext or foreign Account locator fails closed. There is no separate password hash. Password is a required string, with no length/complexity restrictions or insecure-use acknowledgement. NewAccount::passwordless(name) delegates to with_password(name, ""); there is no plaintext credential variant or optional-password login path. Empty and nonempty passwords use the same authenticated envelope and KDF parameters. Empty-string wrapping provides no meaningful confidentiality against a bundle holder, who already knows the password. It does not protect against a malicious bundle holder resealing altered data. No old plaintext credential record or historical key is imported or converted.

_users remains the public Auth boundary for instance-admin capabilities, not a credential directory. _sync retains the daemon’s installation-local bootstrap request lifecycle and derived combined settings; it is not the shared Account inventory. Sync reads native login/activation records at startup, and supported local/RPC preference writes request recomputation. The stored preference change token hashes only sorted activation data, not private keys or encrypted key bytes. Failure to recompute can follow successful local persistence and remains an error; retry explicitly.

Embedded convenience path:

native login anchor + client password -> authenticated root-bundle unlock
native complete bundle + metadata -> client AccountKeyManager -> immutable DatabaseKey
native activation preferences -> this instance's sync coordinator
AccountDB refs/Auth <-> ordinary replicated Entries (no keys/unlock/preferences)

Connected convenience path:

trusted client                           daemon
local label ---------------------------> native login record -> wrapped root credential
password -> unlock -> challenge proof --> public session keyset + exact login UUID
read/CAS complete encrypted bundle -----> only that login's native record
client wrapping key -> reseal update --> atomic bundle + public-metadata replacement
private signer -> signed Entry --------> normal per-tree/proof verification

LocalAccountState and ExchangeLocalAccountState require the authenticated login UUID to match exactly. Proof-only signer connections have no login-record scope. RegisterLocalAccount and ListLocalAccounts require instance-admin on the daemon’s _users Auth. No operation exposes arbitrary native record names or general backend secrets. The unchanged frame-size limit bounds each request/response; oversize state is refused rather than split into partially applied writes. Updated clients require matching support for these operations; older peers may return Unsupported, with no replicated-store fallback.

Bundle format and key lifetimes

The v1 encrypted bundle’s logical JSON payload has only account_id and an explicit ordered, nonempty, unique list of private keys. The external storage envelope contains version, salt, nonce and ciphertext. Version 1 fixes AES-256-GCM, a random 96-bit nonce per sealing operation, and the existing Account Argon2id version 0x13 parameters: 19456 KiB, two iterations, parallelism one, 32-byte output. It retains the Account crypto helper’s base64-text salt convention: the generated 22-character salt text, not the decoded 16 bytes, enters Argon2. The fixed format accepts no caller-supplied work-factor parameters. PasswordStore’s KDF envelopes/upgrades remain independent; this API does not migrate them or claim their upgrade policy.

The complete locator and key list are inside the authenticated ciphertext. The AEAD associated data is the fixed domain/version eidetica:local-credential-bundle:v1. unlock_for additionally compares the authenticated locator to the expected Account ID, so copying a directory hint or encrypted blob cannot rebind it to another Account. Unknown versions/fields, malformed salts/nonces, wrong passwords, tampering and invalid key lists fail closed. Decrypted parser errors use fixed text instead of echoing keys.

Serialization measures the payload before writing into a guarded, preallocated plaintext buffer. The Account key manager calls its existing zeroizing cleanup on drop, including constructor errors after allocation. The completed bundle plaintext buffer and wrapping key use zeroizing owners on success/error/unwind; decrypted JSON is guarded immediately after decryption and released after parsing. The borrowed password remains caller-owned and is not retained. UnlockedCredentials retains private keys and the wrapping key for explicit resealing, not the password. Its signer method returns an independently owned private-key clone. There is no plaintext Debug, Clone or serialization interface on UnlockedCredentials.

Dropping an unlock/session does not destroy independent clones, invalidate existing database handles or revoke Auth. Connected keyed handles retain only their own signer and renew its possession proof after a successful re-login replaces the connection’s public keyset. Login handshakes, key registration and database requests are serialized on the connection so a retained handle cannot race a half-completed authentication exchange. Signing keys use their existing zeroizing owner, but parser/serde/allocator/crypto internal copies, caller-owned passwords, swap, core dumps and a compromised process are outside this cleanup claim. Password rewrap reseals the same keys; older bundles/backups remain usable under their old password. Recovery/export, live revocation and background-device authority are separate work.

Embedded execution

caller-owned keys + Account ID -> CredentialBundle::seal -> encrypted local bytes
                                        |
client password + encrypted bytes -> unlock -> AccountSigner (actual key + SigKey)
                                        |                |
                  AccountSession::from_credentials / from_signer
                                        |
                  session-only key hints and activation (no daemon records)
                                        |
                  open_database -> normal Auth/verified proof -> signed Entry

A direct-key handle needs no Identity. A delegated handle carries the exact supplied path, not a reference-derived guess or an automatic direct-key fallback. A newly selected handle is a new value; old handles are never rebound. A missing public-key hint is bound to the actual signer so name-based service reads cannot fall back to the connection login key; a mismatched supplied public-key hint fails open-time validation. Identity code can supply an exact DatabaseKey/SigKey through this seam without storing local key selection or activation in portable metadata. The Identity membership request lifecycle is not implemented here.

Service execution

client: encrypted bytes -> unlock -> private signer (remains in client)
                                      |
                           SignerChallenge { pubkey }
                                      v
                     daemon: fresh proof challenge only
                                      |
                         TrustedLoginProve { signature }
                                      v
                     session public-key set + cache scope
                                      |
               per-tree Auth gate / signed-entry verification -> storage

AccountSigner::connect creates a fresh connection, proves possession and receives no login credentials. The proof-only path uses the existing challenge verifier, single-use challenges and per-tree authorization; it creates no daemon login record. Its generated signer:<UUID> cache scope is distinct from registered-login UUIDs and is not a credential-storage identity. A failed proof consumes its challenge and does not authenticate. Older v0 peers may return Unsupported or disconnect for the new optional operation; there is no credential-login fallback. Additional actual keys still require existing session-key possession proofs. Reauthentication replaces the client’s proof cache to match the daemon’s reset keyset.

Connected delegated transactions sign on the client, then rely on the daemon’s local proof resolver. Submit stores untrusted input for verification, but a successful commit acknowledgment requires that exact Entry to become Verified. An incomplete proof or rejection is an error, not rollback: stored unverified content can remain for later verification. This reuses the retained Identity validation seam rather than adding an Identity-specific authorization protocol.

The socket remains a filesystem-trusted local API. This does not protect login records from hostile socket callers or shared-file processes. The optional dashboard still unlocks signing keys in daemon memory for its existing web sessions; extracting it into a socket client is not part of this implementation. No stronger live revocation, private-key recovery or isolation guarantee follows from this split.

Regression map

• credential_bundle_tests: explicit scope, wrong/missing signer, cross-Account locator mismatch, password/tamper/format refusal, rewrap and retained signers.

• signer_tests: actual direct and delegated signatures, retained/new handles, wrong-path no-fallback, unauthorized access and real Unix-socket proof-only use without bundle login.

• local_accounts: compare/exchange conflicts, racing inserts, separate records, persistence/reopen and absence from Entry history/tree enumeration; cache clearing/trust reset preserve records.

• SQL schema tests: synthetic v3→v4 retention without backfill and fail-closed missing v4 storage.

• Service signer proof and signed-submit tests: wrong proof, consumed challenges, valid fresh proof and propagation of actual verification refusal.

• local_state_tests: ordinary protected login/key addition/logout/re-login, fresh Account and system histories without key/unlock tables, and portable reference writes as a positive control.

• local_state::tests: distinct independently authorized keys on one shared Account, different installation activation choices, actual signatures and returning login, embedded and through two real service sockets. The fixture preloads verified public data and registration; it does not claim a shipped device-enrollment flow.

• Service native-state test: pre-auth/foreign-login reads and writes denied, non-admin registration/listing denied, own-state reads and compare/exchange conflicts exercised.

AccountSession::from_credentials consumes an unlocked bundle; from_signer accepts one actual signer with explicit Account locator and Auth identity. Caller-held bookkeeping contains public metadata only; actual keys stay in the in-memory key manager, not a plaintext storage variant. Both validate Account access, retain independent immutable acting handles and use a session-owned local-state map rather than native daemon records. Their local-login getters are empty and account_info() is None. They do not configure background sync, and a returned Database has no registered-login sharing preference binding. Added keys and mappings disappear at logout unless the caller explicitly persists the keys in its own store; the original encrypted bundle is unchanged. credential_bundle_tests exercises the complete ordinary bundle, stale-session key additions, returning login and foreign/tampered active bundles. signer_tests also exercises full caller-held sessions, scoped app inputs without personal keys, retained direct/delegated Account handles and exact absence of daemon credential-record writes on actual Unix services.

Encryption

This chapter describes PasswordStore as implemented. It covers content encryption and encrypted derived Store-state caches. Account signing-key encryption is a separate use of Argon2id and AES-256-GCM in account/crypto.rs; its salt handling, envelope, and lifetime are not the PasswordStore format described here. The explicit local bundle format and its memory owners are described in local credential ownership.

The main implementation is in crates/lib/src/store/password_store.rs. Transaction integration is in crates/lib/src/transaction/mod.rs, and the Store-state substrate is in crates/lib/src/store/state.rs plus crates/lib/src/backend/.

Boundaries and ownership

PasswordStore<S> owns the public encryption configuration, encrypted wrapped-Store metadata, and the locked/unlocked state. Opening or initializing it creates a PasswordEncryptor and registers that encryptor with the current Transaction under the Store name.

Transaction owns the data-path boundary:

• it keeps staged Store deltas as plaintext;
• it decrypts historical Entry payloads before CRDT deserialization and merge;
• it encrypts a Store delta immediately before the Entry is built and persisted;
• for record projections, it maps logical keys to physical keys and encrypts/decrypts row values;
• it builds derived state from the immutable Entry history on cache misses.

The backend receives opaque Entry payload bytes and opaque Store-state record keys and values. It compares and orders record keys as bytes and does not interpret the encrypted envelope. In service mode the client performs password derivation, Entry decryption, encrypted-state materialization, and row verification. The daemon stores Entries and user-scoped encrypted cache generations but does not receive the Store password or plaintext content.

Plaintext therefore exists in the client process: wrapped Store configuration after open, staged deltas, decrypted historical deltas, merged CRDT state, logical row keys and values, and values returned to application code. “Encrypted at rest” does not mean plaintext never exists in memory.

Password derivation

Initialization generates a SaltString with SaltString::generate. The value stored in _index is its unpadded base64 text; for the generated v0 format it represents 16 random salt bytes.

The master key is 32 bytes produced by Argon2id version 0x13 with parameters stored in the config:

A compatibility detail is important: the implementation parses the stored text with SaltString::from_b64, then passes salt.as_str().as_bytes() to hash_password_into. The Argon2 salt input is therefore the bytes of the base64 salt string, not the decoded 16 random bytes. Changing that would derive a different key and make existing Stores unreadable.

Older configuration without explicit cost fields uses the same defaults. open rejects unsupported algorithm and KDF names, invalid salts and parameters, wrong-size config nonces, failed config authentication, malformed decrypted metadata, and a wrapped Store type that does not match S. Errors for the last two use fixed text: a malformed-metadata DeserializationFailed carries no parser error, and a TypeMismatch does not echo the decrypted wrapped type.

PasswordEncryptor lazily derives the master key on first data operation and retains it behind the transaction’s encryptor. Cleanup ownership is described in Memory ownership.

Memory ownership

PasswordStore clears specific owned allocations when their owner drops, including on early error returns, future cancellation, and panic unwinding. This is best-effort cleanup of those buffers, not a guarantee that plaintext is erased from process memory.

The private byte owner implements no Debug, Display, Clone, or serde trait. Decrypted wrapped metadata is parsed only to validate it, then dropped; PasswordStore does not retain it.

These are not covered:

• arguments the state check rejects before conversion, and copies the caller retains;
• plaintext returned to the application, including accepted row fields and decrypted Entry payloads, and the plaintext held by Transaction (staged deltas, merged CRDT state, and history projections; see Boundaries and ownership);
• parsed Doc/String values, such as the validated wrapped metadata, whose drop does not wipe;
• serde parser scratch, including a partially deserialized row field and its growth copies;
• Argon2 block memory, AES key schedules, and AES-GCM’s internal copies;
• allocator history, moved stack or register temporaries, swap, hibernation, core dumps, a compromised process, and process exit without destructors.

The registered encryptor, with its password and master key, lives as long as the transaction’s encryptor map. Clones of a Transaction share that map and wrapped Store handles retain clones, so dropping one PasswordStore neither locks the Store nor removes the encryptor; cleanup happens when the last clone drops or the registration is replaced.

Four persistent representations

PasswordStore uses related cryptographic primitives but different byte formats for configuration, Entries, opaque cached state, and projected row caches.

_index configuration

The _index registry entry has type encrypted:password:v0. Its configuration is an atomic Doc with this logical shape:

{
  encryption: {
    algorithm: "aes-256-gcm",
    kdf: "argon2id",
    salt: <base64 text>,
    version: "v0",
    argon2_m_cost: 19456,
    argon2_t_cost: 2,
    argon2_p_cost: 1
  },
  wrapped_config: {
    ciphertext: <base64 text>,
    nonce: <base64 text>
  }
}

Algorithm, KDF, salt, version, and costs are public metadata. wrapped_config encrypts a JSON serialization of WrappedStoreInfo { type, config } with the master AES-256-GCM key and a random 12-byte nonce. The nonce and ciphertext are separate fields and are base64-encoded only because they live inside a Doc. This encryption call supplies no additional authenticated data.

Entry payloads

At commit, the wrapped Store’s serialized delta is encrypted with the master AES-256-GCM key and a fresh random 12-byte nonce:

nonce[12] || AES-GCM ciphertext-and-tag

These are raw subtree payload bytes inside Entry; they are not JSON or base64 wrappers. The plaintext format belongs to the wrapped Store (for example, JSON for Doc-backed Stores or a Yrs binary update). This encryption call also supplies no additional authenticated data. Entry identity, signatures, parents, Store name, and database membership are protected by the Entry/DAG authentication layers, not by row-envelope AAD.

Opaque derived cached state uses the same Encryptor::encrypt format. The merged CRDT state is serialized to JSON, encrypted as nonce || ciphertext-and-tag, and stored at reserved key 0x00. DocStore, YDoc, and any other Store that keeps the default opaque model use this path. For a binary CRDT such as YDoc, this cache plaintext is the CRDT state’s serde JSON representation; it is not necessarily the same bytes as an individual Entry delta.

Projected Table row cache

Table declares projection eidetica/table/rows, version 0. PasswordStore<Table<T>> namespaces that descriptor as eidetica/password/eidetica/table/rows, version 0. The encrypted projection is built after decrypting Entry deltas and applying the Table projection.

Two 32-byte subkeys are derived from the master with BLAKE3’s derive-key mode:

record-key subkey   = derive_key("eidetica/password-store/record-key/v1", master)
record-value subkey = derive_key("eidetica/password-store/record-value/v1", master)

The physical key is BLAKE3 keyed-hash output:

physical_key = keyed_hash(record-key subkey, normalized logical key)  // 32 bytes

The value plaintext is a serde JSON byte serialization of an envelope containing byte arrays:

EncryptedRecordEnvelope { key: logical_key_bytes, value: row_json_bytes }

Because key and value are Rust byte slices, serde JSON represents each as an array of integer bytes. The outer persisted value is not JSON:

nonce[12] || AES-GCM(record-value subkey, envelope_json, AAD)

The AAD is the exact concatenation:

"eidetica/password-store/record-envelope/v1\0"
|| database_id_text_bytes || 0x00 || store_name_utf8
|| 0x00 || physical_key

database_id_text_bytes comes from database_id.to_string(). The Store identity separator and AAD separator make the database/Store/key placement unambiguous for the current format.

On decryption, AES-GCM authenticates the envelope against that AAD. The client then deserializes the envelope and recomputes the physical key from its logical key. Authentication failure is reported as StoreError::DataCorruption("record authentication failed"); a successfully decrypted envelope whose logical key hashes elsewhere is DataCorruption("record envelope key does not match its physical key"). Too-short values and malformed JSON fail as deserialization errors; the malformed-envelope error has fixed text and does not carry the parser error, which can quote decrypted content.

The tests maintainers should start with are:

• unit tests at the end of crates/lib/src/store/password_store.rs for domain separation, nonce uniqueness, Store/physical-key binding, malformed values, and modification;
• crates/lib/src/store/password_store/hardening_tests.rs for buffer cleanup on success, error, cancellation and unwinding, redacted decrypted diagnostics, and fixed persistent-format fixtures;
• crates/lib/tests/it/store/password_store.rs for encrypted Entries, opaque caches, Table row records, physical ordering, staged updates/deletes, and cache clearing;
• crates/lib/tests/it/transaction/record_fallback.rs for cache/history page and cursor equivalence;
• crates/lib/tests/it/store/ydoc_operations.rs for the opaque encrypted YDoc path;
• crates/lib/tests/it/service.rs for warm encrypted Table point reads over the service.

Row semantics and leakage

For an unencrypted Table, a cache record key is the normalized UTF-8 logical key and its value is the JSON row bytes. For an encrypted Table, neither is stored in plaintext. The physical key is stable for the same master key and normalized logical key, so equality and access patterns remain visible. The physical key also defines scan and cursor order; it intentionally does not preserve logical-key sorting or range locality.

Within a transaction, logical mutations are the source for conflict/shadow rules and the physical mutation map is the encrypted overlay. get checks staged logical state first and otherwise fetches the one keyed record. Scans merge staged encrypted records with cached records in physical-key order, decrypt returned records, and expose logical rows with an exclusive physical cursor. The history fallback projects the full plaintext state, applies logical overlays, maps every remaining row to its physical key, then filters and pages in the same physical order.

A same-key update replaces that key in the transaction’s logical projection and in the next materialized generation. It does not overwrite a record in an already published generation, and it does not remove old encrypted Entry payloads or old historical state. Deletion is likewise a CRDT delta/tombstone, not secure erasure of history.

Generation lifecycle

A StoreStateRequest names a generation target by database, Store, lifecycle, scope, projection descriptor, and source key. Historical state uses lifecycle Derived; its source key is the fixed Store tips (a single Entry ID or the deterministic merge-tip cache ID).

A miss follows this sequence:

1. Begin a private Staging namespace.
2. Reconstruct and project verified Entry history in the client for an encrypted Store.
3. Write encrypted records into that private namespace.
4. Publish atomically as Derived.
5. If another builder already published the same target, adopt the winner and discard the private loser.

Published record sets are immutable. A transaction may keep an immutable RecordView, but it never patches the published records during set, update, or delete. New Entry tips form a different source key and therefore resolve or build a different generation.

Clearing derived state unlinks every published derived generation and reclaims the generation unlinked by the previous clear. A reader that already resolved a view may finish against the unlinked generation until the next clear; a new resolve misses and rebuilds from Entries. A view that becomes invalid during a read is resolved once more before history reconstruction. Unsupported custom backends fall back directly to history. Other storage or authentication failures propagate; they are not treated as benign cache misses.

Over the service, encrypted materializations are client-computed and rebound to CacheScope::AccountSession(session_user). They cannot be published into shared scope or another user’s scope. Warm reads use record get/scan operations over opaque views; a warm point read need not scan the row set or fetch Store history.

Security properties and non-properties

PasswordStore provides confidentiality and ciphertext integrity under the password-derived key. For projected rows, AAD and the inner logical-key check bind a valid envelope to the database, Store, and physical key where the client expects it.

Those properties do not make a derived cache authoritative:

• A valid row proves that its envelope was produced under the key for that placement. It does not prove that the generation is the newest one.
• Authentication of individual rows does not prove that the backend returned every row or the correct page boundary.
• A client-computed encrypted cache cannot be semantically verified by the daemon because the daemon lacks plaintext and the password.
• Cache deletion, omission, or corruption can cause an error or force reconstruction; the signed, verified Entry DAG is the recoverable source of truth.
• PasswordStore does not conceal Entry/DAG topology, ciphertext lengths, operation timing, record counts, page sizes, or stable-key equality/access patterns.
• PasswordStore does not provide rollback detection, secure deletion of immutable history, password recovery, or password rotation.

Peer sync is unchanged: peers store and exchange immutable Entries and their authentication data. Derived Store-state records are local/service cache data, not Entry or sync protocol data.

Sync

Eidetica uses a Merkle-CRDT based sync protocol. Peers exchange tips (current DAG heads) and send only the entries the other is missing.

Sync Flow

sequenceDiagram
    participant A as Peer A
    participant B as Peer B

    A->>B: SyncTreeRequest (my tips)
    B->>A: Response (entries you're missing, my tips)
    A->>B: SendEntries (entries you're missing)

1. Peer A sends its current tips for a Tree
2. Peer B compares DAGs, returns entries A is missing plus B’s tips
3. A sends entries B is missing based on the tip comparison

This is stateless and self-correcting - no tracking of previously synced entries.

These tips describe raw DAG possession, not Verified membership. New received Entries are stored Unverified; neither peer authentication nor transfer completion asserts local acceptance. Default Database reads use the retained Verified frontier, which can lag raw tips until local validation succeeds. Write-event cursors also remain raw boundaries. See the canonical verification contract for promotion, partial-history reads and retry limits.

Bootstrap vs Incremental

The same protocol handles both cases:

• Empty tips (new database): Peer sends complete Tree from root
• Has tips (existing database): Peer sends only missing entries

Transport Options

• HTTP: REST API for server-based sync
• Iroh P2P: QUIC-based with NAT traversal for peer-to-peer sync

Both transports implement the same SyncTransport trait. Multiple transports can be enabled simultaneously.

Architecture

graph LR
    App[Application] --> Sync[Sync Module]
    Sync --> BG[Background Thread]
    BG --> TM[TransportManager]
    TM --> HTTP[HTTP Transport]
    TM --> Iroh[Iroh Transport]

The Sync module queues operations for a background thread, which uses a TransportManager to route requests to the appropriate transport based on address type. Failed sends are retried with exponential backoff.

Multi-Transport Support

The TransportManager enables simultaneous use of multiple transports:

• Address-based routing: Each transport declares which addresses it can handle via can_handle_address()
• First-match routing: Requests are routed to the first transport that can handle the address
• Independent servers: Each transport runs its own server on different ports/protocols
• Unified API: accept_connections() starts all registered transports, get_all_server_addresses_async() returns all addresses

Current Limitations

The sync system is currently simple and 1:1. Each peer connection requires manual setup with explicit peer addresses. Planned improvements include:

• Peer discovery
• Address sharing and relay coordination
• Multi-peer sync orchestration

See Bootstrap System for the key exchange flow when joining a database.

Service (Daemon) Architecture

The service module (crate::service) enables running Eidetica as a local daemon over a Unix domain socket. The RPC boundary sits at the storage-operation level: a RemoteConnection forwards every Database-level storage operation to the daemon. Higher-level code (Database, stores, Transaction, Instance) drives I/O through a single Backend trait whose remote implementation (RemoteBackend) wraps the connection, so the call sites are identical for local and connected instances.

Architecture Overview

graph LR
    C1[Client Process 1] -->|Unix Socket| S[Service Server]
    C2[Client Process 2] -->|Unix Socket| S
    D[Web dashboard] -->|web routes only| I[Instance]
    S --> I[Instance]
    I --> B[Backend: SQLite/InMemory]

The server wraps a full Instance (not just a backend) so it can handle both storage operations and write notifications. A client calls Instance::connect("unix://..."), which establishes a RemoteConnection, wraps it as a RemoteBackend, fetches InstanceMetadata over the wire, and constructs an Instance with no local secrets (secrets: None) — signing keys are derived client-side after login, never held by the constructed Instance until a user logs in.

Module Structure

Historical Store state is served through the shared Store-state record substrate rather than a service-layer cache module; every request is bound to the authenticated session (see Store state).

Wire Protocol

The protocol uses length-prefixed JSON frames over a Unix domain socket.

Frame Format

┌──────────────────┬──────────────────────┐
│ Length (4 bytes) │ JSON payload         │
│ big-endian u32   │ (up to 64 MiB)       │
└──────────────────┴──────────────────────┘

Each frame is a 4-byte big-endian length prefix followed by a JSON-serialized payload. Maximum frame size is 64 MiB (MAX_FRAME_SIZE); frames exceeding this are rejected on both read and write. write_frame/read_frame handle serialization and framing; read_frame returns None on clean EOF (before any length-prefix byte) and an UnexpectedEof I/O error when the stream ends mid-prefix or mid-payload.

PROTOCOL_VERSION remains 0, an unstable protocol. Version mismatches fail the handshake; the handshake and acknowledgment still advertise only the version, with no feature-capability negotiation.

Updated service peers tolerate additions at four named extension points:

• An unknown top-level request operation or nested DatabaseOp receives {"Response":"Unsupported"}. The server performs no operation and does not change login or session-key state. A database envelope must still have a valid root_id and identity.
• An explicit Unsupported or unknown variant inside Response consumes exactly one FIFO response slot. That caller receives an I/O error with kind Unsupported; later responses retain their association, including after an abandoned response wait. There is no retry or execution fallback.
• An unknown optional variant inside Notification is ignored without consuming a response slot or invoking callbacks. Known notifications retain their behavior.

These are externally tagged JSON enums: a unit variant is a string; a payload variant is a single-key object. Known payloads still use their original strict serde decoders. Malformed JSON or known payloads, unknown required nested types (including auth identities, read scopes, and DatabaseWrite.source), unknown or ambiguous outer server envelopes, and oversize/truncated frames retire the connection. No authority is inferred from an unknown operation. The existing length prefix and 64 MiB cap are unchanged.

This is not retroactive compatibility. Pre-change v0 binaries have fatal unknown-enum decoders. Common operations keep their named JSON encodings, but a new client sending a new operation to an unpatched v0 daemon may still disconnect. There is no silent downgrade, capability advertisement, or legacy execution fallback. Required wire-shape or semantic changes still need an explicit compatibility decision; #[non_exhaustive] covers Rust source compatibility only, not the wire.

Connection Lifecycle

sequenceDiagram
    participant C as Client
    participant S as Server

    C->>S: Handshake { protocol_version }
    S->>C: HandshakeAck { protocol_version }
    Note over C,S: Version mismatch → server closes connection

    Note over C,S: Connection state: PreAuth

    C->>S: TrustedLoginAccount { username }
    S->>C: TrustedLoginChallenge { challenge, login_id, account_info }
    Note over S: state → AwaitingProof
    C->>S: TrustedLoginProve { signature over challenge }
    S->>C: TrustedLoginOk
    Note over S: state → Authenticated { login_pubkey, session_keyset, login_id }

    opt Register additional per-DB keys
        C->>S: SessionKeyChallenge { pubkey }
        S->>C: SessionKeyChallenge { challenge }
        C->>S: SessionKeyRegister { pubkey, signature }
        S->>C: Ok
        Note over S: keyset insert pubkey
    end

    loop Authenticated request/response
        C->>S: ServiceRequest::AuthenticatedDb(AuthenticatedDbRequest)
        S->>C: ServiceResponse
    end

    Note over C: Client closes connection (EOF)

1. Handshake: client sends Handshake { protocol_version }; server validates and acks. On mismatch the server acks with its own version and closes the connection.
2. Authentication: trusted login (see Security Model below) retrieves a credential record and proves the root key. Alternatively, SignerChallenge { pubkey } returns only a fresh challenge; TrustedLoginProve completes proof of a caller-held signer without a login record. GetInstanceMetadata is also permitted before authentication. Both authentication paths enter the same public-key session state; neither bypasses per-tree Auth.
3. Optional session-key registration: once authenticated, the client may prove possession of additional pubkeys via SessionKeyChallenge/SessionKeyRegister. Each successfully proven pubkey joins the connection’s session_keyset and can then act as the identity on subsequent ops. See Session Keyset below.
4. Authenticated request loop: every storage operation travels inside ServiceRequest::AuthenticatedDb. The server handles requests in connection order. RemoteConnection serializes frame writes, not response waits; its background reader associates responses with pending callers in FIFO order and dispatches notifications separately.
5. Termination: client closes its write half (EOF); the server detects EOF and cleans up.

Cancellation and frame integrity

The pending response slot is enqueued under the writer lock before sending the frame, preserving request/response order even if a response arrives immediately. Cancellation or failure during the frame write retires the whole connection: the client drops its write half, marks the connection dead, and clears pending waiters. A subsequent request cannot append bytes to an incomplete JSON frame. Callers must reconnect rather than reuse that stream.

Once the frame is completely written, dropping only the response wait leaves its FIFO slot in place. The reader consumes that response even when its caller is gone, so the next response cannot be delivered to the wrong request. Other requests can continue using the connection. Neither case rolls back already-dispatched server work or proves that a remote operation did not commit; retry must reconcile persisted state rather than assume cancellation undid it.

Security Model

Client-side signing over the Unix service socket. All socket logins store and serve authenticated encrypted bundles, including empty-password logins, without sending passwords or decrypted signing keys to the daemon. Empty-string wrapping provides no meaningful confidentiality against a bundle holder, including the daemon: client-side unlock is a process boundary, not secrecy for a known password. When the optional dashboard is enabled, web logins decrypt signing keys in the daemon process for their in-memory sessions; the socket protocol remains client-side signing.

• AccountSession keys stay client-side: TrustedLoginAccount returns the user’s full record (account_info, including the encrypted AccountCredentials) in the same round-trip as the challenge. The client derives the wrapping key locally (Argon2id over the password), unlocks the authenticated Account locator/key bundle, signs the challenge, and reads only that login’s current encrypted bundle and metadata from native local state to build its AccountSession — no credential read of _users. The signing key never crosses the socket.
• Authentication via challenge-response: the daemon issues fresh random challenge bytes per login attempt. Successful decryption of the user’s signing key on the client is password verification; the daemon verifies the returned signature against the user’s stored public key. No password is sent over the wire.
• TrustedLogin naming is load-bearing: the flow assumes the caller is already trusted by the socket’s filesystem permissions. Over a network transport this would need a PAKE instead — the name flags that gap deliberately.
• Encrypted stores remain opaque to the daemon: per-database encrypted CRDTs merge as Vec<EncryptedBlob>; the daemon participates in storage and sync without ever holding a content encryption key.
• Filesystem permissions: missing directories in the socket path are created with mode 0700, including under a permissive process umask. The socket is mode 0660.

Caller-owned signers can use AccountSigner without a daemon-held login bundle. A proof-only connection carries a generated cache scope, not a registered local-login identity. The optional dashboard remains an explicit daemon-side signing exception.

The socket directory is the local trust boundary. An existing parent must be owned by the daemon user, not writable by group or others, and reached without symlinked path components. Its group, setgid bit, and traversal permissions can intentionally grant another Unix user access; every process that can connect receives the existing fully trusted service API. Unsafe shared-writable and symlinked layouts are rejected rather than supported. An adjacent advisory lock coordinates cooperating daemon owners, stale sockets are recovered, and graceful or dropped servers remove only the socket identity they bound.

See the crate::service module rustdoc for the full design rationale, including why daemon-side signing (the earlier draft) was rejected.

Request / Response Types

ServiceRequest

The top-level request enum is intentionally flat, keeping the pre-auth surface visible at a glance:

AuthenticatedDbRequest carries an identity claim that the server validates against the connection’s session keyset before dispatch:

pub struct AuthenticatedDbRequest {
    pub root_id: ID,      // database whose auth_settings gate this op
    pub identity: SigKey, // claim; hint pubkey must be in session_keyset
    pub op: DatabaseOp,   // tree-scoped by construction
}

DatabaseOp — the wire surface

Every storage operation rides a single DatabaseOp enum carried in AuthenticatedDbRequest. The server runs its own Database on its local Instance, so verification-on-read, the Verified frontier, and CRDT-state materialization happen server-side by construction. Each variant is tree-scoped through the containing request’s root_id.

VerifyEntrySignature runs the existing historical validator on the daemon, preserving pinned settings, inherited floors, permission bounds and incomplete-evidence errors. The Entry must belong to the requested database; the session-keyset-validated acting identity needs current Read on that database and every claimed delegated tree. The 10-step/64-tip bounds are checked before delegated Read queries. The verdict does not set verification status, acquire missing evidence or fall back to privileged reads; access gates may still perform their normal opportunistic verification. Older v0 peers follow the compatibility limits above.

Operations deliberately not exposed over the wire — update_verification_status, get_instance_secrets, all_roots, and raw verification-status/enumeration queries — live on the concrete local BackendImpl engine (reached via Backend::local_engine), so a RemoteBackend simply does not provide them.

update_verification_status is off-wire by design, not just for tidiness: verification is a local trust decision. If a client could set it over the socket, a client (or anything that reached the socket) could assert “this entry is Verified” without the server having validated it — exactly the caller-asserted-status hole the storage API was hardened to close. So the daemon’s Instance owns verification: it stores everything Unverified and runs Database::verify() itself. The frontier/allow_unverified distinction is applied by the daemon-side Database, and every client connected to the same daemon therefore observes the same verified view.

DatabaseOp::required_permission() -> Permission returns Write(0) for SubmitSignedEntry (advisory only — the per-tree gate is skipped for submit) and for the Store-state staging operations, Admin(0) for SetInstanceMetadata (advisory — gated against _databases instead of root_id), and Read for every other variant. The match is exhaustive, so a new operation cannot inherit Read by falling through.

Published views and private builds use opaque random tokens stored only in the connection context. Every use rechecks the authenticated user, the database, and the connection. Disconnect, explicit abort, and bounded idle expiry delete unpublished private builds. The unchanged 64 MiB frame cap is enforced per request and response, so multi-record state is paged or chunked and a single record that cannot fit is refused with RecordTooLarge.

ServiceResponse

Ticket Bootstrap

AccountSession::join on a connected instance joins a database from a ticket without a client-side sync engine and without sending the account’s private key to the daemon. A bootstrap request proof (SyncRequestAuth) is bound to the answering peer and the requester’s advertised tips, so the exchange takes two requests:

1. The client registers the requesting key in the session keyset, then sends TicketBootstrapPrepare { ticket }. The daemon races the ticket’s address hints with its own transports and returns the chosen address, the peer that answered, and empty tips; Prepare does not read or disclose local database state.
2. The client signs a proof over peer, the database, and tips with the requesting key, and sends it with the requested permission, key name, and optional approver metadata in TicketBootstrap. The daemon requires the proof’s key to be in the session keyset and the proof to cover that exact empty-tip request, then sends it through its own sync engine. The response must be confined to the requested database and every supplied Entry must verify; current requested permission is checked before publishing the tree/peer relationship. Incremental send-back is refused on this path. It re-handshakes the route before recording it, so the address must answer as peer; and if the daemon already holds the database, the proof’s key must already be able to read it there, because the peer store is shared by every user of the daemon.

Pending and rejected results travel as TicketBootstrapOutcome data rather than as ServiceErrors, so the client rebuilds SyncError::BootstrapPending and SyncError::BootstrapRejected exactly. Neither outcome creates Account state or a new serving relationship; existing legacy Account rows are preserved. Only after a successful exchange does the client discover the valid SigKey and record mapping and preferences atomically in its native local record, publishing cache from acknowledged or read-back persisted state. An identical retry does not append equivalent configuration history, and preferences changed during the network wait are not overwritten. Success returns an opened Database only after those client-side writes finish. Pending requires an explicit join retry after approval; rejection is terminal for the request. Requests on a connection are handled in order, so the join occupies its connection until the peer answers. A dropped future or disconnected caller may leave peer-side effects, replicated data or a persisted local account record without its acknowledgement. Already-dispatched daemon work may continue after client EOF, but daemon/process death can interrupt it. Reconnect and retry explicitly; there is no cancellation API or automatic restart-safe completion. The shared frame cancellation rules apply: an interrupted frame retires the connection, while cancelling only its response wait retains FIFO response association. Held/unreadable acquisition and native conflicting-authority/delegated-history recovery remain incomplete; see current recovery limits.

Continuing replication is the daemon’s, and its sync requests are signed with the daemon’s device key. A database that grants only the joining user key, with no global grant, therefore needs separate device-key authorization for later pulls. join does not enroll a device or promise ongoing private background authority. The wildcard-authorized replication test covers global access, not private device enrollment.

Session Keyset

A connection’s cryptographic identity is a set of pubkeys, not a single pubkey. The set is seeded at login with the pubkey that proved TrustedLoginProve (the user’s root pubkey, called login_pubkey) and grows as the client proves possession of additional keys via the registration handshake.

// In service::server
enum ConnectionState {
    PreAuth,
    AwaitingProof { challenge: Vec<u8>, expected_pubkey: PublicKey, .. },
    Authenticated {
        login_pubkey: PublicKey,
        session_keyset: HashSet<PublicKey>,           // includes login_pubkey
        pending_key_challenges: HashMap<PublicKey, Vec<u8>>,
        login_id: String,
        ..
    },
}

No ConnectionState variant holds a PrivateKey (enforced by a structural test).

Why a keyset, not a single pubkey?

An AccountSession has many pubkeys — a root key, a device key, per-DB keys created via AccountSession::add_private_key, etc. Writes signed by a per-DB key produce entries whose tree auth grants that key, not the login key. With a single-pubkey session, reads of those trees would be denied (the login key isn’t a member of the tree the per-DB key authored). The keyset model lets the same connection drive ops on every tree the user actually has a key for, without forcing one connection per key.

Registration handshake

Adding a pubkey to the keyset is a two-step PoP:

1. Client → server: SessionKeyChallenge { pubkey }. Server generates a single-use, pubkey-bound random challenge, stores it in pending_key_challenges[pubkey], returns it.
2. Client signs the challenge with pubkey’s private key, sends SessionKeyRegister { pubkey, signature }. Server verifies, removes the challenge (consumed on success or failure), inserts pubkey into session_keyset on verify, returns Ok.

Each RemoteConnection caches successful registrations in a Mutex<HashSet<PublicKey>> so register_session_key(signing_key) is idempotent — a repeat call for the same key short-circuits without touching the wire. The login pubkey is pre-cached on trusted_login success.

The handle constructors that need per-DB identities register on the caller’s behalf, so most client code never touches the registration API directly:

• Database::create(instance, signing_key, settings) registers signing_key before constructing the returned RemoteBackend-backed handle.
• Database::open_remote(.., identity) is paired by Instance::open_system_db_for_session(root, signing_key), which registers signing_key before opening.
• AccountSession::open_database_with_key(root, key_id) registers the user-held signing key for key_id before routing through Database::open_remote on a connected instance.

Access Control: the three gates

Authenticated requests pass three gates before dispatch:

1. Gate 1 — connection state: the connection must be Authenticated. The identity hint, if any, must have its pubkey field present in the connection’s session_keyset (proof of possession registered earlier). The resolved value becomes the acting pubkey for this op; an absent hint defaults to login_pubkey. The cross-check rejects identity claims for keys the client never proved — an attacker that hijacks an authenticated session still can’t act as a key whose private material isn’t on the client. Submit (DatabaseOp::SubmitSignedEntry) is exempt from this check: an admin transporting a user-signed entry legitimately carries a non-keyset identity, and the verification gate downstream is the real boundary (see Verification-gated submit).
2. Gate 2 — per-tree permission: the server loads auth_settings for the request’s root_id, resolves the acting pubkey’s permission, and rejects the op unless it covers required_permission(). Resolution tries direct membership first; on miss it falls back to the wildcard (*) slot, so a tree’s global grant actually applies to any keyset member that isn’t otherwise listed. GetEntry is gated post-fetch (gate_entry_read): the fetched entry’s owning tree is resolved and Read is required before content is returned, in case the entry’s owning tree differs from the request’s root_id.
3. Gate 3 — cross-tree admin: SetInstanceMetadata rewrites daemon-global pointers, so its gate ignores the request’s root_id and is applied against _databases.auth_settings requiring Admin. The dispatcher special-cases this variant; it fails closed if _databases is unreadable (D8).

Authorization for entry-id-keyed writes is being hardened separately (tracked as D1); see V1 Limitations.

Verification-gated submit

SubmitSignedEntry is verification-gated, not session-gated. The submit handler stores the incoming entry Unverified and runs the server’s own verification pass against the tree’s pinned auth lineage. An attacker without a key the tree’s auth grants cannot produce a Verified entry, and unverified junk is excluded from every default read by the frontier cut — so the per-tree session gate would add no correctness or isolation property here, and would only block legitimate transporters (e.g. an admin session carrying a user-signed genesis).

The core rule for the wire surface: reads are session-gated (confidentiality boundary); submits are verification-gated (integrity boundary).

Error Handling Across the Wire

Errors serialize as ServiceError { module, kind, message }.

Server side: dispatch catches any crate::Error, extracting the error’s module name, discriminant name, and display message.

Client side: service_error_to_eidetica_error() reconstructs the appropriate crate::Error from the (module, kind) pair:

Unrecognized combinations fall back to an Io error carrying the original message, so callers use the same error-handling patterns (e.g. err.is_not_found()) regardless of local vs. remote. A compile-time exhaustive match over crate::Error forces a wire-mapping decision whenever a new variant is added, and a round-trip test asserts every mapped pair survives without hitting the fallback.

Write Coordination

Instance::put_entry() stores an entry through the backend. On a local instance it then fires the local callback registry inline (with previous_tips captured pre-write) and commit().await returns after user callbacks complete. On a connected instance the picture flips: the pre-write get_tips is skipped (the daemon owns the canonical DAG, and reading client-side would also misgate against the connection’s login pubkey rather than the per-DB acting identity), and the local fire is also skipped — the daemon is the sole publisher.

The actual write travels as DatabaseOp::SubmitSignedEntry (verification-gated, see above). Server-side, after the daemon stores and verifies the entry, Instance::fire_write_callbacks runs — for both the daemon’s own consumers (e.g. sync triggers) and a single global callback registered by ServiceServer::run that fans the event out to every subscribed connection. The fan-out builds a Notification::DatabaseWrite { root_id, previous_tips, post_tips, source } and pushes it through each subscriber’s writer channel as ServerFrame::Notification(...). The frame carries cursor brackets only — subscribers that need to enumerate the new entries call Database::ids_added(previous_tips, post_tips) locally, which on a connected instance routes each backend.get(id) through the permission-gated wire read path. This keeps the fan-out constant-size regardless of batch width and closes the post-revocation-leak window (the publisher fan-out doesn’t re-check Read on each event, so cursor-only push means a subscriber whose permission is revoked mid-session can at most learn that some write happened — never the contents).

A connected client subscribes lazily: the first Database::on_write registration for a tree on a given connection spawns a task that sends DatabaseOp::SubscribeWrites. Further registrations on the same tree reuse the subscription. RemoteConnection’s background reader task demuxes inbound ServerFrames — Response frames pop the next pending oneshot in FIFO order; Notification frames upgrade a WeakInstance and route into Instance::fire_write_callbacks, the same dispatch local instances use. The result: connected clients observe writes in daemon-canonical order with full previous_tips, including their own commits; subscriptions are torn down implicitly when the connection drops (a ConnectionGuard Drop in handle_connection scrubs the registry).

Teardown is refcounted with hysteresis rather than immediate. Dropping the last local callback for a tree moves its wire subscription to Idle without any round-trip; a sweep task sends DatabaseOp::UnsubscribeWrites once the idle grace window elapses, and a re-registration before that transitions straight back to Subscribed. Both the sweep and subscribe_writes hold a per-tree fence across their wire round-trips so an Unsubscribe and a racing Subscribe can’t reorder. Client-side teardown hangs off the last user-facing RemoteConnection handle: it carries a drop token that marks the connection dead, aborting the reader task so the socket’s WriteHalf can close and the daemon sees EOF.

Store state

Historical current state is materialized into derived record sets in the shared Store-state record substrate (see Store state). The daemon owns those records; clients resolve views and read points or bounded pages through them, so a Table read fetches the rows it asks for rather than a whole materialized Doc.

Cached state is scoped by CacheScope:

• CacheScope::Shared — daemon-computed state (visible to every authorized user; the bytes were produced by the trusted server-side merge).
• CacheScope::AccountSession(uuid) — client-computed state (scoped to the submitting user’s uuid; only that user can poison their own future reads).

Every client-supplied StoreStateRequest is rebound to the session before it reaches the backend: its database must be the one the request was gated against, a shared-scope request is narrowed to the session user, and a request naming another user’s scope is refused. Resolution then falls back from the user scope to the shared one, so a daemon materialization is reused instead of rebuilt per user, while a client can never publish into shared cached state or another user’s cached state.

Database-Level Wire API

The server runs its own Database on a local Instance, so verification-on-read, the Verified frontier, and CRDT-state materialization happen server-side by construction. Every op is intrinsically tree-scoped through the containing AuthenticatedDbRequest.

Unified Backend seam

Transaction, Store, Database, and Instance all drive I/O through a single Backend trait (crate::instance::backend::Backend) with no local-vs-remote branching at the call sites:

The trait is the intersection of what both implementations can honor with the same meaning. Off-seam local-only operations (instance secrets, verification-status mutation, raw all_roots/get_tree listings) are deliberately not on the trait and live on the concrete in-process engine. They are reached only where one exists, via Backend::local_engine() -> Option<Arc<dyn BackendImpl>> (returns None on a remote backend).

Two implementations exist:

• LocalBackend — wraps an Arc<dyn BackendImpl> (the concrete in-process storage engine). local_engine() returns it.
• RemoteBackend — wraps a RemoteConnection and translates each method to a wire RPC. Carries only an optional acting identity (None = the connection’s session identity); tree-scoped methods take the tree from the caller, and get derives its gating tree server-side from the fetched entry, so no per-handle root is bound and one connection multiplexes every per-database identity via the session keyset. local_engine() returns None.

Database selects the implementation at construction time:

// Local: share the Instance's backend.
ops: instance.backend().clone(),

// Remote (service): a per-handle RemoteBackend bound to the calling identity.
ops: Arc::new(RemoteBackend::new(conn, Some(identity))),

Encrypted-store split

Encrypted stores (wrapped in PasswordStore) are opaque to the daemon — the daemon stores and syncs EncryptedBlob entries without ever holding a content encryption key. The DatabaseOp surface handles the encryption boundary with two complementary primitives:

• GetStoreState { store } — the unencrypted fast path (Doc/Table stores). The server loads the store’s entries, CRDT-merges them locally, materializes the merged state, and returns a WireCrdtValue (serde_json::Value). No decryption; the server works with plaintext because the underlying data is unencrypted.

• GetStoreEntries { store, tips, scope } — the universal primitive for encrypted stores. Returns opaque Entry objects reachable from tips, ordered by subtree height, already verified against the server’s Verified frontier. The client receives encrypted entries it can decrypt and CRDT-merge locally. This works identically for encrypted and unencrypted stores — the server never touches content.

• Store-state resolve/stage/publish and record reads provide a merge fast-path that avoids re-folding entries whose merged state is already materialized. For encrypted Stores, the client decrypts and materializes state, then publishes user-scoped ciphertext. A warm encrypted Table point read uses point-record gets for PasswordStore metadata and the requested row without scanning the row generation or reconstructing history. The daemon cannot validate the plaintext semantics of a client-built encrypted cache. See Store state and Encryption.

The encrypted-store-over-service tests cover both paths: test_database_encrypted_store_roundtrip checks opaque Entry retrieval, while test_warm_encrypted_table_service_point_read_is_lazy proves a warm Table point read stays on record gets and does not reconstruct Store history. Decryption, local merge, and encrypted row-envelope verification are client-only.

Read scope

ReadScope controls the DAG snapshot exposed over the wire:

• Verified (default) — only the maximal all-Verified ancestor-closed prefix of the DAG (the “Verified frontier”). Tips that are still Unverified are replaced by their nearest Verified ancestors; Failed entries are always dropped.
• AllowUnverified — select raw tips with Failed tips dropped, not a recursively sanitized accepted prefix. The caller must explicitly opt in via Database::allow_unverified().

BeginTransaction carries the caller’s ReadScope: selected parents come from the same snapshot as its reads. Selection is not permission to commit against unsettled ancestry: signed local commit validation requires Verified parents, and server-side verification defers a submitted Entry with Unverified parents. GetVerifiedTips itself always returns the server’s Verified frontier, even for a client handle with .allow_unverified(). The verification contract defines the distinction.

Gate scope for AuthenticatedDb

Every DatabaseOp variant is intrinsically tree-scoped via root_id — there are no tree-less operations to gate. Permission follows the required_permission() pattern: Read for queries (GetVerifiedTips, GetStoreState, GetStoreEntries, GetEntry, BeginTransaction, GetStoreTipsUpToEntries, ComputeMergeState), Write for mutation. The per-tree permission gate (Gate 2) always fires for non-submit AuthenticatedDb requests because the tree identity is known before dispatch.

SubmitSignedEntry is the exception — verification-gated, not session-gated, as covered in Access Control above.

One envelope, one op enum

The current wire is the result of a series of consolidations:

• An earlier draft mirrored every storage-trait method onto its own BackendOp variant, carried in a separate ServiceRequest::Authenticated(AuthenticatedRequest) envelope. That surface placed auth/verification/merge above the wire boundary, treating the daemon as a passive key-value store.
• The Database-level DatabaseOp / AuthenticatedDb path was added alongside it so the daemon could own verification-on-read, the Verified frontier, and CRDT-state materialization.
• The collapse landed SetInstanceMetadata into DatabaseOp (dispatcher-special-cased to gate against _databases), folded Get into DatabaseOp::GetEntry, and removed the parallel BackendOp enum, ServiceRequest::Authenticated, and AuthenticatedRequest outright.
• The earlier NotifyEntryWritten RPC was removed: the server’s SubmitSignedEntry handler fires its own write callbacks in-process.

The shape today is a single envelope (AuthenticatedDb(Box<AuthenticatedDbRequest>)) carrying a single op enum (DatabaseOp) over every storage operation.

Feature Gate

#[cfg(all(unix, feature = "service"))]
pub mod service;

The service feature is in the default full set; the unix gate restricts it to platforms with Unix domain sockets.

Testing

Two complementary layers:

• tests/it/service.rs — dedicated integration tests for the service layer: handshake, the TrustedLogin challenge-response, the three gates (unauthenticated rejection, per-tree allow/deny, cross-tree denial), per-user cache isolation, concurrent clients, and the user lifecycle.
• TEST_BACKEND=service — runs the full integration suite through the socket, exercising the RPC layer, serialization, and error reconstruction:

TEST_BACKEND=service cargo nextest run --workspace --all-features
just test service
nix build .#test.service

The full suite passes 1:1 against the service backend. Tests that exercise process-local subsystems — Sync::new (needs the local device key), backend.all_roots / backend.get_verification_status (raw-backend listings that have no wire equivalent), low-level delegation resolution that needs local verification evidence, the CRDT merge cache, and multi-AccountSession-on-one-Instance patterns that would all wedge against a single wire-mode session pubkey — go through always-local helpers regardless of TEST_BACKEND:

• test_local_instance() — like test_instance() but always builds an in-memory local Instance.
• test_local_instance_with_user(username) and test_local_instance_with_user_and_key(username, key_name) — counterparts to the wire-aware versions.
• setup_tree_with_user_key_local() and TestContext::with_local_database() — for tests that poke at local-only backend state.

service::historical_signatures uses a shared fixture for local/socket parity, concrete-key attribution, invalid/incomplete proof, inherited floors and revocation. Its non-admin, Read-only client checks bound acting identity and current Read on every delegation hop, including already-open handles.

The architectural seam — wire-path tests vs. subsystem tests — is explicit at the helper level. The test harness bootstraps the server-side Instance with a passwordless admin user via Instance::create_backend(..., NewAccount::passwordless("admin")), then logs in over the wire as that user. (The old admin/admin auto-bootstrap is gone; production deployments now run eidetica daemon init --username <NAME> with explicit credentials, and the test harness mirrors that shape.)

V1 Limitations

This is a fully trusted local-client v1. Filesystem access to the socket grants the whole service API; the following are deferred with tracked follow-ups:

• Lock-poisoning posture: the per-connection session RwLock and the RemoteBackend per-connection cache mutex are poison-tolerant (handlers recover the guard from a poisoned lock rather than panicking), but other std::sync::Mutex sites in the service path still use lock().unwrap(). Documented at the lock sites; the remaining ones must be audited before serving multiple or untrusted clients.
• Verification status is never asserted over the wire: the service protocol carries no way for a client to assert a verification status; entries arriving over the wire are stored Unverified and earn Verified only via the daemon’s own validation pass. The status model, the pinned-settings validation that makes it staleness-free, the disclosure posture, and the unbuilt authority-reduction (revocation) gap are documented in the Verification Model design doc.
• snapshot() on a connected Database delegates to self.ops(): handles built via Database::create or Database::open_remote carry a RemoteBackend bound to a per-DB identity; reads use that identity (not the connection’s login pubkey). Handles built via plain Database::open on a connected instance share the Instance’s RemoteBackend (which uses the connection’s default session identity) — appropriate for code that wants the connection’s login pubkey to drive reads. The previous heuristic remote-detection in Database::snapshot was replaced with an explicit ops delegation.
• No sync delegation: enable_sync() on a remote Instance is a silent no-op (sync runs daemon-side; the client can’t enable it from over the wire). A future admin-gated EnableSync RPC would let a client ask the daemon to enable its sync subsystem, parallel to the native registration RPC used by InstanceAdmin::create_account.

Future Work

• PAKE for network transport: TrustedLogin is safe only because the socket is filesystem-gated. A network transport must replace it with a password-authenticated key exchange.
• Sync delegation: as above.
• Verified-frontier cursors: WriteEvent cursors are raw DAG frontiers, so a bracket can span Unverified/Failed tips and Database::ids_added enumerates them — consumers must filter on verification status themselves. Narrowing the cursors to the Verified frontier can now read the retained frontier (BackendImpl::verified_snapshot); wiring it into the per-commit path is still future work.
• Notification backpressure: the per-connection writer channel is mpsc::unbounded_channel; a stalled client reader buffers notifications in memory. Switch to a bounded channel with a documented drop-oldest policy if memory growth becomes measurable.
• Derived-key caching: cache the Argon2id-derived encryption key in an OS secret store with a TTL, evolving the daemon into an ssh-agent-like key agent to eliminate repeated password prompts for CLI tools.

Bootstrap

Access requests, approval and verified ticket transfer for keys joining existing databases. Bootstrap access does not enroll a device or grant it separate replication authority.

Architecture

Bootstrap requests are stored in the sync database (_sync), not target databases. A request is auto-approved when the requesting key already holds sufficient authority on the target database; otherwise it enters the manual approval workflow.

Request Flow

sequenceDiagram
    participant Client
    participant Handler
    participant Database

    Client->>Handler: Bootstrap Request (key, permission, proof)
    Handler->>Handler: Verify proof matches claimed key
    Handler->>Handler: Resolve existing authority (direct / global '*' / delegated)

    alt Existing authority sufficient
        Handler-->>Client: BootstrapResponse (approved)
    else Need Manual Approval
        Handler->>Handler: Store request
        Handler-->>Client: BootstrapPending (request_id)
        Note over Client: Admin reviews
        Handler->>Database: Add key on approval
    end

Existing-Authority Auto-Approval

If the requesting key already has sufficient permission on the target database, approval is immediate without adding a new key. The check resolves authority through the same discovery path the rest of the auth layer uses (Database::find_sigkeys), so it recognizes: