Storage

IStorage interface and three backends: JSON file, SQLite, and in-memory.

Overview

@consensus-tools/storage provides the storage abstraction layer for consensus-tools. It defines the IStorage interface and three implementations: JSON file, SQLite, and in-memory. All other packages that need persistence depend on this package.

ℹ

Moved from core in v0.9.0

Storage classes were previously re-exported from @consensus-tools/core. As of v0.9.0, import them directly from @consensus-tools/storage.

Installation

pnpm add @consensus-tools/storage

Requires @consensus-tools/schemas (bundled — installed automatically).

Quick start

import { JsonStorage, SqliteStorage, MemoryStorage } from "@consensus-tools/storage";

const json = new JsonStorage("./state.json");
const sqlite = new SqliteStorage("./state.db");
const memory = new MemoryStorage();

await memory.init(); // all backends require init()

For config-driven setup:

import { createStorage } from "@consensus-tools/storage";

// config must be a full ConsensusToolsConfig with config.local.storage.kind set
const storage = await createStorage(config);

Storage backends

Backend	Persistence	Best for
`JsonStorage`	File (JSON)	Local development, single-process deployments
`SqliteStorage`	SQLite database	Production use with concurrent access
`MemoryStorage`	In-memory (lost on exit)	Tests, quick prototyping, CI environments

API reference

IStorage interface

All backends implement the same contract:

interface IStorage {
  init(): Promise<void>;
  getState(): Promise<StorageState>;
  saveState(state: StorageState): Promise<void>;
  update<T>(fn: (state: StorageState) => T | Promise<T>): Promise<{ state: StorageState; result: T }>;
}

ℹ

Use update() for writes

Prefer update() over getState() + saveState() separately. update() provides mutex protection against concurrent writes and is the primary write API.

Direct usage

import { MemoryStorage } from "@consensus-tools/storage";

const store = new MemoryStorage();
await store.init();

// Atomic update with mutex protection
const { state: updated, result } = await store.update((current) => {
  current.jobs.push(newJob);
  return newJob.id;
});

Storage caps

Limit storage growth by trimming old entries:

import { applyStorageCaps, defaultState } from "@consensus-tools/storage";

const caps: StorageCaps = {
  maxAuditEntries: 1000,
  maxLedgerEntries: 500,
};

const trimmed = applyStorageCaps(state, caps); // keeps most recent entries (FIFO)

Exports reference

Export	Description
`createStorage(config)`	Factory — creates the right backend from config
`JsonStorage`	File-based JSON storage
`SqliteStorage`	SQLite-based storage (requires optional `better-sqlite3`)
`MemoryStorage`	In-memory storage for dev/test
`IStorage`	Storage interface type
`StorageCaps`	Storage capacity limits type
`defaultState()`	Returns an empty default `StorageState`
`applyStorageCaps(state, caps)`	Trim state to capacity limits (FIFO)
`Mutex`	Mutex for concurrent access control

core -- LocalBoard and engines take an IStorage instance
schemas -- StorageState type definition

Previous←personas Nextintegrations→