Overview

Status: pre-alpha. Nothing works end to end yet, and the on-disk format is unstable until v1.0. There is no compatibility promise. Counts on the milestones page are read from PROGRESS.md when the site is built: 321 of 521 roadmap leaves done (62%) in the 2026-11-05 PROGRESS.md snapshot, with 0 marked in progress.

SPEC.md is the design contract (draft v0.1, format version 1.0, unstable). When these pages and the SPEC disagree, the SPEC wins. docs/architecture.md is the map of the code for contributors.

Persia DB is an embedded, single-file, cloud-native search database, written in Rust. Think “SQLite for search”: open a file, index documents, and get ranked results that, by default, never come back empty. The same segment bytes can live in a local file or in object storage (S3, GCS, Azure Blob) and can be served by stateless query nodes on Kubernetes.

A longer definition of the embedded library — in your process, one file, no search server — is What is an embedded search database?. Where the design sits next to SQLite FTS5, Tantivy, and Meilisearch is Comparison. That page is not a benchmark. Persia DB has no persia::Db search API yet (M10); the engine library code can search. How a typo is supposed to avoid an empty page is Typo-tolerant search. Short answers on status, the file format, crash safety, SDKs, and the license are on the FAQ.

A local database is two files (SPEC §4, §9):

<name>.persia — the index container (header and checksummed frames).
<name>.persia.blobs — a content-addressed pack of client files (PDF, images, video), referenced from documents.

The engine (index format, analysis, scoring, query execution) is written in this repository. ADR-0001 records that decision and the closed list of allowed libraries.

What works today

The workspace exists, CI runs, and persia-format implements the byte-level primitives that later layers build on (ROADMAP M0 and M1, both done):

Bounds-checked little-endian readers and writers, canonical LEB128 varints, zigzag signed integers, and checked offset and length types (SPEC §4.1).
Integer codecs used inside future segment sections: delta coding, 128-value bit-packing, postings blocks, and frame-of-reference blocks for numeric columns (SPEC §5.2.1, ADR-0004).
CRC32C, xxh3-64/128, and a helper for CRC-protected fixed-size structures (SPEC §4.1–§4.3).
Block compression codecs None and Zstd, with a decompression-bomb guard on every decode (SPEC §5.2, ADR-0005, ROADMAP 1.4).
Block framing of segment sections: per-block CRC and codec, and a block table that locates every block, so one block can be fetched and decoded on its own (SPEC §5.2, ADR-0006, ROADMAP 1.5).
The container header (magic, versions, feature flags, page size) and the two superblock slots, with the rules for picking the active slot (SPEC §4.2–§4.3, ADR-0007, ROADMAP 1.6).
Frames: encode, decode, scan, and the streaming encoder, including a zstd payload flag and the classes of scan end (SPEC §4.4, ADR-0009, ROADMAP 1.7). The memory-map reader and BlobId / BlobRef are in the same crate.

Those pieces are library code. persia-format implements the on-disk codecs (M1 done, 53/53). persia-storage implements the Storage trait, memory and local-file backends, and the single-file container (M2 done, 46/46). S3, GCS, and Azure have not been started (M12). persia-engine implements collection schema, documents, and manifests (M3 done, 15/15) and a segment builder, writer, and reader (M5 done, 48/48). Its write path is in progress (M6, 49/58): a memtable, a WAL with chunks, group commit, and replay, flush, commit, snapshots, auto-commit, and open checks, deletes and upserts across segments with tombstone sidecars, and segment merging (a merge policy, a segment merger, a background scheduler, and deletion of obsolete objects). Its query path is in progress (M7, 46/51): a query AST, user-text compiler, filter DSL, and query limits, collection statistics (document count, document frequency, average field length), BM25F scoring with static rank and score bounds, matching and score top-k over one view, and a collection search with sorts, pages, facets, and stored fields (CollectionViews::search). Relaxation has not been started (M8). A collection snapshot can be searched as library code. There is no search call on the database handle. persia-analysis implements tokenizers and the built-in analyzers de, en, es, fr, it, keyword, pt, standard, and whitespace (M4 done, 29/29). It does not search. persia-blob exports no API (M9 not started; its dependencies are done). The public persia crate exports no Db and no Collection. The crates are unpublished (publish = false, version 0.0.0). See Getting started and Milestones.

The first release a caller can use, v0.1, is ROADMAP M10. The format freezes at v1.0 (M15).

Why

Five properties the design does not trade away:

Zero friction. Db::open("app.persia") just works. No daemon and no config file for the embedded library. Server configuration, when it exists, is flags, then PERSIA_* environment variables, then an optional TOML file, with production-safe defaults (SPEC §14). Planned: M10, M11.
Own engine. Index format, analysis, BM25F scoring, and query execution live in this repository, behind a small allowed-dependency list (ADR-0001).
Never zero results by default. A query that matches nothing is relaxed step by step, and the response says exactly what changed (SPEC §7.6). The cascade is not started (M8). The query compiler and the BM25F scorer are library code. See Typo-tolerant search and Feature reference.
Cloud-native. Segments are immutable. A local commit and an object-storage commit publish the same segment bytes. Query nodes are stateless. Planned: M12, M13.
Correctness, then speed. Checksums, crash-safe commits, and fuzzed parsers come before performance claims. SPEC §16.2 lists benchmark targets and labels them as targets, not promises. Scalar codec baselines already recorded live in docs/benchmarks.md.

Goals and non-goals

Goals (SPEC §1.1): embedded search with an optional server; BM25F plus filters, facets, and zero-results avoidance; the same segment bytes locally or in object storage; horizontal scale by shard; content-addressed blobs; the embedded Rust API, embedded Go and TypeScript bindings, and Rust and Go network clients; a power loss at any point leaves committed data intact.

Out of scope for v1 (SPEC §1.2):

Text extraction, OCR, and transcription. Clients supply text; blob bytes stay opaque.
Vector or semantic search. The format reserves a feature flag for a later layout (SPEC §18.6).
SQL, joins, multi-document transactions, and cross-collection queries.
More than one writer on the same shard. A lease enforces a single writer (SPEC §11.3).
Encryption at rest inside the file format. That stays with the disk or the storage service.

The types that already exist are quoted on the Code page.