Embedded key-value engine · pure Rust
Keys in, sorted tables out. No lock between the writers.
TessariKV is an LSM storage engine written to become the storage layer under TessariDB. Every structure on a read or write path is lock-free or wait-free wherever a non-blocking form exists, its API is async on tokio, and nothing under it is C or C++.
cargo add tessarikv0.0.2 · early development
Why this engine
The parts of storage that decide everything else.
Lock-free first
Every shared structure on a read or write path is lock-free or wait-free wherever a non-blocking form exists. Any number of tasks write and read at once without a global lock.
Pure Rust, nothing underneath
No C or C++ dependency and no build-time compiler toolchain beyond cargo. Runs on macOS, Linux and Windows.
Async on tokio
Opening, writing, reading, flushing and compacting are async functions; background work runs as tasks, so the engine fits a service rather than owning one.
Transactions without a lock
Snapshot isolation with first committer wins, and opt-in serializable. Conflicts are checked as commits run, with no lock and no single committer.
Durability per write
Each write says how durable it must be before it returns: synced to storage, handed to the operating system, or not logged at all for a bulk load.
Encryption at rest
Every file that holds data or metadata is encrypted and authenticated under a key of its own, with AES-256-GCM or ChaCha20-Poly1305. A wrong key is refused before anything is read.
Keyspaces
Independent sets of keys, each with its own options, tables, levels and compactions, created and dropped while running. One batch may span several and stays atomic.
Building blocks model-checked
The lock-free primitives are each documented with their specification, linearization points and memory-ordering argument, and model-checked under loom.
- compaction
- leveled · universal · FIFO
- codecs
- LZ4 · Snappy · Zstandard · Deflate
- isolation
- snapshot · serializable
- ciphers
- AES-256-GCM · ChaCha20-Poly1305
Lock-free first
Every task writes. None of them waits for a lock.
Writers take their place in the commit order without queueing behind a mutex, the in-memory table is a lock-free skiplist, the block cache is a lock-free CLOCK, and every statistic is one wait-free add on a per-thread stripe. Readers never block writers, and writers never block readers.
The building blocks →- task 1batchseq 42
- task 2batchseq 44
- task 3batchseq 41
- task 4batchseq 43
- write batchput · delete · range delete
- logappended with its durability
- skiplistin memory, lock-free
- flushfrozen, written sorted
- L0 tablenamed by a synced manifest
The write path
A batch, a log record, a node in memory.
A write is an atomic batch of puts, deletes and range deletes, possibly across keyspaces. It is appended to the log with the durability it asked for, lands in the skiplist, and later flushes as a sorted table named by a synced manifest. Recovery replays only the log above the last flush.
- Sync
- survives power loss
- Buffered
- survives a process crash
- NoWal
- for loads you can redo
The read path
Newest first. Skip what cannot hold the key.
A read asks the in-memory tables, then the levels, newest first. A whole-key filter skips an in-memory table that cannot hold the key, a bloom filter skips a file without touching a block, and the lock-free block cache answers before storage does. Every block read from storage has its checksum verified.
The read path, step by step →- memtablefilter: not here
- immutablefilter: not here
- L0 · 000129bloom: skip file
- L1 · 000118block cache · found
- T1
- begin @40
- put acct:7 = 90
- commit → seq 41
- T2
- begin @40
- put acct:7 = 75
- commit → Conflict
Transactions
First committer wins, decided without a lock.
A transaction reads at its own snapshot and keeps its writes private until it commits them as one batch. If a key it writes changed after it began, the commit is refused with Conflict and writes nothing. Serializable transactions also check what they read and scanned.
Encryption at rest
Every file sealed, each under a key of its own.
Tables, the log, the manifest and the options record are encrypted and authenticated with AES-256-GCM, or ChaCha20-Poly1305 where AES instructions are missing, each under a key derived from your provider’s root key. A wrong or missing key is refused before anything is read.
- 000131.sstkey = derive(root, file)user:042 → red9f3a c1e0 77b2 04d9
- 000046.logkey = derive(root, file)put order:17 = oke21c 5a90 3bf7 c8a1
- MANIFESTkey = derive(root, file)L0 += 0001314d07 b6e3 19ac f250
- OPTIONSkey = derive(root, file)multiplier = 10a8f1 02cd 6e94 7b3e
Status
Early development, and saying so.
TessariKV is at 0.0.2. The API and the on-disk format may change in any release before 1.0, and it is not yet the storage layer TessariDB ships on. What it does today is listed feature by feature, each one a line of its README.
let db = Db::open("example", DbOptions::in_memory()).await?;
let mut batch = WriteBatch::new();
batch.put(KeyspaceId::DEFAULT, b"apple", b"red")?;
db.write(&batch, &WriteOptions::default()).await?;
let value = db.get(KeyspaceId::DEFAULT, b"apple", &ReadOptions::default()).await?;
assert_eq!(value.as_deref(), Some(&b"red"[..]));