This repository was archived by the owner on Aug 7, 2026. It is now read-only.
feat(smt): Fjall storage backend for large_forest::PersistentBackend - #1104
Open
battlmonstr wants to merge 68 commits into
Open
feat(smt): Fjall storage backend for large_forest::PersistentBackend#1104battlmonstr wants to merge 68 commits into
battlmonstr wants to merge 68 commits into
Conversation
to support RustRover IDE
Without --reset it is not possible to run the command twice in a row. It fails with an error when running on an existing db.
The existing target runs both smt::large and smt::large_forest unit tests. Running only smt::large is useful until the large_forest module is refactored.
The Makefile declares bench-large-smt-* targets to run benchmarks in the miden-crypto binary (main.rs). There are 2 more related benchmarks: * "criterion" - miden-crypto/benches/large_smt.rs * "codspeed" - benches/smt-codspeed/benches/smt_codspeed.rs It is useful to be able to run them in isolation for alternative backends comparison.
This is needed to make RocksDbConfig backend-agnostic. write_buffer_manager creation relies on rocksdb library.
for reuse across alternative backends
The constants are shared among alternative backends.
Defines traits to be implemented by alternative backends. Adds a RocksKVDB implementation to migrate RocksDB-specific code from RocksDbStorage.
Replaces the hand-rolled RocksDbSubtreeIterator with a chain of six KVDBReader::iter calls (one per subtree depth CF), each mapped to Subtree deserialization.
|
Automated check (CONTRIBUTING.md) Findings:
Recommendations:
Next steps:
|
Author
|
/quality-review |
battlmonstr
force-pushed
the
smt-forest-fjall
branch
2 times, most recently
from
July 30, 2026 16:06
247f78e to
49be44b
Compare
KVDB traits implementation using Fjall database. This backend can be selected by smt-kvdb-fjall feature flag.
Similar to "bench-large-smt-criterion-rocksdb".
Useful for performance comparison of alternative backends.
This is a preparation to schema setup refactoring.
Move merge_batches from smt::large_forest to smt::large to implement KVDBBatch.append().
The KVDB implementations must not depend on any particular schema. The schema configuration is separated to KVDBSchemaFactory implementations. Move related code there: new, configure_smt_cf_options, configure_block_table_options.
The constants are shared among alternative schema factories.
Adds a RocksKVDB implementation to migrate RocksDB-specific code from PersistentBackend.
…ndReader::entries
Previously persistent-forest controlled both PersistentBackend and rocksdb. Now it is repurposed to only control a generic KVDBPersistentBackend, and a new flag controls binding it to RocksDB. persistent-forest-default flag is needed for --all-features builds. In such build it gets enabled, and effectively disables non-default backends.
Fjall database binding of PersistentBackend. This backend can be selected by persistent-forest-fjall feature flag.
battlmonstr
force-pushed
the
smt-forest-fjall
branch
from
July 30, 2026 17:08
49be44b to
b97f837
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This is a follow-up to PR #1101 and depends on it.
#1101 started supporting alternative storage backends for large SMT, and covered
smt::large::storagemodule.This PR covers
smt::large_forest::backend::persistentmodule.PersistentBackend (in mod.rs) used to have business logic to store a large forest SMT. That business logic was wired to RocksDB but in theory could be applied to any key-value storage mechanism.
The PersistentBackend is refactored to work on top of abstract KVDB traits introduced in PR #1101.
A Fjall database backend is added.
Contents
The PR commits are organized in this sequence:
Each intermediate commit produces a valid state in which tests and benchmarks pass.
It is recommended to do code review sequentially commit by commit.
Tests
Benchmarks
macOS 26, Mac Mini M4 Pro 10+4 cores