From 91ff784045195c42b61097ea237512a440f2e36c Mon Sep 17 00:00:00 2001 From: Sasha Denisov Date: Fri, 11 Sep 2026 00:25:12 +0200 Subject: [PATCH 1/5] fix(rag): add flush(), so an index survives the process (#492) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Neither RAG store persisted on demand. qdrant never called EdgeShard.flush(), so points added through addDocument stayed in the shard's in-RAM segment: an index built in one session was gone in the next and the corpus was embedded again on every launch. The reporter measured it on two Android devices and named the missing piece — the segment's id-tracker files never reach disk, which is why the next open fails with "Failed to load ID tracker mappings". close() persisted already, since unloading writes too, but that does not help the case this is about: an Android app the system kills in the background never gets to close anything, and there was no way to persist without giving up the store. flush() joins the VectorStoreRepository contract rather than only the qdrant class. A caller holding the interface — which is what FlutterGemma.initialize hands back — would otherwise need a backend type-check to ask for durability. On web it is NOT a no-op, and assuming it was is what the first draft of this change got wrong. sqlite3 autocommits, so the bytes reach the VFS; whether the VFS wrote them anywhere durable is a separate question, and for the one Flutter web actually gets — IndexedDB, since OPFS needs a dedicated worker — the answer is no. Its xSync is a documented no-op and its own docs describe writes as asynchronous "without any durability guarantees. You can invoke flush". The store now keeps the VFS reference so it can drain it, close() drains too, and the in-memory fallback throws instead of reporting a success it cannot deliver: returning quietly there would have been this very bug wearing a different hat. The qdrant flush translates QdrantException into VectorStoreException like every other method on that class. It is not a VectorStoreException and is not exported from the barrel, so leaking it raw would have made `on VectorStoreException` — the catch the contract asks for — miss every flush failure. Native SQLite really is a no-op: the connection is in autocommit and never opens a transaction. It says so at its own site rather than leaving callers to guess which backends need the call. Core takes a minor bump, not a patch: adding a member to a publicly implementable interface is source-breaking for an external `implements`-er, and the neighbouring filterSchema doc claimed otherwise — that claim is corrected here too. Tested against a real shard, not a mock: 20 documents written, id-tracker files asserted absent before flush and present after, mutation-checked by dropping the call. A failing flush is pinned to VectorStoreException by removing the shard directory under an open store. The reopen-after-close test is relabelled to say what it actually proves, since close() persists on its own and it passes with the flush deleted. --- packages/flutter_gemma/CHANGELOG.md | 3 + .../unconfigured_vector_store.dart | 3 + .../services/vector_store_repository.dart | 49 +++++- packages/flutter_gemma/pubspec.yaml | 2 +- .../flutter_gemma_rag_qdrant/CHANGELOG.md | 3 + .../lib/src/qdrant_edge_client.dart | 16 ++ .../lib/src/qdrant_vector_store.dart | 29 ++++ .../lib/src/qdrant_vector_store_stub.dart | 3 + .../flutter_gemma_rag_qdrant/pubspec.yaml | 4 +- .../test/qdrant_lifecycle_test.dart | 153 +++++++++++++++--- .../flutter_gemma_rag_sqlite/CHANGELOG.md | 3 + .../lib/src/sqlite_vector_store.dart | 8 + .../lib/src/sqlite_vector_store_stub.dart | 3 + .../lib/src/web_sqlite_vector_store.dart | 77 +++++++++ .../lib/src/web_sqlite_vector_store_stub.dart | 5 + .../flutter_gemma_rag_sqlite/pubspec.yaml | 4 +- 16 files changed, 337 insertions(+), 28 deletions(-) diff --git a/packages/flutter_gemma/CHANGELOG.md b/packages/flutter_gemma/CHANGELOG.md index 05a6c6599..f5593d93e 100644 --- a/packages/flutter_gemma/CHANGELOG.md +++ b/packages/flutter_gemma/CHANGELOG.md @@ -1,3 +1,6 @@ +## 1.9.0 +- `VectorStoreRepository` gains `flush()` for stores that buffer writes in memory. + ## 1.8.0 - Whisper output language on `getActiveStt` and `transcribe` (#500). - **Breaking for custom `SpeechRecognizer` implementations**: `transcribe` gained `language:` and the type gained a `language` field. diff --git a/packages/flutter_gemma/lib/core/infrastructure/unconfigured_vector_store.dart b/packages/flutter_gemma/lib/core/infrastructure/unconfigured_vector_store.dart index 00c054bdf..013e3141f 100644 --- a/packages/flutter_gemma/lib/core/infrastructure/unconfigured_vector_store.dart +++ b/packages/flutter_gemma/lib/core/infrastructure/unconfigured_vector_store.dart @@ -53,6 +53,9 @@ class UnconfiguredVectorStore implements VectorStoreRepository { @override Future clear() async => _fail(); + @override + Future flush() async {} + @override Future close() async {} diff --git a/packages/flutter_gemma/lib/core/services/vector_store_repository.dart b/packages/flutter_gemma/lib/core/services/vector_store_repository.dart index fabd67a2f..ffe1ae341 100644 --- a/packages/flutter_gemma/lib/core/services/vector_store_repository.dart +++ b/packages/flutter_gemma/lib/core/services/vector_store_repository.dart @@ -130,12 +130,52 @@ abstract class VectorStoreRepository { /// than report an unreadable corpus as an empty one. Future clear(); + /// Persist everything written so far, without closing the store. + /// + /// Call it once after a bulk index. Writes are not necessarily on disk when + /// [addDocument] returns: a store is free to hold them in memory and settle + /// up later, and one of them does. + /// + /// **Per backend**: + /// - SQLite (native and web): already durable. `sqlite3` runs in autocommit, + /// so each statement is its own transaction, and the web VFS persists to + /// OPFS or IndexedDB. This is a no-op there. + /// - Qdrant: **required.** Points added through the UniFFI shard live in its + /// in-RAM segment until the shard is flushed or unloaded. An index built + /// without this is gone when the process ends, and the corpus is embedded + /// again from scratch on the next launch. + /// + /// [close] persists too, so a store that is closed cleanly does not need + /// this. What it cannot cover is a process that never gets to close — an + /// Android app the system kills in the background is the ordinary case, not + /// the exceptional one — which is what this exists for. + /// + /// Safe on a store that was never initialized, and safe to call repeatedly: + /// implementations must not throw for either — there is nothing pending, so + /// there is nothing to report. + /// + /// Otherwise this method's job is to be **loud**. An implementation that + /// cannot persist — the write failed, or the store is on a backend with no + /// durable storage behind it — must throw [VectorStoreException] rather than + /// return normally. Silence is the failure this method exists to prevent: + /// the caller asked for durability, and a quiet success tells them they have + /// it while the index is still only in memory. + /// + /// The default body is a no-op, for backends that are already durable. Note + /// that every implementation in this repository uses `implements` rather + /// than `extends`, so none of them inherits it; it is here for the contract + /// and for any future implementation that does extend. + Future flush() async {} + /// Close vector store and release resources /// /// **Resource cleanup**: /// - Mobile: Closes SQLite database connection /// - Web: Closes IndexedDB connection /// + /// Persists pending writes on the way out, so an explicit [flush] before + /// this is redundant. + /// /// Idempotent: Safe to call multiple times Future close(); @@ -154,9 +194,12 @@ abstract class VectorStoreRepository { /// The filterable-metadata schema this store was configured with. /// - /// Concrete (bodied) member with a no-op default so that adding it does NOT - /// force an override on existing or external `implements`-ers (this is an - /// `abstract class`, not an `interface class`, so the body is inherited). + /// Concrete (bodied) member with a no-op default, so an implementation that + /// `extends` this class gets it for free. That does NOT spare an + /// `implements`-er: `implements` inherits no bodies, so every store in this + /// repository — all of which use `implements` — declares this itself, and + /// adding a bodied member here is still a source-breaking change for an + /// external `implements`-er. /// Stores that honor [Filter] (qdrant, sqlite/vec0) override [configure] to /// stash the schema and expose it here; everyone else keeps the empty default. FilterSchema get filterSchema => const FilterSchema(); diff --git a/packages/flutter_gemma/pubspec.yaml b/packages/flutter_gemma/pubspec.yaml index 26590c7fe..1b09faad8 100644 --- a/packages/flutter_gemma/pubspec.yaml +++ b/packages/flutter_gemma/pubspec.yaml @@ -1,6 +1,6 @@ name: flutter_gemma description: "Run Gemma and other LLMs on-device in Flutter (Android, iOS, Web, Desktop). Multimodal vision/audio, function calling, thinking mode, GPU, embeddings, RAG." -version: 1.8.0 +version: 1.9.0 resolution: workspace homepage: https://fluttergemma.dev repository: https://github.com/DenisovAV/flutter_gemma diff --git a/packages/flutter_gemma_rag_qdrant/CHANGELOG.md b/packages/flutter_gemma_rag_qdrant/CHANGELOG.md index baac3d65c..9e598897b 100644 --- a/packages/flutter_gemma_rag_qdrant/CHANGELOG.md +++ b/packages/flutter_gemma_rag_qdrant/CHANGELOG.md @@ -1,3 +1,6 @@ +## 1.4.0 +- Add `flush()`; without it an index was lost when the process ended (#492). + ## 1.3.0 - **Breaking:** moved onto the official `qdrant_edge` UniFFI SDK. - **Breaking:** a 1.x store is not readable — remove its files, then re-index. diff --git a/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_edge_client.dart b/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_edge_client.dart index b4afd5a2a..3a752a27e 100644 --- a/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_edge_client.dart +++ b/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_edge_client.dart @@ -299,6 +299,22 @@ class QdrantEdgeClient { } } + /// Write the in-RAM segment out to disk, keeping the shard open. + /// + /// The crate declares this separately from [close]'s `unload()`, and the + /// difference is the whole point: `unload()` also persists, but it ends the + /// shard. This is what a caller reaches for after a bulk index when it wants + /// to keep using the store — and it is what stands between an index and a + /// process the OS kills without warning. + Future flush() async { + _checkOpen(); + try { + _shard.flush(); + } catch (e) { + _rethrow(e); + } + } + /// Close the shard. Idempotent — safe to call more than once. Future close() async { if (_closed) return; diff --git a/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_vector_store.dart b/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_vector_store.dart index 6434a10fe..85f08b93b 100644 --- a/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_vector_store.dart +++ b/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_vector_store.dart @@ -868,6 +868,35 @@ class QdrantVectorStore implements VectorStoreRepository { } } + @override + Future flush() => _serializeLifecycle(_flush); + + Future _flush() async { + // On the lifecycle lane, not beside it: a flush that overlapped a close + // would reach a shard the other call had already unloaded. + // + // No client means nothing was written through this store, so there is + // nothing to persist — including the case where an earlier open failed. + // Callers flush from lifecycle callbacks they cannot make conditional + // (`didChangeAppLifecycleState` and the like), so "not initialized" has to + // be quiet rather than an exception nobody can act on. + final c = _client; + if (c == null) return; + try { + await c.flush(); + } on QdrantException catch (e) { + // Translated, like every other method on this class. `QdrantException` + // is not a `VectorStoreException` and is not exported from the barrel, + // so letting it out would hand the caller a type they cannot name — and + // `on VectorStoreException`, the catch the contract tells them to write, + // would miss every flush failure. + throw VectorStoreException( + 'Failed to flush the qdrant shard: ${e.message}', + e, + ); + } + } + @override Future close() => _serializeLifecycle(_close); diff --git a/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_vector_store_stub.dart b/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_vector_store_stub.dart index 3c3bd575e..75e2c196e 100644 --- a/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_vector_store_stub.dart +++ b/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_vector_store_stub.dart @@ -68,6 +68,9 @@ class QdrantVectorStore implements VectorStoreRepository { 'QdrantVectorStore is native-only; qdrant-edge cannot run on web', ); + @override + Future flush() async {} + @override Future close() async {} } diff --git a/packages/flutter_gemma_rag_qdrant/pubspec.yaml b/packages/flutter_gemma_rag_qdrant/pubspec.yaml index 6840321d9..3067da9e5 100644 --- a/packages/flutter_gemma_rag_qdrant/pubspec.yaml +++ b/packages/flutter_gemma_rag_qdrant/pubspec.yaml @@ -1,6 +1,6 @@ name: flutter_gemma_rag_qdrant description: "qdrant-edge on-device RAG vector store for flutter_gemma, via the official qdrant_edge UniFFI SDK. Opt-in VectorStoreRepository with payload filtering. Native platforms only (no web)." -version: 1.3.0 +version: 1.4.0 homepage: https://fluttergemma.dev repository: https://github.com/DenisovAV/flutter_gemma/tree/main/packages/flutter_gemma_rag_qdrant topics: [rag, qdrant, vector-search, embeddings, on-device] @@ -24,7 +24,7 @@ environment: dependencies: flutter: sdk: flutter - flutter_gemma: ^1.6.1 + flutter_gemma: ^1.9.0 uuid: ^4.0.0 # For the store's owned-subdir path handling (qdrant_edge_v1). path: ^1.9.0 diff --git a/packages/flutter_gemma_rag_qdrant/test/qdrant_lifecycle_test.dart b/packages/flutter_gemma_rag_qdrant/test/qdrant_lifecycle_test.dart index f3906fa8c..acb0f08a0 100644 --- a/packages/flutter_gemma_rag_qdrant/test/qdrant_lifecycle_test.dart +++ b/packages/flutter_gemma_rag_qdrant/test/qdrant_lifecycle_test.dart @@ -498,27 +498,23 @@ void main() { }, ); - test( - 'a marker we cannot READ counts as present, not as absent', - () async { - // The blanket `catch (_)` put "unreadable" in the same bucket as "not - // ours", so a genuine 1.x store whose marker could not be read came up - // as an empty index with no error — back in the silent bucket. - writeLegacyStore(tmp.path); - final marker = File('${tmp.path}/edge_config.json'); - Process.runSync('chmod', ['000', marker.path]); - addTearDown(() => Process.runSync('chmod', ['600', marker.path])); + test('a marker we cannot READ counts as present, not as absent', () async { + // The blanket `catch (_)` put "unreadable" in the same bucket as "not + // ours", so a genuine 1.x store whose marker could not be read came up + // as an empty index with no error — back in the silent bucket. + writeLegacyStore(tmp.path); + final marker = File('${tmp.path}/edge_config.json'); + Process.runSync('chmod', ['000', marker.path]); + addTearDown(() => Process.runSync('chmod', ['600', marker.path])); - final store = QdrantVectorStore(); - addTearDown(store.close); - await expectLater( - store.initialize(tmp.path), - throwsA(isA()), - reason: 'an unreadable 1.x marker was read as "no legacy store"', - ); - }, - skip: Platform.isWindows ? 'chmod semantics differ' : false, - ); + final store = QdrantVectorStore(); + addTearDown(store.close); + await expectLater( + store.initialize(tmp.path), + throwsA(isA()), + reason: 'an unreadable 1.x marker was read as "no legacy store"', + ); + }, skip: Platform.isWindows ? 'chmod semantics differ' : false); }); group('a shard we did not write', () { @@ -911,4 +907,121 @@ void main() { }, ); }); + + group('flush', () { + // #492: the package never called EdgeShard.flush(), so points added through + // addDocument stayed in the shard's in-RAM segment. A corpus indexed in one + // session was gone in the next and had to be embedded again — and the + // reporter's Android trace named the missing piece exactly: + // "Failed to load ID tracker mappings". + // + // That is what these assert on, because it is the observable half of the + // bug that fits in one process. `close()` persists too, so a + // written-then-closed store cannot tell a working flush from a missing one; + // the id-tracker files can, and they are the ones the failure named. + Future write(QdrantVectorStore store, int n) async { + for (var i = 0; i < n; i++) { + await store.addDocument( + id: 'doc$i', + content: 'content $i', + embedding: vec(4, i + 1.0), + ); + } + } + + List idTrackerFiles(Directory root) { + final segments = Directory('${root.path}/$storeDirName/segments'); + if (!segments.existsSync()) return const []; + return segments + .listSync(recursive: true) + .whereType() + .map((f) => f.uri.pathSegments.last) + .where((name) => name.startsWith('mutable_id_tracker.')) + .toList() + ..sort(); + } + + test('writes the id tracker the next open needs', () async { + final store = QdrantVectorStore(); + await store.initialize(tmp.path); + await write(store, 20); + + // Not a weaker "the directory grew": the segment directory is populated + // well before this — vector storage, payload storage and segment.json are + // all already on disk. These two files are the ones that are not. + expect( + idTrackerFiles(tmp), + isEmpty, + reason: 'unflushed points should not have reached the id tracker yet', + ); + + await store.flush(); + + expect(idTrackerFiles(tmp), [ + 'mutable_id_tracker.mappings', + 'mutable_id_tracker.versions', + ]); + await store.close(); + }); + + test('leaves the store usable — it is not a close', () async { + // Scope, stated because it is easy to over-read: this pins that the + // SAME instance keeps serving after a flush, and nothing more. The + // reopen at the end is a sanity check, NOT evidence that flush + // persisted anything — `close()` persists too, so this test still + // passes with the flush call deleted. The id-tracker test above is the + // one that pins persistence, and it is the one mutation-checked. + final store = QdrantVectorStore(); + await store.initialize(tmp.path); + await write(store, 5); + await store.flush(); + + expect((await store.getStats()).documentCount, 5); + await store.addDocument(id: 'after', content: 'a', embedding: vec(4, 9)); + expect((await store.getStats()).documentCount, 6); + await store.close(); + + final reopened = QdrantVectorStore(); + await reopened.initialize(tmp.path); + expect((await reopened.getStats()).documentCount, 6); + await reopened.close(); + }); + + test('is quiet on a store that was never initialized', () async { + // Callers flush from lifecycle callbacks they cannot make conditional, so + // "nothing to persist" must not be an exception they have to catch. + await expectLater(QdrantVectorStore().flush(), completes); + }); + + test('a failure surfaces as VectorStoreException, not QdrantException', () async { + // The type matters as much as the throw. `QdrantException` is not a + // `VectorStoreException` and is not exported from this package's barrel, + // so a caller writing the catch the contract asks for — `on + // VectorStoreException` — would miss every flush failure if the raw type + // leaked. Every other method here translates; this one has to as well. + final store = QdrantVectorStore(); + await store.initialize(tmp.path); + await write(store, 5); + + // Pull the shard directory out from under the open store. The engine + // then fails writing `mutable_id_tracker.mappings` — the same file the + // #492 report could not read on its device. + Directory('${tmp.path}/$storeDirName').deleteSync(recursive: true); + + await expectLater(store.flush(), throwsA(isA())); + try { + await store.close(); + } catch (_) {} + }); + + test('is quiet after close, and when called twice', () async { + final store = QdrantVectorStore(); + await store.initialize(tmp.path); + await write(store, 3); + await expectLater(store.flush(), completes); + await expectLater(store.flush(), completes); + await store.close(); + await expectLater(store.flush(), completes); + }); + }); } diff --git a/packages/flutter_gemma_rag_sqlite/CHANGELOG.md b/packages/flutter_gemma_rag_sqlite/CHANGELOG.md index 52b05efe0..7ffbeca96 100644 --- a/packages/flutter_gemma_rag_sqlite/CHANGELOG.md +++ b/packages/flutter_gemma_rag_sqlite/CHANGELOG.md @@ -1,3 +1,6 @@ +## 1.4.0 +- Add `flush()`; on web it drains the IndexedDB VFS, which never synced on commit (#492). + ## 1.3.1 - Fix web inserts failing on a numeric metadata value or a missing number field. - Fix web filters with a `mustNot` upper bound returning the negative values they exclude. diff --git a/packages/flutter_gemma_rag_sqlite/lib/src/sqlite_vector_store.dart b/packages/flutter_gemma_rag_sqlite/lib/src/sqlite_vector_store.dart index 0bb704f43..10455ee11 100644 --- a/packages/flutter_gemma_rag_sqlite/lib/src/sqlite_vector_store.dart +++ b/packages/flutter_gemma_rag_sqlite/lib/src/sqlite_vector_store.dart @@ -453,6 +453,14 @@ class SqliteVectorStore implements VectorStoreRepository { _detectedDimension = null; } + @override + Future flush() async { + // Nothing to do, and that is a property of sqlite3 rather than an omission + // here: the connection runs in autocommit, so every statement addDocument + // issues is its own transaction and is durable by the time it returns. + // There is no in-memory segment for this store to settle up. + } + @override Future close() async { // Deliberately NOT gated on `_isInitialized` alone. A store whose diff --git a/packages/flutter_gemma_rag_sqlite/lib/src/sqlite_vector_store_stub.dart b/packages/flutter_gemma_rag_sqlite/lib/src/sqlite_vector_store_stub.dart index c28b9ee90..efb7d0c0c 100644 --- a/packages/flutter_gemma_rag_sqlite/lib/src/sqlite_vector_store_stub.dart +++ b/packages/flutter_gemma_rag_sqlite/lib/src/sqlite_vector_store_stub.dart @@ -75,6 +75,9 @@ class SqliteVectorStore implements VectorStoreRepository { 'SqliteVectorStore is not available on web; use WebSqliteVectorStore', ); + @override + Future flush() async {} + @override Future close() async {} } diff --git a/packages/flutter_gemma_rag_sqlite/lib/src/web_sqlite_vector_store.dart b/packages/flutter_gemma_rag_sqlite/lib/src/web_sqlite_vector_store.dart index c759d2216..e6ec12a39 100644 --- a/packages/flutter_gemma_rag_sqlite/lib/src/web_sqlite_vector_store.dart +++ b/packages/flutter_gemma_rag_sqlite/lib/src/web_sqlite_vector_store.dart @@ -25,6 +25,13 @@ import 'package:sqlite3/wasm.dart'; /// The vec0 table is created **lazily** on the first [addDocument] (so the /// embedding dimension can be learned), or recovered from an existing table on /// [initialize]. +/// Which virtual filesystem the web store ended up on. +/// +/// The three differ in durability, not in speed, so [WebSqliteVectorStore.flush] +/// has to know which one it is talking to: one needs draining, one is already +/// on disk, and one cannot persist at all and must say so. +enum _WebPersistence { opfs, indexedDb, inMemory } + class WebSqliteVectorStore implements VectorStoreRepository { static const String _tableName = 'vec_documents'; @@ -42,6 +49,20 @@ class WebSqliteVectorStore implements VectorStoreRepository { int? _detectedDimension; bool _isInitialized = false; + /// Which VFS [_registerPersistentVfs] actually settled on. + /// + /// Not cosmetic: it is the difference between a store that survives a reload + /// and one that does not, and [flush] cannot answer honestly without it. + _WebPersistence _persistence = _WebPersistence.inMemory; + + /// The IndexedDB VFS, kept so [flush] and [close] can drain it. + /// + /// Held deliberately. `sqlite3`'s IndexedDB VFS writes asynchronously and its + /// `xSync` is a documented no-op ("We can't wait for a sync either way"), so + /// a commit does NOT mean the bytes reached IndexedDB. `flush()` on the VFS + /// is the drain, and without a reference there is nothing to call it on. + IndexedDbFileSystem? _idb; + /// Declared filterable-metadata schema (via [configure]). Empty by default, /// so callers that never declare a schema keep the historical behaviour /// (filters are an ignored no-op). @@ -134,6 +155,8 @@ class WebSqliteVectorStore implements VectorStoreRepository { try { final opfs = await SimpleOpfsFileSystem.loadFromStorage(databasePath); sqlite3.registerVirtualFileSystem(opfs, makeDefault: true); + _persistence = _WebPersistence.opfs; + _idb = null; gemmaLog('[WebVectorStore] Using OPFS VFS for persistence'); return; } catch (e) { @@ -147,6 +170,8 @@ class WebSqliteVectorStore implements VectorStoreRepository { dbName: 'flutter_gemma_rag_$databasePath', ); sqlite3.registerVirtualFileSystem(idb, makeDefault: true); + _persistence = _WebPersistence.indexedDb; + _idb = idb; gemmaLog('[WebVectorStore] Using IndexedDB VFS for persistence'); return; } catch (e) { @@ -168,6 +193,8 @@ class WebSqliteVectorStore implements VectorStoreRepository { 'storage. Documents will NOT persist across page reloads.', ); sqlite3.registerVirtualFileSystem(InMemoryFileSystem(), makeDefault: true); + _persistence = _WebPersistence.inMemory; + _idb = null; } /// Reads the embedding dimension back from an existing vec0 table, if any. @@ -485,6 +512,45 @@ class WebSqliteVectorStore implements VectorStoreRepository { } } + @override + Future flush() async { + // NOT a no-op here, unlike the native arm, and the difference is the whole + // reason this method exists on web. + // + // sqlite3 autocommits, so the bytes have reached the VFS. Whether the VFS + // has written them anywhere durable is a separate question, and for the + // one Flutter web actually gets — IndexedDB, since OPFS needs a dedicated + // worker — the answer is no: its `xSync` is a documented no-op ("We can't + // wait for a sync either way"), and `open()` describes its writes as + // asynchronous "without any durability guarantees. You can invoke flush". + // That `flush` is this call. + if (!_isInitialized && _db == null) return; + switch (_persistence) { + case _WebPersistence.indexedDb: + try { + await _idb?.flush(); + } catch (e) { + throw VectorStoreException('Failed to flush the IndexedDB store', e); + } + case _WebPersistence.opfs: + // Synchronous access handles: the write reached storage before the + // statement returned, so there is nothing left to drain. + break; + case _WebPersistence.inMemory: + // Neither OPFS nor IndexedDB was available, so this store cannot + // persist at all. Returning normally would be the #492 defect wearing + // a different hat: the caller asks "make this durable", gets a + // success, and loses the index on reload. The only warning otherwise + // is a gemmaLog that release builds strip. + throw const VectorStoreException( + 'This store is running on an in-memory VFS (neither OPFS nor ' + 'IndexedDB was available, e.g. private browsing or partitioned ' + 'storage), so its documents cannot be persisted. Re-index after ' + 'each reload, or run in a context that allows storage.', + ); + } + } + @override Future close() async { // Deliberately NOT gated on `_isInitialized` alone — same rule as the @@ -495,9 +561,20 @@ class WebSqliteVectorStore implements VectorStoreRepository { if (_db == null && _sqlite3 == null && !_isInitialized) return; try { _db?.close(); + // Drain before dropping the VFS, so the contract's "close persists too" + // is true here as well. `IndexedDbFileSystem.close()` awaits the pending + // writes; without it a caller that only ever calls close() on web loses + // the tail of its index — the same shape of loss #492 reported natively. + // Best-effort by design: close() is a cleanup path callers usually + // cannot act on, and `flush()` is the call that reports failure. + await _idb?.close(); + } catch (e) { + gemmaLog('[WebVectorStore] close() could not drain the VFS: $e'); } finally { _db = null; _sqlite3 = null; + _idb = null; + _persistence = _WebPersistence.inMemory; _isInitialized = false; _detectedDimension = null; } diff --git a/packages/flutter_gemma_rag_sqlite/lib/src/web_sqlite_vector_store_stub.dart b/packages/flutter_gemma_rag_sqlite/lib/src/web_sqlite_vector_store_stub.dart index b87d1c0e0..26fee5985 100644 --- a/packages/flutter_gemma_rag_sqlite/lib/src/web_sqlite_vector_store_stub.dart +++ b/packages/flutter_gemma_rag_sqlite/lib/src/web_sqlite_vector_store_stub.dart @@ -60,6 +60,11 @@ class WebSqliteVectorStore implements VectorStoreRepository { ); } + @override + Future flush() async { + // No-op for stub + } + @override Future close() async { // No-op for stub diff --git a/packages/flutter_gemma_rag_sqlite/pubspec.yaml b/packages/flutter_gemma_rag_sqlite/pubspec.yaml index 11cf90a91..beba775d5 100644 --- a/packages/flutter_gemma_rag_sqlite/pubspec.yaml +++ b/packages/flutter_gemma_rag_sqlite/pubspec.yaml @@ -1,6 +1,6 @@ name: flutter_gemma_rag_sqlite description: "SQLite vector search (sqlite-vec) on-device RAG vector store for flutter_gemma. Opt-in package; implements flutter_gemma's VectorStoreRepository." -version: 1.3.1 +version: 1.4.0 homepage: https://fluttergemma.dev repository: https://github.com/DenisovAV/flutter_gemma/tree/main/packages/flutter_gemma_rag_sqlite topics: [rag, sqlite, vector-search, embeddings, web] @@ -26,7 +26,7 @@ environment: dependencies: flutter: sdk: flutter - flutter_gemma: ^1.6.1 + flutter_gemma: ^1.9.0 sqlite3: ^3.3.0 # Native Assets hook deps — hook/build.dart fetches the per-platform # sqlite-vec (vec0) loadable extension from the native-sqlite-vec-v* GitHub From c7a6447b6c640ebb548c81f5ce24f3706f8e23f6 Mon Sep 17 00:00:00 2001 From: Sasha Denisov Date: Sat, 12 Sep 2026 19:48:14 +0200 Subject: [PATCH 2/5] fix(rag): say what web flush really does, and reach it from the facade MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round two of review, on the fix itself rather than the bug. The web flush was still wrong, and in the opposite direction from round one. It no longer claims to be a no-op — it drains the IndexedDB VFS — but on sqlite3 >= 3.4.0 that drain does not wait. Upstream commit 11be8acb ("Optimize indexeddb flush on idle") replaced a marker queued behind the running batch with a bare `_startWorkingIfNeeded`, whose body is skipped while a batch is in flight, so flush returns in zero event-loop turns. Their own doc still promises to await; 3.3.4 still did. It is an unintended regression, not a change of model. So the code stays and the words change. Capping the dependency below 3.4 was the tempting alternative and is not viable: drift 2.35.0 requires sqlite3 ^3.4.0 and sqlite_async ^3.5.0, so a cap would turn our durability gap into a resolution failure for the most common consumer. Owning the drain ourselves (writeAutomatically: false plus a timer) needs a 3.4 floor anyway, and its own timer manufactures the interleaving that makes flush return early — while widening today's loss window for every web caller that never calls flush. The exposure being documented is bounded: the VFS streams writes continuously, so what an early return misses is the batch in flight, not the index. close() is the stronger drain on web and now says so, in both directions: the contract no longer calls a flush before close redundant in a way that reads as "close is enough", because on qdrant close swallows a failed save and flush is the call that reports it. Also fixed, all found by the same round: - flush() was unreachable from FlutterGemma.rag, so the facade every app uses could not ask for durability at all. Added through the platform interface and all three shells. - close() settled its fields in a `finally` after the await, so isInitialized reported true over a closed database and a flush landing in that window reached a VFS that was closing. - a re-initialize overwrote the VFS field without draining the old one, orphaning its queued pages. - flush()'s guard returned early when a failed re-initialize had left the previous run's VFS holding unwritten pages. - the class dartdoc had attached itself to a private enum I inserted above it. - _rethrow's final `throw e` escaped raw, so anything that is neither an EdgeException nor a UniffiInternalError crossed all twelve wrappers and the store's translation never fired. - the OPFS comment named the wrong mechanism: durability comes from SQLite calling xSync on commit, not from the write having reached storage. - core's bump is marked as breaking for custom implementations, matching how 1.8.0 marked the same shape of change. Tests: the id-tracker assertion could pass on a store that never wrote, so the segment directory is asserted present first. A black-box test reproduces #492 in one process — a copy of the shard taken before the flush reopens empty and silent, after it carries every document — which pins the contract rather than this engine's file names. Mutation-checked: dropping the flush call fails three tests, including that one. The two tests that shell out or delete a mmapped directory are skipped on Windows. The example's sqlite3 dev-dep moves to ^3.5.2 while the package keeps ^3.3.0: the example resolves separately from the workspace, and was pinning 3.3.3 — so the web suite was exercising a flush() that consumers resolving latest do not get. --- packages/flutter_gemma/CHANGELOG.md | 1 + packages/flutter_gemma/example/pubspec.lock | 6 +- packages/flutter_gemma/example/pubspec.yaml | 5 +- .../lib/core/api/flutter_gemma.dart | 8 ++ .../services/vector_store_repository.dart | 32 +++-- .../lib/desktop/flutter_gemma_desktop.dart | 5 + .../lib/flutter_gemma_interface.dart | 6 + .../lib/mobile/flutter_gemma_mobile.dart | 5 + .../lib/web/flutter_gemma_web.dart | 5 + .../lib/src/qdrant_edge_client.dart | 10 +- .../test/qdrant_lifecycle_test.dart | 111 ++++++++++++++---- .../flutter_gemma_rag_sqlite/CHANGELOG.md | 2 +- .../lib/src/web_sqlite_vector_store.dart | 90 ++++++++++---- 13 files changed, 225 insertions(+), 61 deletions(-) diff --git a/packages/flutter_gemma/CHANGELOG.md b/packages/flutter_gemma/CHANGELOG.md index f5593d93e..239864e67 100644 --- a/packages/flutter_gemma/CHANGELOG.md +++ b/packages/flutter_gemma/CHANGELOG.md @@ -1,5 +1,6 @@ ## 1.9.0 - `VectorStoreRepository` gains `flush()` for stores that buffer writes in memory. +- **Breaking for custom `VectorStoreRepository` implementations**: they must declare `flush()`. ## 1.8.0 - Whisper output language on `getActiveStt` and `transcribe` (#500). diff --git a/packages/flutter_gemma/example/pubspec.lock b/packages/flutter_gemma/example/pubspec.lock index f061b7d13..ac6ea59eb 100644 --- a/packages/flutter_gemma/example/pubspec.lock +++ b/packages/flutter_gemma/example/pubspec.lock @@ -241,7 +241,7 @@ packages: path: ".." relative: true source: path - version: "1.8.0" + version: "1.9.0" flutter_gemma_agent: dependency: "direct main" description: @@ -290,14 +290,14 @@ packages: path: "../../flutter_gemma_rag_qdrant" relative: true source: path - version: "1.3.0" + version: "1.4.0" flutter_gemma_rag_sqlite: dependency: "direct main" description: path: "../../flutter_gemma_rag_sqlite" relative: true source: path - version: "1.3.1" + version: "1.4.0" flutter_gemma_speech: dependency: "direct main" description: diff --git a/packages/flutter_gemma/example/pubspec.yaml b/packages/flutter_gemma/example/pubspec.yaml index 47ae16cc3..5701bfab8 100644 --- a/packages/flutter_gemma/example/pubspec.yaml +++ b/packages/flutter_gemma/example/pubspec.yaml @@ -99,7 +99,10 @@ dependency_overrides: dev_dependencies: # The web store integration test opens the same IndexedDB VFS the store # uses, to plant a corpus the store cannot read. - sqlite3: ^3.3.0 + # ^3.5.2, not the package's own ^3.3.0 floor: the single workspace lock + # otherwise resolves 3.3.3, and the web suite would exercise a flush() that + # consumers resolving latest do not get (upstream regression in 3.4.0). + sqlite3: ^3.5.2 integration_test: sdk: flutter flutter_test: diff --git a/packages/flutter_gemma/lib/core/api/flutter_gemma.dart b/packages/flutter_gemma/lib/core/api/flutter_gemma.dart index ca41b2514..676a07875 100644 --- a/packages/flutter_gemma/lib/core/api/flutter_gemma.dart +++ b/packages/flutter_gemma/lib/core/api/flutter_gemma.dart @@ -1112,6 +1112,14 @@ class GemmaRag { Future initialize(String databasePath) => FlutterGemmaPlugin.instance.initializeVectorStore(databasePath); + /// Persist what has been indexed so far, keeping the store open. + /// + /// Call it after a bulk index, and from wherever the app learns it is going + /// away. On qdrant this is what makes an index survive the process at all — + /// without it the points sit in the shard's in-RAM segment and a background + /// kill takes them. See [VectorStoreRepository.flush] for the other backends. + Future flush() => FlutterGemmaPlugin.instance.flushVectorStore(); + /// Add a document; its embedding is computed automatically (needs an active /// embedding model). Future addDocument({ diff --git a/packages/flutter_gemma/lib/core/services/vector_store_repository.dart b/packages/flutter_gemma/lib/core/services/vector_store_repository.dart index ffe1ae341..995e3c1db 100644 --- a/packages/flutter_gemma/lib/core/services/vector_store_repository.dart +++ b/packages/flutter_gemma/lib/core/services/vector_store_repository.dart @@ -137,18 +137,27 @@ abstract class VectorStoreRepository { /// up later, and one of them does. /// /// **Per backend**: - /// - SQLite (native and web): already durable. `sqlite3` runs in autocommit, - /// so each statement is its own transaction, and the web VFS persists to - /// OPFS or IndexedDB. This is a no-op there. /// - Qdrant: **required.** Points added through the UniFFI shard live in its /// in-RAM segment until the shard is flushed or unloaded. An index built /// without this is gone when the process ends, and the corpus is embedded /// again from scratch on the next launch. - /// - /// [close] persists too, so a store that is closed cleanly does not need - /// this. What it cannot cover is a process that never gets to close — an - /// Android app the system kills in the background is the ordinary case, not - /// the exceptional one — which is what this exists for. + /// - SQLite, native: a genuine no-op. The connection is in autocommit and + /// never opens a transaction, so a statement that returned is on disk. + /// - SQLite, web: **not** a no-op, and not a full guarantee either. The VFS + /// Flutter web gets is IndexedDB (OPFS needs a dedicated worker), whose + /// `xSync` does nothing and whose writes are asynchronous by design, so + /// this call is the drain. On `sqlite3` >= 3.4.0 that drain is partial: it + /// returns without awaiting a write batch already in flight (upstream + /// regression). The exposure is bounded — the VFS streams writes + /// continuously, so what is missed is the batch in flight, not the index. + /// + /// [close] persists too, and on web it is the *stronger* drain: it queues + /// behind the running batch on every version. What close cannot cover is a + /// process that never gets to close — an Android app the system kills in the + /// background is the ordinary case, not the exceptional one — which is what + /// this exists for. So the two are not interchangeable: prefer this one + /// while the store stays open, and note that on qdrant a failure is reported + /// by this call and swallowed by [close]. /// /// Safe on a store that was never initialized, and safe to call repeatedly: /// implementations must not throw for either — there is nothing pending, so @@ -173,8 +182,11 @@ abstract class VectorStoreRepository { /// - Mobile: Closes SQLite database connection /// - Web: Closes IndexedDB connection /// - /// Persists pending writes on the way out, so an explicit [flush] before - /// this is redundant. + /// Persists pending writes on the way out, so a [flush] immediately before + /// this adds nothing. It is not a substitute for [flush], though: a store + /// stays usable after flushing and does not after closing, and a failure + /// here is logged rather than thrown — implementations treat close as + /// cleanup the caller usually cannot act on, while [flush] reports. /// /// Idempotent: Safe to call multiple times Future close(); diff --git a/packages/flutter_gemma/lib/desktop/flutter_gemma_desktop.dart b/packages/flutter_gemma/lib/desktop/flutter_gemma_desktop.dart index f2a9318e2..11ade0a8e 100644 --- a/packages/flutter_gemma/lib/desktop/flutter_gemma_desktop.dart +++ b/packages/flutter_gemma/lib/desktop/flutter_gemma_desktop.dart @@ -819,6 +819,11 @@ class FlutterGemmaDesktop extends FlutterGemmaPlugin { ); } + @override + Future flushVectorStore() async { + await ServiceRegistry.instance.vectorStoreRepository.flush(); + } + @override Future addDocumentWithEmbedding({ required String id, diff --git a/packages/flutter_gemma/lib/flutter_gemma_interface.dart b/packages/flutter_gemma/lib/flutter_gemma_interface.dart index 8ee114844..7c0aed8a2 100644 --- a/packages/flutter_gemma/lib/flutter_gemma_interface.dart +++ b/packages/flutter_gemma/lib/flutter_gemma_interface.dart @@ -154,6 +154,12 @@ abstract class FlutterGemmaPlugin extends PlatformInterface { /// Initialize vector store database. Future initializeVectorStore(String databasePath); + /// Persist everything written to the vector store so far, without closing it. + /// + /// See [VectorStoreRepository.flush] for what this means per backend — it is + /// required on qdrant, a no-op on native SQLite, and a partial drain on web. + Future flushVectorStore(); + /// Add document to vector store with pre-computed embedding. Future addDocumentWithEmbedding({ required String id, diff --git a/packages/flutter_gemma/lib/mobile/flutter_gemma_mobile.dart b/packages/flutter_gemma/lib/mobile/flutter_gemma_mobile.dart index c4d61c4b7..210d1765d 100644 --- a/packages/flutter_gemma/lib/mobile/flutter_gemma_mobile.dart +++ b/packages/flutter_gemma/lib/mobile/flutter_gemma_mobile.dart @@ -941,6 +941,11 @@ class FlutterGemmaMobile extends FlutterGemmaPlugin { ); } + @override + Future flushVectorStore() async { + await ServiceRegistry.instance.vectorStoreRepository.flush(); + } + @override Future addDocumentWithEmbedding({ required String id, diff --git a/packages/flutter_gemma/lib/web/flutter_gemma_web.dart b/packages/flutter_gemma/lib/web/flutter_gemma_web.dart index f91c27175..2b3d27336 100644 --- a/packages/flutter_gemma/lib/web/flutter_gemma_web.dart +++ b/packages/flutter_gemma/lib/web/flutter_gemma_web.dart @@ -435,6 +435,11 @@ class FlutterGemmaWeb extends FlutterGemmaPlugin { ); } + @override + Future flushVectorStore() async { + await ServiceRegistry.instance.vectorStoreRepository.flush(); + } + @override Future addDocumentWithEmbedding({ required String id, diff --git a/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_edge_client.dart b/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_edge_client.dart index 3a752a27e..4ed2abf60 100644 --- a/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_edge_client.dart +++ b/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_edge_client.dart @@ -382,7 +382,15 @@ class QdrantEdgeClient { if (e is qe.UniffiInternalError) { throw QdrantException('qdrant-edge internal failure: $e'); } - throw e; + // Everything else is wrapped too, rather than rethrown raw. This used to + // be a bare `throw e`, which meant any failure that is neither an + // `EdgeException` nor a `UniffiInternalError` crossed all twelve wrapper + // methods untouched — so `on QdrantException`, the catch every caller in + // this package writes, missed it, and the store's own translation to + // `VectorStoreException` never fired. The type is the contract; an escape + // hatch that skips it is the same silent-failure shape as the bug this + // package just fixed. + throw QdrantException('qdrant-edge failed with an unexpected error: $e'); } // ---- Filter bridge: qdrant JSON envelope → typed qe.Filter ---------------- diff --git a/packages/flutter_gemma_rag_qdrant/test/qdrant_lifecycle_test.dart b/packages/flutter_gemma_rag_qdrant/test/qdrant_lifecycle_test.dart index acb0f08a0..1bc6a85a0 100644 --- a/packages/flutter_gemma_rag_qdrant/test/qdrant_lifecycle_test.dart +++ b/packages/flutter_gemma_rag_qdrant/test/qdrant_lifecycle_test.dart @@ -915,10 +915,14 @@ void main() { // reporter's Android trace named the missing piece exactly: // "Failed to load ID tracker mappings". // - // That is what these assert on, because it is the observable half of the - // bug that fits in one process. `close()` persists too, so a - // written-then-closed store cannot tell a working flush from a missing one; - // the id-tracker files can, and they are the ones the failure named. + // Two of these pin the fix, from opposite ends. `close()` persists too, so + // a written-then-closed store cannot tell a working flush from a missing + // one — which rules out the obvious test. Instead: + // * the id-tracker files, named by the reporter's own trace, are checked + // directly (white-box, tied to this engine's on-disk layout); + // * a copy of the shard taken mid-session is reopened (black-box, tied + // only to the contract — it reproduces #492 inside one process). + // The second is the one that survives an engine that renames its files. Future write(QdrantVectorStore store, int n) async { for (var i = 0; i < n; i++) { await store.addDocument( @@ -946,6 +950,16 @@ void main() { await store.initialize(tmp.path); await write(store, 20); + // `idTrackerFiles` answers `const []` for a missing directory, so assert + // the directory is THERE first — otherwise the isEmpty below would pass + // on a store that never wrote anything at all, which is the opposite of + // what this test is for. + expect( + Directory('${tmp.path}/$storeDirName/segments').existsSync(), + isTrue, + reason: 'the segment directory should exist before the flush', + ); + // Not a weaker "the directory grew": the segment directory is populated // well before this — vector storage, payload storage and segment.json are // all already on disk. These two files are the ones that are not. @@ -964,6 +978,55 @@ void main() { await store.close(); }); + test('a shard copied mid-session is readable only after a flush', () async { + // The black-box half, and the one that reproduces #492 inside a single + // process. A copy of the directory taken BEFORE the flush is what a + // process killed mid-session leaves on disk — and it reopens EMPTY, with + // no error, which is exactly what the reporter saw. Taken after, the same + // copy carries every document. + // + // This asserts the contract's outcome rather than this engine's file + // names, so it survives a qdrant release that reorganises its segments. + final store = QdrantVectorStore(); + await store.initialize(tmp.path); + await write(store, 20); + + Future snapshot(String suffix) async { + final dir = Directory('${tmp.path}_$suffix')..createSync(); + addTearDown(() { + try { + dir.deleteSync(recursive: true); + } catch (_) {} + }); + final result = await Process.run('cp', [ + '-R', + '${tmp.path}/$storeDirName', + '${dir.path}/$storeDirName', + ]); + expect(result.exitCode, 0, reason: 'cp failed: ${result.stderr}'); + return dir; + } + + final beforeFlush = await snapshot('before'); + await store.flush(); + final afterFlush = await snapshot('after'); + await store.close(); + + final unflushed = QdrantVectorStore(); + await unflushed.initialize(beforeFlush.path); + expect( + (await unflushed.getStats()).documentCount, + 0, + reason: 'this is #492: an unflushed shard reopens empty, and quietly', + ); + await unflushed.close(); + + final flushed = QdrantVectorStore(); + await flushed.initialize(afterFlush.path); + expect((await flushed.getStats()).documentCount, 20); + await flushed.close(); + }, skip: Platform.isWindows ? 'shells out to cp' : null); + test('leaves the store usable — it is not a close', () async { // Scope, stated because it is easy to over-read: this pins that the // SAME instance keeps serving after a flush, and nothing more. The @@ -993,26 +1056,30 @@ void main() { await expectLater(QdrantVectorStore().flush(), completes); }); - test('a failure surfaces as VectorStoreException, not QdrantException', () async { - // The type matters as much as the throw. `QdrantException` is not a - // `VectorStoreException` and is not exported from this package's barrel, - // so a caller writing the catch the contract asks for — `on - // VectorStoreException` — would miss every flush failure if the raw type - // leaked. Every other method here translates; this one has to as well. - final store = QdrantVectorStore(); - await store.initialize(tmp.path); - await write(store, 5); + test( + 'a failure surfaces as VectorStoreException, not QdrantException', + () async { + // The type matters as much as the throw. `QdrantException` is not a + // `VectorStoreException` and is not exported from this package's barrel, + // so a caller writing the catch the contract asks for — `on + // VectorStoreException` — would miss every flush failure if the raw type + // leaked. Every other method here translates; this one has to as well. + final store = QdrantVectorStore(); + await store.initialize(tmp.path); + await write(store, 5); - // Pull the shard directory out from under the open store. The engine - // then fails writing `mutable_id_tracker.mappings` — the same file the - // #492 report could not read on its device. - Directory('${tmp.path}/$storeDirName').deleteSync(recursive: true); + // Pull the shard directory out from under the open store. The engine + // then fails writing `mutable_id_tracker.mappings` — the same file the + // #492 report could not read on its device. + Directory('${tmp.path}/$storeDirName').deleteSync(recursive: true); - await expectLater(store.flush(), throwsA(isA())); - try { - await store.close(); - } catch (_) {} - }); + await expectLater(store.flush(), throwsA(isA())); + try { + await store.close(); + } catch (_) {} + }, + skip: Platform.isWindows ? 'deleteSync fails on the mmapped shard' : null, + ); test('is quiet after close, and when called twice', () async { final store = QdrantVectorStore(); diff --git a/packages/flutter_gemma_rag_sqlite/CHANGELOG.md b/packages/flutter_gemma_rag_sqlite/CHANGELOG.md index 7ffbeca96..900dae6c2 100644 --- a/packages/flutter_gemma_rag_sqlite/CHANGELOG.md +++ b/packages/flutter_gemma_rag_sqlite/CHANGELOG.md @@ -1,5 +1,5 @@ ## 1.4.0 -- Add `flush()`; on web it drains the IndexedDB VFS, which never synced on commit (#492). +- Add `flush()`; on web it drains IndexedDB — only partly on sqlite3 >=3.4 (#492). ## 1.3.1 - Fix web inserts failing on a numeric metadata value or a missing number field. diff --git a/packages/flutter_gemma_rag_sqlite/lib/src/web_sqlite_vector_store.dart b/packages/flutter_gemma_rag_sqlite/lib/src/web_sqlite_vector_store.dart index e6ec12a39..31cc76496 100644 --- a/packages/flutter_gemma_rag_sqlite/lib/src/web_sqlite_vector_store.dart +++ b/packages/flutter_gemma_rag_sqlite/lib/src/web_sqlite_vector_store.dart @@ -25,13 +25,6 @@ import 'package:sqlite3/wasm.dart'; /// The vec0 table is created **lazily** on the first [addDocument] (so the /// embedding dimension can be learned), or recovered from an existing table on /// [initialize]. -/// Which virtual filesystem the web store ended up on. -/// -/// The three differ in durability, not in speed, so [WebSqliteVectorStore.flush] -/// has to know which one it is talking to: one needs draining, one is already -/// on disk, and one cannot persist at all and must say so. -enum _WebPersistence { opfs, indexedDb, inMemory } - class WebSqliteVectorStore implements VectorStoreRepository { static const String _tableName = 'vec_documents'; @@ -150,6 +143,20 @@ class WebSqliteVectorStore implements VectorStoreRepository { WasmSqlite3 sqlite3, String databasePath, ) async { + // A re-initialize overwrites `_idb` below. Drain the previous VFS first: + // dropping it with pages still queued orphans them, and once the field is + // gone nothing else holds a reference that could close it. + final previous = _idb; + _idb = null; + _persistence = _WebPersistence.inMemory; + if (previous != null) { + try { + await previous.close(); + } catch (e) { + gemmaLog('[WebVectorStore] could not drain the previous VFS: $e'); + } + } + // OPFS first — only available in a dedicated web worker; on the main // isolate (the usual Flutter web context) it throws, so we fall through. try { @@ -524,7 +531,28 @@ class WebSqliteVectorStore implements VectorStoreRepository { // wait for a sync either way"), and `open()` describes its writes as // asynchronous "without any durability guarantees. You can invoke flush". // That `flush` is this call. - if (!_isInitialized && _db == null) return; + // + // HOW COMPLETE the drain is depends on the sqlite3 version, and this + // package allows both sides of a regression: + // * < 3.4.0 — `flush()` queues a marker behind the running batch and + // awaits it. A true fence. + // * >= 3.4.0 — `flush()` returns immediately whenever a write batch is + // already in flight, which is the ordinary state right after indexing. + // Upstream commit 11be8acb ("Optimize indexeddb flush on idle") took + // the marker out; its own doc still promises to await. Measured: zero + // event-loop turns on 3.5.2 against seven on 3.3.3. + // + // The exposure is bounded, which is why this is documented rather than + // worked around here: the VFS streams every write into IndexedDB as it + // happens, so what an early return misses is the batch in flight, not the + // index. `close()` is the strong drain on web — it queues behind the + // running batch on every version. + // + // Not gated on `_isInitialized`: a re-initialize that threw leaves this + // store uninitialized while the previous run's VFS still holds unwritten + // pages, and returning quietly there would report success over them. + // Nothing set up at all is the one case that stays quiet, per the contract. + if (_sqlite3 == null && _idb == null) return; switch (_persistence) { case _WebPersistence.indexedDb: try { @@ -533,8 +561,12 @@ class WebSqliteVectorStore implements VectorStoreRepository { throw VectorStoreException('Failed to flush the IndexedDB store', e); } case _WebPersistence.opfs: - // Synchronous access handles: the write reached storage before the - // statement returned, so there is nothing left to drain. + // Nothing to drain, but not for the reason it looks like: an OPFS + // write is NOT durable merely because `writeDart` returned. What makes + // it durable is that SQLite calls `xSync` on commit and this VFS's + // `xSync` is `syncHandle.flush()`. That holds while nothing sets + // `PRAGMA synchronous=OFF` — this store sets no pragma, so the default + // (FULL) applies and every autocommit statement syncs. break; case _WebPersistence.inMemory: // Neither OPFS nor IndexedDB was available, so this store cannot @@ -559,24 +591,29 @@ class WebSqliteVectorStore implements VectorStoreRepository { // `_sqlite3` is named here because the failed-initialize path clears `_db` // but not the WASM instance or the VFS it registered. if (_db == null && _sqlite3 == null && !_isInitialized) return; + // Settle every field BEFORE the await, not in a `finally` after it. While + // that await ran, `isInitialized` still reported true over an + // already-closed database, and a `flush()` landing in that window reached + // a VFS that was closing underneath it. + final db = _db; + final idb = _idb; + _db = null; + _sqlite3 = null; + _idb = null; + _persistence = _WebPersistence.inMemory; + _isInitialized = false; + _detectedDimension = null; try { - _db?.close(); - // Drain before dropping the VFS, so the contract's "close persists too" - // is true here as well. `IndexedDbFileSystem.close()` awaits the pending - // writes; without it a caller that only ever calls close() on web loses - // the tail of its index — the same shape of loss #492 reported natively. + db?.close(); + // The strong drain on web. `IndexedDbFileSystem.close()` queues a close + // item BEHIND the running batch and awaits its completer, so unlike + // `flush()` it is a true fence on every sqlite3 version. Without it a + // caller that only ever calls close() loses the tail of its index. // Best-effort by design: close() is a cleanup path callers usually // cannot act on, and `flush()` is the call that reports failure. - await _idb?.close(); + await idb?.close(); } catch (e) { gemmaLog('[WebVectorStore] close() could not drain the VFS: $e'); - } finally { - _db = null; - _sqlite3 = null; - _idb = null; - _persistence = _WebPersistence.inMemory; - _isInitialized = false; - _detectedDimension = null; } } @@ -590,3 +627,10 @@ class WebSqliteVectorStore implements VectorStoreRepository { return buffer.buffer.asUint8List(); } } + +/// Which virtual filesystem the web store ended up on. +/// +/// The three differ in durability, not in speed, so [WebSqliteVectorStore.flush] +/// has to know which one it is talking to: one needs draining, one is already +/// on disk, and one cannot persist at all and must say so. +enum _WebPersistence { opfs, indexedDb, inMemory } From 6975c73c7d821b91914b3745511d47f8ba57416a Mon Sep 17 00:00:00 2001 From: Sasha Denisov Date: Sat, 12 Sep 2026 20:59:21 +0200 Subject: [PATCH 3/5] fix(rag): keep flush quiet mid-initialize, and our own errors unwrapped MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three findings from a review of the previous commit. - Web flush() threw the in-memory error while initialize() was still running: `_sqlite3` is set before a VFS is chosen, so the guard let the call through while `_persistence` was unset. The guard is now "not initialized and no IndexedDB VFS", and a re-initialize no longer clears `_persistence` up front. - `_rethrow` wrapped the package's own QdrantException a second time, burying "not written by this package" under "unexpected error" and dropping QdrantShardLockedException's type. Ours are rethrown as they are. The foreign-shard test now checks the message; removing the fix fails it. - The example's sqlite3 dev-dep goes back to ^3.3.0. The previous commit said the example lock pinned 3.3.3; it did not — it already resolved 3.5.2, and only the root workspace lock is on 3.3.3. The bump changed nothing. --- packages/flutter_gemma/example/pubspec.yaml | 5 +---- .../lib/src/qdrant_edge_client.dart | 4 ++++ .../test/qdrant_lifecycle_test.dart | 13 ++++++++++++- .../lib/src/web_sqlite_vector_store.dart | 15 +++++++++------ 4 files changed, 26 insertions(+), 11 deletions(-) diff --git a/packages/flutter_gemma/example/pubspec.yaml b/packages/flutter_gemma/example/pubspec.yaml index 5701bfab8..47ae16cc3 100644 --- a/packages/flutter_gemma/example/pubspec.yaml +++ b/packages/flutter_gemma/example/pubspec.yaml @@ -99,10 +99,7 @@ dependency_overrides: dev_dependencies: # The web store integration test opens the same IndexedDB VFS the store # uses, to plant a corpus the store cannot read. - # ^3.5.2, not the package's own ^3.3.0 floor: the single workspace lock - # otherwise resolves 3.3.3, and the web suite would exercise a flush() that - # consumers resolving latest do not get (upstream regression in 3.4.0). - sqlite3: ^3.5.2 + sqlite3: ^3.3.0 integration_test: sdk: flutter flutter_test: diff --git a/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_edge_client.dart b/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_edge_client.dart index 4ed2abf60..fb628b2ff 100644 --- a/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_edge_client.dart +++ b/packages/flutter_gemma_rag_qdrant/lib/src/qdrant_edge_client.dart @@ -372,6 +372,10 @@ class QdrantEdgeClient { /// reached application code as a type the app could not name in a catch. /// 0.8.0-dev.3 exports it, so it can finally be caught here. static Never _rethrow(Object e) { + // Already ours: thrown on purpose inside a wrapper's `try` (openExisting's + // "not written by this package"). Re-wrapping it buried that message + // under "unexpected error" and lost QdrantShardLockedException's type. + if (e is QdrantException) throw e; if (e is qe.ShardLockedEdgeException) { throw QdrantShardLockedException( 'The shard is already open elsewhere (its write-ahead log is held by ' diff --git a/packages/flutter_gemma_rag_qdrant/test/qdrant_lifecycle_test.dart b/packages/flutter_gemma_rag_qdrant/test/qdrant_lifecycle_test.dart index 1bc6a85a0..993d61675 100644 --- a/packages/flutter_gemma_rag_qdrant/test/qdrant_lifecycle_test.dart +++ b/packages/flutter_gemma_rag_qdrant/test/qdrant_lifecycle_test.dart @@ -538,7 +538,18 @@ void main() { addTearDown(store.close); await expectLater( store.initialize(tmp.path), - throwsA(isA()), + throwsA( + isA().having( + (e) => '$e', + 'message', + // The reason must survive the trip out — `_rethrow` used to wrap + // this, our own exception, again as an "unexpected error". + allOf( + contains('not written by this package'), + isNot(contains('unexpected error')), + ), + ), + ), reason: 'a shard it had just loaded was reported as an empty store', ); await expectLater(store.getStats(), throwsA(isA())); diff --git a/packages/flutter_gemma_rag_sqlite/lib/src/web_sqlite_vector_store.dart b/packages/flutter_gemma_rag_sqlite/lib/src/web_sqlite_vector_store.dart index 31cc76496..e94a78f25 100644 --- a/packages/flutter_gemma_rag_sqlite/lib/src/web_sqlite_vector_store.dart +++ b/packages/flutter_gemma_rag_sqlite/lib/src/web_sqlite_vector_store.dart @@ -145,10 +145,11 @@ class WebSqliteVectorStore implements VectorStoreRepository { ) async { // A re-initialize overwrites `_idb` below. Drain the previous VFS first: // dropping it with pages still queued orphans them, and once the field is - // gone nothing else holds a reference that could close it. + // gone nothing else holds a reference that could close it. `_persistence` + // is left for a branch below to set: clearing it here made a flush() that + // landed mid-initialize throw the in-memory error. final previous = _idb; _idb = null; - _persistence = _WebPersistence.inMemory; if (previous != null) { try { await previous.close(); @@ -548,11 +549,13 @@ class WebSqliteVectorStore implements VectorStoreRepository { // index. `close()` is the strong drain on web — it queues behind the // running batch on every version. // - // Not gated on `_isInitialized`: a re-initialize that threw leaves this - // store uninitialized while the previous run's VFS still holds unwritten - // pages, and returning quietly there would report success over them. + // Not gated on `_isInitialized` alone: a re-initialize that threw leaves + // this store uninitialized while its IndexedDB VFS still holds unwritten + // pages, and returning quietly there would report success over them. Not + // on `_sqlite3` either — it is set before any VFS is chosen, so a flush + // landing mid-initialize read `_persistence` unset and threw. // Nothing set up at all is the one case that stays quiet, per the contract. - if (_sqlite3 == null && _idb == null) return; + if (!_isInitialized && _idb == null) return; switch (_persistence) { case _WebPersistence.indexedDb: try { From 6a4ce4ac7cf8b8cbefaa5f28cab9da06eb65b808 Mon Sep 17 00:00:00 2001 From: Sasha Denisov Date: Sun, 13 Sep 2026 00:14:29 +0200 Subject: [PATCH 4/5] =?UTF-8?q?chore(release):=20patch=20versions=20?= =?UTF-8?q?=E2=80=94=20core=201.8.1,=20rag=5Fqdrant=201.3.1,=20rag=5Fsqlit?= =?UTF-8?q?e=201.3.2?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- packages/flutter_gemma/CHANGELOG.md | 2 +- packages/flutter_gemma/example/pubspec.lock | 6 +++--- packages/flutter_gemma/pubspec.yaml | 2 +- packages/flutter_gemma_rag_qdrant/CHANGELOG.md | 2 +- packages/flutter_gemma_rag_qdrant/pubspec.yaml | 4 ++-- packages/flutter_gemma_rag_sqlite/CHANGELOG.md | 2 +- packages/flutter_gemma_rag_sqlite/pubspec.yaml | 4 ++-- 7 files changed, 11 insertions(+), 11 deletions(-) diff --git a/packages/flutter_gemma/CHANGELOG.md b/packages/flutter_gemma/CHANGELOG.md index 239864e67..8b477c1ec 100644 --- a/packages/flutter_gemma/CHANGELOG.md +++ b/packages/flutter_gemma/CHANGELOG.md @@ -1,4 +1,4 @@ -## 1.9.0 +## 1.8.1 - `VectorStoreRepository` gains `flush()` for stores that buffer writes in memory. - **Breaking for custom `VectorStoreRepository` implementations**: they must declare `flush()`. diff --git a/packages/flutter_gemma/example/pubspec.lock b/packages/flutter_gemma/example/pubspec.lock index ac6ea59eb..f034fe913 100644 --- a/packages/flutter_gemma/example/pubspec.lock +++ b/packages/flutter_gemma/example/pubspec.lock @@ -241,7 +241,7 @@ packages: path: ".." relative: true source: path - version: "1.9.0" + version: "1.8.1" flutter_gemma_agent: dependency: "direct main" description: @@ -290,14 +290,14 @@ packages: path: "../../flutter_gemma_rag_qdrant" relative: true source: path - version: "1.4.0" + version: "1.3.1" flutter_gemma_rag_sqlite: dependency: "direct main" description: path: "../../flutter_gemma_rag_sqlite" relative: true source: path - version: "1.4.0" + version: "1.3.2" flutter_gemma_speech: dependency: "direct main" description: diff --git a/packages/flutter_gemma/pubspec.yaml b/packages/flutter_gemma/pubspec.yaml index 1b09faad8..304ad1965 100644 --- a/packages/flutter_gemma/pubspec.yaml +++ b/packages/flutter_gemma/pubspec.yaml @@ -1,6 +1,6 @@ name: flutter_gemma description: "Run Gemma and other LLMs on-device in Flutter (Android, iOS, Web, Desktop). Multimodal vision/audio, function calling, thinking mode, GPU, embeddings, RAG." -version: 1.9.0 +version: 1.8.1 resolution: workspace homepage: https://fluttergemma.dev repository: https://github.com/DenisovAV/flutter_gemma diff --git a/packages/flutter_gemma_rag_qdrant/CHANGELOG.md b/packages/flutter_gemma_rag_qdrant/CHANGELOG.md index 9e598897b..9657380d8 100644 --- a/packages/flutter_gemma_rag_qdrant/CHANGELOG.md +++ b/packages/flutter_gemma_rag_qdrant/CHANGELOG.md @@ -1,4 +1,4 @@ -## 1.4.0 +## 1.3.1 - Add `flush()`; without it an index was lost when the process ended (#492). ## 1.3.0 diff --git a/packages/flutter_gemma_rag_qdrant/pubspec.yaml b/packages/flutter_gemma_rag_qdrant/pubspec.yaml index 3067da9e5..fcb9294ba 100644 --- a/packages/flutter_gemma_rag_qdrant/pubspec.yaml +++ b/packages/flutter_gemma_rag_qdrant/pubspec.yaml @@ -1,6 +1,6 @@ name: flutter_gemma_rag_qdrant description: "qdrant-edge on-device RAG vector store for flutter_gemma, via the official qdrant_edge UniFFI SDK. Opt-in VectorStoreRepository with payload filtering. Native platforms only (no web)." -version: 1.4.0 +version: 1.3.1 homepage: https://fluttergemma.dev repository: https://github.com/DenisovAV/flutter_gemma/tree/main/packages/flutter_gemma_rag_qdrant topics: [rag, qdrant, vector-search, embeddings, on-device] @@ -24,7 +24,7 @@ environment: dependencies: flutter: sdk: flutter - flutter_gemma: ^1.9.0 + flutter_gemma: ^1.8.1 uuid: ^4.0.0 # For the store's owned-subdir path handling (qdrant_edge_v1). path: ^1.9.0 diff --git a/packages/flutter_gemma_rag_sqlite/CHANGELOG.md b/packages/flutter_gemma_rag_sqlite/CHANGELOG.md index 900dae6c2..b3bf29c49 100644 --- a/packages/flutter_gemma_rag_sqlite/CHANGELOG.md +++ b/packages/flutter_gemma_rag_sqlite/CHANGELOG.md @@ -1,4 +1,4 @@ -## 1.4.0 +## 1.3.2 - Add `flush()`; on web it drains IndexedDB — only partly on sqlite3 >=3.4 (#492). ## 1.3.1 diff --git a/packages/flutter_gemma_rag_sqlite/pubspec.yaml b/packages/flutter_gemma_rag_sqlite/pubspec.yaml index beba775d5..4282a7446 100644 --- a/packages/flutter_gemma_rag_sqlite/pubspec.yaml +++ b/packages/flutter_gemma_rag_sqlite/pubspec.yaml @@ -1,6 +1,6 @@ name: flutter_gemma_rag_sqlite description: "SQLite vector search (sqlite-vec) on-device RAG vector store for flutter_gemma. Opt-in package; implements flutter_gemma's VectorStoreRepository." -version: 1.4.0 +version: 1.3.2 homepage: https://fluttergemma.dev repository: https://github.com/DenisovAV/flutter_gemma/tree/main/packages/flutter_gemma_rag_sqlite topics: [rag, sqlite, vector-search, embeddings, web] @@ -26,7 +26,7 @@ environment: dependencies: flutter: sdk: flutter - flutter_gemma: ^1.9.0 + flutter_gemma: ^1.8.1 sqlite3: ^3.3.0 # Native Assets hook deps — hook/build.dart fetches the per-platform # sqlite-vec (vec0) loadable extension from the native-sqlite-vec-v* GitHub From 2666958ac1982f7c99b204696e1e3ed30d6f1a45 Mon Sep 17 00:00:00 2001 From: Sasha Denisov Date: Sun, 13 Sep 2026 00:26:20 +0200 Subject: [PATCH 5/5] docs(rag): document flush(), bump pins to core 1.8.1, rag_qdrant 1.3.1, rag_sqlite 1.3.2 --- CLAUDE.md | 2 +- packages/flutter_gemma/CHANGELOG.md | 3 +-- packages/flutter_gemma/README.md | 5 +++++ .../flutter_gemma/ios/flutter_gemma.podspec | 2 +- .../flutter_gemma/macos/flutter_gemma.podspec | 2 +- packages/flutter_gemma_rag_qdrant/README.md | 8 ++++++++ packages/flutter_gemma_rag_sqlite/README.md | 8 ++++++++ website/content/docs/agent.md | 2 +- website/content/docs/embeddings-and-rag.md | 20 +++++++++++++++++++ website/content/docs/genkit.md | 2 +- website/content/docs/migration.md | 6 +++--- website/content/docs/speech.md | 2 +- 12 files changed, 51 insertions(+), 11 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 1bf5cacd0..81e0d3771 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -150,7 +150,7 @@ Core has NO pigeon (dropped at the 1.0 cut; its value types are hand-written in - **LiteRT-LM**: native libs from `native-v0.16.0` GitHub Release (LiteRT-LM pin `924e79c9`, LiteRT pin `0ff28117`). Android tarball bundles the Qualcomm QNN dispatch stack and Windows tarball bundles Intel NPU dispatch (`LiteRtDispatch.dll` + OpenVino runtime + TBB) for `PreferredBackend.npu` (Qualcomm Snapdragon / Intel LunarLake/PantherLake) — both dispatch libs are **rebuilt from the pin every release**; carrying them forward is what silently broke NPU on both platforms (see the `build-native` skill). v0.16.0: fixes the Android OpenCL per-turn memory leak (LiteRT-LM #2699, #348/#402); v0.15.0 **broke the stream-callback ABI** (4-arg → 2-arg chunk object) with no compat path, handled by a runtime probe in `stream_proxy.c`. Windows discrete GPU works again — the crash was our own dead `litert_link_capi_so` Bazel define, not an upstream regression (#2957 retracted). - **sqlite-vec**: `flutter_gemma_rag_sqlite` fetches the per-platform `vec0` loadable from the `native-sqlite-vec-v` GitHub Release (`sqlite-vec-.tar.gz` + `checksums_sqlite_vec.txt`), SHA256-verified by its `hook/build.dart`. `` names the **upstream sqlite-vec release** the bytes were built from; a letter suffix (`0.1.9-a`) is only for RE-releasing changed bytes under an already-published number. The loadables are NOT committed — `native/sqlite_vec/prebuilt/` is a maintainer override produced by `build_local.sh`, gitignored and `.pubignore`d. - **large_file_handler**: `^0.5.0` (core dep; 0.5.0 declares all 6 platforms — needed for pana platform support + the dart2wasm-clean web graph) -- **Current Version**: core `flutter_gemma` `1.8.0`, `flutter_gemma_rag_sqlite` `1.3.1`, `flutter_gemma_rag_qdrant` `1.3.0`; `flutter_gemma_litertlm` `1.6.3`, `flutter_gemma_mediapipe` `1.0.5`, `flutter_gemma_embeddings` `2.1.1`, `flutter_gemma_speech` `0.5.0`; `flutter_gemma_agent` `0.2.5`, `flutter_gemma_builtin_ai` `0.2.1`, `flutter_gemma_onnx` `0.3.3`; `genkit_flutter_gemma` `0.6.1`, `genkit_hybrid` `0.2.1` +- **Current Version**: core `flutter_gemma` `1.8.1`, `flutter_gemma_rag_sqlite` `1.3.2`, `flutter_gemma_rag_qdrant` `1.3.1`; `flutter_gemma_litertlm` `1.6.3`, `flutter_gemma_mediapipe` `1.0.5`, `flutter_gemma_embeddings` `2.1.1`, `flutter_gemma_speech` `0.5.0`; `flutter_gemma_agent` `0.2.5`, `flutter_gemma_builtin_ai` `0.2.1`, `flutter_gemma_onnx` `0.3.3`; `genkit_flutter_gemma` `0.6.1`, `genkit_hybrid` `0.2.1` - **0.15.2**: embedding unified on LiteRT C API via Dart FFI on all native platforms (Android + iOS + Desktop). Drops `localagents-rag` JVM dep on Android and the separate TFLite C 0.12.7 tarball on Desktop; `TensorFlowLiteC` pod no longer needed on iOS. Single source of truth for `TaskType.prefix` in Dart, fixes cross-platform embedding drift (#264). ## Platform-Specific Setup diff --git a/packages/flutter_gemma/CHANGELOG.md b/packages/flutter_gemma/CHANGELOG.md index 8b477c1ec..bfafaeddd 100644 --- a/packages/flutter_gemma/CHANGELOG.md +++ b/packages/flutter_gemma/CHANGELOG.md @@ -1,6 +1,5 @@ ## 1.8.1 -- `VectorStoreRepository` gains `flush()` for stores that buffer writes in memory. -- **Breaking for custom `VectorStoreRepository` implementations**: they must declare `flush()`. +- Add `VectorStoreRepository.flush()`; custom implementations must declare it (#492). ## 1.8.0 - Whisper output language on `getActiveStt` and `transcribe` (#500). diff --git a/packages/flutter_gemma/README.md b/packages/flutter_gemma/README.md index 8f69127be..d539e71c3 100644 --- a/packages/flutter_gemma/README.md +++ b/packages/flutter_gemma/README.md @@ -1720,8 +1720,13 @@ final results = await FlutterGemmaPlugin.instance.searchSimilar( mustNot: [FieldEquals(key: 'lang', value: 'fr')], ), ); + +// 5. Persist the index while the store stays open (see below) +await FlutterGemmaPlugin.instance.flushVectorStore(); // or FlutterGemma.rag.flush() ``` +**Call `flush()` after indexing.** `flutter_gemma_rag_qdrant` keeps new documents in memory until the store is flushed or closed, so an index built without either is lost when the process ends — an Android app killed in the background is the ordinary case ([#492](https://github.com/DenisovAV/flutter_gemma/issues/492)). On native `flutter_gemma_rag_sqlite` it is a no-op; on web it drains the IndexedDB storage. A store that cannot persist at all throws `VectorStoreException` instead of returning. Custom `VectorStoreRepository` implementations must declare `flush()`. + A field name is checked by the store, in `configure()`. `SqliteVectorStore` is the strict one — `^[A-Za-z][A-Za-z0-9_]*$`, and not a name `vec0` already uses (`id`, `embedding`, `content`, `metadata`, `distance`, `k`) — because the name diff --git a/packages/flutter_gemma/ios/flutter_gemma.podspec b/packages/flutter_gemma/ios/flutter_gemma.podspec index 727e85b2d..01dfa64d5 100644 --- a/packages/flutter_gemma/ios/flutter_gemma.podspec +++ b/packages/flutter_gemma/ios/flutter_gemma.podspec @@ -4,7 +4,7 @@ # Pod::Spec.new do |s| s.name = 'flutter_gemma' - s.version = '1.8.0' + s.version = '1.8.1' s.summary = 'Flutter plugin for running Gemma and other LLMs locally on iOS.' s.description = <<-DESC Core runtime for running Gemma 4, Gemma3n, Gemma 3, FastVLM, Qwen3, diff --git a/packages/flutter_gemma/macos/flutter_gemma.podspec b/packages/flutter_gemma/macos/flutter_gemma.podspec index 776457120..a745e660d 100644 --- a/packages/flutter_gemma/macos/flutter_gemma.podspec +++ b/packages/flutter_gemma/macos/flutter_gemma.podspec @@ -4,7 +4,7 @@ # Pod::Spec.new do |s| s.name = 'flutter_gemma' - s.version = '1.8.0' + s.version = '1.8.1' s.summary = 'Flutter Gemma - Run Gemma AI models locally on desktop' s.description = <<-DESC Flutter plugin for running Gemma AI models locally on macOS using LiteRT-LM. diff --git a/packages/flutter_gemma_rag_qdrant/README.md b/packages/flutter_gemma_rag_qdrant/README.md index d67aa2550..95abcfb78 100644 --- a/packages/flutter_gemma_rag_qdrant/README.md +++ b/packages/flutter_gemma_rag_qdrant/README.md @@ -32,6 +32,7 @@ Then use the unchanged RAG API: await FlutterGemmaPlugin.instance.initializeVectorStore('rag_store'); // a directory await FlutterGemmaPlugin.instance.addDocument(/* ... */); final hits = await FlutterGemmaPlugin.instance.searchSimilar(query: query, topK: 5); +await FlutterGemmaPlugin.instance.flushVectorStore(); // after indexing — see below ``` `QdrantVectorStore` also honors the payload-aware `Filter` DSL on @@ -49,6 +50,13 @@ refuses; if a schema must work on both, keep it inside sqlite's narrower set. ## Behavior notes +- **Call `flushVectorStore()` (or `FlutterGemma.rag.flush()`) after indexing.** + New points stay in the shard's in-memory segment until it is flushed or + closed. A process that ends without either — an Android app killed in the + background — loses them, and the corpus is embedded again on the next launch + ([#492](https://github.com/DenisovAV/flutter_gemma/issues/492)). `close()` + persists too, but logs a failed save; `flush()` throws it as + `VectorStoreException`. - **Cross-platform web is not supported** — `QdrantVectorStore` is native-only. - `enableHnsw` is accepted but a no-op: qdrant decides indexing internally (brute-forces below ~20k points, which is already faster than the Dart HNSW diff --git a/packages/flutter_gemma_rag_sqlite/README.md b/packages/flutter_gemma_rag_sqlite/README.md index d3a32e67f..2fb450aeb 100644 --- a/packages/flutter_gemma_rag_sqlite/README.md +++ b/packages/flutter_gemma_rag_sqlite/README.md @@ -30,6 +30,14 @@ await FlutterGemma.initialize( sorted descending, filtered by `threshold` — the same contract as the qdrant store (vec0 returns distance; the store converts `1 - distance` at the boundary). +`flush()` (`FlutterGemma.rag.flush()`) is a no-op on native: the connection +autocommits, so a statement that returned is on disk. On web it drains the +IndexedDB storage; on `sqlite3` >= 3.4.0 that drain does not wait for a write +batch already in flight (upstream +[sqlite3.dart#408](https://github.com/simolus3/sqlite3.dart/issues/408)), and +`close()` is the stronger drain. When neither OPFS nor IndexedDB is available +the store runs in memory, and `flush()` throws `VectorStoreException`. + ## Declared-column filters `vec0` filters KNN only on **declared, typed metadata columns** (not arbitrary diff --git a/website/content/docs/agent.md b/website/content/docs/agent.md index 291a79a89..c06542310 100644 --- a/website/content/docs/agent.md +++ b/website/content/docs/agent.md @@ -29,7 +29,7 @@ used below). ``` dependencies: - flutter_gemma: ^1.8.0 + flutter_gemma: ^1.8.1 flutter_gemma_agent: ^0.2.5 flutter_gemma_litertlm: ^1.6.3 # an inference engine (LiteRtLmEngine) ``` diff --git a/website/content/docs/embeddings-and-rag.md b/website/content/docs/embeddings-and-rag.md index 2f8179e1b..219935dc6 100644 --- a/website/content/docs/embeddings-and-rag.md +++ b/website/content/docs/embeddings-and-rag.md @@ -114,6 +114,9 @@ for (var i = 0; i < docs.length; i++) { ); } +// 3c. Persist what you indexed while the store stays open (see below) +await FlutterGemma.rag.flush(); + // 4. Semantic search, with optional payload-aware Filter final results = await FlutterGemma.rag.searchSimilar( query: 'quantum entanglement', @@ -132,6 +135,23 @@ final stats = await FlutterGemma.rag.stats(); await FlutterGemma.rag.clear(); ``` +### Persisting the index: `flush()` + +Call `FlutterGemma.rag.flush()` after indexing. What it does depends on the store: + +- **qdrant-edge** — required. New documents stay in memory until the store is + flushed or closed, so an index built without either is lost when the process + ends — an Android app killed in the background is the ordinary case. `close()` + persists too, but only logs a failed save; `flush()` throws it. +- **sqlite-vec, native** — a no-op: every statement is on disk when it returns. +- **sqlite-vec, web** — drains the IndexedDB storage. On `sqlite3` >= 3.4.0 it + does not wait for a write batch already in flight (an upstream regression); + `close()` is the stronger drain there. + +A store that cannot persist at all (the web in-memory fallback) throws +`VectorStoreException` rather than returning. Custom `VectorStoreRepository` +implementations must declare `flush()`. + ## The Filter API `Filter` supports `must` / `should` / `mustNot` lists of conditions: diff --git a/website/content/docs/genkit.md b/website/content/docs/genkit.md index 3a6fdf520..dfa727137 100644 --- a/website/content/docs/genkit.md +++ b/website/content/docs/genkit.md @@ -21,7 +21,7 @@ with the on-device model exactly as it would with any cloud provider. ``` dependencies: genkit_flutter_gemma: ^0.6.0 - flutter_gemma: ^1.8.0 + flutter_gemma: ^1.8.1 # Add the inference engine(s) you need: flutter_gemma_litertlm: ^1.6.3 # .litertlm models (mobile + desktop) + LiteRtEmbeddingBackend flutter_gemma_mediapipe: ^1.0.5 # .task / .bin models (mobile + web) diff --git a/website/content/docs/migration.md b/website/content/docs/migration.md index 194b09421..a52a57010 100644 --- a/website/content/docs/migration.md +++ b/website/content/docs/migration.md @@ -29,12 +29,12 @@ dependencies: ``` dependencies: - flutter_gemma: ^1.8.0 # core — always required + flutter_gemma: ^1.8.1 # core — always required flutter_gemma_litertlm: ^1.6.3 # add if you run .litertlm models (also provides LiteRtEmbeddingBackend) flutter_gemma_mediapipe: ^1.0.5 # add if you run .task / .bin models flutter_gemma_embeddings: ^2.1.1 # add if you compute embeddings (needs a backend, see above) - flutter_gemma_rag_qdrant: ^1.3.0 # add for native on-device RAG (qdrant) - flutter_gemma_rag_sqlite: ^1.3.1 # add for on-device RAG (sqlite-vec; all platforms incl. web) + flutter_gemma_rag_qdrant: ^1.3.1 # add for native on-device RAG (qdrant) + flutter_gemma_rag_sqlite: ^1.3.2 # add for on-device RAG (sqlite-vec; all platforms incl. web) ``` Pick by what you actually used in 0.16.x: diff --git a/website/content/docs/speech.md b/website/content/docs/speech.md index 773147e20..05641bdab 100644 --- a/website/content/docs/speech.md +++ b/website/content/docs/speech.md @@ -30,7 +30,7 @@ inference. ``` dependencies: - flutter_gemma: ^1.8.0 + flutter_gemma: ^1.8.1 flutter_gemma_speech: ^0.5.0 ```