Migrate privstack-storage from DuckDB to SQLite (privstack-db)
Details
Replace DuckDB with SQLite via privstack-db across the entire
privstack-storage crate. This is the largest piece of the DuckDB-to-
SQLite migration.
Key changes:
Cargo.toml: Replace duckdb + uuid deps with privstack-db.
error.rs: Replace duckdb::Error with rusqlite::Error + DbError variants.
Remove the Encryption error variant (encryption now at SQLCipher level).
lib.rs: Remove open_duckdb_with_wal_recovery() and apply_resource_limits()
(SQLite handles WAL recovery automatically). Update re-exports from
scan_duckdb_* to scan_db_*.
event_store.rs: Use privstack_db::open_db_unencrypted/open_in_memory.
Add open_with_conn() for shared connection support. Use
privstack_db::checkpoint(). Translate schema: VARCHAR->TEXT,
BIGINT->INTEGER, TIMESTAMP->INTEGER.
entity_store.rs (~1860 lines rewritten):
- Remove ALL application-level encryption: encryptor field,
encrypt_data_json(), decrypt_data_json(), migrate_unencrypted(),
re_encrypt_all(), base64 helpers. Encryption is now at the SQLCipher
database level.
- Remove open_with_encryptor() constructor, add open_with_conn() for
shared connection pattern.
- SQL dialect translation: VARCHAR->TEXT, BIGINT->INTEGER,
BOOLEAN DEFAULT FALSE->INTEGER DEFAULT 0, VARCHAR[]->TEXT (JSON arrays),
DOUBLE[]->TEXT (JSON arrays), is_trashed=FALSE->is_trashed=0,
information_schema->sqlite_master/PRAGMA, ILIKE->LIKE,
CHECKPOINT->PRAGMA wal_checkpoint(TRUNCATE),
list_cosine_similarity->cosine_similarity (custom fn from privstack-db),
::DOUBLE[] array literals->JSON text strings.
- Tags stored as JSON arrays via serde_json::to_string() instead of
DuckDB array literal format.
- Embeddings stored as JSON text instead of DOUBLE[] arrays.
- Compact: Replace DuckDB ATTACH/COPY FROM DATABASE/DETACH/swap with
SQLite VACUUM INTO via privstack_db::compact().
- Diagnostics: Replace duckdb_tables()/duckdb_views()/duckdb_indexes()
with sqlite_master queries and PRAGMA index_list().
- Schema init: Replace information_schema introspection with
privstack_db::table_exists/column_exists/add_column_if_not_exists.
- Register cosine_similarity custom function on connection open.
Tests: Remove encryption-specific tests (FailingEncryptor,
UnavailableEncryptor, migrate_unencrypted, re_encrypt_all). Replace
uuid::Uuid::new_v4() with atomic counter. All 97 tests pass.
FFI (privstack-ffi): Update callers — replace open_with_encryptor with
open(), remove migrate_unencrypted/re_encrypt_all calls, rename
scan_duckdb_file/compact_duckdb_file to scan_db_file/compact_db_file.
Migrate privstack-vault from DuckDB to SQLite
Details
Replace duckdb with privstack-db (rusqlite/SQLCipher) in the vault crate.
Vault blob encryption (ChaCha20-Poly1305) is preserved — vault blobs
contain secrets (cloud keys, recovery material) that need crypto
protection even within SQLCipher.
Changes:
- Replace duckdb dependency with privstack-db
- Translate SQL types: VARCHAR→TEXT, BIGINT→INTEGER
- Translate param syntax: positional ?→?N
- VaultManager: add open_with_conn() for shared connection support
- VaultManager::open() uses privstack_db::open_db_unencrypted()
- VaultManager::checkpoint() uses privstack_db::checkpoint()
- Update all 3 test files (145 tests): replace duckdb::Connection
with privstack_db::open_in_memory(), update file extensions
All 145 tests pass (32 recovery + 91 integration + 22 unit).
Note: Build may fail at this commit due to privstack-ffi still
referencing removed DuckDB vault APIs — will be resolved when FFI
is migrated.
Add privstack-db crate and migrate BlobStore from DuckDB to SQLCipher
Details
Phase 1 (Foundation):
- Add rusqlite 0.31 with bundled-sqlcipher to workspace dependencies
- Create privstack-db crate: unified SQLCipher connection management with
open_db(), rekey(), checkpoint(), compact(), and helper utilities
- Implement cosine_similarity() as a registered Rust scalar function
(replaces DuckDB's list_cosine_similarity for RAG vector search)
- Add derive_sqlcipher_key() to privstack-crypto for raw hex key formatting
- Align privstack-sync to use workspace rusqlite (resolves libsqlite3-sys
link conflict between bundled and bundled-sqlcipher features)
- 24 tests for privstack-db (encryption, rekey, FTS5, cosine similarity)
Phase 2 (BlobStore Migration):
- Replace duckdb dependency with privstack-db in privstack-blobstore
- Remove all per-entity encryption from BlobStore: encryptor field,
open_with_encryptor(), encrypt/decrypt in store/read, re_encrypt_all(),
migrate_unencrypted(), blob_entity_id() -- at-rest encryption now handled
by SQLCipher at the database file level
- Remove BlobStoreError::Encryption variant (no longer needed)
- Remove privstack-crypto dependency from blobstore
- Translate schema: VARCHAR→TEXT, BIGINT→INTEGER
- Translate SQL: positional params now use ?N syntax
- BlobStore accepts shared Arc<Mutex<Connection>> via open_with_conn()
- 36 tests pass (removed 15 encryption-specific tests, all CRUD/error tests
preserved and passing)
Note: Build may fail at this commit due to privstack-ffi still referencing
removed BlobStore APIs (open_with_encryptor, re_encrypt_all,
migrate_unencrypted) -- will be resolved when FFI is migrated.
Fix UI freeze on Setup Wizard password step due to blocking async call
Details
CompleteSetup() called _biometricService.IsAvailableAsync().GetAwaiter().GetResult()
synchronously on the UI thread, blocking it while the Task.Run-wrapped ObjC
interop checked Touch ID availability. This caused the macOS beachball spinner
after the recovery mnemonic was generated.
Fix: Convert CompleteSetup() to async (CompleteSetupAsync), properly await the
biometric availability check, and update the GoNext() call site to await it.
Get notified about new releases