Changelog

Replace 4 separate database opens (vault.db, blobs.db, entities.db,

events.db) with a single shared connection to privstack.db. All stores

(VaultManager, BlobStore, EntityStore, EventStore) now use

open_with_conn() to share one Arc<Mutex<Connection>>.

datasets.db remains separate and unchanged.

Updated both init_core() and init_with_plugin_host_builder() cfg-gated

init functions. Updated privstack_db_diagnostics() and

privstack_compact_databases() to reflect the new file layout — scanning

privstack.db + datasets.db instead of the old 5-file set.

Phase 7 (Consolidation — partial):

• Remove duckdb workspace dependency from Cargo.toml entirely
• Update all FFI database file extensions: .duckdb → .db
• Update FFI comments: DuckDB → SQLite throughout
• Full workspace compiles cleanly with zero duckdb references
• All tests pass except 2 pre-existing P2P relay timing tests

(dht_sync_code_bidirectional, multi_entity_divergence_real_p2p)

which are network-timing-sensitive and unrelated to the storage

migration

Remaining Phase 7 work (separate commit):

• Consolidate 5 separate .db files into 2 (privstack.db + datasets.db)

with shared connections — requires reworking the FFI startup flow

• Implement one-time DuckDB→SQLite data migration for existing users
• Wire up SQLCipher-authenticated startup flow

Phase 6 of the DuckDB -> SQLite migration. This is the most complex

migration because datasets used DuckDB for OLAP features.

Key changes:

• Replace duckdb dependency with privstack-db + csv crate
• CSV import: replace DuckDB's read_csv_auto() with Rust-side csv crate

parsing with automatic type inference (Integer -> Float -> Text widening)

• Column introspection: replace information_schema.columns with

PRAGMA table_info()

• ALTER COLUMN SET DATA TYPE: replaced with table rebuild pattern

(create new table -> copy with CAST -> drop old -> rename) since

SQLite doesn't support ALTER COLUMN

• DESCRIBE SELECT: replaced with stmt.columns() using rusqlite's

column_decltype feature for type introspection

• ILIKE -> LIKE (SQLite's LIKE is case-insensitive for ASCII by default)
• BOOLEAN columns stored as INTEGER (0/1) with conversion in read/write
• Dry-run mutations use SAVEPOINT/ROLLBACK TO instead of DuckDB's

explicit BEGIN/ROLLBACK transaction pattern

• DDL types normalized: VARCHAR->TEXT, BIGINT->INTEGER, BOOLEAN->INTEGER
• open_datasets_db() removed; replaced by privstack_db::open_db_unencrypted()
• Added open_with_conn() constructor for external connection management

All 73 tests pass, zero warnings. The privstack-ffi crate (which depends

on this crate) compiles cleanly.

Replace DuckDB with SQLite via privstack-db across the entire

privstack-storage crate. This is the largest piece of the DuckDB-to-

SQLite migration.

Key changes:

Cargo.toml: Replace duckdb + uuid deps with privstack-db.

error.rs: Replace duckdb::Error with rusqlite::Error + DbError variants.

Remove the Encryption error variant (encryption now at SQLCipher level).

lib.rs: Remove open_duckdb_with_wal_recovery() and apply_resource_limits()

(SQLite handles WAL recovery automatically). Update re-exports from

scan_duckdb_* to scan_db_*.

event_store.rs: Use privstack_db::open_db_unencrypted/open_in_memory.

Add open_with_conn() for shared connection support. Use

privstack_db::checkpoint(). Translate schema: VARCHAR->TEXT,

BIGINT->INTEGER, TIMESTAMP->INTEGER.

entity_store.rs (~1860 lines rewritten):

• Remove ALL application-level encryption: encryptor field,

encrypt_data_json(), decrypt_data_json(), migrate_unencrypted(),

re_encrypt_all(), base64 helpers. Encryption is now at the SQLCipher

database level.

• Remove open_with_encryptor() constructor, add open_with_conn() for

shared connection pattern.

• SQL dialect translation: VARCHAR->TEXT, BIGINT->INTEGER,

BOOLEAN DEFAULT FALSE->INTEGER DEFAULT 0, VARCHAR[]->TEXT (JSON arrays),

DOUBLE[]->TEXT (JSON arrays), is_trashed=FALSE->is_trashed=0,

information_schema->sqlite_master/PRAGMA, ILIKE->LIKE,

CHECKPOINT->PRAGMA wal_checkpoint(TRUNCATE),

list_cosine_similarity->cosine_similarity (custom fn from privstack-db),

::DOUBLE[] array literals->JSON text strings.

• Tags stored as JSON arrays via serde_json::to_string() instead of

DuckDB array literal format.

• Embeddings stored as JSON text instead of DOUBLE[] arrays.
• Compact: Replace DuckDB ATTACH/COPY FROM DATABASE/DETACH/swap with

SQLite VACUUM INTO via privstack_db::compact().

• Diagnostics: Replace duckdb_tables()/duckdb_views()/duckdb_indexes()

with sqlite_master queries and PRAGMA index_list().

• Schema init: Replace information_schema introspection with

privstack_db::table_exists/column_exists/add_column_if_not_exists.

• Register cosine_similarity custom function on connection open.

Tests: Remove encryption-specific tests (FailingEncryptor,

UnavailableEncryptor, migrate_unencrypted, re_encrypt_all). Replace

uuid::Uuid::new_v4() with atomic counter. All 97 tests pass.

FFI (privstack-ffi): Update callers — replace open_with_encryptor with

open(), remove migrate_unencrypted/re_encrypt_all calls, rename

scan_duckdb_file/compact_duckdb_file to scan_db_file/compact_db_file.

Replace duckdb with privstack-db (rusqlite/SQLCipher) in the vault crate.

Vault blob encryption (ChaCha20-Poly1305) is preserved — vault blobs

contain secrets (cloud keys, recovery material) that need crypto

protection even within SQLCipher.

Changes:

• Replace duckdb dependency with privstack-db
• Translate SQL types: VARCHAR→TEXT, BIGINT→INTEGER
• Translate param syntax: positional ?→?N
• VaultManager: add open_with_conn() for shared connection support
• VaultManager::open() uses privstack_db::open_db_unencrypted()
• VaultManager::checkpoint() uses privstack_db::checkpoint()
• Update all 3 test files (145 tests): replace duckdb::Connection

with privstack_db::open_in_memory(), update file extensions

All 145 tests pass (32 recovery + 91 integration + 22 unit).

Note: Build may fail at this commit due to privstack-ffi still

referencing removed DuckDB vault APIs — will be resolved when FFI

is migrated.