Add category field to dataset metadata for AI-generated dataset isolation
Details
Adds a `category` column to the `_datasets_meta` schema via a new migration,
threaded through the full Rust core → FFI → SDK → Desktop stack. The
ChartDatasetGenerator now tags AI-derived datasets with category "ai-generated"
when calling ImportFromContentAsync. The SDK's IDatasetService and DatasetInfo
types expose the new optional category field with backward-compatible defaults.
Rust changes: schema migration, DatasetMeta.category field, store CRUD/mutations
updated to read/write/copy category, new set_category() helper. FFI request
structs (CreateEmptyRequest, ImportContentRequest) accept optional category.
SDK/Desktop: DatasetInfo gains Category property, IDatasetService methods
CreateEmptyDatasetAsync and ImportFromContentAsync accept optional category
parameter, DatasetService.Mutations threads it to FFI JSON requests.
Version bumps: SDK 1.55.0 → 1.56.0, Desktop 1.55.5 → 1.56.0.
Generate derived datasets for AI insight charts
Details
Instead of referencing the original dataset for chart aggregation (which
fails when the AI uses column names that don't match the underlying table),
the orchestrator now creates small generated datasets from sample data.
For each chart the AI suggests:
1. Aggregates the sample rows client-side using the specified x/y/group
columns and aggregation function (sum/avg/count/min/max)
2. Creates a new dataset via ImportFromContentAsync named
"[AI] {ChartTitle} — {DatasetName}"
3. References the generated dataset ID in the chart block
This eliminates column mismatch errors entirely — the generated dataset
has exactly the columns the chart references. Falls back to the original
dataset when the generated one can't be created.
New file: ChartDatasetGenerator.cs (~170 lines) handles client-side
aggregation and CSV generation for single and grouped chart types.
Desktop 1.55.4 → 1.55.5
Filter chart markers against chart-eligible columns for view insights
Details
When insights are generated from a SQL view, the AI often references
computed columns (e.g., billable_pct) that exist in the view result but
not in the underlying dataset. These charts fail at render time with
"column not found" errors.
Now uses a two-pass approach:
1. Parse chart markers against all analysis columns (allows column name
normalization)
2. Validate parsed charts against EffectiveChartColumns (underlying
dataset) — skip charts with invalid columns instead of emitting them
3. Strip any unparsed [CHART:] markers so raw marker text never appears
in the rendered page
Desktop 1.55.3 → 1.55.4
Improve insight generation prompts with data scientist persona
Details
Updated both cloud and local AI system prompts to frame the AI as a
senior data scientist performing exploratory data analysis. The cloud
prompt now suggests specific section structure (Data Quality, Key
Statistics, Patterns & Trends, Anomalies, Recommendations) and instructs
the AI to use sub-headings, bullet lists, and markdown tables for
well-structured output. Also instructs the model to cite specific column
names, row counts, percentages, and values for concrete analysis.
Desktop 1.55.2 → 1.55.3
Parse markdown headings, bullet lists, and numbered lists in insight pages
Details
BuildSectionBlocks now recognizes:
- ### through ###### headings → heading blocks at the correct level
- Bullet lists (-, *, +) → bullet_list blocks with grouped items
- Numbered lists (1., 2.) → numbered_list blocks with grouped items
Also relaxed chart marker column validation back to analysis columns
instead of chart-eligible columns — the prompt already guides the AI
toward using chart-eligible columns, and strict validation was rejecting
valid view-computed columns that the AI referenced in chart suggestions.
Desktop 1.55.1 → 1.55.2
Get notified about new releases