Conversational Analytics

DataNature's conversational analytics experience lets any user ask questions of governed data in natural language and receive grounded, governed answers — no SQL required. It is delivered through DataNature Voir as a business chat experience and an embedded copilot for builders.

Unlike a naive "text-to-SQL" prompt, voir-analytics is a retrieval-grounded, multi-stage reasoning pipeline. It does not ask a model to hallucinate a query against a schema it has never seen. Instead it discovers the relevant catalogs, schemas, tables, columns, and even categorical filter values through semantic search over the Business Semantics layer, then composes a query grounded in that retrieved context, validates it against the user's access scope, executes it read-only on Trino, and finally narrates and visualizes the result.

voir is exposed two ways:

• As a first-class capability inside any Voir Agent that includes a lakehouse-flavored connector.
• As a dedicated sub-agent, ie analytical-agent.

Key features

• Natural-language to governed SQL — plain-English questions become validated Trino SQL, grounded in your actual schema.
• Semantic schema discovery — vector search over catalog/schema/table/column metadata surfaces only the relevant objects, keeping prompts small and precise.
• Categorical value resolution — resolves fuzzy filter phrases ("customers in the bagmati region", "premium tier") to the exact stored values via an entity-resolution index, eliminating WHERE region = 'Bagmati' misses.
• Read-only safety guard — a SQL parser rejects any non-SELECT statement (INSERT/UPDATE/DELETE/DDL) before execution.
• Scope enforcement — when invoked inside a scoped agent, every table reference is checked against the agent's catalog/schema/table allowlist; out-of-scope SQL is rejected with SCOPE_VIOLATION.
• Automatic visualization — tabular results are offered as bar/line/pie/area charts with summary statistics (Total / Avg / Max / Min).
• Narrative reporting — the report generator summarizes a result set into an executive-readable narrative.
• Streaming answers — tokens, tool calls, and results stream to the client as NDJSON events; users see progress in real time and can interrupt.
• Full provenance — the SQL, catalog, schema, and row counts travel with the answer for audit and re-execution.

Enhancing the Analytical Agent

1. Metadata Enrichment

What it is — Metadata enrichment is the process of augmenting raw schema information (table names, column names, data types) with additional descriptive context that helps the agent reason about what the data actually means.

Why it matters — Raw schemas are often cryptic. A column named cust_seg_cd or amt_ltd tells a language model very little. Without meaning, the agent guesses — and guesses produce wrong joins, wrong filters, and wrong aggregations.

What gets enriched

• Business descriptions — a plain-language explanation of what each table and column represents.
• Synonyms / aliases — the everyday words a user might use ("revenue," "sales," "turnover") mapped to the technical column.
• Relationships — how tables connect (which keys join to which), so the agent knows valid paths between entities.
• Units, formats, and conventions — currency, timezones, date granularity, whether a flag is Y/N vs 1/0, etc.
• Usage hints — common filters, typical groupings, or caveats ("this table contains soft-deleted rows; filter on the active flag").

The payoff — Enriched metadata narrows the search space. The agent spends less effort inferring intent and more effort composing correct queries, which improves both accuracy and consistency across similar questions.

2. Vector Ingestion

What it is — Vector ingestion is the process of converting textual assets — table descriptions, column notes, example questions, documentation — into numerical embeddings and storing them so they can be retrieved by semantic similarity.

Why it matters — When a user asks a question, the agent doesn't need every table in the database — it needs the relevant ones. Keyword matching is brittle ("income" won't match "revenue"). Embeddings capture meaning, so a question about "earnings last quarter" can surface the right financial tables even when the words don't literally match.

How the flow works (conceptually)

1. Descriptive text is chunked into meaningful units.
2. Each chunk is passed through an embedding model to produce a vector.
3. Vectors are stored alongside a reference to their source (which table/column they describe).
4. At query time, the user's question is embedded and compared against the stored vectors to retrieve the most semantically similar items.

Good practices

• Keep chunks focused — one coherent idea per chunk retrieves better than a giant blob.
• Re-ingest when descriptions change, so the semantic index stays fresh.
• Include example questions as embeddings — they often match user phrasing more closely than dry schema descriptions.

3. Categorical Value Ingestion

What it is — Categorical value ingestion captures the actual distinct values that live inside low-cardinality columns — statuses, categories, regions, types, codes — and makes them available to the agent.

Why it matters — A user asks "show me all cancelled orders," but the database stores the status as CANC, Cancelled, or 3. The agent cannot know this from the schema alone. If it filters on the literal word "cancelled," the query returns nothing. Knowing the real domain values lets the agent map natural language to the exact stored representation.

What gets ingested

• The set of distinct values for each categorical column (typically only where cardinality is low enough to be meaningful).
• Optionally, a human-friendly label for each code value.

Important guardrails

• Only ingest categorical values for columns where the value set is small and stable. High-cardinality columns (IDs, free text, timestamps) should be excluded — they add noise, not signal.
• Refresh periodically, since new categories can appear over time.
• Be mindful of sensitive values; avoid ingesting anything that shouldn't be exposed to the model.

The payoff — Correct value mapping is one of the highest-leverage fixes for "the query runs but returns nothing" failures. It directly translates user vocabulary into valid WHERE-clause literals.

4. Selecting Tables to Enhance SQL Agent Performance

The core problem — As a database grows to hundreds or thousands of tables, feeding all of them to the agent becomes counterproductive:

• It overwhelms the context window.
• It increases the chance the agent picks a wrong-but-plausible table.
• It slows down every query and raises cost.

Good table selection is therefore one of the most important levers for both accuracy and latency.

Strategies for narrowing the candidate set

1. Semantic retrieval first — use the vector index (Section 2) to shortlist only the tables semantically related to the user's question. Instead of "all tables," the agent reasons over a focused handful.
2. Relevance scoring / ranking — rank candidate tables by a combination of semantic similarity, historical usage, and metadata quality, then pass only the top-N to the agent.
3. Curate a high-value subset — not all tables are equal. Prioritize the "golden" tables that answer most real questions — well-documented, clean, frequently queried — and de-prioritize staging, temporary, or deprecated tables.
4. Exclude noise explicitly — filter out audit tables, backups, internal system tables, and anything the agent should never query. A smaller, cleaner surface reduces mistakes.
5. Lean on relationships — once a primary table is chosen, use known join relationships to pull in only the connected tables needed to answer, rather than the whole schema.

Practical guidelines

• Quality over quantity — a few well-described, correctly selected tables beat a huge unfiltered dump every time.
• Tune N — too few tables and you miss the answer; too many and you dilute accuracy and slow the agent. Find the balance empirically.
• Measure and iterate — track which tables the agent selects vs. which it should have selected, and refine the enrichment and retrieval accordingly.
• Feedback loops — successful queries can inform future selection (this table reliably answers this kind of question).

Putting It Together

These four pieces reinforce each other:

• Metadata enrichment gives the data meaning.
• Vector ingestion makes that meaning retrievable by intent.
• Categorical value ingestion grounds natural language in real stored values.
• Table selection ensures the agent works over a focused, relevant slice of the database.

Invest in all four, and the SQL agent produces more accurate queries, fails less often on empty results, and responds faster — because it reasons over the right, well-understood, and appropriately scoped information.