Skip to content

Using Your Workspace ​

What DataNature is ​

DataNature is a lakehouse data and AI platform. It gives your organisation a workspace containing everything needed to:

  1. ingest data,
  2. transform it,
  3. govern it,
  4. serve it to SQL, BI and AI users.

All data lives as Apache Iceberg tables on S3-compatible object storage. Several engines read the same tables:

EngineUsed for
Trino ("Lakehouse")Interactive SQL, CSV export, external JDBC, dbt
Spark 4 (Connect)Notebooks, heavy dbt transforms
DN Warehouse (StarRocks)Fast serving, semantic-layer materialized views

Core concepts ​

ConceptWhat it is
WorkspaceYour organisation's isolated environment. It has a display name and a slug (e.g. acme-ab12cd). The slug shows up in host names, catalog names and SQL user names.
LakehouseThe Iceberg catalog, called lakehouse in SQL. Address tables as lakehouse.<schema>.<table>.
Schema (namespace)A folder of tables. Ingestion writes to bronze_<source>, and dbt builds silver and gold schemas.
PipelineA Connect definition that copies tables from a source database into bronze.
JobA schedule that runs pipelines, dbt models and cube refreshes in order.
dbt projectYour transformation code (SQL models). It lives in the workspace's Version Control (git).
CubeA semantic-layer definition of measures and dimensions over a gold table. It can be materialized in the DN Warehouse.
PolicyA governance rule (access, column masking or row filter) enforced by the query engines.
LakebaseThe workspace's internal PostgreSQL. It stores platform configuration. Treat it as read-only and internal. Don't build on it.

Medallion layers ​

LayerSchema namingBuilt byContents
Bronzebronze_<source> (from the pipeline's destination name)ConnectRaw copy of source tables. All columns nullable, lower-case names.
Silvere.g. default_silver, or your dbt schema namedbtCleaned, typed, de-duplicated, joined
Golde.g. default_gold / golddbtBusiness-level facts and aggregates. Cubes and dashboards should read from here.

Signing in ​

  • Every surface uses single sign-on: the portal and all tools (BI, notebooks, orchestration, query UI, catalog). Sign in once with your platform username and password.
  • Password rules on hardened installs:
    • at least 12 characters, with upper-case, lower-case, a digit and a symbol;
    • it must not contain your username.
  • Lockout: 5 wrong passwords locks the account temporarily, with the wait growing up to about 15 minutes.
    • ⚠️ A BI tool or SQL client that keeps retrying an old saved password will lock you out. Update saved passwords in your tools straight after changing yours.
  • Sessions expire after about 30 minutes idle, and at most about 10 hours. Save your work in notebooks and editors.
  • Pending account actions (a forced password change, verifying your profile) block external tools such as DBeaver or JDBC from signing in. Sign in to the portal once in a browser and complete them.
  • After a platform upgrade, if a page behaves oddly, do a hard refresh (Ctrl+Shift+R / ⌘+Shift+R). The browser may be caching the old UI.

Roles and permissions ​

Access is set per page. Each page has one of four levels:

LevelMeaning
No accessThe page is hidden
ViewRead-only
UseWork with it: run, create your own items
ManageAdminister it: settings, other people's items, enable or disable
  • The workspace owner has full access to everything.
  • Admins define custom roles in Settings → Roles and assign people in Settings → Organization.
  • Data access is separate from page access. Tables, rows and columns are governed by policies (Access & Policies). You might be able to open the Workbench and still not see a table.

Things that surprise people:

  • Semantic Models needs Use or higher. With View only, you get a 403 on cubes. This is intentional.
  • Cube materialized views (mv_* tables) can't be queried directly in SQL. Use the semantic layer.
  • If you can see pages but every save fails with something like Required: write, current: read, you've been given a viewer role. Ask an admin.

There is no in-product access request form yet. To get access, ask a workspace admin: anyone with Manage on Data & AI Security for data, or on Settings → Roles for pages.

SidebarPagePurpose
OverviewCommand CenterHealth of compute, storage, tools and workloads
RecentsThings you recently touched
ApplicationsTool launcherOpen, enable or scale tools, and view connection details
Semantic ModelsCubes, warehouse browser, query playground
MarketplaceShared assets
Data EngineeringConnectIngestion: sources, destinations, pipelines, reconciliation
JobsScheduling ingestion + dbt + cube refresh
WorkbenchDBT Editor, Workflow (DAG repo), Query Editor (SQL)
VoirChat, Twins, Skills, ConnectorsAI analytics agents (licensed)
DocuContextDocument knowledge base (licensed)
AtlasMonitoring, Runs, Alerts, Logs & AuditRun health, dbt run history, alerts, audit trail
Atlas → Data & AI SecurityGovernance: access, masking, row filters, cube security, PII guardrails
Account menuSettingsOrganization, roles, API keys, license, email

Items marked Soon in the sidebar (Streaming, Activation, ODS & Data Sharing, Secrets, Version Control) are not available yet.

Tools (Applications) ​

Each tool card on Applications has Open and Connection details. Admins also get Enable, Disable and Scale down.

ToolAddressWhat you use it for
Query engine (Trino)query.<domain>SQL over the lakehouse. Also the JDBC endpoint for DBeaver or BI tools.
BI (Superset)bi.<domain>Dashboards and charts
Orchestration (Airflow)orchestration.<domain>Viewing DAG runs and task logs. You normally drive runs from Jobs.
Notebooks (JupyterHub)notebook.<domain>Python / PySpark notebooks
Transform (Spark)transform.<domain>Spark UI. Spark itself is used from notebooks and dbt.
Dataflow (ingestion runtime)dataflow.<domain>The engine behind Connect. You rarely need to open it.
Catalog (DataHub)governance.<domain>Search, lineage and documentation. Not on every install.
DN Warehousevia WorkbenchFast serving SQL (StarRocks)
MLflow, Milvus, Redisin-cluster onlyFrom notebooks only. See BI, Voir AI & ML Tools.

Your first hour: a suggested path ​

  1. Connect → Sources → New connection. Add your source database and click Test connection.
  2. Connect → Destinations. Create a destination, e.g. crm, which writes to bronze_crm.
  3. Connect → Pipelines → + New pipeline. Pick two small tables as Full load, then Deploy. See Ingestion (Connect).
  4. Jobs → + Create job → Ingest only. Set Trigger now, or add a schedule.
  5. Workbench → Query Editor → Lakehouse. Run SELECT * FROM lakehouse.bronze_crm.<table> LIMIT 50.
  6. Workbench → DBT Editor. Write a silver model and run it. See Transformation (dbt).
  7. Jobs. Turn it into an Ingest + Transform job so the whole chain runs on a schedule.

The end-to-end path ​

 Source DB ──► Connect pipeline ──► bronze_<source>.*      (Iceberg, lakehouse)
 (Oracle/MSSQL/                         │
  MySQL/Postgres)                       ▼
                                 dbt models ──► silver / gold   (Iceberg or DN Warehouse)
                                        │
                                        ▼
                              Semantic cubes (+ optional MVs)
                                        │
             ┌──────────────┬───────────┼──────────────┬──────────────┐
             ▼              ▼           ▼              ▼              ▼
        SQL Workbench   Notebooks    Voir AI       Superset BI   External JDBC
             ▲
     Jobs schedule the whole chain.  Governance (Ranger) applies to every query path that runs as you.

Conventions used in these docs ​

  • UI labels are in bold, as they appear on screen: Data Engineering → Jobs.
  • <workspace> is your workspace slug, e.g. acme-ab12cd. <safe> is the same with dashes as underscores: acme_ab12cd.
  • <domain> is your platform's domain name. Tool addresses look like query.<domain>, or query-<platform>.<domain> on some installs.
  • ⚠️ marks a rule that has caused data loss or an outage when ignored.
  • "Contact your admin" means a workspace admin or platform operator has to act; you can't fix it from your seat.

Your install may differ. Self-hosted and air-gapped installs are upgraded on their own schedule. If a feature here doesn't match your screen, your install may be on an older release. Ask your admin which one.

Built with VitePress · DataNature, a product by Dlytica Inc.