Apache Spark

Apache Spark is the distributed processing engine within DataNature Data Engineering.

Apache Spark · distributed · PB-scale

• Distributed engine for petabyte-scale processing, aggregation, and feature engineering.
• Drives Silver and Gold transformations alongside dbt, integrated in the workbench.
• Runs containerised on the platform and is governed by Atlas.