Today, at its annual Data + AI Summit, Databricks announced that it is open-sourcing its core declarative ETL framework as Apache Spark Declarative Pipelines, making it available to the entire Apache ...
As Apache Spark becomes more widely adopted, the focus has been on creating higher-level APIs that provide increased opportunities for automatic optimization. In the talk below, Michael Armbrust, ...