📄️ Environment
Env specific variables
📄️ Extract
This step is optional and useful only if you intend to extract data from a SQL Database into
📄️ Scheduling
Scheduling Spark jobs using the native job schedulers such as Airflow fits the following cases:
📄️ Connections
Connections are defined in the connections section under the root attribute application.
📄️ Filesystem
A filesystem is the location where datasets and Starlake Data Pipeline metadata used for ingestion are stored.
📄️ Load
Load Strategy