📄️ Bootstrap project
Create sample project
📄️ Infer schemas
In this section you will learn how to describe the samples data created when bootstrapping a new project.
📄️ Extract
This step is optional. If you are not interested in extracting data from a relational database, you can skip this step.
📄️ Load
In this section you will learn how to load and transform data using the samples files created when bootstrapping a new project and the [schemas
📄️ Transform
Now that our file is successfully loaded and available as a table, we usually need to join them and expose KPIs.
📄️ Lineage
Entity relations
📄️ Access Control
Once loaded in the data warehouse, the data is accessible to all users. You may want to restrict access to some tables or rows to some users or groups of users.
📄️ Orchestration
Now that we have seen how to load and transform data, we will see how to orchestrate the tasks using the starlake command line tool.
📄️ About Metrics
During ingestion, Starlake may produce metrics for any attribute in the dataset. Currently, only top level attributes are supported.
📄️ Expectations
Expectations are available for your load and transform jobs. They are used to validate the data that is being loaded into your data warehouse. Expectations are defined in the expectations section of your job configuration file and
📄️ Documentation
To build your online documentation, you can use the starlake CLI to generate a static website from your markdown files.