Skip to main content

Install the CLI

Download

To install the starlake CLI, you need to download the starlake script from github. The script will in turn download required dependencies and copy them to the bin subdirectory.

$ mkdir $HOME/starlake
$ cd $HOME/starlake
$ curl -O hhttps://raw.githubusercontent.com/starlake-ai/starlake/master/distrib/starlake.sh
$ chmod +x starlake.sh
$ ./starlake.sh
$

This will download starlake dependencies and display commands supported by the CLI.

Starlake Version 0.5.2
Usage:
starlake [command]
Available commands =>
bqload
jdbcload
esload
extract
splitfile
import
infer-schema
jobs2gv
kafkaload
load
metrics
parquet2csv
transform
watch
xls2yml
yml2ddl
yml2gv
yml2xls

The following files and folders should now have been created.


.
├── starlake.sh
├── version.info
├── bin
│ └── spark
│ ├── ...
│ ├── conf
│ │ ├── ...
│ │ ├── log4j2.properties
│ ├── jars
│ │ ├── ...
│ │ ├── spark-3.1-bigquery-0.27.0-preview.jar
│ │ ├── starlake-spark3_2.12-0.5.2-assembly.jar
└── └── └── ...

note

Any library you may need (Snowflake or Oracle client) will need to copied in the bin/spar/jars folder.

That's it ! We now need to create a new project.

Specific Version

To install a specific version (0.5.2 in the example below), set the COMET_VERSION environment variable before running starlake. Subsequent runs will use this version:

$ COMET_VERSION=0.5.2 ./starlake.sh # version 0.5.3 will be installed
$ ./starlake.sh ... # version 0.5.3 will be running