For each Meltano installation, if you need to customize environment variables, this is done with the .env
file that is created with each new installation.
By default, Meltano shares anonymous usage data with the Meltano team using Google Analytics. We use this data to learn about the size of our user base and the specific Meltano features they are (not yet) using, which helps us determine the highest impact changes we can make in each weekly release to make Meltano even more useful for you and others like you.
If you'd prefer to use Meltano without sending the team this kind of data, you can disable tracking entirely using one of these methods:
When creating a new project, pass --no_usage_stats
to meltano init
:
meltano init PROJECT_NAME --no_usage_stats
In an existing project, disable the send_anonymous_usage_stats
setting in the meltano.yml
file:
send_anonymous_usage_stats: false
To disable tracking in all projects in one go, set the MELTANO_DISABLE_TRACKING
environment variable to True
:
# Add to `~/.bashrc`, `~/.zshrc`, etc, depending on the shell you use:
export MELTANO_DISABLE_TRACKING=True
When anonymous usage tracking is enabled, Meltano tracks the following events:
meltano init {project name}
meltano ui
meltano elt {extractor} {loader} --transform {skip, only, run}
meltano add {extractor, loader, transform, model, transformer, orchestrator}
meltano discover {all, extractors, loaders, transforms, models, transformers, orchestrators}
meltano install
meltano invoke {plugin_name} {plugin_args}
meltano select {extractor} {entities_filter} {attributes_filter}
meltano schedule add {name} {extractor} {loader} {interval}
meltano permissions grant --db {postgres, snowflake} --dry
Beyond the invocation of these commands and the identified command line arguments, Meltano does not track any other event metadata, plugin configuration, or processed data.
Finally, Meltano also tracks anonymous web metrics when browsing the Meltano UI pages.
If you want to evaluate Meltano's anonymous usage tracking strategy for yourself, you can check meltano.core.tracking.GoogleAnalyticsTracker and all the places that it is used.
Until role-based access control is implemented in Meltano, we need to prevent user editing of certain settings from the UI. View this tap-gitlab
environment variable setup example to learn how to work with this current limitation.
By default, Meltano uses a SQLite database named ./meltano/meltano.db
as its system database.
You can choose to use a different system database backend or configuration using the --database-uri
option of the meltano
command, or the MELTANO_DATABASE_URI
environment variable:
# SQLite (absolute path required, notice the `3` slashes before the path)
export MELTANO_DATABASE_URI=sqlite:////path/to/system_database.db
# PostgreSQL:
export MELTANO_DATABASE_URI=postgresql://username:password@host:port/database
The following are the environment variables currently available for customization for Flask.
Update your .env
file in your project directory with the desired customizations.
export FLASK_PROFILE = ""
export FLASK_ENV = ""
These variables are specific to Flask-OAuthlib and work with OAuth authentication with GitLab.
Update your .env
file in your project directory with the desired customizations.
# GitLab Client ID
export OAUTH_GITLAB_APPLICATION_ID = ""
# GitLab Client Secret
export OAUTH_GITLAB_SECRET = ""
For more information on how to get these from your GitLab application, check out the integration docs from GitLab.
By default, the API and Airflow services listen with following host/port combinations.
API: http://0.0.0.0:5000
Airflow: http://0.0.0.0:5010
To change the host/port configuration on which the API or Airflow server listen, update your .env
in your project directory with the following configuration:
Airflow integration
Meltano expects Airflow to be available on the same host it is currently exposed from.
Meltano doesn't support connecting to a remote Airflow instance.
# Meltano API configuration
export MELTANO_API_HOSTNAME="0.0.0.0"
export MELTANO_API_PORT="5000"
# Airflow configuration
export AIRFLOW__WEBSERVER__WEB_SERVER_HOST="0.0.0.0"
export AIRFLOW__WEBSERVER__WEB_SERVER_PORT="5010"