Databricks Connect setup¶
Although Databricks Connect is NOT required when coding in notebooks, you may find it useful when working with the datalake management commands.
In the [rootpackage]/_config/bundles/databricksbundle.yaml project bundle config, add the following configuration:
parameters:
databricksbundle:
databricks_connect:
connection:
address: 'https://dbc-123.cloud.databricks.com'
token: 'abcd123456'
cluster_id: '0416-084917-doles835'
Storing tokens and other sensitive information in YAML configs is generally not a good idea. Try moving the token to your environment variables and the .env file located in the project root:
parameters:
databricksbundle:
databricks_connect:
connection:
address: 'https://westeurope.azuredatabricks.net'
token: '%env(DBX_TOKEN)%'
cluster_id: '0416-084917-doles835'
How to test Databricks connection?¶
To test that your local configuration works properly, activate the virtual environment and run:
$ console dbx:test-connection --env=dev
--env
option.