Databricks

​

Setup

​

Step 1: Get Databricks connection details

To connect ClarityQ to your Databricks workspace using a service principal, you’ll need the following information:

1. Server hostname
2. HTTP path
3. Service principal Application ID
4. Service principal OAuth secret

​

Step 2: Find your Databricks connection details

​

Server Hostname

1. Log in to your Databricks workspace
2. In the sidebar, click SQL Warehouses (or Compute → SQL Warehouses)
3. Click on your SQL warehouse name
4. Go to the Connection Details tab
5. Copy the Server hostname (e.g., your-workspace.cloud.databricks.com)

​

HTTP Path

1. In the same Connection Details tab of your SQL warehouse
2. Copy the HTTP path (e.g., /sql/1.0/warehouses/abc123def456)

​

Step 3: Create service principal credentials

Create a dedicated service principal for ClarityQ (recommended for production environments):

​

Create service principal for ClarityQ

1. In your Databricks workspace, go to Settings → Identity and access
2. Click the Service principals tab
3. Click Add service principal
4. Enter name: clarityq-service-principal
5. Click Add
6. Click on the newly created service principal
7. Copy the Application ID - you’ll need this for ClarityQ connection (UUID format like 065e73cc-a713-4ae0-8212-ba7f98e32bee)
8. Go to the Secrets tab
9. Click Generate secret
10. Set expiration date (recommended for security)
11. Click Generate and copy the secret immediately
12. In the same service principal details page, go to Permissions tab
13. Click Grant access or Add permissions
14. Search for and select Service Principal: User
15. Save the changes
16. This allows the service principal to execute queries (required for ClarityQ)

​

Grant data access permissions to ClarityQ

Note: These steps require workspace admin or account admin permissions. If you don’t have these permissions, ask your Databricks administrator to perform these steps. Grant SELECT permissions to your ClarityQ service principal for the data you want to analyze: Step 1: Navigate to your catalog

1. Go to Catalog in your Databricks workspace
2. Choose the catalog containing data you want ClarityQ to access

Step 2: Grant SELECT permissions

1. Select the Permissions tab
2. Click the Grant button
3. Search for and select your clarityq-service-principal
4. Check the SELECT privilege
5. Click Grant to save

Granular permission options:

• Whole catalog: Grants ClarityQ access to all schemas and tables in the catalog
• Specific schema: Navigate to the schema level and repeat the above steps to grant access to specific schemas only
• Specific tables: Navigate to individual tables and grant access to only the tables you want ClarityQ to analyze

For metadata discovery (recommended):

• Also grant SELECT permissions on information_schema so ClarityQ can discover available tables and columns

​

Step 4: Configure connection in ClarityQ

In the ClarityQ interface, fill out the connection form with the following fields:

​

Required Fields

• Connection Name: Choose a name for this connection (e.g., “Production Databricks”)
• Server Hostname: Your Databricks server hostname (e.g., your-workspace.cloud.databricks.com)
• HTTP Path: Your SQL warehouse HTTP path (e.g., /sql/1.0/warehouses/abc123def456)
• Application ID: Your service principal Application ID (UUID format, e.g., 065e73cc-a713-4ae0-8212-ba7f98e32bee)
• OAuth Secret: The service principal OAuth secret generated in Step 3

​

Step 5: Test the connection

Verify the connection in ClarityQ to ensure:

• Databricks workspace accessibility
• SQL warehouse availability
• Token authentication
• Catalog and schema access
• Query execution capabilities

​

Connection parameters reference

​

Required parameters

Parameter	Type	Description	Example
connection_name	string	Unique identifier for this connection	Production Databricks
server_hostname	string	Databricks workspace hostname	your-workspace.cloud.databricks.com
http_path	string	SQL warehouse HTTP path	/sql/1.0/warehouses/abc123def456
application_id	string	Service principal Application ID (UUID)	065e73cc-a713-4ae0-8212-ba7f98e32bee
oauth_secret	string	Service principal OAuth secret	dapis...

​

Troubleshooting

​

Common connection issues

Authentication failed

• Verify your service principal OAuth secret is correct and not expired
• Check if the service principal has necessary permissions through Admin Settings
• Ensure the user/service principal exists and is active

Service principal access issues

• Most common cause: Service principal not added to workspace yet
  • Ask your Databricks administrator to add the service principal to the workspace
• Missing permissions: Ask your admin to use the role-based approach described above
• Insufficient permissions: Ensure ClarityQ has Service Principal User role + read access to desired data