# DOCUMENTATION STRUCTURE
Source: https://docs.clarityq.ai/DOCUMENTATION_STRUCTURE
# ClarityQ Documentation Structure
## 1. Get Started
* What is ClarityQ
* Connect Your Data Warehouse *(overview + links to individual warehouse guides)*
* Build Your Context Layer
* Your First Chat
* Best Practices
## 2. Data Warehouse Connections
* BigQuery
* Snowflake
* Redshift
* PostgreSQL
* Databricks
* MaxCompute
## 3. Context Layer
### Overview
* What is the Context Layer
* Recommended Build Order
### Table Catalog
* What is the Table Catalog
* Setting Up Your Table Catalog
* Daily Discovery Jobs *(what automated jobs discover — new tables, columns, schema changes — and how to act on findings)*
* Building Entities from Tables
### Event Catalog
* What is the Event Catalog
* Setting Up Your Event Catalog
* Daily Discovery Jobs *(automated event/parameter discovery and how to act on changes)*
* Missing Description Wizard *(how to use the wizard to fill in missing descriptions across your catalog)*
* Syncing Events to the Semantic Catalog
### Semantic Catalog
* What is the Semantic Catalog
* Dimensions
* Metrics
* Features
* Segments
* Building Your First Component
* Validation & Status
### Memory
* Using Memory
### Skills
* Using Skills
## 4. Ask Anything
* How Ask Anything Works
* Asking Questions
* Understanding Responses
* Sharing & Collaboration
* Chat Errors & Troubleshooting
## 5. Automations
* What are Tasks
* Creating & Scheduling Tasks
* Task Delivery & Notifications
## 6. Integrations
### Slack
* Connecting Slack *(installing the ClarityQ Slack app, OAuth flow, selecting a channel)*
* Using ClarityQ in Slack *(receiving task deliveries and notifications in Slack)*
## 7. Administration
* User Management & Roles
* Organization Settings
* Filters
* Security & Compliance
## 8. Support
* FAQ
* Get Help
# Context Hub
Source: https://docs.clarityq.ai/admin/context-hub
A checklist of the setup work that makes a new workspace ready to answer questions.
The Context Hub is a single page showing what still needs doing before a workspace is ready to use. It lists the setup steps, tracks which are done, and gives you one place to upload the documents that help complete them.
It's aimed at **onboarding a new workspace**. Once a workspace is set up, you won't need it day to day.
## The Setup Tasks
The Hub tracks these steps:
| Task | What it covers |
| - | - |
| **Data warehouse connection** | The warehouse connection itself |
| **Add table documentation** | Descriptions for your tables and their columns |
| **Add tables to your Context Layer** | Choosing which tables ClarityQ can use |
| **Add event & parameter documentation** | Descriptions for your tracked events |
| **Add metric definitions** | The KPIs your team measures |
| **Provide example questions & answers** | Real questions your team asks, with the answers you expect |
Which tasks appear depends on how your workspace is set up — a workspace without an Event Catalog won't show the events task.
## Task Status
Each task shows one of four states:
| Status | What it means |
| - | - |
| **Missing** | Nothing has been provided yet |
| **Processing** | ClarityQ is working through what you supplied |
| **Partial** | Some of it is done, and there's more to review |
| **Completed** | Nothing further is needed |
A **Partial** task offers a shortcut to review what's left — for table and event documentation, that takes you straight into the wizard that fills the gaps.
## Uploading Files
Tasks can be completed by uploading what you already have — a metrics spreadsheet, a glossary document, a data dictionary. The Hub has a files panel where you can upload these, see what's attached to each task, and remove files you no longer need.
This is separate from [Files](/context-layer/files) in the Context Layer: Hub files are there to get the workspace set up, not to be used when answering questions.
## Related
* [Build Your Context Layer](/get-started/build-your-context-layer) — what you're setting up and why
* [Recommended Build Order](/context-layer/building/recommended-build-order) — the order to tackle it in
# Filters
Source: https://docs.clarityq.ai/admin/filters
Org-wide and table-level rules that scope every query ClarityQ runs against your warehouse.
Filters let your team set rules that ClarityQ applies automatically when it queries your data — for example, *"always exclude test accounts"* or *"restrict this table to the past 24 months"*. They keep answers consistent across every chat and dashboard, without requiring each user to remember the rules.
ClarityQ supports two kinds of filters.
## Mandatory Filters
A mandatory filter is a SQL condition attached to a table in the [Table Catalog](/context-layer/table-catalog/what-is-the-table-catalog). It scopes every question that reaches that table, so nobody has to remember the rule.
It takes effect in two ways:
* **In components built on the table** — when a metric, segment, or entity is built, the filter is written into its SQL. Every answer using that component carries it.
* **In queries written against the raw table** — the filter travels with the table's definition, and the agent applies it and tells you which filters it used in that answer.
Common uses:
* Excluding internal accounts, test users, or staff emails.
* Restricting time ranges (for example, only the last 24 months of events).
* Enforcing tenancy on multi-tenant tables so each product team sees only its own slice.
### Setting a Mandatory Filter
You set these yourself, and you need write access to the Table Catalog — **Admins**, **Analyst Contributors**, **Analysts**, and **Integration Managers** have it.
* **On one table** — open the table in the Table Catalog, go to its settings, and use the **Mandatory Filters** tab.
* **On tables you're adding** — the **Add Tables to the Context** wizard has a mandatory filters step, so you can set them as you bring tables in. Filters set there apply only to the tables being added, not to tables already in the catalog.
Write the condition on its own, without the `WHERE` keyword and without parameters:
```sql theme={null}
is_test = false
```
ClarityQ checks the condition is valid SQL and rejects it if not.
### Changing Mandatory Filters
Changing a table's mandatory filters is **not applied retroactively** — anything already built on that table keeps the filtering it had when it was created. What that means depends on the table:
* **The Events table** — Changing its mandatory filters only affects **future** feature creation. Existing features are not updated automatically and must be reviewed manually.
* **Any other table** — The change is not applied automatically to existing entities, dimensions, metrics, or segments that use the table.
Because a filter change doesn't propagate to existing components, review the impact before you make it. After changing the filters, use the [Context Builder](/context-layer/building/how-the-context-layer-builder-works) to find everything that references the table and update it as needed, so answers stay consistent and accurate.
## Question Filters
Question filters live at the workspace level and scope what the agent considers when it answers an Ask Anything question. Unlike mandatory filters, they're visible in the chat UI when relevant — for example, picking a country or app version before the agent runs the query.
People reach them through **More Filters** in the chat input.
Question filters are set up alongside the rest of your workspace configuration. Reach out to your ClarityQ contact when you want to add or change one.
## When to Use Which
Mandatory filters apply everywhere and aren't optional — use them to enforce rules everyone must follow. Question filters give analysts choices that change the scope of an answer — use them when there are several valid ways to slice the data and you want the user to pick.
# Organization Settings
Source: https://docs.clarityq.ai/admin/organization-settings
Where admins manage users, integrations, API keys, and usage for the whole organization.
**Organization Settings** is where an Admin manages everything that applies across the organization rather than to a single workspace. Open it from the settings entry in your profile menu.
It's available to **Admins** only. Other roles don't see it.
## What's In There
| Section | What you do there |
| - | - |
| **User Management** | Invite people, set their roles, and manage usage limits. See [User Management and Roles](/admin/user-management-and-roles) |
| **External Access** | Connect your Slack workspace so ClarityQ can answer and deliver there ([Connecting Slack](/integrations/slack/connecting-slack)), and turn **MCP access** on or off so members can ask ClarityQ from AI assistants ([Turning On MCP Access](/integrations/clarityq-mcp-server/connecting-ai-assistants#turning-on-mcp-access)) |
| **MCP** | Add MCP connectors, which appear to your team as Apps. See [Connecting MCP Servers](/integrations/mcp/connecting-mcp-servers) |
| **API** | Create and revoke API keys for programmatic access |
| **Usage** | Track prompts and ClarityQ Tokens across the organization. See [Usage Dashboard](/admin/usage-dashboard) |
## Working Across Workspaces
An organization can hold more than one workspace — for example, one for your consumer app and another for your B2B product. Each workspace has its own [Context Layer](/context-layer/what-is-the-context-layer), its own dashboards and automations, and its own user permissions.
To move between them, use the workspace dropdown at the top of the sidebar. It lists every workspace you have access to in the current organization.
# Security and Compliance
Source: https://docs.clarityq.ai/admin/security-and-compliance
Authentication options, access control, and how ClarityQ handles your data.
This page covers the security-relevant features available in the ClarityQ product. Certifications and policies (SOC 2, GDPR, sub-processor lists) are maintained outside the product — reach out to your ClarityQ contact for the current documentation.
## Authentication
ClarityQ uses Auth0 for identity. The following sign-in methods are supported:
* **Email and password** — Standard email-based sign-in.
* **Google sign-in** — One-click sign-in for organizations using Google Workspace.
* **SAML SSO** — Federated sign-in through your identity provider (Okta, Azure AD, OneLogin, or any SAML-compliant IdP). When enabled, ClarityQ provisions new users automatically the first time they sign in through your IdP — there's no separate invite step.
To enable SAML SSO for your organization, contact your ClarityQ team. Once configured, the **Invite User** button on the User Management page is replaced with an **SSO Enabled** badge.
## Access Control
Permissions are enforced by [role](/admin/user-management-and-roles). Every API call and UI action checks the calling user's role against the permission required by the action. Roles can be assigned org-wide and overridden per workspace.
Programmatic access uses workspace-scoped **API keys**. A key carries no role of its own and reaches only ClarityQ's public API endpoints, not the rest of the product. Keys expire after a year by default and each has a daily prompt limit. They're created, listed, rotated, and revoked under **Organization Settings → API**, which is Admin only.
**Per-user data permissions.** Roles decide which features a person can use, not which rows they can see. When your data warehouse already enforces row-level access, a connection can run each query under the asking user's own warehouse identity instead of the shared service account, so those grants, row filters and masks apply inside ClarityQ too. This is available for [Databricks](/data-warehouse/databricks#run-queries-as-the-asking-user-per-user-permissions) and [Trino](/data-warehouse/trino#run-queries-as-the-asking-user-per-user-permissions), and the two work differently. With Databricks, each person connects their own account once under **Settings → Apps**. With Trino, nothing is asked of the user — ClarityQ runs the query as them using the service connection, provided Trino allows that impersonation. Either way, results are cached per user and shared dashboards resolve per viewer.
## Data Handling
* **Warehouse connections** — ClarityQ reads from your data warehouse using the credentials you provide during [warehouse setup](/get-started/connect-your-data-warehouse). Credentials are stored encrypted and used only by the agent and discovery jobs.
* **Mandatory filters** — Apply org-wide rules that scope every query the agent runs (see [Filters](/admin/filters)).
* **User deletion** — Removing a user permanently deletes their chats, saved queries, dashboards, and other personal content. There's no recovery, so transfer ownership of anything the team still needs before deleting the account.
## Asking for More
Audit logs, IP allowlists, region selection, and similar controls aren't surfaced in the product UI today. If your organization needs any of these, reach out — we'll let you know what's available and what's on the roadmap.
# Usage Dashboard
Source: https://docs.clarityq.ai/admin/usage-dashboard
See how your organization consumes ClarityQ Tokens, by user, workspace, and type.
The Usage dashboard shows how your organization is using ClarityQ: how many prompts your team sent, how many ClarityQ Tokens they consumed, and where that consumption came from.
Find it under **Organization Settings → Usage**. It's available to **Admins**.
What you see depends on how your organization is metered. Organizations on ClarityQ Tokens see CQT alongside prompts, as described below. Organizations metered in prompts see the same page in prompts only — two summary cards rather than four, and no CQT line on the chart.
## ClarityQ Tokens (CQT)
**ClarityQ Tokens (CQT)** are the unit ClarityQ meters usage in. Every answer consumes CQT based on the work it took to produce — the models involved and any compute the agent used along the way. A short question costs less than a deep multi-step analysis.
Two counters run side by side:
* **Prompts** — how many questions were asked.
* **CQT** — how much those questions actually consumed.
Both appear throughout the dashboard, because the interesting number is usually the relationship between them: how heavy an average prompt is.
**Background** prompts aren't sent by a person. They come from ClarityQ's own automated jobs and processes — the work that keeps your workspace current behind the scenes. They consume CQT like any other prompt, but they aren't added to your prompt count, so the Background row shows CQT with no prompts against it.
Separately, usage by ClarityQ staff working inside your organization is excluded from your numbers entirely.
## Choosing a Period
The period selector at the top drives the whole page. Options are **Current billing period**, last calendar month, the last 7, 28, 60, or 90 days, or a custom range.
Everything below compares against the equivalent preceding window, so you can see whether consumption is climbing.
Usage recorded before your organization moved to ClarityQ Tokens isn't shown. A 90-day view of an organization that switched last week shows last week.
## The Summary Cards
Four cards sit above the chart:
| Card | What it tells you |
| - | - |
| **Prompts used** | Total questions asked in the period |
| **CQT used** | Total tokens consumed |
| **Avg daily burn (CQT / day)** | Your run rate — the fastest way to project the rest of the month |
| **Avg CQT per prompt** | How heavy a typical prompt is |
Each shows the previous period's value and the change between them.
## Volume Over Time
The **Volume over time** chart plots prompts as bars against CQT as a line, so you can see when the two diverge — a spike in CQT without a matching spike in prompts means the questions got heavier, not more frequent.
Turn on **Compare** to overlay the previous period.
## The Tabs
Four tabs break the same period down different ways. In each one you can search and filter, and clicking a row jumps to the **Prompts** tab filtered to that selection — the quickest way to answer "what exactly did this person run?"
* **Prompts** — every individual prompt, with its date, user, workspace, type, and CQT cost.
* **Users** — consumption per person, split by type, with each user's share of the organization. It also shows how many of your seats were actually active, meaning they sent at least one prompt.
* **Workspaces** — consumption per workspace, with a daily chart you can switch between CQT and prompts.
* **Types** — consumption per type, with a daily chart you can view in absolute terms or as a percentage mix.
### Prompt Types
A prompt's **type** records where it was sent from and who sent it:
| Type | Where it comes from |
| - | - |
| **Ask Anything** | Someone asking a question in the ClarityQ app |
| **Builder** | Someone working in the Context Builder |
| **Embedded** | Someone using ClarityQ embedded in another application |
| **Slack** | Someone asking ClarityQ in Slack |
| **API** | Your own systems calling the ClarityQ API |
| **Scheduled tasks** | An automation running on its schedule, with nobody present |
| **Diagnosis** | An investigation into an answer someone rated thumbs-down |
| **Background** | ClarityQ's own automated jobs and processes |
The **Types** tab is how you answer "where is our usage actually going?" — for example, whether scheduled tasks quietly account for more than your team's day-to-day questions.
The "% of org" column always divides by your whole organization for the period. In a filtered view those percentages won't add up to 100% — that's intentional.
## Exporting
Each tab has an **Export to CSV** button. The export matches the tab's current filters and ignores pagination, so you always get the full set rather than the page on screen.
Exports are capped at 100,000 rows. If you hit that, narrow the period or the filters.
## Related
* [Usage Limits and Top-Ups](/admin/usage-limits-and-top-ups) — setting limits and extending them
* [User Management and Roles](/admin/user-management-and-roles) — who holds the Admin role
# Usage Limits and Top-Ups
Source: https://docs.clarityq.ai/admin/usage-limits-and-top-ups
How monthly limits work, what users see when they hit one, and how to extend them.
Usage limits keep monthly consumption predictable. They work at two levels, and both reset at the start of each calendar month.
## The Two Levels
**Your organization's monthly limit** is set by ClarityQ as part of your agreement. You can't edit it in ClarityQ — instead you extend it for the current month by buying a top-up.
**Per-user limits** are yours to manage, and they're optional. When they're enabled for your organization you can set a default limit that applies to everyone, override it for individuals, and grant extra to someone who needs it.
The two counters measure slightly different things. Your organization's limit counts **everything** your org consumed, API calls included. A per-user limit counts what that person did themselves — in Ask Anything, the Context Builder, Slack, scheduled tasks, answer diagnosis, and the embed. API usage and background work don't count against an individual.
## What Users See
At **80%** of their limit, a user gets a warning in the chat telling them roughly how much is left and when it resets. They can dismiss it and keep working.
At the limit, prompts stop. The composer is disabled and a banner explains when access returns. It can't be dismissed — the only ways out are a reset, a top-up, or extra granted by an admin. When more room is made available, the block lifts on its own without a reload.
Scheduled tasks are skipped while a limit is reached, and **resume automatically** when it resets or more room is added. Nobody has to restart them.
## Admin Notifications
Admins are notified when the organization crosses **80%** of its limit, and again when it's reached. Email always goes out; the Slack message only if your organization has connected Slack. The 80% notice is your cue to top up if you expect to need more before the month ends.
## Managing Limits
Limits live under **Organization Settings → User Management**, alongside the people they apply to.
An **org usage card** at the top shows your remaining balance for the period, a breakdown of where consumption came from, and buttons to **Set Limits**, **Buy Top-up**, and view **History**. The history panel lists every limit change and top-up in the current month.
The users table gains a usage column, and — when per-user limits are on — a limit column showing each person's ceiling, any extra they've been granted, and whether they've hit it.
### Setting Per-User Limits
**Set Limits** opens a dialog where you set the **default limit per user**. It applies to everyone across all workspaces, and it does not apply to API calls.
Two options are worth knowing:
* **Exempt admins from this limit** — admins skip the per-user limit. Your organization's limit still applies to them.
* **Clear existing user overrides and top-ups** — resets everyone to the default. Usage already recorded is untouched.
To help you pick a number, the dialog shows your average usage per user over the last 30 days, how many people exceed that average, and what your heaviest user consumed.
You can't set a limit below what someone has already used this month. ClarityQ tells you who's over and what the minimum workable limit is. Limits are never applied retroactively or deferred to next month.
### Overriding One User
Use **Set usage limit** on a row to give an individual a different ceiling, or no limit at all. An override stays until you change or remove it.
## Top-Ups
There are two kinds, and they solve different problems.
### Org Top-Up — Extends Your Organization's Limit
**Buy Top-up** on the org usage card raises your organization's limit for the **current month**. It covers all usage, including API. Pick a preset amount or enter a custom one; the dialog converts it into your billing unit and shows your new limit.
Two things to know:
* **No payment is taken at the time.** ClarityQ includes it in your monthly invoice based on your consumption.
* **Top-ups expire at the end of the month.** They raise this month's ceiling, not your ongoing limit.
This button appears when your organization extends its limit manually. Some organizations are instead configured for **automatic top-ups**, in which case the card says so and the ceiling is raised for you rather than blocking.
### Grant Extra — Unblocks One Person
**Grant extra** on a user's row gives that person a one-time amount on top of their limit for the current month. It's **free of charge** and doesn't touch your organization's limit — it's how you unblock a single user without buying anything.
If they're currently paused, they can send prompts again immediately and their scheduled tasks resume on their own.
## Related
* [Usage Dashboard](/admin/usage-dashboard) — seeing where consumption comes from
* [User Management and Roles](/admin/user-management-and-roles) — roles and access
# User Management and Roles
Source: https://docs.clarityq.ai/admin/user-management-and-roles
Invite teammates, assign roles, and control who can do what across your workspaces.
User management is where you invite teammates to ClarityQ, set what each person can do, and remove access when someone leaves. It lives under **Organization Settings → User Management**.
Organization Settings is **Admin only**. No other role can open it.
## Inviting Users
If your organization signs in with **email/password or Google**, click **Invite User**. The wizard has two steps:
1. **User details** — their email, full name, and which workspaces they should have access to.
2. **Roles** — a default role for the person, and optional per-workspace overrides.
The invitee receives an email and is added to the organization once they accept.
If your organization uses **SSO**, the **Invite User** button is hidden — the **SSO Enabled** badge appears instead. New teammates are provisioned automatically the first time they sign in through your identity provider.
## Roles
Every user has an **org-wide role** that applies across all workspaces by default. You can override a user's role per workspace when their responsibilities differ in different parts of the business.
| Role | What it can do |
| - | - |
| **Admin** | Full control — manage users and roles, edit every catalog component, configure integrations, and use Ask Anything. |
| **Integration Manager** | Connect data warehouses and manage the Table Catalog. Cannot edit semantic components or chat. |
| **Analyst Contributor** | Read and write across all catalogs (Table, Event, Semantic), use Ask Anything, build dashboards and automations. |
| **Analyst** | Same as Analyst Contributor but **read-only** in the Semantic Catalog — useful when you want analysts to use definitions without editing them. |
| **Business User** | Read access to catalogs, full Ask Anything, dashboards, and automations. Cannot edit definitions. |
Roles are fixed — they can't be renamed or customized. If your organization needs a different mix of permissions, assign the closest fit or reach out for guidance.
### Context Builder Access
Editing the Context Layer happens in the [Context Builder](/context-layer/building/how-the-context-layer-builder-works), which requires write access to the Semantic Catalog. By default the following roles can open the Builder, hold the lock, and deploy changes:
* **Admin**
* **Analyst Contributor**
**Analyst**, **Integration Manager**, and **Business User** can read the published Context Layer but can't open the Builder. Per-workspace overrides apply here too — a user can have Builder access in one workspace without having it across the org.
The Builder is also gated by a feature flag per workspace. If the **Context Builder** entry doesn't appear in your sidebar even though your role permits it, contact ClarityQ to enable it for the workspace.
## Per-Workspace Overrides
When a user's role differs between workspaces, edit their row and assign a per-workspace role. The User Management table shows **+N overrides** next to anyone with non-default roles in some workspaces.
You can also tick **+Integration Manager access** on a workspace to layer warehouse and Table Catalog management on top of whatever role they hold there.
**Admin** is an organization-level role and can't be overridden per workspace. An Admin is an Admin everywhere.
## Removing Users
Deleting a user removes their access immediately. Their conversation history and private content are permanently deleted — there's no recovery.
### Transferring Their Dashboards
If the user owns public dashboards, the delete dialog tells you how many and offers to **transfer ownership** to someone else. Pick a person, or choose to delete the dashboards along with the user.
Transfer only covers dashboards the new owner can already access. Anything they can't access is transferred to **you** instead, so no dashboard is left without an owner. The dialog spells out the split before you confirm, and the confirmation tells you exactly what moved where.
Everything else still cascades. The user's conversation history and their private dashboards and items are permanently deleted, whether or not you transfer their public dashboards. This can't be undone.
## API Keys
For programmatic access — for example, calling ClarityQ from your data pipeline — generate an API key under **Organization Settings → API**.
A key is scoped to a single workspace and works only with ClarityQ's public API endpoints, such as asking a question, reading conversations, and managing scheduled tasks. It does **not** carry the permissions of whoever created it, and it can't be used to reach the rest of the product.
Keys expire after a year by default, and each one has a daily prompt limit. From the API page you can see when a key was last used and how much it has consumed, change its limit, rotate it, or revoke it.
# Ask a question
Source: https://docs.clarityq.ai/api-reference/api/ask-a-question
post /api/v1/products/{product_id}/ask
Submit a natural-language question to the ClarityQ agent. The request is processed asynchronously; use the returned conversation_id to poll for results. The optional data_partition restricts the analyzed data to a UTC date range, like the date filter in the ClarityQ app; when omitted, the product's configured default date filter is applied (the same default the ClarityQ app uses), or no date filter if the product has no default.
# Get conversation status
Source: https://docs.clarityq.ai/api-reference/api/get-conversation-status
get /api/v1/products/{product_id}/conversations/{conversation_id}/status
Retrieve the current status of a conversation.
# Get conversation summary
Source: https://docs.clarityq.ai/api-reference/api/get-conversation-summary
get /api/v1/products/{product_id}/conversations/{conversation_id}/summary
Retrieve a summary of the conversation's last assistant response, including any SQL queries that were executed and a text summary.
# Get last completed task run summary
Source: https://docs.clarityq.ai/api-reference/api/get-last-completed-task-run-summary
get /api/v1/products/{product_id}/tasks/{task_id}/last-run-summary
Retrieve the SQL results and text summary from the last completed run of a task.
# Get Prompt Usage
Source: https://docs.clarityq.ai/api-reference/api/get-prompt-usage
get /api/v1/products/{product_id}/usage/prompts
Return prompt usage for the organization that owns the given product as flat count buckets, always grouped by product. Use the repeatable breakdown_by parameter to control dimensions (user, origin, usage_type, date); by default counts are broken down by origin, aggregated over the whole window. Use usage_type to attribute prompts to the channel that produced them (TASK for scheduled tasks, ASK_ANYTHING, BUILDER, API, SLACK, EMBED, DIAGNOSIS), or task for one row per scheduled task, carrying the task's own id and name. Both are served from the metering ledger and cannot be combined with origin, and both state the organization's billing unit: an organization billed in CQT reads the cqt fields, one billed in prompts reads prompt_count and gets no cqt at all. The API key authorizes its own product, but the response covers every product in that organization so a single key gives org-wide visibility. Counts cover all prompt origins unless the origin parameter is supplied. Requires prompt usage tracking to be enabled for the organization. Dates are optional and default to the last 30 days.
Returns prompt usage for the organization that owns the given product as flat count buckets,
always grouped by product. Use the repeatable `breakdown_by` parameter to control dimensions —
`user`, `origin` (`APP`, `SLACK` or `API_KEY`), `usage_type` and `task` (both below) and `date`
(UTC calendar day). By default counts are broken down by `origin`, aggregated over the whole
window.
Your API key authorizes its own product, but the response covers **every product in that
organization**, so a single key gives org-wide visibility. Requires prompt usage tracking to
be enabled for your organization.
Both dates are optional: `end_date` defaults to today (UTC) and `start_date` to 30 days
before `end_date`, so calling with no date parameters returns the last 30 days. The response
always echoes the resolved window.
Dimension fields appear on rows only when the matching `breakdown_by` value was requested.
```text Example request theme={null}
GET /api/v1/products/{product_id}/usage/prompts
?start_date=2026-05-01&end_date=2026-05-31
&breakdown_by=user&breakdown_by=origin
```
```json Example response theme={null}
{
"start_date": "2026-05-01",
"end_date": "2026-05-31",
"breakdown_by": ["user", "origin"],
"total_prompts": 12,
"rows": [
{
"product_id": "00000000-0000-0000-0000-000000000000",
"product_name": "Acme Analytics",
"user_id": "11111111-1111-1111-1111-111111111111",
"user_email": "jane@acme.com",
"origin": "APP",
"prompt_count": 12
}
]
}
```
## Attributing usage to scheduled tasks
`origin` tells you how a prompt arrived — from the app, from Slack, or through an API key. It
does not tell you what produced it: a scheduled task runs under its owner and is reported as
`APP`, indistinguishable from that person typing a question.
Break down by `usage_type` instead to attribute usage to the channel that produced it:
| Value | What it covers |
| - | - |
| `TASK` | Scheduled task runs |
| `ASK_ANYTHING` | Questions asked in the app |
| `BUILDER` | Context layer work in the app |
| `API` | Questions sent through the API |
| `SLACK` | Questions asked in Slack |
| `EMBED` | Questions asked in an embedded analyst |
| `DIAGNOSIS` | Automatic root-cause runs on a negatively rated answer |
Combine it with `date` or `user` as usual, and add the `usage_type` query parameter to narrow
to a single value. Those seven are the values that carry prompts; `BACKGROUND` exists as a usage
type but is never a billable prompt, so filtering on it returns `422` rather than an empty
response.
```text Example request theme={null}
GET /api/v1/products/{product_id}/usage/prompts
?start_date=2026-05-01&end_date=2026-05-31
&breakdown_by=usage_type&breakdown_by=date
&usage_type=TASK
```
```json Example response theme={null}
{
"start_date": "2026-05-01",
"end_date": "2026-05-31",
"breakdown_by": ["usage_type", "date"],
"total_prompts": 5,
"unit": "PROMPTS",
"rows": [
{
"product_id": "00000000-0000-0000-0000-000000000000",
"product_name": "Acme Analytics",
"usage_type": "TASK",
"date": "2026-05-01",
"prompt_count": 4
},
{
"product_id": "00000000-0000-0000-0000-000000000000",
"product_name": "Acme Analytics",
"usage_type": "TASK",
"date": "2026-05-02",
"prompt_count": 1
}
]
}
```
### One row per scheduled task
`usage_type` tells you how much came from scheduled tasks. `breakdown_by=task` tells you **which**
task: one row each, carrying the task's own id and name.
```text Example request theme={null}
GET /api/v1/products/{product_id}/usage/prompts
?start_date=2026-05-01&end_date=2026-05-31
&breakdown_by=task
```
```json Example response theme={null}
{
"start_date": "2026-05-01",
"end_date": "2026-05-31",
"breakdown_by": ["task"],
"total_prompts": 3,
"unit": "CQT",
"total_cqt": 340,
"rows": [
{
"product_id": "00000000-0000-0000-0000-000000000000",
"product_name": "Acme Analytics",
"task_id": "task-Ab3xY9Kq2Z",
"task_name": "Yesterday's Ads ARPU Report",
"prompt_count": 2,
"cqt": 300
},
{
"product_id": "00000000-0000-0000-0000-000000000000",
"product_name": "Acme Analytics",
"task_id": "task-7Rt2Wm5Pd1",
"task_name": "Weekly Retention",
"prompt_count": 1,
"cqt": 40
}
]
}
```
A task's rows are keyed on the task itself, not on the title of the conversation each run
happened to create, so renaming a conversation — or one task running nightly — still reports one
row per task. Combine with `date` or `user` to split it further.
`task_id` matches the `id` of the task in the Tasks API, so these counts join straight onto your
own list of scheduled tasks.
`breakdown_by=task` covers scheduled task runs only — nothing else has a task to be attributed
to. A `usage_type` filter that contradicts that (anything other than `TASK`) returns `422`.
`origin` cannot be combined with `usage_type` or `task` in one request — they are measured
separately and their totals are not guaranteed to match to the prompt. Asking for both returns
`422`.
### Which number to read
A `usage_type` or `task` breakdown states your organization's billing unit in `unit`:
* **`PROMPTS`** — read `prompt_count` on each row and `total_prompts` on the response.
* **`CQT`** — read `cqt` on each row and `total_cqt` on the response. Rows still carry
`prompt_count`, but CQT is what you are billed on. Usage from before your organization moved
to CQT billing is never included, so a window that starts earlier is reported from the switch
date onwards and the response echoes the adjusted `start_date`.
`cqt` and `total_cqt` are omitted entirely for organizations billed in prompts.
# Submit External Feedback
Source: https://docs.clarityq.ai/api-reference/api/submit-external-feedback
post /api/v1/products/{product_id}/external-feedback
Store an arbitrary evaluation/feedback JSON payload from an external system. The payload is stored as-is without validation. The optional external_id and conversation_id are sender-side identifiers kept as opaque strings for later correlation.
Store an evaluation or feedback payload produced by your own tooling — for example a
pipeline that scores ClarityQ's answers against your reference data.
The `payload` field accepts **any JSON** and is stored exactly as sent, without validation,
so your format can evolve freely without breaking the integration. The optional `source`,
`external_id`, and `conversation_id` fields are kept as opaque strings you can use to
correlate feedback records with your own systems later.
```text Example request theme={null}
POST /api/v1/products/{product_id}/external-feedback
Content-Type: application/json
{
"source": "my-eval-pipeline",
"external_id": "QUESTION_42",
"conversation_id": "your-conversation-reference",
"payload": {
"passed": true,
"overall_score": 95,
"strengths": ["Accurate metric calculations"],
"weaknesses": ["Could present SQL more explicitly"]
}
}
```
```json Example response theme={null}
{
"id": "811918ff-2dd0-4106-8afd-b26a1f18f461",
"created_at": "2026-08-13T09:54:58Z"
}
```
# Submit Feedback
Source: https://docs.clarityq.ai/api-reference/api/submit-feedback
put /api/v1/products/{product_id}/conversations/{conversation_id}/feedback
Submit or replace the rating and/or comment on the conversation's latest answer. The feedback is stored on the answer exactly like the in-app rating, so API and in-app feedback are interchangeable. The conversation must be COMPLETED. The call is idempotent: a repeat call fully replaces prior feedback. At least one of rating or comment is required.
Submit or replace feedback — a thumbs rating and/or a free-text comment — on a conversation's
**latest answer**. The feedback is stored on the answer exactly like the in-app 👍 / 👎, so
feedback left through the API and through the ClarityQ app are interchangeable and appear in the
same places.
You only pass the `conversation_id`: the server resolves which answer to rate from the
conversation's latest message. The conversation must be **completed** — submitting before the
answer is ready returns `409`.
The call is **idempotent**: submitting again fully replaces the previous feedback for that
answer. Provide at least one of `rating` or `comment`; send `"rating": null` to clear a rating.
```text Example request theme={null}
PUT /api/v1/products/{product_id}/conversations/{conversation_id}/feedback
Content-Type: application/json
{
"rating": "POSITIVE",
"comment": "Matched our dashboard exactly."
}
```
```json Example response theme={null}
{
"conversation_id": "9f1c2b4a-3d5e-4f6a-8b7c-1d2e3f4a5b6c",
"rating": "POSITIVE",
"comment": "Matched our dashboard exactly.",
"updated_at": "2026-07-01T12:34:56Z"
}
```
# API Reference
Source: https://docs.clarityq.ai/api-reference/index
ClarityQ public API endpoints, authenticated via API keys.
# API Reference
The ClarityQ API allows you to interact with ClarityQ programmatically using API keys. All endpoints require authentication via the `X-API-Key` header.
**Base URL:** `https://api.clarityq.ai`
## Endpoints
Submit a natural-language question to the ClarityQ agent.
Poll for the status of an ongoing conversation.
Retrieve the summary of a completed conversation.
Get the summary of the last completed task run.
Pull org-wide prompt usage by product, broken down by user, origin and day.
# Asking Questions
Source: https://docs.clarityq.ai/ask-anything/asking-questions
How to ask effective questions using mentions, skills, date ranges, and filters.
Ask Anything supports several input tools that help you ask precise, well-scoped questions. This page covers each one and when to use it.
## Plain Language Questions
Type your question in plain English. The agent translates business language into SQL, so ask the way you'd ask a colleague on the data team.
**Examples:**
* "What was our conversion rate last week?"
* "Show me the top 10 events by volume this month"
* "Compare daily active users between iOS and Android"
## Mentions (`@`)
Type `@` to reference a specific component from your Semantic Catalog. A dropdown appears with your available metrics, features, segments, and entities — and, under a **Files** group, any [files](/context-layer/files) in your Context Layer.
Mentions ensure the agent uses the exact business definition you intend — rather than searching and potentially picking the wrong one.
**Example:** *"What is @Monthly Revenue broken down by country?"*
Mentions are the most reliable way to point the agent to the right component. Use them whenever you're referencing a specific metric, segment, or entity.
## Files in a Chat
Files work in both directions in a conversation: you can bring one in, and you can ask the agent to produce one.
### Attaching a File to Your Question
Use the attach button in the composer, or paste or drag a file straight into it. Attachments belong to **that conversation only** — they aren't added to your Context Layer and nobody else sees them.
You can attach documents, spreadsheets, images, and data files — PDF, Word, PowerPoint, Excel, CSV, JSON, Markdown, PNG, and JPEG among them.
Reach for an attachment when the file matters to this question and nothing else:
* A spreadsheet of extra data to join against your warehouse numbers
* An image of a report or dashboard you want recreated
* A document whose figures you want checked against the data
**Example:** attach a CSV of target numbers and ask *"how did we do against these targets last quarter?"*
If you find yourself attaching the same file repeatedly, it belongs in your Context Layer instead. Files there are described once and the agent finds them on its own in every session. See [Files](/context-layer/files).
### Referencing a Context Layer File
For files already in your Context Layer, type `@` and pick from the **Files** group rather than re-uploading. That points the agent at a specific file even if it might not have chosen it on its own.
### Asking the Agent to Create a File
You can ask for a file back and download it from the chat:
* *"Export this as a CSV"*
* *"Put this summary in a PDF"*
The agent generates the file and it appears in the conversation, ready to download. Files the agent creates belong to that conversation — they aren't added to the Files page.
## Skills (`/`)
Type `/` to invoke a skill — a reusable analysis methodology that guides the agent through a multi-step workflow. The agent follows the skill's instructions step by step, asking clarifying questions and validating results along the way.
**Example:** *"/funnel\_analysis"* — The agent asks which events to include, runs the analysis, and validates the output.
## Date Range
Use the date picker dropdown to scope your question to a specific time period. Presets include:
* Today, Yesterday
* This Week, Last Week
* Last 7, 14, 28, 60, 90, 180 days
* This Calendar Month, Last Calendar Month
* Custom date range
If no date range is selected, the agent uses its judgment based on the question.
## Filters
Pre-configured product filters appear above the input field. These constrain the data the agent queries — for example, filtering by platform, region, or product.
Product filters are configured by the ClarityQ team during onboarding. Contact [support@clarityq.ai](mailto:support@clarityq.ai) to add or modify filters.
## Follow-Up Questions
Ask Anything is conversational. Within the same session, you can ask follow-ups that build on previous answers:
* "Break that down by platform"
* "What about last month?"
* "Exclude test users"
* "Show me that as a line chart"
The agent retains the full context of your conversation.
## Editing Messages
If you want to refine a question without starting over, hover over your message and click the **Edit** icon. Edit is best for small corrections — clarifying wording, adjusting a mention, or fixing a filter. The agent re-runs the analysis with your updated question, and you can navigate between message versions using the arrow buttons.
# Connecting Your Data Warehouse Account
Source: https://docs.clarityq.ai/ask-anything/connecting-your-data-warehouse-account
Some connections run every query under your own data warehouse identity. Connect your account once so your warehouse applies your permissions to what you ask.
Most connections run queries as a service account, and ClarityQ's own permissions decide what you can reach. Some are configured to run each query **as the person asking**, so your data warehouse applies your own grants, row filters and masks. On those connections you connect your warehouse account to ClarityQ once, and only you can use it.
## Do I need to connect?
You will know: on a connection that runs queries as you, a new chat shows a **Connect your Databricks account** card above the message box, and an existing chat shows a notice on the message box. Click **Connect Databricks** and you come straight back to the chat afterwards, with a confirmation naming the account you connected.
To manage your connections, open **Settings → Apps** and look at the **Data Warehouse** section at the top. Only connections that run queries as you are listed; the ones that use a service account need nothing from you and do not appear. Each card shows one of:
* **Not connected** — click **Connect** before asking questions on it.
* **Connected**, with *Signed in as* your warehouse account. **Reconnect** refreshes the sign-in; **Disconnect** removes it.
* **Connected**, with a warning that your sign-in expires in a few days. **Reconnect** to keep access.
* **Sign-in expired** — your sign-in lapsed or was revoked in the warehouse. **Reconnect** to see data again.
Two days before your sign-in lapses, a banner at the top of ClarityQ reminds you to reconnect; you can dismiss it once per sign-in. If it lapses in the middle of a conversation, the message box says your sign-in has expired and offers **Reconnect**, and dashboards show a connect prompt in place of their widgets. ClarityQ never runs your question as the service account instead.
## Connecting
From the card in Ask Anything, the notice on the message box, a dashboard that asks you to connect, or from **Settings → Apps → Data Warehouse**. You are sent to your data warehouse's sign-in page. Sign in with **your own** account, the one that matches your ClarityQ email. ClarityQ refuses a sign-in from a different account.
Approve the consent screen. ClarityQ asks only for the ability to run SQL and to stay signed in; it cannot change data or settings in your warehouse.
You land back in ClarityQ with the connection marked **Connected**. Your questions, dashboards and scheduled tasks on this connection now run as you.
## What runs as you
* **Questions in Ask Anything**, including retries and corrections.
* **Dashboards**: every viewer sees the rows their own account is allowed to see, even on a dashboard someone else built.
* **Scheduled tasks you own**: they keep running as you while your sign-in is valid. If it expires, the task fails until you reconnect; your workspace admin can lengthen the sign-in on the warehouse side.
Catalog discovery and sample values are shared for the whole workspace and are collected by the service account.
## Embedded ClarityQ
Signing in to your warehouse needs a full-page redirect, which cannot run inside an embedded view. A workspace whose connection runs queries as the asking user is therefore not available when ClarityQ is embedded in another application. Ask your workspace admin if you need it there.
## Disconnecting
**Disconnect** removes ClarityQ's access to your account for that connection, and clears the results ClarityQ had cached for you on it, so dashboards stop showing rows you fetched earlier. Until you connect again, questions on it are refused. Your warehouse admin can also revoke ClarityQ from the warehouse side, which shows up here as **Reconnect required**.
# Diagnosing Answers
Source: https://docs.clarityq.ai/ask-anything/diagnosing-answers
Investigate why an answer was wrong and get a suggested fix to your Context Layer.
When an answer isn't right, a thumbs-down does more than register a complaint. If you have builder access, ClarityQ can **diagnose** what went wrong — investigating the conversation that produced the answer against your [Context Layer](/context-layer/what-is-the-context-layer) — and suggest a concrete fix you can apply on the spot.
Diagnosis is aimed at the people who maintain the model. Anyone can rate an answer, but the diagnosis flow appears only for users with **builder access**. Other users' thumbs-down ratings are still recorded to help improve accuracy.
Diagnosis runs on regular Ask Anything chats. It isn't available in Builder chats, in an [embedded](/integrations/embed/embedding-clarityq) ClarityQ, or in workspaces that answer across several other workspaces — those have no Context Layer of their own to diagnose. A thumbs-down there records your feedback as normal.
## Starting a Diagnosis
1. Give an answer a **thumbs-down** in Ask Anything.
2. In the feedback form, select the reasons that describe what went wrong (for example a logic error or a missing filter) and optionally add a note.
3. Click **Send and suggest fix**.
That hands the answer to a dedicated diagnosis agent. The agent works in the background — it can take a little while, and you're free to keep chatting while it runs. You'll be notified when the results are ready to review.
## What the Diagnosis Investigates
The diagnosis agent re-reads the conversation up to the answer you rated and inspects your Context Layer — the metrics, entities, and definitions the answer relied on. It checks whether those components exist and whether their definitions are correct, so it can tell a **fixable modeling problem** apart from a **one-off reasoning error**.
The investigation is **read-only**. It looks at your Context Layer but never changes it — any fix is applied separately, through the Builder, only if you choose to.
## What You Get Back
A diagnosis ends in one of three ways:
* **A fix to apply.** The diagnosis explains what went wrong and recommends a change to your Context Layer. Click **Fix now** to jump into the [Context Builder](/context-layer/building/how-the-context-layer-builder-works) with the suggested fix pre-loaded, or **Dismiss** to leave things as they are.
* **Nothing to fix.** The answer matched your Context Layer, so there's no definition to change — the failure was a one-off. Your feedback is still recorded.
* **Couldn't finish.** If the investigation errors out, your feedback is saved and you can **Try again**.
## Applying the Fix
**Fix now** doesn't change anything on its own — it opens the Builder pre-loaded with the suggested correction so you can review it, then release it through your draft and deploy. See [Context Layer Versioning](/context-layer/building/context-layer-versioning) for how changes are released.
This works for **versioned components** — entities, metrics, segments, and entity dimensions — which are the components the Builder edits.
### When the fix is on a feature
[Features](/context-layer/semantic-catalog/features) are managed live rather than through the Builder, so they can't be fixed from a Builder prompt. When every part of a recommended fix targets a feature, the button changes to **Fix manually** and the panel notes that features are managed manually in the Semantic Catalog.
Clicking it takes you to **Semantic Catalog → Features**, where you make the change on the feature itself. Nothing is pre-filled, so keep the diagnosis text in view as you go — it says exactly what to change.
If a fix spans a feature *and* something versioned, you get the normal **Fix now** handoff into the Builder.
## Diagnoses in Shared Conversations
When a conversation is [shared](/ask-anything/sharing-and-collaboration), any answer that was diagnosed is marked with an indicator on its rating control. Opening it shows the diagnosis and its recommended fix **read-only** — there are no Fix now, Dismiss, or Try again controls, since the shared view isn't the place to act on them.
This lets you send someone the evidence of what went wrong, not just the bad answer.
# Errors & Troubleshooting
Source: https://docs.clarityq.ai/ask-anything/errors-and-troubleshooting
Common errors in Ask Anything and how to resolve them.
Occasionally, the agent may encounter errors when answering a question. Here's what you might see and how to handle it.
## SQL Errors
The agent generates SQL based on your Semantic Catalog and question. If the generated query fails:
* **The error is displayed inline** in the conversation with details about what went wrong
* **The agent may retry** — it can detect the error, adjust the query, and re-run automatically
* **If retries fail**, the agent will explain the issue and may ask for clarification
**Common causes:**
* Referencing a column or table that doesn't exist in the warehouse
* Incorrect date formats or ranges
* Permission issues on specific tables
**What you can do:**
* Edit your question to be more specific
* Use `@` mentions to point the agent to the correct component
* Check if the relevant table is in your Table Catalog and has an approved status
## Connect Your Data Warehouse Account
On connections that run queries as the person asking, a new chat shows a **Connect your Databricks account** card above the message box, and an existing chat shows a notice on the message box, until you connect. If your sign-in expires mid-conversation, the next answer stops the same way. The agent does not retry; nothing about the question is wrong.
Click **Connect Databricks**, sign in with your own warehouse account and ask again. Connections can also be managed under **Settings → Apps → Data Warehouse**. See [Connecting Your Data Warehouse Account](/ask-anything/connecting-your-data-warehouse-account).
## Token Limit Warnings
Long conversations or very complex analyses can approach the token limit. If this happens, you'll see a warning. Start a new conversation to continue with a fresh context.
## Query Timeout
Some queries against large tables may take longer than expected. If a query times out:
* Try narrowing the date range
* Add filters to reduce the data scanned
* Check if mandatory filters are set on the table to limit the data scope
## Agent Errors
If the agent encounters an unexpected error during analysis, an error message will appear in the conversation. You can:
* **Retry** by asking the same question again
* **Rephrase** the question with more specificity
* **Start a new conversation** if the issue persists
## Aborting a Request
If the agent is taking too long or you want to change your question, click the **Stop** button (appears while the agent is working) to abort the current request. You can then edit your question or ask a new one.
## Getting Help
If you encounter persistent errors or unexpected behavior, reach out to [support@clarityq.ai](mailto:support@clarityq.ai) with:
* The question you asked
* The error message you received
* The workspace and date range you were querying
# How Ask Anything Works
Source: https://docs.clarityq.ai/ask-anything/how-ask-anything-works
What happens behind the scenes when you ask ClarityQ a question.
When you ask a question in Ask Anything, ClarityQ's AI agent goes through a structured workflow to find the most accurate answer. Understanding this workflow helps you ask better questions and interpret results more effectively.
## The Agent Workflow
Every question goes through these steps:
The agent searches your Semantic Catalog for relevant components — metrics, features, segments, and entities that match the business concepts in your question.
The agent examines each promising result in detail, reviewing its properties, dimensions, measures, and logic to determine what's relevant.
Components that will be used in the analysis are imported, similar to loading the right definitions before writing a query. If a component has clarification hooks, the agent may ask you a question before proceeding.
The agent builds and runs a SQL query against your data warehouse, applying the appropriate date range, filters, and constraints.
Results are returned with a written insight, data table, and — when the visualization toggle is on — rich visualizations including charts and full visual reports.
The agent performs the search, review, and import steps silently. You only see the final results — SQL, data, visualizations, and the written analysis.
## How the Agent Uses Your Context Layer
The quality of answers depends directly on your Context Layer:
| Context Layer Component | How the Agent Uses It |
| - | - |
| **Semantic Catalog** | Primary source for business definitions — metrics, features, segments, dimensions, and entities |
| **Table Catalog** | Falls back to raw table/column metadata when no semantic component exists |
| **Memory** | Injects business rules, naming conventions, and domain knowledge into the agent's reasoning |
| **Skills** | When invoked with `/`, follows step-by-step analysis methodologies with clarifications and validation |
## Parallel Execution
The agent runs independent operations in parallel — for example, searching for multiple concepts simultaneously or viewing several components at once. This makes complex analyses faster without sacrificing accuracy.
## When the Agent Asks for Clarification
If your question is ambiguous or the agent discovers multiple valid interpretations during its search, it will ask a clarification question before executing. For example:
* "Do you mean **daily active users** or **monthly active users**?"
* "Which region should I filter to — US only or global?"
Answer the clarification to continue. This ensures accuracy rather than guessing.
# Sharing & Collaboration
Source: https://docs.clarityq.ai/ask-anything/sharing-and-collaboration
Share conversations, send to analysts, and collaborate on data questions.
ClarityQ is built for team collaboration. You can share your conversations, request analyst reviews, and keep track of important chats.
## Sharing Conversations
Click the **Share** button in the top-right of a conversation to generate a shareable link. Anyone with the link can view the conversation in read-only mode — including the question, SQL, data tables, and visualizations.
## Starring Conversations
Click the **Star** icon to mark a conversation as a favorite. Starred conversations appear in a dedicated section for easy access.
## Send to Analyst
If you need a deeper review or want a data analyst to take over your question:
1. Click **Send to Analyst** in the conversation
2. Select an analyst from the list
3. Add an optional note explaining what you need
The conversation appears in the analyst's **Request Queue**. Once an analyst picks it up, they can:
* Review your original question and the agent's response
* Run additional queries or analysis within the same conversation
* **Approve** the result — confirming the analysis is correct
* **Decline** the result — with a mandatory explanation of what's wrong
* **Mark as Fixed** — if they resolved an issue with the original answer
You'll see the analyst's review status and any comments they leave on the conversation.
Send to Analyst is useful when you want a human expert to validate a complex analysis, investigate an unexpected result, or refine the agent's output before sharing with stakeholders.
# Understanding Responses
Source: https://docs.clarityq.ai/ask-anything/understanding-responses
What you see when ClarityQ answers your question — from reasoning to final visualization.
When you ask a question in Ask Anything, the agent works through several visible steps before arriving at a final answer. Here's what you'll see and what it means.
## Reasoning
The first thing you'll often see is the agent's **thinking** — a collapsible block that shows how the agent interprets your question. This is where it breaks down what you're asking, identifies the key concepts, and plans its approach.
Reading the thinking block is useful when you want to understand *why* the agent chose a particular approach, or when the final answer doesn't match your expectations and you want to see where the reasoning diverged.
An answer often contains several thinking blocks. Only the **first** one is expanded for you — the rest start collapsed, so the answer stays readable. Click any of them to open it.
## Searching Your Semantic Catalog
Next, the agent searches your Semantic Catalog for relevant business definitions — metrics, features, segments, and entities that match the concepts in your question. You'll see a brief note showing what the agent searched for and how many results it found.
This step is how ClarityQ connects your plain-language question to the business logic your team has defined. If you asked about "churn," the agent is looking for your organization's specific definition of churn — not guessing.
## Reviewing Semantic Components
After searching, the agent reviews the components it found. You'll see a list of the semantic items being examined — with their names, types, and descriptions. You can click any component to view its full definition in the Semantic Catalog.
The list is grouped to stay readable: skills first, then metrics, then entities, segments, and features. When the agent looks at an entity's dimensions, those dimensions appear in parentheses next to the entity they belong to rather than as separate items. [Files](/context-layer/files) the agent read appear here too, and clicking one downloads it.
This is where the agent decides which metrics, dimensions, and entities to use in its analysis. If you notice the agent picked the wrong component, you can follow up with a correction or use an `@` mention to be more specific.
## Asking for Clarification
Sometimes — especially when your question could mean more than one thing — the agent pauses and asks you a question before continuing. You'll see a card with multiple choice options or a text field.
For example, the agent might ask which time granularity you want (daily vs. weekly), or which of two similar metrics you're referring to. Answer the question and the agent continues from where it left off.
## Running Queries
Once the agent knows what to analyze, it executes one or more SQL queries against your data warehouse. For each query, you'll see:
* **A title** describing what the query does
* **The SQL code** in a collapsible block — expand it to see exactly what was run
* **Execution time and cost** — how long the query took and what it cost
* **A results table** — interactive, with sortable columns
The agent often runs multiple queries in sequence, building on earlier results. For example, it might first query overall numbers, then break them down by dimension, then compare to a previous period.
If a SQL query fails, the agent detects the error and usually retries with a corrected query automatically. You'll see both the failed attempt and the retry.
## The Final Answer
After all the analysis is complete, the agent presents its findings:
* **A written insight** summarizing what the data shows, putting numbers in context — for example, noting that a metric is "down 3.2% compared to last week" or "at its highest point in 90 days"
* **Visualizations** — when the visualization toggle is on, every numeric answer includes rich visuals. Depending on the complexity of the question, this can be a single chart or a full visual report with multiple charts, all automatically selected to best fit the data
This is the part you'll share with stakeholders or use to make decisions. Everything above it — the thinking, searching, querying — is the transparent trail showing how the agent got there.
## Working with Results
Once you have your answer, you can:
* **Ask follow-up questions** — "Break that down by country" or "What about last month?"
* **Challenge the result** — If something looks off, tell the agent. Ask it to re-check assumptions or approach the question differently.
* **Edit your question** — Hover over your message and click edit to refine it. The agent re-runs the analysis with your updated wording.
* **Check the SQL** — Expand any SQL block to verify the query logic yourself.
## Rating Responses
Rate any response using the feedback buttons — thumbs up, neutral, or thumbs down. You can optionally add a text reason. Ratings help improve ClarityQ's accuracy over time.
If you have builder access, a thumbs-down can also kick off a [Diagnosis](/ask-anything/diagnosing-answers) — an investigation into what went wrong that suggests a fix to your Context Layer.
# Creating & Scheduling Tasks
Source: https://docs.clarityq.ai/automations/creating-and-scheduling-tasks
How to create, schedule, edit, and manage automated tasks.
## Creating a Task from Ask Anything
The easiest way to create a task is to ask the agent directly within a conversation. For example:
* *"Schedule this analysis to run every Monday at 9am"*
* *"Turn this into a daily report and send it to Slack"*
* *"Create a weekly task that checks revenue per user every Friday"*
The agent proposes a task with an optimized prompt, schedule, and delivery settings. You review the details and approve — the task is created without leaving the conversation.
Creating tasks from a conversation is especially effective because the agent already has the context of your analysis. It can optimize the prompt based on what worked in the current chat.
## Creating a Task Manually
You can also create tasks from the Automations page:
Navigate to **Automations** from the sidebar and click **Create Task**.
Give the task a descriptive name — for example, "Daily KPI Summary" or "Weekly Churn Report."
Enter the question you want the agent to answer. Write it the same way you'd ask in Ask Anything.
Choose a frequency and time:
* **Daily** — Runs every day at the specified time
* **Weekly** — Runs on a selected day of the week
* **Monthly** — Runs on a selected day of the month
The time is set in your local timezone.
Choose where results should be sent — Email, Slack, or both. Results always appear in the in-app Results tab regardless of other delivery settings. See [Task Delivery & Notifications](/automations/task-delivery-and-notifications) for details.
Save the task. You can check **Test after creation** to run a test immediately — see below for what this means.
### Testing a Task
When you check **Test after creation**, ClarityQ executes the prompt immediately so you can verify the output. The test result appears in the **Results** tab under Automations — but it is **not delivered** to Slack or email recipients. This lets you review exactly what the task will produce before it goes out to anyone on its scheduled cadence.
**Test after creation** and **Run Now** behave differently. Test after creation is a dry run — results are only visible in the Results tab. **Run Now** triggers a full execution that delivers results to all configured channels (Slack, email) just like a scheduled run would.
## Managing Tasks
### Editing a Task
Click on a task to open it for editing. You can update the name, prompt, schedule, and delivery settings. If you change the schedule, the next run time is recalculated automatically.
### Pausing and Resuming
* **Pause** — Temporarily stops a task from running. No runs will occur while paused.
* **Resume** — Reactivates a paused task. The next run time is recalculated from the current schedule.
### Running Manually
Click **Run Now** to trigger an immediate execution outside the regular schedule. This is useful for testing or getting an on-demand result.
### Deleting a Task
Delete a task to permanently remove it and all its results.
Deleting a task cannot be undone. All associated results are permanently removed.
## Tips for Effective Tasks
* **Be specific in your prompt** — The agent runs without follow-up questions, so make the prompt clear and self-contained
* **Use `@` mentions** — Reference specific metrics or segments to ensure consistent results every time
* **Use `/` to invoke skills** — If the analysis you want to automate is defined as a skill, reference it with `/skill_name` in the prompt. The agent will follow the skill's methodology on every execution.
* **Start with daily tasks** — Monitor the first few results to make sure they match your expectations before reducing frequency
* **Test first** — Test after creating or editing to verify the output before the first scheduled execution
# Task Delivery & Notifications
Source: https://docs.clarityq.ai/automations/task-delivery-and-notifications
How task results are delivered via email, Slack, and in-app.
When a task completes, ClarityQ generates a summary of the key findings and delivers it to your configured channels. You must enable at least one delivery channel (email or Slack) when creating a task.
## Delivery Channels
### In-App (Always On)
Every task result automatically appears in the **Results** tab under Automations. This cannot be disabled. Unread results are marked with a dot indicator.
Click any completed result to open the full conversation — including the agent's written insight, data tables, visualizations, and SQL.
### Email
When email delivery is enabled, ClarityQ sends a summary email to the specified recipients after each execution. The email includes:
* Task name
* Summary of key findings (3-4 sentences)
* Workspace name for context
* A link to view the full report in ClarityQ
You can add multiple email recipients when creating or editing a task.
### Slack
When Slack delivery is enabled, ClarityQ posts a message to the selected Slack channel after each execution. The message includes:
* Task name
* Summary of key findings
* A **View Chat** button linking to the full conversation in ClarityQ
Slack delivery requires a connected Slack workspace. See [Connecting Slack](/integrations/slack/connecting-slack) to set up the integration.
## Failed Results
If a task execution fails, ClarityQ sends a failure notification to the same channels configured for the task, so you're aware even when something goes wrong. Failed results show an error indicator in the Results tab with details on hover.
## Sharing Results
You can share any completed result by clicking the **Share** button, which generates a link to the full conversation.
# What are Tasks?
Source: https://docs.clarityq.ai/automations/what-are-tasks
Automated, scheduled questions that deliver results on a recurring basis.
Tasks are scheduled automations that run an AI-powered prompt on a recurring basis and deliver the results to you automatically. Instead of manually asking the same question every day or week, create a task and let ClarityQ handle it.
## How Tasks Work
A task is a combination of:
* **A prompt** — The question you want ClarityQ to answer (e.g., "What were yesterday's key metrics?")
* **A schedule** — When to run it (daily, weekly, or monthly at a specific time)
* **Delivery channels** — Where to send the results (email, Slack, and/or in-app)
When a task runs, the agent executes your prompt just like it would in Ask Anything — searching the Semantic Catalog, running SQL, and generating results. It then produces a summary and delivers it to your configured channels.
## Task Statuses
| Status | Meaning |
| - | - |
| **Active** | Running on schedule. The next run time is shown in the task list. |
| **Paused** | Temporarily disabled. No runs will occur until you resume it. |
## Task Results
Every time a task executes, the results appear in the **Results** tab. Each result includes:
* **Summary** — A concise 3-4 sentence overview of the key findings
* **Status** — Running, Completed, or Failed
* **Timestamp** — When the task executed, shown in your timezone
* **Full conversation** — Click a completed result to view the complete agent response, including data tables and visualizations
Unread results are marked with a dot so you can easily see which ones you haven't reviewed yet.
# Building Entities from Tables
Source: https://docs.clarityq.ai/context-layer/building/building-entities-from-tables
Create semantic entities from your Table Catalog to power the Semantic Catalog.
Entities are the core business objects in your Semantic Catalog — they represent the things your organization cares about, like users, orders, sessions, or transactions. Entities are built from tables in your Table Catalog.
## What Is an Entity?
An entity maps one or more database tables to a business concept. It contains:
* **Dimensions** — All the properties that describe the entity — attributes, measures, and grouping axes. Anything you can say about an entity is a dimension: country, platform, revenue amount, session duration, user email, plan name.
For example, a `users` table might become a **Users** entity with dimensions like `country`, `signup_source`, `lifetime_value`, `email`, and `plan_name`.
## How to Build Entities
Open the Table Catalog and select the tables you want to create entities from. You can select multiple tables at once.
The agent inspects the selected tables — sampling data, understanding the grain of each table, and identifying relationships between them.
The agent suggests one or more entities based on the tables you selected, each with a name, description, SQL, and proposed dimensions. A single table may become one entity, or multiple tables may be combined into one entity with joins.
Review each proposed entity and its dimensions. You can accept them as-is, ask the agent to adjust names, descriptions, or dimensions, or remove entities you don't need.
## Best Practices
* **Start with your most queried tables** — Build entities for the tables your team asks about most
* **Use meaningful names** — Entity and property names should reflect business terminology, not database column names
* **Dimension descriptions come from the Table Catalog** — The descriptions you approved on columns in the Table Catalog carry over to dimensions automatically, so invest in good column descriptions early
* **Review nested fields** — For tables with STRUCT/RECORD columns, select the specific nested fields that matter rather than including the entire structure
## Adding Dimensions to Existing Entities
When new columns are discovered in the Table Catalog — for example, after a schema change or a daily discovery job — they don't automatically become dimensions on existing entities. You need to add them manually.
From the Table Catalog, select the new columns and choose an existing entity to add them to. The agent creates the dimensions using the column descriptions from the Table Catalog.
When new columns are discovered by [daily discovery jobs](/context-layer/table-catalog/daily-discovery-jobs), you'll be notified and they'll appear in the Missing Description Wizard. Once described and approved, add the relevant ones as dimensions to your entities so the agent can use them in analyses.
# Building Features from the Event Catalog
Source: https://docs.clarityq.ai/context-layer/building/building-features-from-the-event-catalog
How events become queryable features in the Semantic Catalog.
Features are the Semantic Catalog components that represent product events — things like `signup_completed`, `purchase_made`, or `level_finished`. They're created by syncing approved events from the [Event Catalog](/context-layer/event-catalog/what-is-the-event-catalog) into the Semantic Catalog.
Unlike metrics, entities, and skills, features are **not built through the Context Builder**. They come from the Event Catalog's sync process.
## How Features Are Created
Events must have an approved description before they can be synced. Use the [Missing Description Wizard](/context-layer/event-catalog/missing-description-wizard) to fill in descriptions efficiently.
Select one or more approved events and choose **Sync to Semantic Catalog**. See [Syncing Events to the Semantic Catalog](/context-layer/event-catalog/syncing-events-to-the-semantic-catalog) for the full walkthrough.
Each synced event becomes a Feature. The event's parameters and user properties become Dimensions on that Feature. Descriptions carry over automatically.
## What Gets Created
| Source | Becomes |
| - | - |
| The event itself | A Feature named after the event |
| Each event parameter | A Dimension on that Feature |
| Each user property | A Dimension on that Feature |
## Sync Options
When syncing, you choose between two options:
* **Sync and Approve** — The Feature is immediately available to everyone in your workspace.
* **Sync Only** — The Feature is created in a private state, visible only to you until you manually approve it. Use this when you want to review or adjust the Feature before exposing it to the team.
## Keeping Features Up to Date
Once an event is synced, new parameters and user properties are added as Dimensions automatically as they're approved in the Event Catalog. You don't need to re-sync the parent event each time.
When you edit an event's description in the Event Catalog after it's been synced, ClarityQ prompts you to update the corresponding Feature's description as well. The same applies when editing a parameter or user property description — you'll be asked whether to update the matching Dimension in the Semantic Catalog.
## Adding Table Columns as Dimensions
Beyond synced parameters and user properties, you can add columns from your events table as dimensions on features. This is useful for columns that exist in the table but aren't tracked as event parameters — for example, a `session_id`, `device_model`, or `app_version` column.
When adding table columns, you can choose to add them to:
* **All features** — The dimension is linked to every feature and automatically links to future features as well. This creates a **shared** dimension.
* **Specific features** — The dimension is linked only to the features you select.
## Adding Custom Dimensions
You can also create dimensions with custom SQL that don't come from a table column or event parameter. This is useful for derived values — for example, a dimension that extracts a nested field, applies a transformation, or combines multiple columns.
Custom dimensions can be scoped the same way: shared across all features, or linked to specific ones.
## Common Dimensions vs Feature-Specific Dimensions
A dimension that's linked to **two or more features** is automatically classified as a **common dimension**. A dimension linked to only one feature is considered feature-specific.
Common dimensions appear in a dedicated **Common Dimensions** tab in the Semantic Catalog, making them easy to find and manage separately from the dimensions nested under individual features.
This distinction matters because common dimensions represent properties that apply broadly across your product events — like `country`, `platform`, or `user_id` — while feature-specific dimensions are unique to a single event.
## Managing Common Dimensions
From the Common Dimensions view, you can manage how a dimension relates to each of its linked features:
* **Link to more features** — Extend a common dimension to additional features it isn't linked to yet.
* **Remove from a feature** — Deletes the link between the dimension and that feature. If the dimension has no remaining feature links, it's deleted entirely.
* **Duplicate and Detach** — Creates a copy of the dimension linked only to that specific feature, breaking the shared link. Future changes to the common dimension no longer affect this feature. Use this when a feature needs its own version of a dimension that diverges from the shared definition.
# Building Product Memory
Source: https://docs.clarityq.ai/context-layer/building/building-memory
How to create, edit, and manage product-level memory entries through the Context Builder.
[Product memory](/context-layer/memory) stores durable facts, definitions, and rules that apply to everyone working on a product. The **Context Builder** is where you create and manage these entries.
Product memory is for context that doesn't belong in the Semantic Catalog but matters every time someone asks a question — business definitions, naming conventions, data quirks, and querying preferences.
## What Belongs in Product Memory
Good product memory entries are short, factual, and broadly applicable:
* *"All monetary amounts are stored in cents — divide by 100 to get dollars."*
* *"When asked about 'last month', use the previous full calendar month, not trailing 30 days."*
* *"The `users` table includes both real users and bot accounts. Always filter by `is_bot = false` unless explicitly asked about bots."*
**Don't use product memory for:**
* Metric or dimension definitions — those belong in the [Semantic Catalog](/context-layer/semantic-catalog/what-is-the-semantic-catalog)
* Repeatable analysis workflows — those belong in [Skills](/context-layer/skills)
* Personal preferences — those belong in [personal memory](/context-layer/memory#personal-memory)
## Creating a Product Memory Entry
You can add product memory from two places:
* **Memory page → Product Memory tab** — Click **Add memory** to start a guided flow.
* **Builder chat** — Type `/product-memory add` or describe what you want the agent to remember in natural language (e.g., *"remember that revenue should always exclude refunds"*).
Each entry has a **title** (a short label identifying the topic) and **content** (the actual fact, rule, or instruction to remember). The agent structures both for you.
Before saving, the agent checks for existing entries that might overlap or contradict the new one and warns you if it finds any.
## Editing and Deleting
* **Edit** — From the Memory page, click the edit icon on any entry. Or type `/product-memory edit ` in a Builder chat.
* **Delete** — Click the delete icon on the Memory page, or type `/product-memory delete ` in chat.
Product memory follows the same draft-and-deploy cycle as other Context Layer components — changes are staged in your draft and only go live when you deploy.
Keep each memory item focused on a single rule or definition. Don't bundle unrelated policies into one entry.
# Building Metrics
Source: https://docs.clarityq.ai/context-layer/building/building-metrics
How to create, edit, and ship metrics through the Context Builder.
A [metric](/context-layer/semantic-catalog/metrics) is a business calculation the agent can reach for whenever a question calls for it — revenue, daily active users, conversion rate, marketing spend. The **Context Builder** is where you create new metrics, edit existing ones, and adjust how they're calculated.
You can provide as much or as little as you want — a name, description, and SQL, or just a plain-language request. The agent completes whatever is missing, adapts your SQL to ClarityQ standards, validates the result, and saves it to your draft. You review what was created and deploy it when you're ready.
## Starting a Metric
You can start a new metric from three places:
* **Semantic Catalog → Metrics** — Click **Add Metric** in the top right. ClarityQ opens a Builder chat with the command pre-filled.
* **Builder chat directly** — Type `/metric add` in any Builder chat.
* **From an Ask Anything conversation** — If you've been exploring a question in Ask Anything and realize you need a new metric, continue the conversation in the Builder. The agent already has the context of your analysis, so you can type `/metric add` and it will use what you've discussed to build the metric.
## How the Agent Builds the Metric
Describe what you want to measure in plain language — for example, "total cashout amount in dollars" or "daily active users." The agent takes it from there:
1. **Checks for duplicates** — Searches your existing metrics to avoid creating a near-duplicate. If a similar metric already exists, it suggests editing it instead.
2. **Identifies the source** — Finds the right table and column in your Table Catalog and confirms with you.
3. **Determines time behavior** — Figures out whether the metric should aggregate over a date range (e.g., *revenue this month*), apply to a single point in time (e.g., *DAU on Tuesday*), or be time-independent. If the wording is ambiguous, the agent asks a clarifying question.
4. **Writes and validates** — Generates the SQL, validates it against your data, and saves the metric to your draft.
If something doesn't look right — a typo in a column name, an aggregation that doesn't make sense on the chosen column — the agent surfaces the issue and you can correct course in chat.
## Editing and Deleting Metrics
From the **Metrics** tab of the Semantic Catalog:
* **Edit** — Open the metric and click **Edit** in the row menu, or type `/metric edit ` in a Builder chat. The agent loads the current definition and walks you through the changes.
* **Delete** — Choose **Delete** from the row menu, or type `/metric delete `. Deletion removes the metric from the catalog once your draft is deployed.
## Adding Several Metrics at Once
Building a dozen metrics one at a time is slow. Two faster paths:
* **List them in the chat** — Ask the agent to add several metrics in one message (*"add installs, marketing\_spend, cashout\_amount, and ARPU"*). Before creating anything the agent shows you a single proposal table — name, description, and the formula for any composites like ARPU — and waits for your confirmation.
* **Attach a file** — Drop a simple `.xlsx` spreadsheet (single tab) or a `.csv` file with your definitions into the Builder message. The agent reads the file, confirms what it understood, and creates the metrics in bulk.
When uploading a file, include only metrics — don't mix metrics with other component types like entities or segments. This keeps the agent focused and reduces the chance of errors. Bulk creation via chat or file upload works the same way for other component types.
## After Creating
A newly created metric lands in your draft and is only available to you until you deploy. To make a metric live for everyone:
1. **Test in Chat** to spot-check the answer ClarityQ returns when the metric is used (see [How the Context Layer Builder works](/context-layer/building/how-the-context-layer-builder-works#drafts-and-versioning)).
2. **Run the audit** (`/audit-changes`) on your draft so any broken references or validation issues surface before you ship.
3. **Deploy** the draft from the orange banner at the top of the screen. The metric (along with any other draft changes) becomes part of the published catalog and available to everyone in your organization.
When a question could be answered with an existing metric instead, the agent flags the overlap and offers to edit the existing metric rather than create a near-duplicate. Lean into that — fewer, well-defined metrics keep the catalog easier to reason about.
# Building Segments
Source: https://docs.clarityq.ai/context-layer/building/building-segments
How to create and edit reusable user cohort definitions through the Context Builder.
A [segment](/context-layer/semantic-catalog/segments) is a reusable cohort definition — a named group of users matching specific criteria. The **Context Builder** is where you create new segments, edit existing ones, and refine their SQL.
You can provide a name, description, and SQL, or just describe the cohort you want in plain language. The agent completes whatever is missing, adapts your SQL to ClarityQ standards, validates the result, and saves it to your draft.
## Starting a Segment
You can start a new segment from three places:
* **Semantic Catalog → Segments** — Click **Add Segment** in the top right. ClarityQ opens a Builder chat with the command pre-filled.
* **Builder chat directly** — Type `/segment add` in any Builder chat.
* **From an Ask Anything conversation** — If you've been analyzing a user cohort in Ask Anything and want to make it reusable, continue in the Builder. The agent already has the context and can encode the cohort definition into a segment.
## How the Agent Builds the Segment
Describe the cohort you want — for example, "users who made a purchase in the last 30 days" or "churned users who haven't logged in for 60 days." The agent takes it from there:
1. **Checks for duplicates** — Searches your existing segments to avoid creating a near-duplicate. If a similar segment already exists, it suggests editing it instead.
2. **Identifies the source** — Finds the right tables and user identifier column in your data, and confirms with you.
3. **Determines time behavior** — Figures out whether the segment should scope to a date range (e.g., *users active in the last 28 days*), a single point in time (e.g., *subscribers as of March 1*), or be timeless (e.g., *users who ever verified their email*). If the wording is ambiguous, the agent asks a clarifying question.
4. **Writes and validates** — Generates the SQL, validates that it returns a distinct list of user identifiers, and saves the segment to your draft.
The SQL must return exactly one column — the user identifier — with distinct values. No extra columns, no metric values, no breakdowns. The agent enforces this automatically.
## Editing and Deleting Segments
From the **Segments** tab of the Semantic Catalog:
* **Edit** — Open the segment and click **Edit** in the row menu, or type `/segment edit ` in a Builder chat. The agent loads the current definition and walks you through the changes.
* **Delete** — Choose **Delete** from the row menu, or type `/segment delete `. Deletion removes the segment from the catalog once your draft is deployed.
## Adding Several Segments at Once
Like metrics, you can create multiple segments in one go:
* **List them in the chat** — Ask the agent to add several segments in one message (*"add segments for high spenders, churned users, and new signups"*). The agent shows a proposal with each segment's name, description, and cohort criteria, and waits for your confirmation.
* **Attach a file** — Drop a simple `.xlsx` spreadsheet (single tab) or a `.csv` file with your definitions into the Builder message. The agent reads the file, confirms what it understood, and creates the segments in bulk.
When uploading a file, include only segments — don't mix segments with other component types. This keeps the agent focused and reduces the chance of errors.
## After Creating
A newly created segment lands in your draft and is only available to you until you deploy. To make a segment live for everyone:
1. **Test in Chat** to verify the segment works as expected — for example, ask *"how many high spenders were there last month?"* and confirm the count makes sense.
2. **Run the audit** (`/audit-changes`) on your draft to catch any issues before deploying.
3. **Deploy** the draft from the orange banner at the top of the screen. The segment becomes part of the published catalog and available to everyone in your organization.
# Building Skills
Source: https://docs.clarityq.ai/context-layer/building/building-skills
How to create and edit custom skills through the Context Builder.
A [skill](/context-layer/skills) is a reusable analytical methodology the agent follows step by step when invoked with `/skill_name`. The **Context Builder** is where you create new skills, edit existing ones, and refine them based on real usage.
You can provide a full methodology or just describe the analysis you want to capture. The agent structures the skill for you — writing the content, defining clarifications, and setting up the todo list.
## Starting a Skill
You can start a new skill from three places:
* **Skills page** — Click **Add Skill** in the top right. ClarityQ opens a Builder chat with the command pre-filled.
* **Builder chat directly** — Type `/skill add` in any Builder chat.
* **From an Ask Anything conversation** — If you've just run an analysis in Ask Anything that you'd like to make repeatable, continue in the Builder. The agent already has the context and can encode the methodology into a skill.
## What the Agent Creates
The agent builds a skill with the following fields:
* **Name** — A `snake_case` identifier that becomes the slash command (e.g., `/cohort_analysis`)
* **Description** — A one-line summary shown in the slash command picker. This is what triggers the agent to import the skill, so it includes what the skill does, when to use it, and trigger phrases
* **Content** — The methodology itself in structured Markdown — rules, steps, constraints, SQL patterns, and gotchas specific to your data
* **Clarifications** — Questions the agent asks the user before running the skill, to gather any required inputs
* **Todo list** — The checklist the agent works through during execution
Before writing anything, the agent checks for existing skills that cover the same topic. If one exists, it suggests editing it instead of creating a near-duplicate.
## Clarifications, Todo List, and References
Skills have three supporting mechanisms that control how the agent behaves when running them. You can use clarifications, a todo list, or both — depending on what the skill needs.
### Clarifications
Questions the agent asks the user **before** executing the skill. Use clarifications when the skill needs one or two small inputs that change the analytical path — for example, which date range to use or which user segment to focus on.
Keep clarifications minimal. If you find yourself adding more than two or three, the skill's content should handle the branching instead.
### Todo List
A checklist the agent works through **during** execution. Each todo item maps to a step or rule in the skill's content, keeping the agent on track as it progresses through the analysis.
When the agent creates a skill, it typically generates the right todo list automatically based on the content. You can review and adjust the items if needed.
In chat, the todo list appears as a visible checklist with checkboxes. As the agent completes each step, it checks off the corresponding item — so you can follow along and see exactly where the agent is in the workflow.
Use a todo list when the skill has multiple steps and you want to ensure none are skipped.
### References
Supplementary documentation that loads on demand. When the agent imports a skill, it sees the names and descriptions of its references — but the full content only loads when the agent decides it needs more detail. This keeps skills concise while supporting deep documentation when needed.
References are ideal for information that's too detailed or too long to put in the skill's content but that the agent may need to consult during execution — business logic edge cases, known data quirks, analysis guidelines, or output formatting rules.
**Example:** A `/funnel_analysis` skill's content defines the general methodology — how to build a funnel query, how to calculate drop-off rates, and how to present results. Its references contain special requirements for specific funnels: "Onboarding funnel" might document that step 2 must exclude users who skipped the tutorial, while "Purchase funnel" might note that the checkout step has a known 30-second delay before the event fires. The agent loads the relevant reference only when the user asks about that particular funnel.
## The Description Matters Most
The description is the only thing the agent reads on every message to decide whether to import a skill. The content and hooks only load if the description triggers first.
The agent usually writes a good description automatically, but it's worth reviewing to make sure it captures the right scope — specific enough to trigger on relevant questions, but not so broad that it fires on unrelated ones.
A good description follows this pattern: *what it does + when to use it + trigger phrases.* For example: *"Analyze user retention by cohort. Use when: 'retention', 'cohort analysis', 'D7 retention'."*
## Editing Skills
Edit an existing skill from the Skills page (click **Edit** in the row menu) or type `/skill edit ` in a Builder chat. The agent loads the current definition and walks you through the changes. Each save creates a new version.
## Testing and Iterating
Before deploying a skill, test it in Ask Anything by invoking it with `/skill_name`. Review the agent's output — did it follow the right steps? Did it miss anything? Did it produce the answer you expected?
If something is off, go back to the Builder and refine the skill. Repeat until the results are consistent and accurate. First drafts rarely come out perfect, and that's fine — the iteration is what makes a skill reliable.
You don't need to have the perfect skill in mind before you start. Chat with the agent in the Builder — describe the analysis you're trying to capture, brainstorm the steps, debate edge cases, and iterate on the content together. The agent will refine the skill with you until it's right. The conversation *is* the drafting process.
The best skills come from encoding workflows your team already runs manually. If you find yourself explaining the same analysis more than once, that's a skill.
## Skill Scope
All custom skills are scoped to the **workspace level** — once deployed, they're available to everyone working in the workspace.
# Context Layer Versioning
Source: https://docs.clarityq.ai/context-layer/building/context-layer-versioning
How drafts, versions, and releases work across the Context Layer.
Every change to the Context Layer goes through a versioned release process. You never edit the live catalog directly — instead, changes are staged in a draft, tested, and deployed as a new version. This gives your team a clear history of what changed, when, and by whom.
## How Versions Work
The Context Layer always has one **published version** — the version everyone sees and that the agent uses for Ask Anything answers. When you start editing in the Builder, a **draft** is created alongside it. The draft is private to you until you deploy it.
Each time you deploy, the version number increments and your draft becomes the new published version. The previous version isn't deleted — it stays in the version history permanently.
## What Gets Versioned
A single version captures the state of all Context Layer components managed through the Builder:
* Entities and their dimensions
* Metrics
* Segments
* Skills
* Product memory entries
All changes within a single draft — whether you edited one metric or fifty components — are deployed together as one version.
## What's Not Versioned
Some parts of the Context Layer live outside the draft-and-deploy cycle:
* **Table Catalog** — Tables and columns are discovered directly from your data warehouse and updated by daily discovery jobs. Changes take effect immediately.
* **Event Catalog** — Events and parameters are synced from your data source. Like the Table Catalog, changes are live as soon as they're made.
* **Features** — Features promoted from the Event Catalog are managed separately and not part of the versioned draft. This includes any [Rules](/context-layer/rules) on a feature or a feature-only dimension — they're edited live and take effect immediately. Rules on entities, metrics, segments, and entity dimensions are versioned like the components they belong to.
* **Personal memory** — Personal memory entries are saved and applied immediately in your conversations, without going through a deploy.
These components don't need the review-and-deploy workflow because they're either driven by external data sources or scoped to an individual user.
## The Change Set
Every deployed version includes an auto-generated description of what changed. The change set records every component that was:
* **Created** — New components added in this version
* **Modified** — Existing components that were updated
* **Deleted** — Components that were removed
You can view the change set from the version history at any time — useful for auditing a deploy after the fact or understanding when a particular change was introduced.
## Version History
To view the version history, click the **All versions** link at the top of the **Semantic Catalog**, **Skills**, or **Product Memory** screens. The page lists every deployed version with its:
* Version number
* Timestamp
* Author (who deployed it)
* Auto-generated description of the changes
### Actions on Each Version
* **View Changes** — See the detailed list of components created, modified, or deleted in that specific version. Available on every version.
* **Compare to Current** — See the cumulative changes between that version and the current published version. Useful for understanding how the catalog has evolved over time. Available on any version that isn't the current one.
* **Rollback** — Immediately restores the catalog to that version's state and publishes it as a new version. This is not a draft — the rollback takes effect right away. Rolling back doesn't erase the versions in between — the history is append-only, so you always have a complete audit trail. **Admins** can roll back to any version. **Contributors** can only roll back if every version between the current one and the target was deployed by them — they cannot roll back past another user's deploy.
## Common Scenarios
### "What changed in the last deploy?"
Open version history and click **View Changes** on the latest version. You'll see exactly what was created, modified, or deleted.
### "Something broke — what changed since the last known good version?"
Find the version you trust in the history and click **Compare to Current**. This shows every cumulative change between that version and what's live now.
### "I need to undo a bad deploy"
Find the version you want to restore and click **Rollback**. The catalog is immediately restored to that version's state and published as a new version. The bad version stays in the history — nothing is lost.
Rollback publishes a new version on top of the current one. It doesn't delete or rewrite history. Your version timeline always moves forward.
# How the Context Layer Builder works
Source: https://docs.clarityq.ai/context-layer/building/how-the-context-layer-builder-works
The Context Builder (aka the Builder) is where your team drafts, validates, and deploys changes to the Context Layer.
The **Context Builder** is a dedicated chat where your team creates and edits the components that live in your Context Layer — metrics, entities, segments, dimensions, skills, and product memory items. The Builder treats every change as part of a single draft version of your catalog: you take the lock on the Context Layer, make your edits, audit the updates, and deploy a new version when you're ready.
The Builder's separate chat is accessed from the **Context Builder** entry in the sidebar or from any **Edit** or **Add** action in the Semantic Catalog. Built-in [skills](/context-layer/skills) like `/metric`, `/entity`, `/segment`, `/skill`, and `/product-memory` — with subcommands such as `add`, `edit`, and `delete` — are the skills that allow you to build and maintain every component of your Context Layer.
Building requires write access to the Context Layer — **Admins** and **Analyst Contributors** by default. See [User Management and Roles](/admin/user-management-and-roles) for the full breakdown.
## The Build Cycle
Every set of changes follows the same cycle:
```
Lock → Build → Test → Audit → Deploy
```
1. **Lock** — Take ownership of the catalog so no one else edits at the same time
2. **Build** — Create, edit, or delete components in the Builder chat
3. **Test** — Use Ask Anything to verify your changes produce the expected answers
4. **Audit** — Run `/audit-changes` to catch broken references, duplicates, and gaps
5. **Deploy** — Publish your draft as a new version for everyone
The sections below and the following pages explain each step in detail.
You can also arrive at the Builder from a [Diagnosis](/ask-anything/diagnosing-answers): when you thumbs-down an answer in Ask Anything and click **Fix now**, the Builder opens with the suggested fix pre-loaded, ready to review and build.
## The Lock
Editing the Context Layer is a single-writer operation — only one user can hold the draft at a time. When you click **Start Editing**, ClarityQ acquires the lock for the entire catalog (not for an individual component).
While the lock is yours:
* You can run as many Builder commands as you want — each one edits the same draft.
* Other users are unaffected — they continue using the published version as normal. If someone else tries to start editing, they'll see that the Context Layer is locked, who holds the lock, and when the editing session started.
The lock is released automatically when you **Deploy** or **Discard** your draft. If the lock holder is unreachable (closed the tab, away from work), an **admin** can press **Unlock** on the lock card to free the catalog. Note that unlocking **discards the current draft** — any changes the previous user made are lost.
Only one Builder chat can be active at a time. Starting a new Builder chat while another is open prompts you to switch — past chats stay in your history.
## Drafts and Versioning
Each change you make in the Builder lands on a **draft** version of your catalog. By default the catalog has a single **published** version that everyone uses; while a draft is open, a second state exists alongside it:
| State | Who sees it |
| - | - |
| **Published** | Everyone — what the agent uses for Ask Anything answers. |
| **Draft** | Only you (the lock holder) — visible across the entire workspace, including Ask Anything, so you can test your changes before deploying. |
The draft owner sees an orange banner across the top of ClarityQ, telling you which version of the catalog you're editing as a private draft. The banner gives you four actions:
* **New Build** — Start another Builder chat against the same draft.
* **Test in Chat** — Open an Ask Anything chat that uses your draft instead of the published catalog. Ask the questions your changes are meant to answer — for example, after redefining your DAU metric, ask *"what was DAU last week?"* — and make sure the agent picks up your new definition and returns what you expect. Other people's chats keep using the published version, so your testing stays isolated.
* **View changes** — See a detailed list of all changes between the current published version and your draft (created, modified, and deleted components).
* **Discard** — Throw the draft away. Every edit you made in this session is removed and the catalog snaps back to its current published version. The lock is released so someone else can start a new draft. There's no undo — once a draft is discarded it can't be recovered.
* **Deploy** — Promote the draft to a new published version of the catalog. The version number increments, your changes go live for everyone in your organization, and the lock is released. ClarityQ also snapshots the change set with an auto-generated description so the version history records what changed.
## The Pre-Deploy Audit Gate
When you press **Deploy**, ClarityQ checks whether your draft has been audited since its last edit. If not, you see a prompt asking you to either run an audit first or deploy without one.
The audit (`/audit-changes`) is a separate Builder skill that focuses on the changes introduced in your current draft — looking for broken references, dead tables, duplicate definitions, orphaned components, and similar gaps. Running it before deploying catches mistakes that would otherwise reach every user.
Treat the audit as the equivalent of CI for your catalog — run it before every deploy and resolve any findings first.
## Version History and Rollback
To reach the version history, click the **All versions** link that appears at the top of the **Semantic Catalog**, **Skills**, and **Product Memory** screens. The page lists every deployed version with its timestamp, author, and the auto-generated description for the change set.
Each row offers one or more actions:
* **View Changes** — Available on every version. Opens a detailed list of changes between that version and the one immediately before it, showing every component created, modified, or deleted in that release. Use it to audit a specific deploy — *"what changed when we shipped v4?"*
* **Compare to Current** — Available on any version that isn't the current published one. Opens a diff between that version and the currently published version, so you can see the cumulative drift since a known good state — *"what's changed in the catalog since v3?"*
* **Rollback** — Available on any version that isn't the current published one. Immediately restores the catalog to that version's state and publishes it as a new version. Rolling back doesn't erase the versions in between — the history is append-only.
# Recommended Build Order
Source: https://docs.clarityq.ai/context-layer/building/recommended-build-order
The step-by-step order for building your Context Layer effectively.
The Context Layer components build on each other. Following this order ensures each step has the inputs it needs and avoids rework.
## The Order
Everything starts here. [Connect your warehouse](/get-started/connect-your-data-warehouse) so ClarityQ can access your data. Configure [mandatory filters](/get-started/connect-your-data-warehouse#mandatory-filters) if needed.
After connecting, ClarityQ automatically discovers your tables and columns. Review the results, enrich descriptions, and [build entities](/context-layer/building/building-entities-from-tables) from your key tables. Daily discovery jobs will keep the catalog up to date as your schema evolves.
If your product tracks events, set up the Event Catalog. Run the [Missing Description Wizard](/context-layer/event-catalog/missing-description-wizard) to fill in gaps, then [sync events to the Semantic Catalog](/context-layer/event-catalog/syncing-events-to-the-semantic-catalog) so they become queryable.
Define your core business logic — [dimensions](/context-layer/semantic-catalog/dimensions), [metrics](/context-layer/semantic-catalog/metrics), [features](/context-layer/semantic-catalog/features), and [segments](/context-layer/semantic-catalog/segments). Start with the components your team asks about most. ClarityQ's AI agent actively helps build these using your existing definitions and data structure.
Store business rules, naming conventions, and domain context that doesn't live in your database. Start with [personal memory](/context-layer/memory) for the conventions you find yourself repeating, and add product memory as it rolls out.
Once your team has identified repeatable analysis patterns, [encode them as skills](/context-layer/skills). This is an ongoing process — add skills as new workflows emerge.
## When Can I Start Asking Questions?
You don't need to complete every step before using Ask Anything. As a rule of thumb:
| Stage | What You Can Do |
| - | - |
| **Table Catalog reviewed + a few entities built** | Ask basic questions about your data |
| **Event Catalog synced** | Ask about product events and user behavior |
| **Core metrics and dimensions defined** | Ask business questions with accurate, consistent answers |
| **Memory added** | Get answers that reflect your organization's specific context and terminology |
| **Skills created** | Run complex, multi-step analyses consistently |
The Context Layer is a living system — keep building and refining it over time for increasingly accurate results.
# Daily Discovery Jobs
Source: https://docs.clarityq.ai/context-layer/event-catalog/daily-discovery-jobs
How ClarityQ keeps your Event Catalog in sync with the events flowing into your warehouse.
Your product instrumentation isn't static — teams ship new events, deprecate old ones, and add parameters along the way. ClarityQ runs daily discovery jobs against your events table to keep the Event Catalog aligned with what your warehouse actually contains.
## What the Jobs Discover
Each day, ClarityQ scans the most recent window of events in your warehouse and updates the catalog with what it finds:
| Change | What Happens |
| - | - |
| **New event** | Added in **Pending** approval status, **Active** data status. AI generates a suggested description when it recognises the event. |
| **New parameter** | Added under the source event, and to the **Common Parameters** tab if it appears on more than one event. |
| **New user property** | Added to the **User Properties** tab. |
| **Updated parameter values** | Distinct values and value counts are refreshed for parameters and user properties. |
| **Updated platforms** | New platform / version combinations seen for an event are added to its **Platforms** column. |
| **No longer seen** | Items absent from the recent window flip to **Inactive** data status. Existing entries are preserved — nothing is deleted. |
## What the Job Runs
Each daily run executes the following, in order:
Scan the events table for new events, parameters, and user properties within the discovery window, and re-calculate which existing items are still active.
Update the distinct values and value counts for parameters and user properties. By default, only items missing values are refreshed — ask ClarityQ to switch to a full refresh on every run if you want it.
Generate AI-suggested descriptions for any item still missing one — common parameters first, then user properties, then events.
## Where Findings Surface
* **Event Catalog tabs** — New items appear in Events, Common Parameters, and User Properties, with their suggested descriptions pre-filled.
* **Missing-descriptions badge** — The sidebar shows a running count of items still needing a description, leading into the [Missing Description Wizard](/context-layer/event-catalog/missing-description-wizard).
* **Email notifications** — Configured recipients get a daily summary of what was discovered, including new-item counts.
## How to Act on Findings
When the job finds something new:
* **Review** the suggested description, then accept it, edit it, or write your own.
* **Approve** the event, parameter, or user property to move it to **Approved** status.
* **Hide** anything you want to keep out of the catalog and out of the agent's reach.
* **Sync** approved events to the [Semantic Catalog](/context-layer/event-catalog/syncing-events-to-the-semantic-catalog) so the agent can use them.
## Notes on the Discovery Window
The default window is roughly two days. Events that fired before then aren't picked up by daily runs. For a launch event that happened months ago, ask ClarityQ to trigger a one-time backfill against the date range you need.
An **Inactive** item isn't gone from your warehouse history — it just hasn't fired during the most recent window. If something flips to Inactive unexpectedly, check whether it was renamed, deprecated, or filtered out by an upstream change.
# Missing Description Wizard
Source: https://docs.clarityq.ai/context-layer/event-catalog/missing-description-wizard
Batch through every event, parameter, and user property still missing a description in one guided pass.
The Missing Description Wizard is the fastest way to get your catalog from raw event names to fully described. It collects every item still missing a description — events, common parameters, user properties, and any tables that need one — and walks you through them one at a time, accepting AI suggestions, edits, or your own descriptions as you go.
Use it during onboarding to clear the initial backlog, and again whenever daily discovery surfaces new items.
## Starting the Wizard
The sidebar shows a **Complete** button with the current count of items still missing a description. Click it to open the wizard.
On the first run you'll see a welcome screen that explains the flow and offers to apply ClarityQ's AI-suggested descriptions where they exist. Continue when you're ready.
## How the Wizard Works
The wizard groups items by type and walks through them in order, showing one item per screen.
If your Table Catalog has tables missing descriptions, the wizard starts there. Each table shows its columns and any AI suggestion.
Every event in **Pending** status without an approved description. The wizard shows the event name, type, volume, and a suggested description when ClarityQ has one.
Common parameters and user properties are surfaced together. For each one you see its source events, distinct values, and AI suggestion.
For each item you can:
* **Accept the suggestion** — Use the ClarityQ-suggested description as-is.
* **Edit and save** — Adjust the suggestion or write your own description.
* **Skip** — Move to the next item without saving. The item stays in **Pending** until you come back to it.
When you save a description, the item moves to **Approved** status. If it's a parameter or user property whose parent event has already been synced, it auto-syncs as a [Dimension](/context-layer/semantic-catalog/dimensions) on the related [Feature](/context-layer/semantic-catalog/features).
## Progress and Resuming
The header shows your position in each section (for example, **3/12**) along with **Back**, **Next**, and **Exit** buttons. You can leave the wizard at any time — your progress is remembered, so the next session picks up where you left off.
## Tips
Start by skimming the AI suggestions before editing. For events ClarityQ recognises (especially standard GA4 events and their common parameters), the suggestion is usually accurate and you can accept it in a single click.
Approving items in the wizard doesn't sync the parent events to the Semantic Catalog. Once events are described and approved, head to the Events tab and [sync them](/context-layer/event-catalog/syncing-events-to-the-semantic-catalog) so the agent can use them.
# Setting Up Your Event Catalog
Source: https://docs.clarityq.ai/context-layer/event-catalog/setting-up-your-event-catalog
How to connect ClarityQ to your product analytics events and run the initial discovery.
The Event Catalog reads from a single table in your data warehouse — the one your analytics platform writes events to. Setting it up tells ClarityQ which table to read, which rows to keep, and which events or properties to ignore.
Setup is done with ClarityQ during onboarding. Your [data warehouse](/get-started/connect-your-data-warehouse) needs to be connected first; once the source is configured, [daily discovery jobs](/context-layer/event-catalog/daily-discovery-jobs) take over.
## What ClarityQ Needs From You
Share the following with your ClarityQ contact:
| Input | What it is |
| - | - |
| **Events table** | The fully qualified path of the warehouse table that holds your events (e.g., `my-project.analytics_123456.events_*`) |
| **Filters** | SQL `WHERE` conditions used to scope ingestion (e.g., restrict to a single app bundle, exclude staging traffic, drop pre-launch events) |
| **Ignore patterns** | Regex patterns for events, parameters, or user properties you don't want catalogued — useful for internal debug events, deprecated names, or fields you'd rather keep out of the agent's reach |
| **Version rules** | Patterns identifying which platform-version combinations to include or exclude (e.g., skip beta builds) |
| **Datetime format** | The format your event date column uses, if it's not the default `YYYY-MM-DD` |
## What Happens After Setup
ClarityQ scans the events table, identifies every distinct event name, parameter, and user property in the configured window, and creates entries in the catalog.
The Events, Common Parameters, and User Properties tabs populate. Everything starts in `Pending` approval status with `Active` data status.
For supported platforms, ClarityQ pre-fills descriptions where it recognises the event or property. The rest are left for you to describe.
Use the [Missing Description Wizard](/context-layer/event-catalog/missing-description-wizard) to fill in any items still without a description, then approve them.
Once an event is approved, [sync it to the Semantic Catalog](/context-layer/event-catalog/syncing-events-to-the-semantic-catalog) so the agent can use it in metrics, segments, and ad-hoc questions.
If your events table is the GA4 BigQuery export, ClarityQ already ships with descriptions for the standard GA4 events (`session_start`, `screen_view`, `first_visit`, and so on) and the common GA4 parameters and user properties — they come back pre-filled, leaving you only your custom events to describe.
The daily discovery job looks at a rolling two-day window, so newly-instrumented events appear within a day. If you need to backfill an older period — for example, to catch events from a launch that happened months ago — ask ClarityQ to run a one-time backfill against the date range you care about.
# Syncing Events to the Semantic Catalog
Source: https://docs.clarityq.ai/context-layer/event-catalog/syncing-events-to-the-semantic-catalog
Promote approved events from the Event Catalog into the Semantic Catalog so the agent can use them in analyses.
Syncing turns an approved event in the Event Catalog into a queryable component in the [Semantic Catalog](/context-layer/semantic-catalog/what-is-the-semantic-catalog). After sync, the agent reaches for the event by name in metrics, segments, and ad-hoc questions, without you having to point at the raw event name in your warehouse.
The sync is a one-way action you trigger when an event is ready — typically after you've described and approved it in the Event Catalog.
## What Gets Created
Each synced event creates the following:
| Source | Becomes |
| - | - |
| The event itself | A [Feature](/context-layer/semantic-catalog/features) named after the event |
| Each of the event's parameters | A [Dimension](/context-layer/semantic-catalog/dimensions) on that Feature |
| Each user property carried by the event | A Dimension on that Feature |
Descriptions and ownership carry over: the event's description becomes the Feature's description, each parameter and user property description becomes the corresponding Dimension's description, and the approver is recorded on the new components.
## Running a Sync
Make sure the event is in **Approved** status. Pending events can't be synced — they need an approved description first.
From the Events tab (or the Event page), select one or more events and choose **Sync to Semantic Catalog**.
A confirmation dialog shows the count of selected events and offers two actions: **Sync and Approve** or **Sync Only** (see below for the difference).
ClarityQ reports how many events synced successfully, how many were skipped (already synced), and any that failed with the reason.
## Sync and Approve vs Sync Only
| Action | New component status | Visibility |
| - | - | - |
| **Sync and Approve** | `Approved` | Available to everyone in your workspace immediately |
| **Sync Only** | `Private` | Visible only to you in the Semantic Catalog until manually approved |
Use **Sync Only** when you want to review the resulting Feature before exposing it to the rest of the team — for instance, to tweak its name or rework the dimensions before publishing.
## Auto-sync for Parameters and User Properties
Once an event is synced, its parameters and user properties stay in lock-step automatically:
* **Newly approved descriptions** on a parameter or user property attached to a synced event auto-sync as Dimensions on the existing Feature.
* **Newly discovered parameters** on a synced event are added as Dimensions once they're approved.
This keeps the Semantic Catalog current as your catalog fills in, without you having to re-sync the parent event each time.
## Notes
Re-syncing an already-synced event is a no-op — the existing Feature is preserved. To rebuild a Feature (for example, after renaming the event or restructuring its parameters), delete the Feature from the Semantic Catalog and sync the event again.
When you edit an event's description after it's been synced, ClarityQ prompts you to update the corresponding Feature's description as well. The same applies to parameter and user property descriptions — you'll be asked whether to update the matching Dimension.
# What is the Event Catalog?
Source: https://docs.clarityq.ai/context-layer/event-catalog/what-is-the-event-catalog
A real-time inventory of your product analytics events, including all their parameters and associated user properties.
The Event Catalog is ClarityQ's repository for your product analytics events — the user actions and product behaviors tracked in your warehouse (think `signup_completed`, `purchase_made`, `screen_viewed`). It automatically discovers events as they appear, captures the parameters and user properties they carry, and gives you one place to describe and approve them before they flow into the Semantic Catalog.
Without an Event Catalog, ClarityQ would only know your events by their raw names. With it, every event has a business meaning attached, and the agent can reason about user behavior in your product's terms.
## What It Contains
The Event Catalog is organized into three tabs.
### Events
For each event tracked in your product, the Event Catalog stores:
* **Event Name** — The identifier as it appears in your warehouse (e.g., `signup_completed`)
* **Description** — User-provided or AI-suggested explanation of when the event fires
* **Type** — `Automatic` for events the analytics platform collects out of the box (e.g., GA4's `session_start`, `screen_view`) or `Custom` for events your team instrumented
* **Platforms** — The platforms the event has been observed on (e.g., iOS, Web, Android)
* **Volume** — Total occurrence count
* **Params** — The number of the event parameters; click the number to open the parameters list
* **First Fired** — When the event was first seen
* **Last Fired** — When the event was most recently seen
* **Data Status** — Whether the event is still active in your warehouse (see [Statuses](#statuses) below)
Click the **arrow** next to an event name — or the number in the **Params** column — to expand the row in place. The expanded view lists every parameter the event carries, split into `Custom` and `Automatic` tabs. Each parameter shows its name, approval status, description, data type, distinct value count, and example values — and you can edit a parameter's description directly from there.
Click the **event name** itself to open the **Event page**, the full detail view for a single event. The page shows the event's approval status and approver, description, type, data status, volume, last modified date, the platform and version where the event first and last fired, and the timestamps of its first and last occurrence. Below the metadata is the full parameters table — the same one shown in the expanded row. From this page you can edit the event's description, hide or unhide it, and [sync it to the Semantic Catalog](/context-layer/event-catalog/syncing-events-to-the-semantic-catalog) once it's approved.
### Common Parameters
Parameters are the attributes attached to events (e.g., `button_id` on a click event). The Common Parameters tab lists parameters that appear across multiple events — describe each one once, and ClarityQ understands it everywhere it shows up.
For each parameter:
* **Name** — The parameter identifier as it appears on events (e.g., `button_id`)
* **Description** — User-provided or AI-suggested
* **Type** — `Automatic` for parameters the analytics platform attaches to events out of the box (e.g., GA4's `page_location`, `ga_session_id`) or `Custom` for parameters your team added
* **Data Type** — The parameter's SQL data type (e.g., `STRING`, `INTEGER`, `FLOAT`, `BOOLEAN`)
* **# Values** — The number of distinct values seen for this parameter
* **Platforms** — The platforms the parameter has been observed on (e.g., iOS, Web, Android)
* **% Events** — The share of your events that include this parameter
* **# Events** — The number of distinct events that include this parameter
* **First Fired** — When the parameter was first seen
* **Last Fired** — When the parameter was most recently seen
* **Data Status** — Whether the parameter is still active in your warehouse (see [Statuses](#statuses) below)
* **Approval** — Who approved the description
* **Status** — Approval status (`Pending` or `Approved`)
Click the **arrow** next to a parameter name — or the number in the **# Values** column — to expand the row in place. The expanded view shows up to ten example values seen for that parameter, useful for spot-checking what it actually contains without leaving the table.
Click the **parameter name** itself to open the **Common Parameter page**, the full detail view for a single parameter. The page shows the parameter's approval status and approver, description, type, data type, data status, **% Events** (share of events that include it), and **# Events** (count of events that include it), along with the platforms and versions where the parameter first and last appeared and the timestamps of its first and last occurrence. Below the metadata, two sections give you context: a **Values** section listing example values, and an **Events with This Parameter** table showing every event that includes the parameter, with each event's name, type, approval status, description, and the parameter's description as used in that specific event. From this page you can edit the parameter's description (which applies everywhere it's used) and hide or unhide it.
### User Properties
User properties are user-level attributes that accompany events (e.g., `subscription_tier`, `country`). They're tracked separately from event parameters because they describe the user, not the event. The Event Catalog stores the same fields for each user property as it does for common parameters.
Click the **arrow** next to a user property name — or the number in the **# Values** column — to expand the row in place and see up to ten example values inline.
Click the **user property name** to open the **User Property page**, which mirrors the Common Parameter page: full metadata (approval status and approver, description, type, data type, data status, **% Events**, **# Events**, platforms and versions, first and last fired), a **Values** section, and an **Events with This User Property** table listing every event the user property appears on. From this page you can edit the user property's description (which applies everywhere it's used) and hide or unhide it.
## Statuses
Every event, parameter, and user property has two statuses.
**Approval status** reflects where the item is in its lifecycle:
* **Pending** — No approved description yet
* **Approved** — Has a verified description and is ready to sync to the Semantic Catalog
* **Synced** — Already synced to the Semantic Catalog as a [Feature](/context-layer/semantic-catalog/features) (for events) or [Dimension](/context-layer/semantic-catalog/dimensions) (for parameters and user properties)
**Data status** reflects whether it's still active in your warehouse:
* **Active** — Seen during the most recent discovery window
* **Inactive** — Not seen recently (the event may have been deprecated or renamed)
## How It Fits into the Context Layer
The Event Catalog is updated continuously by [daily discovery jobs](/context-layer/event-catalog/daily-discovery-jobs) that scan your warehouse for new events, parameters, user properties, and value changes.
Once an event has an approved description, you can [sync it to the Semantic Catalog](/context-layer/event-catalog/syncing-events-to-the-semantic-catalog). The event becomes a [Feature](/context-layer/semantic-catalog/features), and its parameters and user properties become [Dimensions](/context-layer/semantic-catalog/dimensions) — making them available in metrics, segments, and ad-hoc questions. From that point on, the agent draws on these components automatically whenever a question touches that part of your product, with no need to point it at the raw event.
Whether you're setting up the catalog for the first time or keeping up with newly-discovered items afterward, the [Missing Description Wizard](/context-layer/event-catalog/missing-description-wizard) batches everything that's still missing a description into a single guided workflow.
# Files
Source: https://docs.clarityq.ai/context-layer/files
Bring documents, spreadsheets, and images into ClarityQ so the agent can use them.
Files let you give the agent knowledge that doesn't live in your warehouse. Upload a CSV of extra data, a catalog you maintain in a spreadsheet, a PDF of business definitions, or an image of a dashboard you want to replicate — and the agent can use it when it answers.
Files work in both directions. You can bring files in, and you can ask the agent to produce them: a CSV of results, a PDF summary, and other formats it can generate in the chat.
## Two Ways to Bring a File In
Where you put a file decides how long it lasts and who else benefits from it.
| | **Context Layer files** | **Chat attachments** |
| - | - | - |
| Where you add it | The **Files** page | The chat composer |
| How long it lasts | Permanently, across every session | That conversation only |
| Who can use it | You, or everyone in the workspace once approved | Just that chat |
| Use it for | Reference data the agent should always know | A one-off file relevant to one question |
The rest of this page is about Context Layer files. For attaching a file to a single conversation — and for asking the agent to generate one — see [Files in a Chat](/ask-anything/asking-questions#files-in-a-chat).
## The Files Page
Files live under **Context Layer → Files** in the sidebar. The table lists each file with its **File name**, **Size**, **Description**, **Uploaded by**, **Last Modified**, and **Status**.
You can search by name or description, and filter by uploader or status. The uploader filter starts set to **you**, so you see your own files first — clear it to see everything in the workspace.
A capacity bar shows how much of your workspace's file storage is used, broken down by approved, private, and excluded files.
## Uploading Files
Click **Upload file**, or drag files onto the table.
Supported formats:
* **Documents** — PDF, Word (DOCX), OpenDocument (ODT), RTF, EPUB, PowerPoint (PPTX)
* **Text** — Markdown, TXT, HTML
* **Data** — CSV, TSV, JSON, Excel (XLSX)
* **Images** — JPEG, PNG, GIF, WEBP
Limits:
* **50 MB** per file
* **10 files** per upload
* **5 GB** total per workspace by default — your workspace may be configured with a different capacity
Uploads run one at a time, and a progress toaster tracks them. You can keep working while they finish, and cancel any that haven't started.
## Descriptions Make Files Discoverable
A file's description is how the agent decides whether the file is relevant to a question. This matters more than anything else on this page.
For text-based files, ClarityQ reads the file and **generates a description automatically**. While that's happening the row shows as processing.
For files it can't read as text — images, mostly — a **wizard opens after your upload finishes** and asks you to describe each one. You can step through them, or skip: a file with no description simply won't be found automatically, though you can still bring it into a chat by `@mentioning` it by name.
You can edit any description later from the Description column.
## File Statuses
Every file has one of three statuses. New uploads always start as **Private**.
| Status | What it means |
| - | - |
| **Private** | The agent uses it only in *your* chats. |
| **Approved** | Part of the shared workspace context — the agent uses it for everyone with access to the workspace. |
| **Excluded** | Hidden from the agent entirely. It stays on the Files page and still counts toward your storage capacity, and you can re-include it at any time. |
**Downloading** is narrower than seeing the row. Anyone who can see an **Approved** file can download it. A **Private** file's contents stay with the person who uploaded it — admins included.
A private file's *contents* are restricted to its uploader, but its **row is visible to everyone in the workspace** — name, description, size, and who uploaded it. Treat "private" as controlling who can open the file, not who can see that it exists. Don't upload a file whose *name* or *description* is itself sensitive.
There are two exceptions where someone else can download a private file — both cases where you already handed it over deliberately: when you `@mentioned` it in a chat you shared, and when you `@mentioned` it in a request you sent to an analyst for review.
## Managing Files
Each row has a menu: **Download**, **Approve**, **Make private**, **Exclude**, **Re-include**, and **Delete**. You can also select several files and act on them together.
Who can do what:
| Action | Who |
| - | - |
| Upload | Anyone with Ask Anything access |
| Approve / Make private | The file's uploader |
| Exclude / Re-include / Delete | **Admin** or **Analyst Contributor** on any file; everyone else on their own private files |
| Edit description | The uploader, or an Admin/Analyst Contributor on an approved file |
Approving a file makes it available to anyone in the org with access to the workspace. Making it private again removes that access. Deleting is permanent.
Files are **live** — they take effect as soon as you change them. Unlike metrics or entities, they aren't part of the draft-and-deploy cycle and have no version history. See [Context Layer Versioning](/context-layer/building/context-layer-versioning).
## Using Files in a Chat
Type `@` in the chat and pick from the **Files** group to point the agent at a specific file. This works even for files with no description, and it's the reliable way to say "use this one."
In a sent message, clicking a file mention downloads the file. Private files are marked so you can tell them apart at a glance.
Behind the scenes the agent reads each file's name, description, and type first, and downloads the contents only when it actually needs them. Excluded files are never fetched.
Files must be enabled for your workspace. If you don't see **Files** in the sidebar, contact your ClarityQ representative. Files aren't available in workspaces that answer across several other workspaces.
# Memory
Source: https://docs.clarityq.ai/context-layer/memory
Persistent context that ClarityQ retains across conversations.
Memory is how ClarityQ remembers things about you and your product between chats. It stores the rules, conventions, and preferences that shape how your data should be interpreted — the kind of context that doesn't live in your database but matters every time someone asks a question.
Without Memory, you'd have to repeat the same context in every conversation. With it, ClarityQ answers in your terms, applies your defaults, and respects rules you've taught it once.
## The Two Levels
Memory operates at two levels. Each level scopes who a memory applies to and who can change it.
| Level | Scope | Examples |
| - | - | - |
| **Personal** | Just you | Your timezone, your preferred chart style, the cohorts you analyze most often |
| **Product** | Everyone working on a specific product | The product's primary KPIs, naming conventions, common filters, internal account exclusions |
## What Belongs in Memory
Good memory entries are short, factual, and durable. They describe a rule or preference that applies broadly, not a one-off fact tied to a single question.
Examples of useful memories:
* "Treat all monetary amounts as USD unless a currency column is present."
* "When asked about 'last month', interpret it as the previous full calendar month, not the trailing 30 days."
* "I work in EMEA — default reports to UTC+1."
Don't use Memory for:
* **Metric and dimension definitions** — these belong in the [Semantic Catalog](/context-layer/semantic-catalog/what-is-the-semantic-catalog), where they're versioned and validated.
* **Repeatable analysis workflows** — these belong in [Skills](/context-layer/skills).
* **One-off facts** only relevant to a single chat — keep those in the conversation itself.
## Adding and Maintaining Memory
Personal and Product memory have slightly different management flows.
### Personal Memory
Personal memory is added and edited entirely in Ask Anything conversations. Tell ClarityQ to remember something during a chat — for example, *"remember that I prefer line charts over bar charts"* — and the agent saves it to your personal memory. To update or remove an entry, ask the agent to make the change the same way.
The **Memory page** lists everything that's been saved on your behalf. Each entry has a **Name** and a **Description** (the memory itself). Delete entries directly from the page when you no longer want them. The **Enable Memory** toggle on the page stops the agent from loading any personal memories without deleting them.
### Product Memory
Product memory has two entry points:
* **Memory page → Product Memory tab** — Click **Add memory** to start a guided flow.
* **Ask Anything** — Type `/product-memory add` (or `/product-memory edit`, `/product-memory delete`) in chat.
Each product memory entry has a **title** (a short label identifying the topic) and **content** (the durable fact, rule, or instruction the team needs ClarityQ to remember).
Product memory follows the same draft-and-release cycle as the rest of the Semantic Catalog: changes are staged as drafts, reviewed, and released as a new version. Use the **edit** and **delete** icons on each row to update or remove entries.
Only Admins can edit Product memory. Anyone in the product can read it, and the agent uses it in every conversation in the product.
## Keep Memory Short
Because memory is loaded into *every* conversation, length has a cost: a long entry is carried on every turn and competes for the agent's attention with everything else. Short, factual entries work better than long ones.
A useful ceiling is around **1,500 characters** per entry. If you ask the agent to save something longer — a multi-step methodology, a worked example, a block of SQL — it will offer you a choice before saving: **Build as skill (Recommended)** or **Keep as product memory**. A [skill](/context-layer/skills) loads only when it's needed, which is why it's the better home for procedures.
The choice is always yours. If you keep it as memory, the agent saves exactly what you asked for and mentions the trade-off once. Running `/audit-changes` also flags memory entries that have grown too long, so you can trim them or move them into a skill later.
## How Memory Is Used
Unlike other parts of the Context Layer that the agent searches on demand, memory is **always loaded** into every conversation. The agent has your personal and product memory available from the start and applies your conventions automatically, without you having to restate them.
# Rules
Source: https://docs.clarityq.ai/context-layer/rules
Mandatory instructions attached to a single Semantic Catalog item, applied whenever the agent uses it.
A rule is a named instruction attached to one Semantic Catalog item — an entity, metric, dimension, segment, or feature. Whenever the agent uses that item to answer a question, it must follow the item's rules. Rules capture the constraints that keep an item used correctly: required filters, business logic, data-quality caveats, and warnings.
Unlike [Memory](/context-layer/memory), which applies across the whole workspace, a rule is **local to the item it's attached to**. The agent only sees a rule when it uses that specific item, and it's surfaced automatically — you don't have to restate it.
## What a Rule Contains
Each rule has two fields:
| Field | Purpose |
| - | - |
| **Name** | A short label for the rule (e.g., *Exclude internal accounts*, *Revenue is in USD*). |
| **Content** | The instruction itself — what the agent must do or account for when using the item. |
There are no rule types or categories — a rule is simply a named instruction. Write the content as a direct instruction, for example *"Always exclude rows where `is_test = true`"* or *"This metric only covers web traffic; note that in any answer that uses it."*
## Which Items Can Have Rules
Rules can be attached to **entities, metrics, dimensions, segments, and features**. [Skills](/context-layer/skills) do not carry rules — a skill describes a procedure, and its own clarifications and todos play that role.
An item can have any number of rules. They're listed in the **Rules** section on the item's detail page.
## How the Agent Uses Rules
When the agent brings an item into a conversation — because a question needs it — it reads that item's rules and treats them as mandatory instructions for the rest of the answer. Rules are never loaded globally; they only apply when their item is actually in use.
Rules also travel between an item and its parent, so a rule can't be missed just because the agent approached the data from a different direction:
* **Looking at an entity brings in its dimensions' rules too.** The agent doesn't have to open each dimension separately to see them. Each rule is labelled with the dimension it belongs to.
* **Looking at a dimension brings in its entity's rules — and the entity's base query.** This is the important one. A dimension's own SQL is usually just a column selected from a table, while the filters that matter — excluding test accounts, a date floor, a currency condition — live in the entity's `WHERE` clause. The agent is told to carry those filters into whatever query it writes, so using a dimension on its own can't quietly drop them.
The practical consequence: a required filter is usually best written as a rule on the **entity**, where it protects every dimension underneath it, rather than repeated on each dimension.
## Managing Rules
Rules are managed from the **Rules** section on an item's detail page in the Semantic Catalog. The section lists each rule by name and content, with controls to add, edit, and delete for users who can edit the catalog. How a change is saved depends on whether the item is versioned.
### Versioned items — entities, metrics, segments, and entity dimensions
For these items, rules are added and changed through the **Builder chat**, so the change goes through the same draft-and-release cycle as any other catalog edit. As with any Builder work, you first need to be editing the Context Layer — click **Start Editing** to take the lock if you don't already hold it. See [How the Builder Works](/context-layer/building/how-the-context-layer-builder-works) for the lock and [Context Layer Versioning](/context-layer/building/context-layer-versioning) for drafts and deploys.
There are two ways to start:
* **From the item's page** — use the **Add rule**, **Edit**, or **Delete** control in the item's Rules section. This opens the Builder chat with the request already filled in (for example, *"add new rule to revenue metric"*), so you just describe the rule.
* **By typing in the Builder chat** — ask for the change in your own words, naming the item and what the rule should do, for example *"add a rule to the revenue metric: always exclude test users where `is_test = true`"* or *"edit the Required Filters rule on the players entity."*
Either way, the agent then:
1. **Finds the item** the rule belongs to. If your description matches more than one item, or none, it asks you to pick or clarify rather than guessing.
2. **Checks the rule fits** — that the columns or concepts it names really exist on the item, and that it doesn't contradict an existing rule or a memory. If something's off, it tells you instead of creating the rule.
3. **Proposes the rule** with a name and content for you to approve. Approve it and the rule is written into your draft; decline and the agent revises it — nothing is saved until you approve.
The rule goes live for everyone when you **deploy** your draft, along with any other changes in it.
### Live items — features and feature dimensions
Features, and dimensions that belong only to a feature, are not versioned — they're managed live (see [Features](/context-layer/semantic-catalog/features)). Their rules are edited directly on the item page: **Add Rule** opens a short form with a **Name** and **Content** field, and the change is saved immediately, with no draft or deploy. Deleting a rule on a live item takes effect right away and can't be undone.
Editing rules requires permission to edit the Context Layer. Analysts and business users see an item's rules read-only, so they can understand how the agent treats that data.
## Rules vs Memory vs Skills
Rules overlap with Memory and Skills, so it helps to keep the distinction clear:
| Use a… | When the instruction is… | Scope |
| - | - | - |
| **Rule** | About how to use *one specific item* correctly | That item only |
| **[Memory](/context-layer/memory)** | True across the whole workspace, regardless of item | Workspace-wide |
| **[Skill](/context-layer/skills)** | A repeatable *procedure* with steps | A named workflow |
A quick test: if the instruction only makes sense when a particular item is in play, it's a rule. If it holds no matter which item the agent uses, it belongs in Memory.
# Dimensions
Source: https://docs.clarityq.ai/context-layer/semantic-catalog/dimensions
Properties used to filter and group data in your analyses.
A dimension is any property that describes an [entity](/context-layer/semantic-catalog/entities). Dimensions serve as attributes, measures, and grouping axes all in one — country, platform, plan type, revenue amount, session duration, signup date are all dimensions.
The agent uses dimensions to filter, group, and analyze data. When someone asks *"break down revenue by country"*, the agent uses the `country` dimension. When someone asks *"show me only iOS users"*, it applies the `platform` dimension as a filter.
## What a Dimension Contains
Each dimension has:
* **Name** — A `snake_case` identifier (e.g., `country`, `platform`, `plan_type`)
* **Description** — What the dimension represents and how it can be used for analysis (e.g., *"Country the user is in. Filter cohorts, group revenue by country."*)
* **SQL** — A fully qualified SQL statement that selects this dimension from the entity's data sources
* **Type** — The data type (e.g., `VARCHAR`, `INTEGER`, `DATE`)
* **Data sample** — A few real values from the data (e.g., `"US"`, `"UK"`, `"DE"`) to help the agent understand the domain
* **Rules** — Optional instructions the agent must follow whenever it uses this dimension (see [Rules](/context-layer/rules))
## How Dimensions Are Created
Dimensions can be created in several ways:
* **During entity creation** — When you build an entity through the [Context Builder](/context-layer/building/how-the-context-layer-builder-works), the agent inspects the source tables and proposes dimensions for columns useful for filtering and analysis.
* **From the Table Catalog** — Select columns from a table and choose an existing entity to add them to as dimensions.
* **In the Builder chat** — Type `/dimension add` to add a dimension to an existing entity.
Not every column needs to be a dimension. Focus on columns your team actually uses for filtering and analysis. Internal IDs, raw timestamps, and technical fields can usually be left out.
## How the Agent Uses Dimensions
The agent combines dimensions with metrics and segments at query time:
* **Grouping** — *"revenue by country"* → the agent adds `country` to the GROUP BY
* **Filtering** — *"only premium users"* → the agent adds a WHERE clause using the relevant dimension
* **Breakdowns** — *"compare iOS vs Android"* → the agent groups by `platform` and presents the comparison
The data sample helps the agent understand valid values — so when a user says "US" the agent knows to filter `country = 'US'` rather than guessing.
## Rules on Dimensions
A dimension can carry [Rules](/context-layer/rules) — named instructions the agent must follow whenever it uses that dimension.
For example, on a `plan_type` dimension:
**Trials are not free users** — *"`trial` is a separate plan. Never fold trial users into `free` when grouping or filtering by plan."*
When the agent uses a dimension it also picks up the rules and base query of the [entity](/context-layer/semantic-catalog/entities) the dimension belongs to. So a filter that applies to the whole entity belongs on the entity, and a dimension rule is for what's true of that property alone — its valid values, how it should be grouped, or a quirk in how it was recorded.
Rules are managed from the **Rules** section on the dimension's page. See [Rules](/context-layer/rules) for how adding and editing works.
# Entities
Source: https://docs.clarityq.ai/context-layer/semantic-catalog/entities
Core data sources in the Semantic Catalog that represent your business objects.
An entity represents a business concept in your Semantic Catalog — users, orders, sessions, transactions, campaigns. Unlike tables in your data warehouse, which reflect how your data is stored, entities reflect how your business thinks about the data.
An entity can draw from **one or multiple tables**. For example, a `users` entity might join your `accounts` table with your `profiles` table and your `subscriptions` table to create a single, complete view of a user. The entity's SQL contains the necessary joins, so the rest of the Semantic Catalog works with the business concept — not the underlying table structure.
Entities are the foundation of the Semantic Catalog. Metrics query data from entities, segments filter users defined by entities, and dimensions belong to entities.
## What an Entity Contains
Each entity has:
* **Name** — A `snake_case` identifier describing the business concept (e.g., `users`, `sessions`, `transactions`)
* **Description** — A plain-language explanation of what the entity represents and what each row means (e.g., *"One row per user account. Contains profile data, signup info, and account status."*)
* **SQL** — A fully qualified SELECT statement that defines the entity's data, including any joins across multiple tables
* **Dimensions** — The properties on this entity that can be used for filtering, grouping, and analysis (see [Dimensions](/context-layer/semantic-catalog/dimensions))
* **Rules** — Optional instructions the agent must follow whenever it uses this entity, such as a filter that must always be applied (see [Rules](/context-layer/rules))
## How Entities Are Created
Entities are built through the [Context Builder](/context-layer/building/how-the-context-layer-builder-works) — either from the Table Catalog or from scratch.
* **From the Table Catalog** — Select one or more tables from the Table Catalog and ask the agent to turn them into entities. The agent inspects the tables, understands their structure, and creates the entity with the appropriate joins and dimensions. See [Building Entities from Tables](/context-layer/building/building-entities-from-tables).
* **In the Builder chat** — Type `/entity add` and specify the tables you want to create entities from. The agent inspects the tables, samples the data, and proposes entity definitions with dimensions.
The agent validates that the source table exists, checks the data grain (what each row represents), and warns about potential issues like missing filters or ambiguous joins.
## How the Agent Uses Entities
When answering questions, the agent searches for relevant entities, uses their SQL as the data source, and applies the entity's dimensions for filtering and grouping. The entity's description helps the agent decide which entity to use when multiple sources could answer a question.
## Rules on Entities
An entity can carry [Rules](/context-layer/rules) — named instructions the agent must follow whenever it uses that entity. They hold the constraints that keep it queried correctly: required filters, business logic, and data-quality caveats.
For example, on a `users` entity:
**Exclude test accounts** — *"Always filter `is_test = false`. Internal QA accounts are otherwise counted as real users."*
A rule on an entity also reaches its dimensions. When the agent uses one of the entity's dimensions on its own, it still sees the entity's rules and its base query — so a required filter can't be dropped by coming at the data from a dimension. That makes the entity the right home for any filter that must apply to everything built on it.
Rules are managed from the **Rules** section on the entity's page. See [Rules](/context-layer/rules) for how adding and editing works.
# Features
Source: https://docs.clarityq.ai/context-layer/semantic-catalog/features
Tracked product events promoted from the Event Catalog into the Semantic Catalog.
A feature represents a tracked product event that has been promoted from the [Event Catalog](/context-layer/event-catalog/what-is-the-event-catalog) into the Semantic Catalog. Events like `signup_completed`, `purchase_made`, or `level_finished` become features once synced, making them available to the agent for answering questions.
## What a Feature Contains
Each feature has:
* **Name** — Derived from the event name
* **Description** — Carried over from the event's approved description in the Event Catalog
* **Dimensions** — The event's parameters and user properties, each becoming a dimension that can be used for filtering and grouping
* **Rules** — Optional instructions the agent must follow whenever it uses this feature (see [Rules](/context-layer/rules))
## How Features Differ from Other Components
Features are **not created through the Context Builder**. They come from the Event Catalog's sync process:
1. An event is approved in the Event Catalog (with a description)
2. The event is synced to the Semantic Catalog
3. It becomes a Feature, with its parameters as dimensions
This is different from entities (built from the Table Catalog), metrics (built in the Builder), and segments (built in the Builder).
## Why Features Exist Alongside the Event Catalog
The Event Catalog and Features serve different purposes. The Event Catalog is a **discovery and documentation tool** — it catalogs every event your product tracks, helps you describe and approve them, and keeps up with new events as they ship.
But the agent doesn't query the Event Catalog directly when answering questions. It works with the **Semantic Catalog**. Features are the bridge: they promote approved events from the Event Catalog into the Semantic Catalog, where the agent can find them, understand their parameters as dimensions, and use them in SQL queries.
Beyond that, events in the Event Catalog are raw — they only contain parameters and user properties as they arrive from your tracking implementation. The underlying events table is often messy, with implementation quirks, naming inconsistencies, and data quality issues.
Features are the **modeled, clean version** of those events. When an event becomes a Feature:
* Implementation errors are corrected
* Only the relevant data is included — noise is stripped away
* Full SQL definitions explain how to query the event correctly
* Descriptions provide business context on what the event means and when it fires
* You can add dimensions beyond just the event's parameters — from table columns or custom definitions
Think of the Event Catalog as the source of truth for *what events exist*, and Features as the curated, queryable versions the agent actually uses.
## How the Agent Uses Features
When someone asks about user behavior — *"how many users completed signup last week?"* or *"what are the top events by volume?"* — the agent finds the relevant feature and uses it to build the query. The feature's dimensions allow breakdowns like *"signups by platform"* or *"purchases filtered to premium plan"*.
## Keeping Features Current
Once an event is synced, new parameters and user properties are added as dimensions automatically as they're approved in the Event Catalog. You don't need to re-sync the parent event.
## Rules on Features
A feature can carry [Rules](/context-layer/rules) — named instructions the agent must follow whenever it uses that feature. So can **each of its dimensions**: a rule on the feature applies whenever the feature is used, while a rule on one of its dimensions applies to that property and is added from that dimension's own **Rules** section.
For example, on a `purchase_made` feature:
**Exclude sandbox purchases** — *"Test-store transactions are present in the data before March 2026. Filter them out for any date range reaching back that far."*
And on that feature's `price` dimension:
**Price is in cents** — *"Values are stored in cents. Divide by 100 before presenting an amount."*
Because features are managed live rather than versioned, rules on a feature and on its dimensions are edited directly on the page and take effect immediately — there's no draft or deploy step. This differs from entities, metrics, and segments, whose rule changes go through the Builder and a versioned release.
See [Building Features from the Event Catalog](/context-layer/building/building-features-from-the-event-catalog) for the full workflow, and [Syncing Events to the Semantic Catalog](/context-layer/event-catalog/syncing-events-to-the-semantic-catalog) for the sync process details.
# Metrics
Source: https://docs.clarityq.ai/context-layer/semantic-catalog/metrics
Standalone business calculations — your KPIs and derived numeric values.
A metric is a standalone business calculation in the Semantic Catalog — a KPI or derived numeric value like revenue, daily active users, conversion rate, or marketing spend.
Unlike dimensions (which belong to entities), metrics are top-level components. Each metric has its own SQL that returns a single number, and the agent combines metrics with dimensions at query time to produce breakdowns and comparisons.
## What a Metric Contains
Each metric has:
* **Name** — A `snake_case` identifier (e.g., `revenue`, `dau`, `conversion_rate`)
* **Description** — What the metric measures in business terms (e.g., *"Total processed cashout amount in dollars over the date window."*)
* **SQL** — A complete, executable query that returns exactly one row with one numeric column
* **Rules** — Optional instructions the agent must follow whenever it uses this metric, such as what the number leaves out (see [Rules](/context-layer/rules))
## Time Behavior
Every metric is parameterized for time in one of three ways:
* **Window metric** — Aggregates over a date range selected by the user (e.g., *revenue this month*, *signups in the last 7 days*). This is the most common type.
* **Point metric** — Calculated for a single date or moment (e.g., *DAU on Tuesday*, *active accounts as of March 1*).
* **Timeless metric** — Independent of time entirely (rare — only for genuinely timeless values like *total accounts ever created*).
The agent applies the appropriate date parameters automatically based on the user's question and the date range selected in Ask Anything.
## How the Agent Uses Metrics
When someone asks a question that involves a metric, the agent:
1. Finds the metric by searching the Semantic Catalog
2. Runs the metric's SQL with the appropriate date parameters
3. Combines it with dimensions if the user asked for a breakdown (e.g., *"revenue by country"*)
4. Applies segment filters if requested (e.g., *"revenue for premium users"*)
Because the SQL is pre-defined, everyone gets the same answer to the same question — regardless of who asks.
## Rules on Metrics
A metric can carry [Rules](/context-layer/rules) — named instructions the agent must follow whenever it uses that metric.
For example, on a `revenue` metric:
**Revenue is gross** — *"This metric does not subtract refunds. When a question asks about net revenue, say so rather than presenting this number as net."*
Rules are the place for the caveats you'd otherwise repeat in every question: what the number does and doesn't include, a currency assumption, or a date before which the data is incomplete.
Rules are managed from the **Rules** section on the metric's page. See [Rules](/context-layer/rules) for how adding and editing works.
## Building Metrics
Metrics are created through the [Context Builder](/context-layer/building/building-metrics). You describe what you want to measure, and the agent drafts the SQL, validates it, and saves it to your draft. See [Building Metrics](/context-layer/building/building-metrics) for the full guide.
# Segments
Source: https://docs.clarityq.ai/context-layer/semantic-catalog/segments
Reusable cohort definitions that filter users across any analysis.
A segment is a reusable cohort definition in the Semantic Catalog — a named group of users matching specific criteria. Once defined, a segment can be applied as a filter to any question, metric, or analysis.
Examples: *premium users*, *users who churned in the last 30 days*, *high spenders*, *users from organic acquisition*.
## What a Segment Contains
Each segment has:
* **Name** — A `snake_case` identifier (e.g., `premium_users`, `churned_users`, `high_spenders`)
* **Description** — Who is included in the segment, in business terms (e.g., *"Users with at least one in-app purchase over the date window."*)
* **SQL** — A complete, executable query that returns a distinct list of user identifiers matching the cohort criteria
* **Rules** — Optional instructions the agent must follow whenever it applies this segment (see [Rules](/context-layer/rules))
The SQL must return exactly one column — the user identifier — with distinct values. No extra columns, no metric values, no breakdowns. The segment purely defines *who* is in the group.
## Time Behavior
Like metrics, segments use date parameters in their SQL:
* Most segments use `[DATE_FROM]` and `[DATE_TO]` to scope the cohort to a date window (e.g., *users who made a purchase in the last 30 days*)
* Some segments are timeless (e.g., *users from a specific country*) and don't need date parameters
The agent applies the date parameters based on the user's question and the selected date range.
## How the Agent Uses Segments
When someone asks *"what was revenue for premium users last week?"*, the agent:
1. Finds the **revenue** metric
2. Finds the **premium users** segment
3. Joins the segment's user list with the metric's query as a filter
4. Returns revenue scoped to that cohort
Segments are composable — the agent can combine them with any metric, entity, or dimension without pre-built combinations.
## Rules on Segments
A segment can carry [Rules](/context-layer/rules) — named instructions the agent must follow whenever it applies that segment.
For example, on a `premium_users` segment:
**Paying plans only** — *"Covers monthly and annual subscribers. Trials are excluded, so this is not the same as 'users with a subscription'."*
Rules keep a cohort from being quietly misread — spelling out who is in it, who isn't, and when it shouldn't be used.
Rules are managed from the **Rules** section on the segment's page. See [Rules](/context-layer/rules) for how adding and editing works.
## Building Segments
Segments are created through the [Context Builder](/context-layer/building/how-the-context-layer-builder-works). Type `/segment add` in the Builder chat or describe the cohort you want to define. The agent drafts the SQL, validates that it returns distinct user identifiers, and saves it to your draft.
# What is the Semantic Catalog?
Source: https://docs.clarityq.ai/context-layer/semantic-catalog/what-is-the-semantic-catalog
The business logic layer that defines how your organization measures and analyzes data.
The Semantic Catalog is where your business logic lives in ClarityQ. It's a structured collection of components that define how your organization thinks about its data — not in terms of tables and columns, but in terms of business concepts like revenue, active users, churn, and conversion.
When someone asks a question in Ask Anything, the agent searches the Semantic Catalog first. It finds the relevant definitions, uses them to build accurate SQL, and returns answers grounded in your team's agreed-upon business logic.
## Why It Matters
Without a Semantic Catalog, the agent would have to guess what "revenue" or "active user" means by looking at raw tables. Different people might get different answers to the same question depending on which table the agent picks or how it interprets a column.
The Semantic Catalog eliminates that ambiguity. When your team defines "revenue" as a metric with specific SQL, every question about revenue uses that exact definition — consistently, for everyone.
## Components
The Semantic Catalog contains five types of components:
### Entities
Entities represent your core data sources — the tables or views that contain the data your team queries. Each entity has a name, description, SQL definition, and a set of dimensions that describe its columns. Entities are the foundation that metrics, segments, and features build on.
[Learn more →](/context-layer/semantic-catalog/entities)
### Dimensions
Dimensions are the properties that describe an entity — anything you can say about it. They serve as attributes, measures, and grouping axes all in one. Country, platform, plan type, revenue amount, session duration, signup date — these are all dimensions. They belong to entities and are used for filtering, grouping, and analysis.
[Learn more →](/context-layer/semantic-catalog/dimensions)
### Metrics
Metrics are standalone business calculations — KPIs and derived numeric values like revenue, daily active users, or conversion rate. Each metric has its own SQL that returns a single number, and can be combined with dimensions at query time for breakdowns.
[Learn more →](/context-layer/semantic-catalog/metrics)
### Features
Features represent tracked product events promoted from the [Event Catalog](/context-layer/event-catalog/what-is-the-event-catalog). When an event is synced, it becomes a Feature with its parameters available as dimensions. Features answer questions about user behavior — what happened, how often, and with what properties.
[Learn more →](/context-layer/semantic-catalog/features)
### Segments
Segments are reusable cohort definitions — named groups of users matching specific criteria. Each segment has SQL that returns a distinct list of user identifiers. Once defined, segments can be applied to any question as a filter.
[Learn more →](/context-layer/semantic-catalog/segments)
## How Components Work Together
The agent combines components at query time to answer questions. For example, when someone asks *"what was revenue last week by country for premium users?"*, the agent:
1. Finds the **revenue** metric
2. Applies the **country** dimension from the relevant entity
3. Filters to the **premium users** segment
4. Builds and runs the SQL
No one has to pre-build that specific combination — the Semantic Catalog makes it composable.
## Rules on Components
Any component — an entity, metric, dimension, segment, or feature — can carry [Rules](/context-layer/rules): named instructions the agent must follow whenever it uses that component, such as a required filter or a data-quality caveat. Rules travel with the component and are applied automatically, so the right constraints are enforced no matter how a question reaches the item.
## Building the Semantic Catalog
Components are created and managed through the [Context Builder](/context-layer/building/how-the-context-layer-builder-works). The agent helps you build them — you describe what you need, and it drafts the definition, validates the SQL, and saves it to your draft for review and deploy.
See [Building the Context Layer](/context-layer/building/recommended-build-order) for the recommended approach.
# Skills
Source: https://docs.clarityq.ai/context-layer/skills
Reusable workflows the agent follows — invoked by you or automatically by the agent.
A skill is a named, repeatable workflow that captures the steps, assumptions, and clarifying questions that make a particular task correct. You can invoke a skill explicitly with `/skill_name` in chat, or the agent can recognize from a skill's description that it's relevant to a question and invoke it automatically.
Skills live alongside [Memory](/context-layer/memory) and the [Semantic Catalog](/context-layer/semantic-catalog/what-is-the-semantic-catalog) in your Context Layer. Where Memory holds durable facts and the Semantic Catalog holds metric and dimension definitions, Skills hold *procedures*.
## What a Skill Contains
| Field | Required | Purpose |
| - | - | - |
| **Name** | Yes | Short identifier used to invoke the skill (e.g., `weekly-retention`). Up to 64 characters. |
| **Description** | Yes | One-line summary shown in the slash command picker. Also used by the agent to decide when to invoke the skill automatically. |
| **Content** | Yes | The methodology itself, written in Markdown, with as much detail as the task needs. |
| **Clarifications** | Optional | Questions the agent asks before running, to gather any required inputs. |
| **Todos** | Optional | The task list the agent works through while executing the skill, kept in sync with the content. |
| **References** | Optional | Supplementary documentation the agent can consult on demand while running the skill. |
A skill can chain to another by mentioning it as `@skill_name` in its content — useful for shared sub-procedures like cohort scoping or experiment extraction.
## Built-in Skills
ClarityQ ships with skills for building and maintaining your Context Layer. They're available in the **Builder chat** to admins and contributors:
| Skill | What it does |
| - | - |
| `/entity` | Create or edit an entity (a data source or table definition). |
| `/dimension` | Create or edit a dimension on an entity. |
| `/metric` | Create or edit a metric. |
| `/segment` | Create or edit a segment (a reusable user cohort). |
| `/skill` | Create or edit a custom skill. |
| `/product-memory` | Create, edit, or remove a product-level memory entry — facts and rules that apply to everyone working on the product. |
| `/audit-changes` | Run a health check focused on the changes introduced in your current draft — surfaces broken references, gaps, and inconsistencies. |
Use these when you want guided creation or editing — the agent walks you through the right SQL, naming conventions, and validations.
## Custom Skills
You can also write **custom skills** for the workflows your team runs repeatedly: a weekly retention drilldown, a launch readout template, a product-area QBR. Custom skills are managed on the **Skills** page.
All custom skills are scoped to the **workspace level** — once deployed, they're available to everyone working in the workspace.
## Invoking a Skill
There are three ways a skill can be invoked:
* **Type `/` in chat** — Type `/` followed by the skill name, or just `/` to open the skills picker and filter as you type.
* **Skills dropdown** — Select a skill from the skills dropdown available from the add menu in the chat input box.
* **Automatically by the agent** — When the agent recognizes from a skill's description that it matches the user's question, it imports and follows the skill on its own — no manual invocation needed.
## When to Use Skills vs Memory vs Rules
* Use **Skills** for repeatable procedures with steps ("how we calculate weekly retention").
* Use **[Memory](/context-layer/memory)** for durable facts and conventions that hold across the whole workspace ("we exclude internal accounts").
* Use **[Rules](/context-layer/rules)** for instructions tied to one specific catalog item ("always filter this metric to paid plans").
If you find yourself explaining the same workflow twice, that's a skill. If an instruction only matters when a particular item is used, that's a rule.
# Daily Discovery Jobs
Source: https://docs.clarityq.ai/context-layer/table-catalog/daily-discovery-jobs
How ClarityQ automatically detects schema changes and keeps your Table Catalog up to date.
Your data warehouse schema isn't static — tables get new columns, data changes, and structures evolve. ClarityQ runs daily discovery jobs to keep your Table Catalog in sync with your warehouse.
## What the Jobs Discover
Each day, ClarityQ re-scans your cataloged tables and detects:
| Change | What Happens |
| - | - |
| **New columns** | Added to the catalog in **Pending** status. An email notification is sent to relevant stakeholders. |
| **Removed columns** | Marked as **Deleted** in the catalog. |
| **Updated column values** | Sample values are refreshed for categorical columns (up to 50 distinct values per column). |
| **Table metadata changes** | Row counts, table size, and last modified timestamps are updated. |
| **Schema changes** | Data type modifications and structural changes are detected and reflected. |
## How to Act on Findings
After a daily job runs, review the changes from the Table Catalog:
### New Columns
New columns appear with **Pending** status. For each one:
* **Review** the column name, data type, and sampled values
* **Accept or edit** the AI-suggested description
* **Approve** the column to confirm it's ready for use
* **Hide** it if it's irrelevant or sensitive
### Removed Columns
Columns that no longer exist in your warehouse are marked as **Deleted**. Review these to confirm they were intentionally removed from the source.
### Updated Values
Column value samples are refreshed automatically. If you see unexpected values, it may indicate a data quality issue upstream.
## Notifications
ClarityQ notifies you when discovery jobs find changes:
* **Email** — Sent when new columns are detected on your tables
* **Slack** — Schema change notifications (if [Slack is connected](/integrations/slack/connecting-slack))
## Manual Refresh
You can trigger a re-discovery at any time without waiting for the daily job:
* **Refresh a single table** — From the table's detail view, trigger a schema refresh
* **Reset column values** — Re-fetch all column values for a specific table
During discovery, the table's state shows as **In Progress**. The catalog auto-refreshes every few seconds so you can track progress in real time.
# Setting Up Your Table Catalog
Source: https://docs.clarityq.ai/context-layer/table-catalog/setting-up-your-table-catalog
How to select tables and run the initial discovery process.
After you [connect your data warehouse](/get-started/connect-your-data-warehouse), the setup wizard directs you to the Table Catalog. From here, you add the tables you want ClarityQ to catalog.
## Adding Tables
Click **Add Tables** in the Table Catalog to open the table selection wizard. The wizard walks you through three steps:
Browse your warehouse's schemas and choose the tables you want ClarityQ to catalog. You don't need to add all tables — focus on the ones relevant to your analytics.
Set SQL WHERE conditions that are automatically applied to every query ClarityQ runs against the selected tables. Useful for scoping to a specific product, excluding deleted records, or filtering out stale data. See [Mandatory Filters](#mandatory-filters) below.
See a summary of the tables that were added to your catalog.
The Table Catalog page then displays the tables you added, along with their discovered schemas, columns, and metadata.
Your organization's admin sets a **maximum number of tables** that can be added to a workspace's context. The agent only uses tables that are in the Table Catalog — tables in your warehouse that haven't been added are invisible to ClarityQ.
Start with your most important tables — the ones your team queries most often. You can always add more tables later by clicking **Add Tables** again.
## What Happens During Discovery
When you add tables, ClarityQ runs a discovery workflow for each one:
1. **Schema extraction** — Queries your warehouse for the table's structure, columns, data types, and constraints
2. **Column value sampling** — Fetches up to 50 distinct values for categorical columns
3. **Metadata collection** — Captures row counts, table size, partitioning, clustering, and timestamps
4. **AI description generation** — Automatically suggests descriptions for tables and columns that don't have one
Discovery runs in parallel across multiple tables. Each table's state is tracked independently — you'll see an **In Progress** spinner while it's running, a **Completed** checkmark when done, or an **Error** indicator if something went wrong.
## Mandatory Filters
Mandatory filters let you define SQL WHERE conditions that are **automatically applied to every query** ClarityQ runs against your tables. They're useful for scoping your data — for example, restricting to a specific product, excluding deleted records, or filtering out stale data.
**Examples:**
```sql theme={null}
bundle_id = 'com.myapp'
```
```sql theme={null}
is_deleted = false AND created_at >= '2024-01-01'
```
### When to use them
* You have multi-tenant tables and want ClarityQ to only see data for your product
* You need to exclude test data, deleted records, or irrelevant rows
* You want to limit the date range ClarityQ queries to avoid scanning historical data you don't need
### How they work
* **Set during table selection** — The table selection wizard lets you define mandatory filters that apply to all newly added tables.
* **Edit per table** — After setup, you can adjust filters for individual tables from the table settings.
* **Validated before saving** — ClarityQ runs a dry-run validation against your warehouse to make sure the filter syntax is correct.
* **Enforced automatically** — Every query ClarityQ generates for a filtered table includes these conditions in the WHERE clause. Users don't see or need to think about them.
Mandatory filters set during table selection apply only to **newly added tables**. Existing tables are not affected — edit their filters individually from the table settings.
## Table Settings
After a table is cataloged, you can configure it from the table settings:
* **Table type** — Classify the table (Events, Fact, Dimension, Lookup, Aggregate, Raw Data). This affects how ClarityQ uses the table.
* **Mandatory filters** — Edit the SQL WHERE conditions for this specific table.
* **Analytics engine** — For Events tables, specify the analytics platform (e.g., GA4) to enable platform-specific enrichment.
* **Descriptions** — Edit table and column descriptions, or accept AI-generated suggestions.
## Reviewing Tables and Columns
After discovery, tables and columns start in **Pending** status. Review them to confirm they're accurate:
* **Approve** tables and columns you've reviewed
* **Hide** columns that are irrelevant or sensitive — they won't appear in the catalog or be used by the agent
* **Edit descriptions** to add business context that helps ClarityQ understand what the data means
# What is the Table Catalog?
Source: https://docs.clarityq.ai/context-layer/table-catalog/what-is-the-table-catalog
A live inventory of your data warehouse tables, columns, and metadata.
The Table Catalog is ClarityQ's centralized metadata repository for your data warehouse. It automatically discovers, catalogs, and enriches your tables and columns with business context — giving ClarityQ the foundational understanding it needs to answer questions about your data.
## What It Contains
### Tables
For each table in your warehouse, the Table Catalog stores:
* **Fully qualified name** — The complete path to the table in your warehouse (e.g., `project.dataset.table_name`), along with the table name as it appears in the database
* **Description** — User-provided or AI-suggested description of what the table contains
* **Table type** — A user-assigned classification that helps ClarityQ understand how to use the table: Events, Fact, Dimension, Lookup, Aggregate, or Raw Data
* **Statistics** — Row count, size, creation date, and last modified timestamp
* **Mandatory filters** — Optional SQL WHERE conditions that are automatically applied to every query against this table (see [Mandatory Filters](/get-started/connect-your-data-warehouse#mandatory-filters))
### Columns
For each column, the Table Catalog captures:
* **Data type** and whether it's nullable or repeated
* **Distinct value count** and up to 50 sample values for categorical columns
* **Description** — User-provided or AI-suggested
* **Nested fields** — Full support for STRUCT/RECORD types with hierarchical column display
### Statuses
Every table and column has a status that reflects whether it's ready to be synced to the Semantic Catalog:
* **Pending** — Does not yet have an approved description
* **Approved** — Has an approved description and is ready to be synced to the Semantic Catalog
Tables also have a processing state during discovery:
* **In Progress** — Currently being discovered or updated
* **Completed** — Discovery finished successfully
* **Error** — Something went wrong during discovery (with a detailed error message)
## How It Fits into the Context Layer
The Table Catalog is the first building block of the Context Layer. It feeds into the Semantic Catalog in two ways:
1. **Entity building** — Create entities (core business objects) directly from your tables and columns
2. **Event syncing** — Tables classified as "Events" can sync to the Semantic Catalog via the [Event Catalog](/context-layer/event-catalog/what-is-the-event-catalog)
# What is the Context Layer?
Source: https://docs.clarityq.ai/context-layer/what-is-the-context-layer
The Context Layer teaches ClarityQ how your business thinks about its data.
The Context Layer is the foundation of ClarityQ. It's a structured model that bridges the gap between your raw database tables and the business concepts your team uses every day — metrics, events, segments, dimensions, and more.
Without a Context Layer, ClarityQ would only see tables and columns. With it, ClarityQ understands what "daily active users" means in your organization, how "churn" is calculated, which events matter, and how your data relates to your business logic.
The Context Layer is a prerequisite for using Ask Anything. The more complete and accurate your context, the better ClarityQ's answers will be.
## The Components
The Context Layer is made up of six components, each serving a distinct purpose:
### Table Catalog
The Table Catalog is a live inventory of the tables and columns in your data warehouse. ClarityQ automatically discovers your schema and keeps it up to date through daily discovery jobs. From here, you build entities that feed into the Semantic Catalog.
[Learn more →](/context-layer/table-catalog/what-is-the-table-catalog)
### Event Catalog
The Event Catalog maps your product's tracked events, parameters, and user properties. Events are synced to the Semantic Catalog, where they become queryable business concepts. A built-in Missing Description Wizard helps you fill in gaps across your catalog.
[Learn more →](/context-layer/event-catalog/what-is-the-event-catalog)
### Semantic Catalog
The Semantic Catalog is where business logic lives. It contains the components that define how your organization measures and analyzes data:
* **Entities** — Core business objects that represent your data sources (e.g., users, orders, sessions)
* **Dimensions** — Properties used to filter and group data (e.g., country, platform, plan type)
* **Metrics** — Business KPIs and calculations (e.g., revenue, retention rate, conversion)
* **Features** — Tracked product behaviors promoted from the Event Catalog (e.g., `signup_completed`, `purchase_made`), with their event parameters available as dimensions
* **Segments** — Reusable cohort definitions (e.g., power users, churned customers)
ClarityQ's AI agent actively helps build these components using your existing definitions and data structure.
[Learn more →](/context-layer/semantic-catalog/what-is-the-semantic-catalog)
### Skills
Skills are reusable analysis methodologies encoded as step-by-step instructions. When invoked, the AI agent follows them precisely — asking clarifying questions, executing the analysis, and validating results. Skills ensure complex workflows run consistently every time.
[Learn more →](/context-layer/skills)
### Memory
Memory is persistent context that ClarityQ retains across conversations. It operates at two levels — personal and product — and stores business rules, naming conventions, and domain knowledge that doesn't live in your database.
[Learn more →](/context-layer/memory)
### Files
Files are documents, spreadsheets, and images you upload so the agent can use them — reference data, catalogs you maintain outside the warehouse, or business definitions written down elsewhere. Once described, the agent finds them on its own when they're relevant.
[Learn more →](/context-layer/files)
## How the Components Connect
```
Data Warehouse
↓
Table Catalog ──→ Entities ──→ Semantic Catalog
↓ ↑
Event Catalog ─── Sync ───────────┘
↑
Memory ───── Context ─────────────┘
↑
Skills ───── Workflows ───────────┘
↓
Ask Anything
```
The Table Catalog and Event Catalog feed into the Semantic Catalog — one through entity building, the other through event syncing. Memory and Skills enrich the agent's understanding and capabilities. Together, they power Ask Anything.
# Asking About a Dashboard's Data
Source: https://docs.clarityq.ai/dashboards/asking-about-dashboard-data
Dig deeper into what your dashboard shows by asking questions with its filters applied.
In ClarityQ, dashboards are not endpoints — they're starting points. A dashboard gives you an overview, but the real exploration happens when you start asking questions about what you see. This is what makes ClarityQ dashboards different: you can have a conversation with your dashboard.
When you ask a question about a dashboard, the agent is fully aware of what you're looking at — which dashboard, which filters you've applied, and which date range you've selected. You don't need to re-explain the context. Just ask about what caught your attention and the agent picks up from where you are.
## How to Ask
### From the Dashboards page
You're viewing a saved dashboard, adjusting the date range and filters, and you notice something unexpected — a sudden drop in a metric or an unusual spike. Click **Ask** in the dashboard's action bar to open a chat that inherits the dashboard's current filters. You can immediately ask *"why did EMEA revenue drop last week?"* without re-setting the date range or filters you were already looking at.
### From a dashboard in chat
When you're viewing a dashboard in the right panel during a conversation, the same flow works inline. Change the dashboard's filters to focus on what interests you, then type your question. The agent answers in the context of the dashboard with those specific filters applied.
After you send a message, a **filters line** appears below your question showing which dashboard and filter set the agent is referencing. Hover over it to see the exact filters the agent used — so you always know what data scope the answer is based on.
If the dashboard has multiple versions in the chat (from iterating with the agent), only the **latest version** has interactive filters. Older versions are visible but their filters are locked. Make sure you're viewing the latest version before changing filters and asking questions.
# Creating and Editing Dashboards
Source: https://docs.clarityq.ai/dashboards/creating-and-editing-dashboards
How to create dashboards with the AI agent, edit them, and work with filters.
## Artifacts vs Dashboards in Chat
Before diving into creation, it's important to understand the distinction between the two types of visual output in chat:
* **Artifacts** are static visualizations displayed **inline** in the conversation. They show the data as it was at the moment the agent generated them.
* **Dashboards** are interactive visualizations that appear as a **reference card** in the conversation. Clicking the card opens the dashboard in the **right panel**, where you can interact with filters, change date ranges, and see live data.
## Creating a Dashboard
### From an artifact in chat
The simplest way to create a dashboard is to turn an existing artifact into one. When the agent generates a visualization as part of an answer, click the **Make it a dashboard** button on the artifact. The agent converts the static visualization into an interactive dashboard — replacing hardcoded data with dynamic SQL queries and adding a date range picker and filters.
You can also ask the agent directly — for example, *"turn this into a dashboard"*.
### With the `/dashboard` command
For a dashboard built from scratch, use the `/dashboard add` command with a description of what you want:
* *"/dashboard add retention dashboard with main retention metrics and trends, with filters for country and platform"*
* *"/dashboard add weekly KPI overview showing installs, revenue, churn rate, and DAU with a date range picker and platform breakdown"*
* *"/dashboard add funnel dashboard showing signup to first purchase conversion with step-by-step drop-off rates"*
The agent builds the full dashboard — visualizations, date range picker, dynamic SQL queries, and semantic component tagging.
Be specific about what you want to see. Mention the metrics, breakdowns, filters, and time ranges you care about. The more detail you give, the closer the first draft will be to what you need.
## Saving a Dashboard
When a dashboard is created in chat, it **only exists within that conversation**. To make it persistent and accessible from the Dashboards page, you need to save it.
Click the **Save Dashboard** button at the top of the dashboard panel. You'll be prompted to give the dashboard a name and an optional description. Once saved, the dashboard appears in your **My Dashboards** view. It starts private — only you can open it until you [share it](/dashboards/sharing-and-managing-dashboards).
Until you save, the dashboard lives only in the chat. If you close the conversation, you can still reopen it and save later — but the dashboard won't appear on the Dashboards page until you do.
## Dashboard Versions in Chat
As you iterate on a dashboard in the same conversation — asking the agent to add charts, change filters, or restructure the layout — each update creates a **new version** in the chat. You'll see multiple dashboard reference cards in the conversation, one for each version.
* **Navigate between versions** by clicking different reference cards — each opens that version in the right panel
* **Only the latest version is interactive** — older versions are visible but their filters are locked
* **Save saves the version you're viewing** — whichever version you have open when you click **Save Dashboard** is the one that gets persisted and becomes accessible on the Dashboards page
## Editing a Dashboard
If a saved dashboard needs changes, use **Edit in Chat** from the dashboard's action bar. This opens a chat where the agent has the full context of the dashboard. Describe what you want to change:
* *"Add a table showing the top 10 countries by revenue"*
* *"Change the bar chart to a line chart"*
* *"Add a platform filter dropdown"*
* *"Remove the KPI card for churn rate"*
The agent modifies the dashboard and you can preview the result in the right panel. Click **Save Dashboard** again to persist the updated version.
## Filters
### Date Range (Required)
Every dashboard has a date range picker with a **default preset** that the agent selects based on the dashboard's content:
* **Trend and time series dashboards** default to a range like Last 7 days, Last 14 days, or Last 28 days
* **Single-day snapshot dashboards** default to Yesterday
Available presets include: Today, Yesterday, This Week, Last Week, Last 7/14/28/90/180 days, This Calendar Month, Last Calendar Month, and a Custom date range.
Changing the date range re-runs all queries and updates every visualization.
### Additional Filters (Optional)
The agent adds filters based on the data and your request. If you don't specify which filters to include, the agent uses its judgment — if the data groups by a categorical column (like country or platform), it becomes a filter candidate.
Filter types include:
* **Dropdown filters** — Select from a list of values (e.g., country, platform)
* **Multi-select filters** — Choose multiple values at once
* **Cascading filters** — A child filter's options depend on the parent's selection (e.g., selecting a country updates the available cities)
Additional filters default to **"All"**, meaning no filtering is applied until you make a selection.
Filters can have their options **hardcoded** (queried once when the dashboard is built, faster to load) or **dynamic** (queried at runtime whenever a parent filter changes, always current).
### Breakdown Filters
Breakdown filters let you choose what dimension to group all the metrics in the dashboard by. Instead of filtering *out* data, a breakdown filter splits it — adding a row for each value of the selected dimension.
For example, a KPI table showing installs, revenue, and churn could have a breakdown filter with options like **Country**, **Platform**, or **Install Date**. Selecting "Country" adds a row for each country, showing all metrics per country. Switching to "Platform" regroups the same metrics by platform.
Breakdown filters are especially useful for tables where you want to explore the same set of metrics across different slices without creating a separate dashboard for each one.
### Changing Filter Defaults
You can change the default date range preset and filter values by asking the agent in **Edit in Chat** — for example, *"change the default date range to last 28 days"* or *"default the platform filter to iOS"*.
You cannot change the set of available date range presets — those are fixed. But you can change which preset is selected by default, and you can add, remove, or restructure additional filters through Edit in Chat.
## Advanced Tips
### Refreshing the Data
Dashboard results are cached, so opening a dashboard doesn't re-query your warehouse every time. When you need the current numbers — after a pipeline run, say — click the **refresh** button in the dashboard's action bar. It re-runs the queries with the cache bypassed and **keeps your current filter selections**; the page doesn't reload and nothing resets. The fresh results become the new cached results for everyone who opens the dashboard next.
### Performance
ClarityQ optimizes dashboard queries for efficiency and caches results to minimize warehouse load. If a dashboard still feels slow:
* **Use a shorter default date range** — A dashboard defaulting to Last 180 days will query much more data than one defaulting to Last 7 days. Ask the agent to change the default if you don't need the full range on every load.
* **Keep dashboards focused** — The agent optimizes queries so that multiple components can share data from fewer queries, but fewer components still means fewer queries overall. Consider splitting large dashboards into multiple smaller ones.
### Pagination
For tables with many rows, the agent adds pagination automatically. Be aware that larger page sizes mean more data transferred per query. If a table feels slow, consider reducing the page size or adding filters to narrow the result set.
### Filters
* **Cascading filters support up to 2 levels** — A parent filter can drive one child filter, but you can't chain three or more levels
* **Hardcoded filter options load faster** than dynamic ones — For stable lists like countries or platforms, hardcoded options avoid an extra query on every load
* **Changing a parent filter resets its child** — If you change the country, the city filter resets to "All"
### Error Handling
If a SQL query fails inside a dashboard — for example, because a table was renamed or a column was removed — that specific component shows an error state. The rest of the dashboard continues to work. Use **Edit in Chat** to fix the broken query.
### Iterating with the Agent
Dashboards rarely come out perfect on the first try. Use **Edit in Chat** to refine rather than starting over. Describe what's wrong — *"the revenue chart is showing gross instead of net"* or *"the table needs a column for percentage change"* — and the agent adjusts the existing dashboard rather than rebuilding from scratch.
# Scheduling Dashboards
Source: https://docs.clarityq.ai/dashboards/scheduling-dashboards
Automate recurring dashboard snapshots and analyses delivered via email or Slack.
You can schedule a dashboard to run automatically on a recurring basis, delivering results via email, Slack, or both. There are two ways to set this up:
## Through chat (recommended)
Describe what you want in natural language while viewing the dashboard — for example:
* *"Send me a daily snapshot of this dashboard for yesterday's data"*
* *"Every Monday morning, send me insights from this dashboard for the previous week"*
The agent automatically generates the right prompt, sets up the schedule, and configures the delivery. This is the easiest way because you don't need to think about how to phrase the task prompt.
## Through the Schedule button
Click **Schedule** from the dashboard's action bar. You'll be asked to write the prompt yourself — this is the instruction the agent will follow on each run. The prompt should describe in words which filter values to apply and what you want from the task.
For example: *"Run this dashboard with the date range set to yesterday. Send the resulting dashboard as a snapshot."*
## Common Use Cases
**Periodic snapshot** — Receive the dashboard with specific filters applied on a regular cadence.
> *"Send me a daily snapshot of this dashboard for yesterday's data every morning at 9am"*
The task runs the dashboard with the date filter set to yesterday and delivers the resulting view.
**Recurring analysis** — Have the agent analyze what the dashboard shows and deliver written insights.
> *"Every Monday, send me the key insights and notable changes you can find in this dashboard for the previous week"*
The task runs the dashboard, the agent reviews the data, and delivers a written summary highlighting trends, anomalies, and notable changes — not just the dashboard itself.
# Sharing and Managing Dashboards
Source: https://docs.clarityq.ai/dashboards/sharing-and-managing-dashboards
Share dashboards with your team, work on them together, and keep track of every version.
Every dashboard has an owner — the person who created it — and an access list that decides who else can open it. Access is granted per dashboard, to named people or to the whole workspace.
## Sharing a Dashboard
Only the owner can change who has access. The owner clicks **Share** and gets the access dialog. When anyone else clicks Share, ClarityQ copies a link to the dashboard instead — handy for pointing a colleague at it, though the link only opens for people who already have access.
The dialog has two parts.
**People with access** — add people by name or email and set each one to **Viewer** or **Editor**. You're listed first as the owner. To take someone off the list, set them to **Remove access**.
**General access** — a single **Everyone in the workspace** row, set to **No access**, **Can View**, or **Can Edit**. Use this when a dashboard should be open to the whole team rather than to named people.
Level and general-access changes apply the moment you make them. When you add people you can tick **Notify people** and include a short message — they get an email saying you shared the dashboard with them.
Someone can have access by more than one route — named individually *and* through the workspace. When that happens they get the higher of the two levels.
## Access Levels
| Level | What it allows |
| - | - |
| **Viewer** | Open the dashboard, change its filters, download it |
| **Editor** | Everything a Viewer can, plus save changes, open **Edit in Chat**, and restore an earlier version |
| **Owner** | Everything an Editor can, plus manage access, rename, archive, and **Send to Analyst** |
Your own level is shown next to the dashboard's name, so you can always tell whether you can edit what you're looking at.
Someone without access is told the dashboard isn't shared with them, and pointed at the owner to ask for it.
If you lose access to a dashboard, it also disappears from your **Starred** view.
## Working on a Dashboard Together
Granting someone **Editor** lets them work on the dashboard with you. They can save changes over it, open **Edit in Chat** to refine it with the agent, and restore an earlier version.
This is not live co-editing. Nobody is locked out and there's no simultaneous editing — people take turns, and if two of you save the same dashboard, ClarityQ asks what to do (see below).
Managing access, renaming, archiving, and **Send to Analyst** stay with the owner.
## Version History
Every save creates a version. Open **Version history** from the dashboard's menu to see them all, newest first, each showing the date and who saved it.
To go back, pick **Restore this version**. Restoring deletes nothing: the version you restore is copied forward as a new version, so the one you were on stays in the history and can be restored again later.
Owners and editors can restore. Anyone with access can read the history.
Version history lists versions — it doesn't compare them. There's no side-by-side view, so restoring is the way to go back.
## When Someone Else Saved First
If the last person to save a dashboard was someone other than you, ClarityQ won't quietly write over their work. It tells you who saved it and when, and offers two ways forward:
* **Replace their version** — saves the way any save works. Yours becomes the current version and theirs stays in the version history, so nothing is lost.
* **Save as new dashboard** — leaves theirs untouched and creates your own copy. The copy starts private and owned by you, and the access list isn't carried over.
This applies whenever someone else saved last, even if that was weeks ago — not only when two of you are working at the same moment.
## Starring
Click the **Star** icon to add a dashboard to your **Starred** view. This is your personal collection of frequently used dashboards — it doesn't affect visibility for anyone else.
## Download as PDF
Click **Download PDF** from the action bar to generate a snapshot of the dashboard at its current filter state. Useful for including dashboard views in presentations or email reports.
## Inspect Mode
Click **Inspect** to see which semantic components (metrics, entities, segments, features) the dashboard uses, along with all the SQL queries it runs — including the current filter values as parameters.
This is especially useful for analysts who want to verify the correctness of a dashboard's logic — both for their own dashboards and for dashboards sent to them for review by other users.
## Send to Analyst
If you want an analyst to review your dashboard — for example, to verify the underlying queries — click **Send to Analyst**. The analyst receives the dashboard in their request queue and can review it, suggest changes, and approve or decline.
## Duplicating
Click **Duplicate** to create a copy of any dashboard you have access to. The copy is yours, starts private, and includes the full dashboard code. Use this to adapt someone else's dashboard to your own needs without affecting the original.
## Archiving
Click **Archive** to hide a dashboard from all views without deleting it. You can **Unarchive** later to restore it. If other people currently have access, ClarityQ tells you how many will lose it before you confirm.
# What are Dashboards?
Source: https://docs.clarityq.ai/dashboards/what-are-dashboards
Interactive, AI-generated visualizations powered by live data from your warehouse.
Dashboards in ClarityQ are interactive visualizations that pull live data from your data warehouse. Unlike the visualizations the agent creates during a chat — which are static snapshots of a single answer — dashboards run real SQL queries and update dynamically as you change filters and date ranges.
Every dashboard includes a **date range picker** and can have additional filters like dropdowns and text search. They contain charts, tables, KPI cards, and other visual components — all generated by the AI agent based on your natural language request.
## How Dashboards Work
When you create a dashboard, the agent:
1. Builds the visualization code with charts, tables, and KPI cards
2. Adds a date range picker with standard presets (last 7 days, this month, custom range, etc.)
3. Replaces any hardcoded data with dynamic SQL queries that run against your warehouse
4. Tags the dashboard with the semantic components it uses (metrics, entities, segments, features)
5. Adds optional filters if relevant to the data
Every time someone opens the dashboard or changes a filter, the queries re-run and the visualizations update with fresh data. Query results are cached to keep things fast — repeated visits with the same filters load instantly without hitting the warehouse again.
## Dashboard Views
You'll find a **Dashboards** area in the sidebar menu. Inside, dashboards are organized into three views:
| View | What it shows |
| - | - |
| **Starred** | Dashboards you've marked as favorites |
| **My Dashboards** | Dashboards you created |
| **Shared with me** | Dashboards you can open but don't own — shared with you directly, or with the whole workspace |
## Status and Visibility
These are two separate things. A dashboard's **status** is about its lifecycle:
| Status | What it means |
| - | - |
| **Live** | In use and visible to anyone it's shared with |
| **Archived** | Hidden from all views, and can be restored |
Who can open it is controlled separately, by its access list. A **shared status** sums that up at a glance:
| Shared status | Who can open it |
| - | - |
| **Private** | Only you |
| A number of people | The people you've shared it with |
| **Workspace** | Everyone in the workspace |
A new dashboard starts private. See [Sharing and Managing Dashboards](/dashboards/sharing-and-managing-dashboards) for how to give others access.
# Amazon Athena
Source: https://docs.clarityq.ai/data-warehouse/athena
Connect Amazon Athena to ClarityQ
## Setup
### Prerequisites in AWS
Before connecting Athena to ClarityQ, you need to set up the required AWS resources and permissions.
#### Step 1: Create an IAM user for ClarityQ
1. Go to the [AWS Management Console](https://console.aws.amazon.com/)
2. Navigate to **IAM** → **Users**
3. Click **Create user**
4. Enter a descriptive name (e.g., `clarityq-athena-user`)
5. Click **Next**
#### Step 2: Assign permissions
Attach the following permissions to the IAM user. You can create a custom IAM policy with the minimum required permissions:
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AthenaAccess",
"Effect": "Allow",
"Action": [
"athena:StartQueryExecution",
"athena:GetQueryExecution",
"athena:GetQueryResults",
"athena:StopQueryExecution",
"athena:GetWorkGroup"
],
"Resource": "*"
},
{
"Sid": "GlueAccess",
"Effect": "Allow",
"Action": [
"glue:GetDatabase",
"glue:GetDatabases",
"glue:GetTable",
"glue:GetTables",
"glue:GetPartitions"
],
"Resource": "*"
},
{
"Sid": "S3StagingAccess",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": [
"arn:aws:s3:::your-staging-bucket",
"arn:aws:s3:::your-staging-bucket/*"
]
},
{
"Sid": "S3DataReadAccess",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::your-data-bucket",
"arn:aws:s3:::your-data-bucket/*"
]
}
]
}
```
**Why does ClarityQ need `s3:PutObject` if it only reads data?**
Athena is a serverless query engine — it doesn't store data itself. When you run a query, Athena reads
your data from S3, processes it, and writes the results to a separate S3 staging directory. The `s3:PutObject`
permission on the staging bucket is required by Athena for this results step. ClarityQ does not write to your
data buckets.
**Important:**
* Replace `your-staging-bucket` with the S3 bucket for Athena query results.
* Replace `your-data-bucket` with the S3 bucket(s) where your source data lives. If your data spans multiple buckets, add each one to the `S3DataReadAccess` statement.
* If you use **AWS Lake Formation** to manage data access, the `S3DataReadAccess` statement may not be needed — Lake Formation can grant table-level permissions independently of S3 IAM policies.
The policy above uses `"Resource": "*"` for Athena and Glue, which grants access to all workgroups and databases.
For tighter security, you can scope permissions to specific resources by replacing `"*"` with resource ARNs.
**Athena — restrict to a single workgroup:**
```json theme={null}
{
"Sid": "AthenaAccess",
"Effect": "Allow",
"Action": [
"athena:StartQueryExecution",
"athena:GetQueryExecution",
"athena:GetQueryResults",
"athena:StopQueryExecution",
"athena:GetWorkGroup"
],
"Resource": "arn:aws:athena:::workgroup/"
}
```
**Glue — restrict to a specific database and its tables:**
```json theme={null}
{
"Sid": "GlueAccess",
"Effect": "Allow",
"Action": [
"glue:GetDatabase",
"glue:GetDatabases",
"glue:GetTable",
"glue:GetTables",
"glue:GetPartitions"
],
"Resource": [
"arn:aws:glue:::catalog",
"arn:aws:glue:::database/",
"arn:aws:glue:::table//*"
]
}
```
Replace ``, ``, ``, and `` with your values.
The Glue `catalog` resource is required for any Glue API call to succeed.
To attach the policy:
1. In the IAM user creation flow, select **Attach policies directly**
2. Click **Create policy**, paste the JSON above, and save it (e.g., `ClarityQAthenaAccess`)
3. Attach the newly created policy to the user
4. Click **Next** then **Create user**
#### Step 3: Generate access keys
1. Select the IAM user you just created
2. Go to the **Security credentials** tab
3. Under **Access keys**, click **Create access key**
4. Select **Third-party service** as the use case
5. Click **Create access key**
6. **Save both the Access Key ID and Secret Access Key** — the secret key will not be shown again
#### Step 4: Set up the S3 staging directory
Athena requires an S3 location to store query results. If you don't already have one:
1. Go to [Amazon S3](https://console.aws.amazon.com/s3/)
2. Click **Create bucket**
3. Choose a name (e.g., `clarityq-athena-results`) and select the same region as your Athena workgroup
4. Keep the default settings (Block all public access enabled) and click **Create bucket**
Make sure the IAM user has read/write access to this bucket (already covered in the policy above).
**Workgroup query results location vs. ClarityQ S3 Staging Directory**
Athena workgroups can have their own **"Query result location"** configured in the AWS console.
If your workgroup has **"Override client-side settings"** enabled, the workgroup's location is used
and the S3 Staging Directory value you enter in ClarityQ is ignored.
Make sure the IAM policy grants `S3StagingAccess` permissions to whichever bucket actually receives
the query results — whether that's the bucket configured in ClarityQ or the one set on the workgroup.
#### Step 5: Verify your Athena workgroup
1. Go to [Amazon Athena](https://console.aws.amazon.com/athena/)
2. Navigate to **Workgroups** in the left sidebar
3. Note the workgroup name you want ClarityQ to use (default is `primary`)
4. Ensure the workgroup's **State** is **Enabled**
5. If you scoped the IAM policy to a specific workgroup (see [Restrict permissions to specific resources](#optional-restrict-permissions-to-specific-resources) above), make sure the workgroup name matches the ARN in your policy
***
### Configure connection in ClarityQ
In the ClarityQ interface, fill out the connection form with the following fields:
#### Required Fields
* **Connection Name**: Choose a name for this connection (e.g., "Production Athena")
* **AWS Region**: The AWS region where your Athena workgroup runs (e.g., `us-east-1`)
* **S3 Staging Directory**: The S3 path for query results (e.g., `s3://your-staging-bucket/athena-results/`)
* **Database**: The Athena database (Glue catalog database) to query (e.g., `default`). You can find your database name in the **Athena Query Editor** under the "Database" dropdown in the left sidebar
* **AWS Access Key ID**: The access key ID generated in Step 3
* **AWS Secret Access Key**: The corresponding secret access key
#### Optional Fields
* **Catalog**: The data catalog name (defaults to `AwsDataCatalog` — only change this if you use a custom or federated catalog)
* **Workgroup**: The Athena workgroup to use (defaults to `primary`)
## Troubleshooting
### Common connection issues
**Access Denied on S3 staging directory**
* Ensure the IAM user has `s3:PutObject` and `s3:GetObject` permissions on the staging bucket
* Verify the S3 path includes a trailing slash (e.g., `s3://bucket/prefix/`)
* Confirm the bucket is in the same region as your Athena workgroup
**Database not found**
* Check that the database exists in the AWS Glue Data Catalog for the specified region
* Verify the catalog name is correct (use `AwsDataCatalog` unless you have a custom catalog)
**Query execution timeout or workgroup errors**
* Confirm the workgroup exists and is enabled in the Athena console
* Ensure the IAM user has `athena:GetWorkGroup` permission for the specified workgroup
**InvalidIdentityToken or credential errors**
* Double-check that the Access Key ID and Secret Access Key are correct
* Ensure the IAM user is active (not deactivated) in the AWS console
* If you rotated keys, update the credentials in ClarityQ
# BigQuery
Source: https://docs.clarityq.ai/data-warehouse/bigquery
Connect Google BigQuery to ClarityQ
## Setup
### Connection Options
ClarityQ offers two ways to connect to BigQuery:
1. **Using ClarityQ service account**
2. **Using your own service account**
***
### Option 1: Using ClarityQ Service Account
For quick setup and testing, you can grant access to ClarityQ's managed service account instead of creating your own.
#### Step 1: Grant access to ClarityQ service account
1. Go to [Google Cloud Console](https://console.cloud.google.com/)
2. Navigate to **IAM & Admin** → **IAM**
3. Click **Grant Access**
4. Add the ClarityQ service account email (this will be displayed in the ClarityQ UI):
```
clarityq-[unique-id]@clarityq-platform.iam.gserviceaccount.com
```
**Note**: The exact email address will be shown in the ClarityQ connection interface.
5. Assign the appropriate BigQuery roles:
* **BigQuery Job User**
* **BigQuery Data Viewer**
* **BigQuery Metadata Viewer**
* **BigQuery Resource Viewer**
* **Service Usage Consumer**
#### Step 2: Configure connection in ClarityQ
In the ClarityQ interface, fill out the connection form:
**Basic Fields:**
* **Connection Name**: Choose a name for this connection (e.g., "BigQuery via ClarityQ SA")
* **Project ID**: Your Google Cloud project ID
**Service Account Configuration:**
Leave "Use Own Service Account" toggle disabled (default). You'll see a message like:
> To connect Google BigQuery, please grant access to ClarityQ service account:
Click "Generate ClarityQ Service Account" to get the email address, then copy the exact service account email shown (each connection has a unique identifier).
***
### Option 2: Using Your Own Service Account
#### Step 1: Create a service account
1. Go to [Google Cloud Console](https://console.cloud.google.com/)
2. Navigate to **IAM & Admin** → **Service Accounts**
3. Click **Create Service Account**
4. Enter a descriptive name (e.g., `clarityq-service-account`)
5. Add a description for future reference
6. Click **Create and Continue**
#### Step 2: Assign permissions
Assign the appropriate BigQuery roles to your service account:
1. In the role assignment step, assign these roles:
* **BigQuery Job User**
* **BigQuery Data Viewer**
* **BigQuery Metadata Viewer**
* **BigQuery Resource Viewer**
* **Service Usage Consumer**
2. Click **Continue** then **Done**
#### Step 3: Generate service account key
1. Select your service account from the list
2. Go to the **Keys** tab
3. Click **Add Key** → **Create New Key**
4. Choose **JSON** format
5. Click **Create**
6. Save the downloaded JSON file securely
#### Step 4: Configure connection in ClarityQ
In the ClarityQ interface, fill out the connection form:
**Basic Fields:**
* **Connection Name**: Choose a name for this connection (e.g., "Production BigQuery")
* **Project ID**: Your Google Cloud project ID
**Service Account Configuration:**
Toggle "Use Own Service Account" to choose your authentication method:
**Own Service Account (toggle enabled):**
* **Service Account Key**: Upload your JSON service account key file (drag & drop or browse)
**Optional: Quota Project Configuration**
Toggle "Use Quota Project" if you want to specify a different project for billing:
* **Quota Project ID**: Project ID to use for billing (if different from main project)
***
## Troubleshooting
### Common connection issues
**BigQuery API not enabled**
* Error: `BigQuery API has not been used in project ... before or it is disabled`
* The BigQuery API must be enabled at the project level — this is separate from IAM permissions
* Go to [Google Cloud Console > APIs & Services](https://console.cloud.google.com/apis/library/bigquery.googleapis.com) and click **Enable** for your project
* Wait a few minutes after enabling, then retry the connection
***
### Service account key structure
The `service_account_key` must contain:
```json theme={null}
{
"type": "service_account",
"project_id": "your-project-id",
"private_key_id": "key-id",
"private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
"client_email": "service-account@project.iam.gserviceaccount.com",
"client_id": "123456789",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/..."
}
```
# ClickHouse
Source: https://docs.clarityq.ai/data-warehouse/clickhouse
Connect ClickHouse to ClarityQ
## Setup
### Step 1: Create a dedicated user for ClarityQ
Connect to ClickHouse as an admin (a SQL client, or the **SQL Console** on ClickHouse Cloud) and create a service account:
```sql theme={null}
-- Create the ClarityQ user
CREATE USER clarityq_user IDENTIFIED WITH sha256_password BY 'your_secure_password';
```
On ClickHouse Cloud you can instead issue a JWT for the account and use **Access Token** authentication in Step 4.
### Step 2: Grant database permissions
Configure the required permissions for ClarityQ to discover schemas, query data, and cancel its own queries:
```sql theme={null}
-- Grant read access to the databases ClarityQ should access
GRANT SELECT ON analytics.* TO clarityq_user;
GRANT SELECT ON events.* TO clarityq_user;
-- Or grant access to specific tables only
GRANT SELECT ON analytics.orders TO clarityq_user;
-- Grant query cancellation, so stopping a question stops the work on your server
GRANT SELECT(query_id, user, query) ON system.processes TO clarityq_user;
```
ClickHouse filters its metadata views by these grants, so the account's `SELECT` privileges determine which tables ClarityQ discovers.
### Step 3: Configure network access
Ensure your ClickHouse server allows connections from the following ClarityQ IP addresses:
* 44.218.213.75
* 54.161.37.107
* 54.208.212.67
On ClickHouse Cloud, select your service, go to **Settings** → **Security** and add these addresses to the **IP access list**. A new service starts with no addresses allowed, so ClarityQ reaches it once they are added.
On a self-hosted server, allow inbound connections on the HTTP port (`8443` or `8123`) in your firewall or security group.
### Step 4: Configure connection in ClarityQ
In the ClarityQ interface, fill out the connection form with the following fields:
#### Required Fields
* **Connection Name**: Choose a name for this connection (e.g., "Production ClickHouse")
* **Host**: Your ClickHouse hostname (e.g., `your-instance.clickhouse.cloud`)
* **Port**: `8443` (default, pre-filled) for TLS, `8123` for plaintext HTTP
* **Connect over TLS**: Enabled (default). Disable it for a self-hosted server that serves the plaintext HTTP port on a private network
* **User**: `clarityq_user` (the user you created in Step 1)
#### Optional Fields
* **Database**: Leave empty to catalogue every database the user can read, or set it to confine ClarityQ to one database (e.g., `analytics`)
#### Authentication Type
Choose your authentication method using the toggle buttons:
**Password Authentication (default):**
* **Password**: Enter the password you set for `clarityq_user`
**Access Token Authentication:**
* **Access Token (JWT)**: Paste a JWT issued for `clarityq_user` (ClickHouse Cloud)
Set the **Port** to match the transport whenever you toggle **Connect over TLS**: ClickHouse serves TLS on `8443` and plaintext on `8123`.
### Step 5: Test the connection
Verify the connection in ClarityQ to ensure:
* Authentication succeeds
* Databases are visible to `clarityq_user`
* Table metadata can be discovered
* Query execution works correctly
## Troubleshooting
### Common connection issues
**Connection times out**
* Add the ClarityQ IP addresses to the Cloud IP access list, firewall or security group (see Step 3)
* Enter the hostname on its own, with the scheme and port left out of the field
**Connection refused or TLS handshake error**
* Match the **Port** to the transport: `8443` with **Connect over TLS** enabled, `8123` with it disabled
* Behind a proxy, use the port your ClickHouse URL uses
**Database not found**
* Check the spelling of the **Database** field against the databases listed in the error message
* Clear the field to catalogue every database the user can read
**Database is not visible to the user**
* Grant the user access to it: `GRANT SELECT ON your_database.* TO clarityq_user`
**User has no grants on any database**
* Grant `SELECT` on the databases ClarityQ should read (see Step 2)
**Cannot read system.databases**
* Allow the user to read ClickHouse's metadata views, which ClarityQ uses to discover schemas
**Authentication failed**
* Re-enter the credential your server uses, either Password or Access Token — one credential is stored per connection
* For access tokens, issue a fresh JWT if the previous one has expired
**Tables missing after discovery**
* Clear the **Database** field to catalogue every database instead of the one you named
* Grant the user `SELECT` on the missing databases
* Query `system` and `information_schema` tables by name; they stay out of the catalog
**Stopping a question leaves the query running**
* Grant `SELECT(query_id, user, query) ON system.processes` so ClarityQ can cancel with `KILL QUERY` (see Step 2)
# Databricks
Source: https://docs.clarityq.ai/data-warehouse/databricks
Connect Databricks to ClarityQ
## Catalog options
ClarityQ supports two Databricks catalog types:
* **Unity Catalog**: the modern, recommended governance layer for Databricks. Supports both serverless and classic SQL warehouses.
* **Legacy Hive Metastore**: the original `hive_metastore` catalog. Requires a **classic (cluster-based) SQL warehouse**, as serverless SQL warehouses do not support the legacy Hive Metastore.
You'll select your catalog type in the ClarityQ connection form (Step 4).
## Setup
### Step 1: Get Databricks connection details
To connect ClarityQ to your Databricks workspace using a service principal, you'll need the following information:
1. **Server hostname**
2. **HTTP path**
3. **Service principal Application ID**
4. **Service principal OAuth secret**
### Step 2: Find your Databricks connection details
#### Server Hostname
1. Log in to your Databricks workspace
2. In the sidebar, click **SQL Warehouses** (or **Compute** → **SQL Warehouses**)
3. Click on your SQL warehouse name
4. Go to the **Connection Details** tab
5. Copy the **Server hostname** (e.g., `your-workspace.cloud.databricks.com`)
#### HTTP Path
1. In the same **Connection Details** tab of your SQL warehouse
2. Copy the **HTTP path** (e.g., `/sql/1.0/warehouses/abc123def456`)
If you plan to use the **Legacy Hive Metastore**, make sure your HTTP path points to a **classic (cluster-based) SQL warehouse**, not a serverless one. Serverless SQL warehouses do not support the legacy Hive Metastore.
### Step 3: Create service principal credentials
Create a dedicated service principal for ClarityQ (recommended for production environments):
#### Create service principal for ClarityQ
1. In your Databricks workspace, go to **Settings** → **Identity and access**
2. Click the **Service principals** tab
3. Click **Add service principal**
4. Enter name: `clarityq-service-principal`
5. Click **Add**
6. Click on the newly created service principal
7. **Copy the Application ID** - you'll need this for the ClarityQ connection, where it will be entered in the **Client ID** input field (UUID format like `065e73cc-a713-4ae0-8212-ba7f98e32bee`)
8. Go to the **Secrets** tab
9. Click **Generate secret**
10. Set expiration date (recommended for security)
11. Click **Generate** and copy the secret immediately
12. In the same service principal details page, go to **Permissions** tab
13. Click **Grant access** or **Add permissions**
14. Search for and select **Service Principal: Use**
15. Save the changes
16. This allows the service principal to execute queries (required for ClarityQ)
#### Grant SQL Warehouse access
You need to grant the service principal permission to use the SQL Warehouse:
1. In the sidebar, click **SQL Warehouses** (or **Compute** → **SQL Warehouses**)
2. Click on your SQL warehouse name
3. Go to the **Permissions** tab
4. Search for and select `clarityq-service-principal`
5. In the permission dropdown, select **Can use**
6. Click **Save**
#### Grant data access permissions to ClarityQ
**Note**: These steps require **workspace admin** or **account admin** permissions. If you don't have these permissions, ask your Databricks administrator to perform these steps.
Choose the tab that matches your catalog type:
Grant SELECT permissions to your ClarityQ service principal for the data you want to analyze:
**Step 1: Navigate to your catalog**
1. Go to **Catalog** in your Databricks workspace
2. Choose the catalog containing data you want ClarityQ to access
**Step 2: Grant SELECT permissions**
1. Select the **Permissions** tab
2. Click the **Grant** button
3. Search for and select your `clarityq-service-principal`
4. Check the **SELECT** privilege
5. Click **Grant** to save
**Granular permission options:**
* **Whole catalog**: Grants ClarityQ access to all schemas and tables in the catalog
* **Specific schema**: Navigate to the schema level and repeat the above steps to grant access to specific schemas only
* **Specific tables**: Navigate to individual tables and grant access to only the tables you want ClarityQ to analyze.
**For metadata discovery (recommended):**
Also grant SELECT permissions on `information_schema` so ClarityQ can discover available tables and columns.
The legacy Hive Metastore does not use the Unity Catalog permissions UI. Instead, you must grant permissions via SQL.
Run the following SQL statements in a Databricks SQL editor or notebook, replacing `` with your service principal's Application ID:
```sql theme={null}
GRANT USAGE ON CATALOG hive_metastore TO ``;
GRANT USAGE ON SCHEMA hive_metastore.default TO ``;
GRANT SELECT ON SCHEMA hive_metastore.default TO ``;
GRANT READ_METADATA ON SCHEMA hive_metastore.default TO ``;
```
Replace `default` with the name of each schema you want ClarityQ to access, and repeat for additional schemas as needed.
The `READ_METADATA` grant is required for the Hive Metastore. It allows ClarityQ to discover table names, column names, and column types, equivalent to the `information_schema` SELECT access used in Unity Catalog. Without it, ClarityQ can authenticate successfully but won't be able to see which tables exist.
### Step 3b (optional): Register ClarityQ as an OAuth app for per-user permissions
Skip this step unless you want each person's queries to run under their own Databricks identity (see [Run queries as the asking user](#run-queries-as-the-asking-user-per-user-permissions) below). It requires a **Databricks account admin**.
In the Databricks **account console** (not the workspace settings), go to **Settings → App connections** and click **Add connection**. Only account admins see this page.
* **Application Name**: `ClarityQ`
* **Redirect URLs**, one per line: `https://app.clarityq.ai/api/v1/integrations/data-warehouse/oauth/callback`
Use **Add scope** to add `sql` and `offline_access`. `sql` lets ClarityQ run statements on your SQL warehouses; `offline_access` issues the refresh token that keeps a person connected between sessions. The `openid`, `email` and `profile` scopes may already be listed; they are harmless. `all-apis` is not needed.
* Keep **Generate a client secret** checked. ClarityQ authenticates to Databricks as a confidential client.
* **Enable single-use refresh tokens** is optional. With it on, every refresh issues a new refresh token, so people who use ClarityQ regularly are never asked to reconnect; only someone idle for longer than the refresh token lifetime is. ClarityQ handles the rotation.
* **Access token TTL**: the default 60 minutes is fine (allowed range 5 to 1,440).
* **Refresh token TTL**: the default 10,080 minutes is 7 days. When it lapses without use, the person reconnects and their scheduled tasks stop until they do. Raise it, up to 129,600 minutes (90 days), if that is too short for your team.
Click **Add**, then copy the **Client ID** and the **Client secret**. The secret is shown once; you will paste both into the ClarityQ connection form as **OAuth App Client ID** and **OAuth App Client Secret**.
Everyone who will ask questions on this connection must exist as a user in the Databricks account, with **Can use** on the SQL warehouse and `SELECT` on the data they may see. Their Databricks email must match their ClarityQ login email.
### Step 4: Configure connection in ClarityQ
In the ClarityQ interface, fill out the connection form with the following fields:
#### Required Fields
* **Connection Name**: Choose a name for this connection (e.g., "Production Databricks")
* **Server Hostname**: Your Databricks server hostname (e.g., `your-workspace.cloud.databricks.com`)
* **HTTP Path**: Your SQL warehouse HTTP path (e.g., `/sql/1.0/warehouses/abc123def456`)
* **Client ID**: Your service principal Application ID (UUID format, e.g., `065e73cc-a713-4ae0-8212-ba7f98e32bee`)
* **Client Secret**: The service principal OAuth secret generated in Step 3
* **Catalog type**: Select **Unity Catalog** (default) or **Legacy Hive Metastore**
* Choose **Legacy Hive Metastore** only if your data resides in the original `hive_metastore` catalog and you are using a classic (cluster-based) SQL warehouse
#### Run queries as the asking user (per-user permissions)
Leave this **off** and every query runs as the service principal above: Databricks sees one identity, and ClarityQ's own permissions decide what people can reach.
Turn it **on** and ClarityQ runs each person's queries under their own Databricks identity. Unity Catalog then applies that user's grants, **row filters** and **column masks**, and your Databricks audit log records the real asker rather than the service principal. Two people asking the same question can legitimately get different rows.
Databricks has no way for one identity to act as another, so this works through OAuth: each person signs in to Databricks once from ClarityQ and allows it to run SQL on their behalf. That is why the toggle asks for three more fields, taken from [Step 3b](#step-3b-optional--register-clarityq-as-an-oauth-app-for-per-user-permissions):
* **OAuth App Client ID**
* **OAuth App Client Secret**
* **Refresh token TTL (minutes)** — copy the value from the OAuth app. Leave the default 10080 (7 days) if you did not change it there. ClarityQ uses it to warn people two days before their sign-in lapses.
What changes when it is on:
* Each user connects their Databricks account once, from the **Connect your Databricks account** card Ask Anything shows on a new chat, or under **Settings → Apps → Data Warehouse** (see [Connecting Your Data Warehouse Account](/ask-anything/connecting-your-data-warehouse-account)). Until they do, nothing runs for them on this connection. ClarityQ never falls back to the service principal for a person's query.
* Cached results are kept separate per user, so nobody is served rows that were filtered for someone else.
* Shared dashboards resolve per viewer: the author sees their rows, a colleague with wider access sees theirs.
* Scheduled tasks run under their owner's connected account. If that account's sign-in expires, the task fails until the owner reconnects.
* Schema discovery, sample values and other background work have no asking user, so they continue to run as the service principal. Its grants decide what the Context Layer can see.
Policy stays where you already keep it. ClarityQ writes nothing into Unity Catalog and holds no copy of your grants; it states who is asking, and Databricks decides.
### Step 5: Test the connection
Verify the connection in ClarityQ to ensure:
* Databricks workspace accessibility
* SQL warehouse availability
* Token authentication
* Catalog and schema access
* Query execution capabilities
## Connection parameters reference
### Required parameters
| Parameter | Type | Description | Example |
| - | - | - | - |
| `connection_name` | string | Unique identifier for this connection | `Production Databricks` |
| `server_hostname` | string | Databricks workspace hostname | `your-workspace.cloud.databricks.com` |
| `http_path` | string | SQL warehouse HTTP path | `/sql/1.0/warehouses/abc123def456` |
| `client_id` | string | Service principal Application ID, entered as **Client ID** in ClarityQ | `065e73cc-a713-4ae0-8212-ba7f98e32bee` |
| `client_secret` | string | Service principal OAuth secret, entered as **Client Secret** in ClarityQ | `dapis...` |
### Optional parameters
| Parameter | Type | Default | Description |
| - | - | - | - |
| `catalog` | string | Unity Catalog | Set to `hive_metastore` for the legacy Hive Metastore |
| `impersonate_users` | boolean | `false` | Run each query under the asking user's own Databricks identity |
| `oauth_client_id` | string | — | Client ID of the OAuth app registered for ClarityQ; required when `impersonate_users` is on |
| `oauth_client_secret` | string | — | Client secret of that OAuth app, stored in ClarityQ's secret manager; required when `impersonate_users` is on |
| `refresh_token_ttl_minutes` | integer | `10080` | Must match the OAuth app's **Refresh token TTL**; drives the reconnect reminder. Maximum `129600` |
## Troubleshooting
### Common connection issues
**Authentication failed**
* Verify your service principal OAuth secret is correct and not expired
* Check if the service principal has necessary permissions through Admin Settings
* Ensure the user/service principal exists and is active
**Service principal access issues**
* **Most common cause**: Service principal not added to workspace yet
* Ask your Databricks administrator to add the service principal to the workspace
* **Missing permissions**: Ask your admin to use the role-based approach described above
* **Insufficient permissions**: Ensure ClarityQ has **Service Principal Use** role + **read access** to desired data
* **SQL Warehouse access denied**: Ensure the service principal has **Can use** permission on the SQL Warehouse (see Step 3)
**Per-user permissions issues**
* **A connect prompt in a chat or on a dashboard**: the connection runs queries as the asking user and this person has not connected yet, or their sign-in expired. They connect from the **Connect Databricks** button in the chat, or under **Settings → Apps → Data Warehouse**.
* **"Please sign in to Databricks as …"**: the Databricks account used on the consent screen is not the ClarityQ user's own. Sign out of Databricks and connect again with the matching email.
* **User can connect but gets `Access Denied` or no rows**: with per-user permissions on, it is that user's grants that apply, not the service principal's. Check their `SELECT` grants, **Can use** on the warehouse, and your row filters.
* **Row filters not applied**: Unity Catalog row filters and column masks apply to tables, not views, and only on Unity Catalog tables on a SQL warehouse. Nothing in `hive_metastore` is filtered.
* **Scheduled task suddenly fails**: the owner's refresh token expired (7 days by default on the OAuth app). The owner reconnects, or the account admin raises the refresh token lifetime.
* **"Databricks did not return a refresh token"**: the OAuth app is missing the `offline_access` scope.
**Legacy Hive Metastore issues**
* **Tables not visible**: Ensure `READ_METADATA` has been granted on the schema. Without it, ClarityQ can connect but cannot discover tables.
* **Serverless warehouse errors**: The legacy Hive Metastore is not supported on serverless SQL warehouses. Switch to a classic (cluster-based) SQL warehouse.
* **Permission denied on GRANT statements**: Hive Metastore grants require workspace admin or the `GRANT` privilege. Ask your administrator to run the SQL grant statements.
# MaxCompute
Source: https://docs.clarityq.ai/data-warehouse/maxcompute
Connect Alibaba Cloud MaxCompute to ClarityQ
## Setup
### Step 1: Get your Alibaba Cloud credentials
To connect ClarityQ to your MaxCompute project, you'll need the following information:
1. **Access Key ID**
2. **Access Key Secret**
3. **MaxCompute Endpoint**
4. **Project Name**
### Step 2: Create a RAM user for ClarityQ
Create a dedicated RAM (Resource Access Management) user for ClarityQ to securely access your MaxCompute data:
1. Log in to the [Alibaba Cloud Console](https://ram.console.alibabacloud.com/)
2. Navigate to **RAM** → **Users**
3. Click **Create User**
4. Enter the login name `clarityq`
5. Check **Permanent AccessKey** ("Create an AccessKey ID and Secret for API or SDK access")
6. A confirmation prompt will appear — check **I confirm that it is necessary to create an AccessKey**
7. Click **OK**
8. **Copy the Access Key ID and Access Key Secret** immediately — the secret is only shown once
### Step 3: Assign permissions to the RAM user
Grant the ClarityQ RAM user the necessary permissions to read your MaxCompute data.
First, find your Main Account UID and the RAM user's UID:
1. Go to the [RAM Console](https://ram.console.alibabacloud.com/) → **Users**
2. Click on the `clarityq` user
3. Copy the **User UID** (a numeric ID) — this is your ``
4. Your **Main Account ID** can be found by hovering over your avatar in the top-right corner of the Alibaba Cloud Console — this is your ``
Then run the following SQL commands in the MaxCompute SQL editor, replacing ``, ``, and `` with your values:
```sql theme={null}
-- Add the RAM user to your MaxCompute project
ADD USER RAM$:;
-- Create a read-only role for ClarityQ
CREATE ROLE clarityq_reader;
-- Grant the role permission to run queries, list tables, and read all table data
GRANT CreateInstance, List ON PROJECT TO ROLE clarityq_reader;
GRANT Describe, Select ON TABLE * TO ROLE clarityq_reader;
-- Assign the role to the ClarityQ user
GRANT clarityq_reader TO RAM$:;
```
To grant access to specific tables only (instead of all tables):
```sql theme={null}
GRANT Describe, Select ON TABLE TO ROLE clarityq_reader;
```
### Step 4: Find your MaxCompute endpoint
The MaxCompute endpoint is region-specific. Use the endpoint that corresponds to the region your project is deployed in:
1. Go to the [MaxCompute Endpoints documentation](https://www.alibabacloud.com/help/en/maxcompute/user-guide/endpoints)
2. Find the row matching your project's region
3. Copy the appropriate endpoint URL
### Step 5: Configure connection in ClarityQ
In the ClarityQ interface, fill out the connection form with the following fields:
#### Required Fields
* **Connection Name**: Choose a name for this connection (e.g., "Production MaxCompute")
* **Access Key ID**: Your RAM user's Access Key ID
* **Access Key Secret**: Your RAM user's Access Key Secret
* **Endpoint**: Your region-specific MaxCompute endpoint (e.g., `https://service.us-east-1.maxcompute.aliyun.com/api`)
* **Project Name**: Your MaxCompute project name (e.g., `my_analytics_project`)
### Step 6: Test the connection
Verify the connection in ClarityQ to ensure:
* Alibaba Cloud authentication succeeds
* MaxCompute project is accessible
* Table metadata can be discovered
* Query execution works correctly
## Connection parameters reference
### Required parameters
| Parameter | Type | Description | Example |
| - | - | - | - |
| `connection_name` | string | Unique identifier for this connection | `Production MaxCompute` |
| `access_key_id` | string | Alibaba Cloud RAM user Access Key ID | `LTAI5t...` |
| `access_key_secret` | string | Alibaba Cloud RAM user Access Key Secret | `gKxR8...` |
| `endpoint` | string | Region-specific MaxCompute service endpoint (see Step 4) | `https://service.us-east-1.maxcompute.aliyun.com/api` |
| `project_name` | string | MaxCompute project name | `my_analytics_project` |
## Troubleshooting
### Common connection issues
**Authentication failed**
* Verify your Access Key ID and Secret are correct and active
* Ensure the RAM user has **Permanent AccessKey** enabled
* Check that the Access Key has not been disabled or deleted in the RAM console
**Project not found**
* Confirm the project name is spelled correctly (case-sensitive)
* Verify the endpoint region matches the region where your project was created
* Ensure the RAM user has been granted access to the project
**Permission denied on tables**
* Confirm the RAM user has been granted **SELECT** permissions at the project level
* For project-level permissions, ensure the `ADD USER` command was run before granting table access
* Check if the project owner has restricted access via security policies or label-based security
# PostgreSQL
Source: https://docs.clarityq.ai/data-warehouse/postgres
Connect PostgreSQL to ClarityQ
## Setup
### Step 1: Create a dedicated database user
Connect to your PostgreSQL database and create a service account for ClarityQ:
```sql theme={null}
-- Create the ClarityQ user
CREATE USER clarityq_user WITH PASSWORD 'your_secure_password';
-- Grant database connection privilege
GRANT CONNECT ON DATABASE your_database TO clarityq_user;
```
### Step 2: Configure schema and table permissions
Grant the necessary permissions for ClarityQ to discover schemas, query data, and perform analytics:
```sql theme={null}
-- Grant schema usage for data schemas
GRANT USAGE ON SCHEMA public TO clarityq_user;
GRANT USAGE ON SCHEMA your_schema TO clarityq_user;
-- Grant schema access for metadata discovery
GRANT USAGE ON SCHEMA information_schema TO clarityq_user;
GRANT USAGE ON SCHEMA pg_catalog TO clarityq_user;
-- Grant SELECT on existing tables for data querying
GRANT SELECT ON ALL TABLES IN SCHEMA public TO clarityq_user;
GRANT SELECT ON ALL TABLES IN SCHEMA your_schema TO clarityq_user;
-- Grant SELECT on system tables for schema introspection
GRANT SELECT ON ALL TABLES IN SCHEMA information_schema TO clarityq_user;
GRANT SELECT ON ALL TABLES IN SCHEMA pg_catalog TO clarityq_user;
-- Grant access to views for complete schema discovery
GRANT SELECT ON ALL TABLES IN SCHEMA information_schema TO clarityq_user;
```
### Step 3: Configure network access
Ensure your PostgreSQL database allows connections from the following ClarityQ IP addresses:
* 44.218.213.75
* 54.161.37.107
* 54.208.212.67
Configure your firewall, security groups, or database access rules to allow inbound connections from these IPs on port 5432.
### Step 5: Add connection in ClarityQ
In the ClarityQ interface, fill out the connection form with the following fields:
* **Connection Name**: Choose a name for this connection (e.g., "Production PostgreSQL")
* **Host**: Your PostgreSQL server hostname or IP address (e.g., `db.example.com`)
* **Port**: `5432` (default, pre-filled)
* **Database**: Your database name (e.g., `analytics_db`)
* **User**: `clarityq_user` (the user you created in Step 1)
* **Password**: The password you set for `clarityq_user`
## Connection parameters reference
### Required parameters
| Parameter | Type | Description | Example |
| - | - | - | - |
| `connection_name` | string | Unique identifier for this connection | `Production DB` |
| `host` | string | Database server hostname or IP address | `db.example.com` |
| `database` | string | Target database name | `analytics_db` |
| `user` | string | PostgreSQL username | `clarityq_user` |
| `password` | string | User password | `secure_password` |
### Optional parameters
| Parameter | Type | Default | Description |
| - | - | - | - |
| `port` | number | `5432` | PostgreSQL server port |
| `schema` | string | `public` | Default schema |
# Amazon Redshift
Source: https://docs.clarityq.ai/data-warehouse/redshift
Connect Amazon Redshift to ClarityQ
## Setup
### Step 1: Gather cluster information
1. Log into [AWS Management Console](https://console.aws.amazon.com/)
2. Navigate to **Amazon Redshift** → **Clusters**
3. Select your cluster
4. Note the following:
* **Endpoint**: e.g., `mycluster.abc123.us-west-2.redshift.amazonaws.com`
* **Port**: Usually `5439`
* **Database**: Your database name
### Step 2: Configure network access
Choose one of the following connection methods:
#### Option 1: Direct IP Access
Allow ClarityQ to connect by configuring your security group:
1. Navigate to **EC2** → **Security Groups**
2. Find the security group attached to your Redshift cluster
3. Add an inbound rule:
* **Type**: Custom TCP
* **Port**: 5439
* **Source**: Add these ClarityQ IP addresses:
* `44.218.213.75`
* `54.161.37.107`
* `54.208.212.67`
#### Option 2: AWS PrivateLink
Establish a private connection without exposing traffic to the public internet:
**Step 2a: Create a Network Load Balancer (if you don't have one)**
If you don't already have a Network Load Balancer for your Redshift cluster:
1. **Navigate to EC2 Load Balancers**
* In AWS Console, search for "EC2" and select **EC2**
* In the left navigation pane, under **Load Balancing**, click **Load Balancers**
2. **Create Network Load Balancer**
* Click **Create Load Balancer**
* Select **Network Load Balancer** and click **Create**
3. **Configure Load Balancer**
* **Load balancer name**: Enter a name (e.g., `redshift-nlb`)
* **Scheme**: Select **Internal** (for private connectivity)
* **IP address type**: Select **IPv4**
* **VPC**: Choose the same VPC as your Redshift cluster
* **Availability Zones**: Select the subnets where your Redshift cluster is accessible
4. **Configure Listener**
* **Protocol**: TCP
* **Port**: 5439 (Redshift default port)
* **Target group**: Create a new target group with:
* **Target type**: IP addresses
* **Protocol**: TCP
* **Port**: 5439
* **VPC**: Same as your Redshift cluster
* **Add your Redshift cluster's IP as a target**
5. **Create the Load Balancer**
* Review settings and click **Create load balancer**
* Wait for it to become **Active** (takes a few minutes)
**Step 2b: Create an Endpoint Service**
1. **Open AWS Console and navigate to VPC**
* Go to [AWS Management Console](https://console.aws.amazon.com/)
* In the search bar, type "VPC" and select **VPC**
2. **Access Endpoint Services**
* In the left navigation pane, scroll down to **Virtual private cloud**
* Click **Endpoint services**
3. **Create the Endpoint Service**
* Click **Create endpoint service**
4. **Configure the Endpoint Service**
* **Service name**: Leave this empty (AWS will auto-generate)
* **Load balancer type**: Select **Network**
* **Available load balancers**: Choose your Network Load Balancer from Step 2a
* **Require acceptance for endpoint**: Check this box (recommended for security)
* **Supported IP address types**: Select **IPv4** (default)
5. **Review and Create**
* Click **Create endpoint service**
* AWS will generate a service name in the format: `com.amazonaws.vpce..vpce-svc-`
* **Important**: Copy this service name - you'll need it for Step 2e
**Step 2c: Authorize ClarityQ Access**
1. In your Endpoint Service settings, add ClarityQ's AWS account as "Allow principals":
```
arn:aws:iam::452271769350:root
```
**Step 2d: Configure Security Group for PrivateLink**
If your NLB has an associated security group, you must allow inbound traffic from ClarityQ's VPC CIDR ranges:
1. Navigate to **EC2** → **Security Groups**
2. Find the security group attached to your Network Load Balancer
3. Add inbound rules:
* **Type**: Custom TCP
* **Port**: 5439
* **Source**: Add these ClarityQ VPC CIDR ranges:
* `10.2.0.0/16` (Production)
* `10.1.0.0/16` (Staging)
* `10.0.0.0/16` (Development)
These CIDR ranges are required because PrivateLink traffic appears to originate from ClarityQ's VPC. Without these rules, connections will time out even if the endpoint service is correctly configured.
**Step 2e: Note Service Details for ClarityQ Configuration**
You'll need these details when configuring the connection in ClarityQ:
* **Service name**: Your endpoint service name
* **AWS Region**: The region where your service is located
* **Cluster details**: Endpoint, port, database name, and schema
### Step 3: Create a dedicated user
Connect to your Redshift cluster and create a service account:
```sql theme={null}
-- Create user with password
CREATE USER clarityq_user
WITH PASSWORD 'your_secure_password'
CONNECTION LIMIT 10;
```
### Step 4: Grant database permissions
Configure the required permissions for ClarityQ to connect, discover schemas, and query your data:
```sql theme={null}
-- Grant database connection
GRANT CONNECT ON DATABASE your_database TO clarityq_user;
-- Grant schema discovery and usage (for all schemas ClarityQ should access)
GRANT USAGE ON SCHEMA public TO clarityq_user;
GRANT USAGE ON SCHEMA your_schema TO clarityq_user;
-- Grant table read access for data querying
GRANT SELECT ON ALL TABLES IN SCHEMA public TO clarityq_user;
GRANT SELECT ON ALL TABLES IN SCHEMA your_schema TO clarityq_user;
```
### Step 5: Configure connection in ClarityQ
In the ClarityQ interface, fill out the connection form with the following fields:
#### For Option 1 (Direct IP Access):
**Basic Fields:**
* **Connection Name**: Choose a name for this connection (e.g., "Production Redshift")
* **Host**: Your Redshift cluster endpoint (e.g., `mycluster.abc123.us-west-2.redshift.amazonaws.com`)
* **Port**: `5439` (default, pre-filled)
* **Database**: Your database name (e.g., `analytics_db`)
* **User**: `clarityq_user` (the user you created in Step 3)
* **DB Schema**: `public` (default, pre-filled)
#### For Option 2 (AWS PrivateLink):
**Basic Fields:**
* **Connection Name**: Choose a name for this connection (e.g., "Production Redshift PrivateLink")
* **Host**: The DNS name of your VPC Endpoint Service (e.g., `com.amazonaws.vpce.us-west-2.vpce-svc-abc123def456.us-west-2.vpce.amazonaws.com`)
* **Port**: `5439` (default, pre-filled)
* **Database**: Your database name (e.g., `analytics_db`)
* **User**: `clarityq_user` (the user you created in Step 3)
* **DB Schema**: `public` (default, pre-filled)
#### Authentication Method (for both options)
Choose one of the following by toggling "Use IAM Authentication":
**Password Authentication (default):**
* **Password**: Enter the password you set for `clarityq_user`
**IAM Authentication (toggle enabled):**
* **Cluster Identifier**: Your Redshift cluster name (e.g., `mycluster`)
* **Region**: AWS region (e.g., `us-west-2`)
* **Access Key ID**: AWS access key from Step 4 (Method 2)
* **Secret Access Key**: AWS secret key from Step 4 (Method 2)
## IAM authentication setup
For enhanced security, configure IAM-based authentication:
### Step 1: Create IAM policy
Create a policy for Redshift access:
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"redshift:GetClusterCredentials"
],
"Resource": [
"arn:aws:redshift:us-west-2:123456789012:dbuser:mycluster/clarityq_user"
]
}
]
}
```
### Step 2: Attach policy to IAM user or role
1. Create or select an IAM user/role
2. Attach the policy created above
3. Generate access keys if using IAM user
### Step 3: Configure in ClarityQ
Use the IAM configuration with:
* `cluster_identifier`: Your cluster name
* `region`: AWS region
* `access_key_id` and `secret_access_key`: AWS credentials for the IAM user/role with the policy created above
# Snowflake
Source: https://docs.clarityq.ai/data-warehouse/snowflake
Connect Snowflake to ClarityQ
## Setup
### Step 1: Create a dedicated user for ClarityQ
Connect to Snowflake and create a service account:
```sql theme={null}
-- Create a user with the public key authentication
CREATE USER clarityq_user
RSA_PUBLIC_KEY=''
DEFAULT_ROLE = clarityq_role
COMMENT = 'User for ClarityQ to access specific table';
```
### Step 2: Create and configure a role
Set up appropriate permissions through a dedicated role:
```sql theme={null}
-- Create a dedicated role with limited permissions
CREATE ROLE clarityq_role;
-- Grant SELECT permission on the specific table
GRANT USAGE ON DATABASE ALIEN_EVENTS TO ROLE clarityq_role;
GRANT USAGE ON SCHEMA ALIEN_EVENTS.PUBLIC TO ROLE clarityq_role;
GRANT SELECT ON TABLE ALIEN_EVENTS.PUBLIC.ALIEN_EVENTS TO ROLE clarityq_role;
-- Or grant access to all the tables in the schema
GRANT SELECT ON ALL TABLES IN SCHEMA ALIEN_EVENTS.PUBLIC TO ROLE clarityq_role;
-- Assign the role to the user
GRANT ROLE clarityq_role TO USER clarityq_user;
```
### Step 3: Create a ClarityQ warehouse (or use an existing one)
```sql theme={null}
-- Create or use existing warehouse
CREATE WAREHOUSE IF NOT EXISTS COMPUTE_WH
WITH WAREHOUSE_SIZE = '{XSMALL | SMALL | MEDIUM | LARGE}'
AUTO_SUSPEND = 300
AUTO_RESUME = TRUE;
-- Grant usage permission
GRANT USAGE ON WAREHOUSE COMPUTE_WH TO ROLE CLARITYQ_ROLE;
```
### Step 4: Configure connection in ClarityQ
In the ClarityQ interface, fill out the connection form with the following fields:
#### Basic Fields (Required for all connections)
* **Connection Name**: Choose a name for this connection (e.g., "Production Snowflake")
* **Account**: Your Snowflake account identifier (e.g., `xy12345.us-east-1`)
* **User**: `clarityq_user` (the user you created in Step 1)
* **Warehouse**: Your warehouse name (e.g., `COMPUTE_WH`)
* **Database**: Your database name (e.g., `ANALYTICS_DB`)
* **DB Schema**: Schema name (e.g., `public`)
* **Role**: `clarityq_role` (the role you created in Step 2)
#### Authentication Type
Choose your authentication method using the toggle buttons:
**Password Authentication:**
* **Password**: Enter the password you set for `clarityq_user`
**Private Key Authentication:**
* **Private Key**: Paste your RSA private key (multi-line text field)
**Key-pair authentication setup:**
For enhanced security, configure RSA key-pair authentication:
*Generate RSA key pair:*
```bash theme={null}
# Generate private key
openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8 -nocrypt
# Generate public key
openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub
```
*Configure in Snowflake:*
```sql theme={null}
-- Set public key for user
ALTER USER clarityq_user SET RSA_PUBLIC_KEY='';
```
## Connection parameters reference
### Required parameters
| Parameter | Type | Description | Example |
| - | - | - | - |
| `account` | string | Snowflake account identifier with region | `xy12345.us-east-1` |
| `user` | string | Snowflake username | `clarityq_user` |
| `warehouse` | string | Compute warehouse name | `COMPUTE_WH` |
| `database` | string | Target database name | `ANALYTICS_DB` |
### Optional parameters
| Parameter | Type | Default | Description |
| - | - | - | - |
| `connection_name` | string | - | Custom name for this connection |
| `db_schema` | string | `public` | Default schema within database |
| `role` | string | - | Snowflake role to use |
### Authentication methods
Choose one of the following authentication methods:
#### Password authentication (default)
| Parameter | Type | Description |
| - | - | - |
| `password` | string | User password |
#### Key-pair authentication
| Parameter | Type | Description |
| - | - | - |
| `private_key` | string | Private key in PEM format |
# Trino
Source: https://docs.clarityq.ai/data-warehouse/trino
Connect Trino to ClarityQ
ClarityQ talks to your Trino coordinator over its HTTP API, so any deployment that speaks the standard Trino protocol works — a self-managed cluster on Hadoop, Kubernetes or bare metal, as well as commercial distributions such as Starburst.
Because Trino is a federation engine, one connection is enough for a whole cluster: by default ClarityQ discovers every catalog it exposes, and queries can span them. ClarityQ sends one SQL statement and Trino performs any cross-source work itself.
## Filling in the connection form
The fields below appear in the order the form asks for them.
### Connection Name
Any label you will recognise later, for example `Production Trino`. It has no effect on the connection.
### Coordinator Host
The hostname of your Trino **coordinator** — the node that accepts client queries. Not a worker, and not a load balancer that fronts workers.
```
trino.example.com
```
### Port
`443` for a TLS-terminated coordinator, which is the normal production setup and the default here. Clusters behind a proxy sometimes publish a different port; use whatever your Trino URL uses.
ClarityQ always connects over HTTPS.
### Default Catalog (optional)
How wide ClarityQ looks:
* **Empty** — every catalog on the cluster is discovered. Start here.
* **Set** — discovery is confined to that catalog.
`system`, `jmx`, `tpch` and `tpcds` are skipped either way, and stay queryable by full name.
### User
The username ClarityQ authenticates as. Trino attributes every query to it, so it is the name that appears in your cluster's query history and the identity your access-control rules apply to. A dedicated service account is recommended; it needs read access only, and the grants are listed further down this page.
### Authentication
Pick whichever your coordinator uses:
* **Password** — for clusters using LDAP or a password file. Paste the service account's password.
* **Access Token** — a JWT issued by the OAuth2/OIDC provider protecting your coordinator (Keycloak and similar). Paste a token for the service account; a long-lived one avoids re-entering it.
Credentials are stored in ClarityQ's secret manager, never in the database, and are only ever sent over HTTPS.
### Run queries as the asking user (per-user permissions)
Leave this **off** and every query runs as the service account above: your cluster sees one identity, and ClarityQ's own permissions decide what people can reach.
Turn it **on** and ClarityQ instead runs each query as the person asking it. Your access-control layer — Ranger, OPA, Immuta or Trino's file-based rules — then applies that user's grants, row filters and column masks. Your cluster's query history also records the real asker rather than the service account, so your audit trail stays meaningful. Two people asking the same question can legitimately get different rows, and someone without access to a table gets a permission error.
It requires two things on your side:
1. The service account must be allowed to act as your end users (an `impersonation` rule in file-based access control, or the equivalent in Starburst and Ranger).
2. Your cluster must authorise those users under the name ClarityQ sends, which is **the user's full ClarityQ email address** — `dana@yourcompany.com`, not `dana`. Either write your rules against the email, or map it to your internal username in your identity provider.
Scheduled refreshes and catalog discovery have no asking user, so they continue to run as the service account.
Cached results are also kept separate per user, so nobody is served rows that were filtered for someone else.
Policy stays where you already keep it. ClarityQ writes nothing into your cluster and holds no copy of your grants — it states who is asking, and your engine decides. Existing Ranger or OPA policies apply to ClarityQ unchanged, with no permission model to duplicate.
## What to grant the service account
Through your access-control layer, the account needs:
1. `SELECT` on every table ClarityQ should analyze
2. Read access to `information_schema` in every catalog you want discovered — that is how ClarityQ finds tables
3. Permission to cancel its own queries, so aborting a question stops the work on your cluster instead of leaving it running
4. An `impersonation` rule, only if you enable per-user permissions
No write, DDL or admin privileges are needed. ClarityQ only reads.
## Network access
Allow inbound connections to the coordinator from ClarityQ:
* 44.218.213.75
* 54.161.37.107
* 54.208.212.67
## Connection parameters reference
| Parameter | Required | Default | Description |
| - | - | - | - |
| `connection_name` | yes | — | Label for this connection |
| `host` | yes | — | Trino coordinator hostname |
| `port` | no | `443` | Coordinator port |
| `catalog` | no | — | Confines discovery to this catalog, and resolves unqualified names against it; empty discovers every catalog |
| `user` | yes | — | Service account username |
| `password` | one of | — | LDAP / password-file credential |
| `access_token` | one of | — | JWT credential |
| `impersonate_users` | no | `false` | Run each query as the asking user |
## How ClarityQ uses the connection
* Table discovery reads `information_schema` — in the catalog you named, or in each catalog returned by `SHOW CATALOGS` when you named none. A catalog that cannot be read is skipped rather than failing discovery, so one misconfigured connector does not hide the rest.
* A cluster-wide scan is time-boxed. On a cluster with very many catalogs it can stop early, and the catalogs it did not reach are named in ClarityQ's logs; naming a catalog on the connection avoids the scan entirely.
* Partition detection probes the connector's `"$partitions"` metadata table, so date-partitioned Hive, Iceberg and Delta tables get partition-pruned filters automatically. Tables partitioned by separate `year`/`month`/`day` columns are recognised as partitioned but are not date-filtered, since no single column identifies a date.
* Row counts and table sizes come from `SHOW STATS`; running `ANALYZE ` on important tables improves them.
* Nested `ROW` and `ARRAY(ROW(...))` columns are expanded, so event-parameter style columns are described in the catalog.
* Cancellation calls `DELETE /v1/query/{id}`, which your cluster authorises as killing a query the account owns. If that is not permitted, ClarityQ falls back to `CALL system.runtime.kill_query`, which needs permission to execute that procedure. Either one is enough.
## Troubleshooting
| Symptom | Likely cause |
| - | - |
| Test connection hangs, then fails | The coordinator is not reachable from ClarityQ's IPs, or the port is wrong |
| `Access Denied` on a table | The account being used lacks `SELECT` there — check your Ranger, OPA, Immuta or file-based rules. With per-user permissions on, that is the asking user, not the service account |
| `Catalog '…' is not available on this cluster` | The catalog field is misspelled; the message lists the catalogs that do exist. Leave the field empty to use all of them |
| `Cannot impersonate user …` | Per-user permissions is on but no impersonation rule grants it |
| Tables missing after discovery | You named a catalog, so only that one is searched; or the service account lacks access to the catalog; or it is a benchmark catalog (`tpch`, `tpcds`) that discovery skips — those remain queryable as `catalog.schema.table` |
| Tables a user cannot query still appear in the catalog | Discovery runs as the service account, so the tree reflects what it can see. Per-user rules are enforced when the query runs |
| Cancelling appears to do nothing | The account may not be allowed to kill its own queries (see the note above) |
| Users see more or fewer rows than expected | With per-user permissions on, results reflect that user's own grants — check that ClarityQ emails map to your Trino usernames |
# Best Practices
Source: https://docs.clarityq.ai/get-started/best-practices
Tips for getting the most accurate and useful answers from ClarityQ.
These tips will help you get better results from ClarityQ — whether you're asking your first question or running complex analyses daily.
## Asking Questions
### Be concise and intentional
Translate business questions into clear analysis questions. The clearer the question, the better the analysis.
### Ask ClarityQ how to investigate your problem
If you're unsure where to start, ask ClarityQ what questions you should ask.
For example: *"I see a drop in revenue per user over the last 4 weeks. What should I ask to get the most accurate analysis?"*
### Split complex questions into smaller ones
Break down big or multi-part questions and use follow-ups. This helps ClarityQ reason more effectively and reduces confusion.
## Using the Semantic Layer
### Reference semantic layer components with @mentions
Prefer metrics and segments from the Semantic Catalog. Type `@` and select from the popup to ensure consistency and correctness across your analyses.
## Managing Conversations
### Start a new session when changing context
If you move to a different problem or business area, start a fresh chat to avoid context bleed from previous questions.
### Use Edit when you want to refine, not restart
Edit is best for small corrections — clarifying a question, adjusting a filter, fixing wording, or changing a metric. No need to start over for minor changes.
### Turn off visualizations unless needed
Save tokens and focus on reasoning first. Enable charts only when you need to visualize results.
## Validating Results
### Validate complex results
ClarityQ improves constantly, but no system is 100% accurate. Always sanity-check complex queries and conclusions.
### If something looks odd in a visualization, say it
Tell the agent what seems wrong and ask it to fix, explain, or regenerate the visualization.
### Challenge results that don't make sense
You can investigate further, disagree with the agent, and ask it to re-check assumptions or analyze the data from a different angle.
## What's Next?
Set up your data warehouse connection.
Learn about the Context Layer and its components.
# Build Your Context Layer
Source: https://docs.clarityq.ai/get-started/build-your-context-layer
Understand the Context Layer and the recommended order for building it.
Before anyone on your team can ask questions in ClarityQ, the **Context Layer** needs to be built. The Context Layer is what teaches ClarityQ how your business thinks about its data — it bridges the gap between raw database tables and the business concepts your team uses every day.
The Context Layer is a prerequisite for using Ask Anything. The more complete and accurate your context, the better ClarityQ's answers will be.
## What's in the Context Layer?
The Context Layer is made up of six components:
A live inventory of the tables and columns in your data warehouse. ClarityQ automatically discovers and indexes your schema.
A catalog of product events, parameters, and user properties — synced from your analytics tracking and mapped to the semantic catalog.
The core business logic layer. Define metrics, features, segments, and dimensions that reflect how your organization measures and analyzes data.
Persistent context that ClarityQ retains across conversations — at the personal and product level.
Reusable analysis methodologies that guide the AI agent through complex, multi-step workflows consistently.
Documents, spreadsheets, and images you upload so the agent can draw on knowledge that doesn't live in your warehouse.
## Recommended Build Order
The components build on each other. Follow this order for the best results:
Once you've [connected your data warehouse](/get-started/connect-your-data-warehouse), choose which tables ClarityQ can use. Open **Add Tables to the Context** from the Table Catalog and pick them — there's a limit on how many a workspace can hold, so choose the ones that matter. ClarityQ then profiles those tables and suggests descriptions for you to review, and you build entities from them.
If your product tracks events (e.g., via Mixpanel, Amplitude, or custom tracking), set up the Event Catalog. Use the Missing Description Wizard to fill in gaps, then sync events to the Semantic Catalog.
This is where you define business logic. Create entities, dimensions, metrics, features, and segments that map to how your team actually talks about data. ClarityQ's AI agent actively helps build these components — leveraging your existing definitions, documentation, and data structure.
Add the knowledge that doesn't live in your database — business rules, naming conventions, domain-specific context. Memory improves answer quality over time.
Once your team has established repeatable analysis patterns, encode them as skills. The agent will follow these step by step for consistent, high-quality results.
You don't need to complete every component before using Ask Anything. Once your Table Catalog and a few core Semantic Catalog components are in place, you can start asking questions — and continue building context as you go.
# Connect Your Data Warehouse
Source: https://docs.clarityq.ai/get-started/connect-your-data-warehouse
Set up a secure connection between ClarityQ and your data warehouse.
The first step to using ClarityQ is connecting your data warehouse. ClarityQ uses **read-only access** to query your data — it never writes to or modifies your database.
## Supported Data Warehouses
Google BigQuery
Snowflake
Amazon Redshift
PostgreSQL
Databricks
Alibaba Cloud MaxCompute
Amazon Athena
Trino (incl. Starburst)
ClickHouse
## Setup Process
You need permission to manage the data warehouse connection, which **Admins** and **Integration Managers** have.
1. **Configure credentials** — Follow the platform-specific guide to create a service account or user with read-only access.
2. **Start the wizard** — It opens on a welcome screen; click **Start**.
3. **Select your data warehouse** — Choose from the supported platforms above.
4. **Enter your connection details**, then click **Test Connection**. The **Finish** button stays disabled until the test passes, so you can't save a connection that doesn't work.
The setup wizard walks you through each step. Most connections take under 10 minutes to configure.
## What Happens After You Connect
Connecting a warehouse doesn't give ClarityQ your tables — you choose them. The wizard's final screen offers **Add tables now**, which opens the **Add Tables to the Context** wizard in the [Table Catalog](/context-layer/table-catalog/what-is-the-table-catalog).
That wizard runs in four steps: select the tables, add table documentation, define mandatory filters, and review what you added. See [Setting Up Your Table Catalog](/context-layer/table-catalog/setting-up-your-table-catalog) for the full walkthrough.
## What Every Connection Includes
* **Read-only access** — ClarityQ never writes to your database
* **Encrypted data transmission** via SSL/TLS
* **Authentication options** tailored to each platform (service accounts, IAM roles, dedicated users)
* **Connection pooling** for optimal performance
* **Troubleshooting steps** in each platform guide
## Per-User Permissions
By default every query runs as the service account you configure, and ClarityQ's [roles](/admin/user-management-and-roles) decide what people can reach. If your data warehouse already enforces who may see which rows, [Databricks](/data-warehouse/databricks#run-queries-as-the-asking-user-per-user-permissions) and [Trino](/data-warehouse/trino#run-queries-as-the-asking-user-per-user-permissions) connections can instead run each query under the asking user's own identity, so your warehouse's grants, row filters and masks apply to what they ask. Each person then [connects their own account](/ask-anything/connecting-your-data-warehouse-account) once.
# What is ClarityQ?
Source: https://docs.clarityq.ai/get-started/what-is-clarityq
AI-powered data analytics that learns your business context and lets anyone ask questions in natural language.
ClarityQ is an AI-powered data analytics platform that lets anyone in your organization ask questions about your data in plain English and get accurate answers, insights, and visualizations — no SQL or data expertise required.
ClarityQ connects to your data warehouse and builds a **Context Layer** — a rich semantic model of your business data. Our AI agent actively helps build this layer, leveraging your existing documentation, semantic definitions, and data structure to accelerate the process. Once context is in place, your data team stays focused on high-impact work while product, marketing, operations, and business teams get the answers they need — instantly and independently.
## How It Works
ClarityQ works in two phases: **building context** and **asking questions**.
Connect ClarityQ to your data warehouse — BigQuery, Snowflake, Athena, Redshift, PostgreSQL, Databricks, MaxCompute, Trino, or ClickHouse. ClarityQ uses read-only access to query your data securely.
ClarityQ learns your business by building a **Context Layer** — a semantic model that maps your raw data to business concepts like metrics, features, segments, and dimensions. This is what makes ClarityQ's answers accurate and relevant to your organization.
Once context is built, anyone on your team can ask data questions in plain English. ClarityQ's AI agent translates questions into SQL, executes queries, and returns answers with charts, tables, and insights.
Schedule recurring questions as automated tasks, share conversations with your team, and build up memory so ClarityQ gets smarter over time.
## Key Capabilities
Ask data questions in natural language. ClarityQ generates SQL, runs queries, and returns answers with rich visualizations — no SQL or data expertise required.
A searchable semantic layer that maps your raw data to business concepts. Includes table catalog, event catalog, semantic catalog, files, skills, and memory.
Turn any answer into an interactive dashboard that pulls live data, with filters and date ranges your team can explore.
Schedule recurring questions as tasks and receive answers automatically via email or Slack on a cadence you define.
A two-tier memory system (personal and product) that retains context across conversations and improves answers over time.
Define entities, metrics, features, segments, and dimensions that reflect how your business thinks about data — not how your database stores it.
Encode your team's best analysis methodologies into reusable skills. The AI agent follows them step by step — asking clarifying questions, executing the analysis, and validating results — so complex workflows run consistently every time.
## Who Is ClarityQ For?
| Role | How ClarityQ Helps |
| - | - |
| **Data Analysts** | Build and maintain the context layer, embed ClarityQ into daily workflows, and run sophisticated analyses in minutes instead of hours or days. |
| **Product Managers** | Get instant answers about user behavior, feature adoption, and funnel metrics without waiting for the data team. |
| **Marketing Teams** | Analyze campaign performance, cohort behavior, and conversion rates in natural language. |
| **Business & Operations** | Track KPIs, monitor trends, and generate reports on demand. |
| **Data Engineers** | Maintain the context layer and keep business definitions aligned with the underlying data. |
# Your First Chat
Source: https://docs.clarityq.ai/get-started/your-first-chat
Ask your first data question in ClarityQ and understand what to expect.
Once your data warehouse is connected and you have some Context Layer components in place, you're ready to ask your first question.
## Starting a New Conversation
Navigate to **Ask Anything** from the sidebar. You'll see a welcome screen with the prompt: *"What would you like to ask your data today?"*
ClarityQ may display **suggested questions** based on your semantic catalog — click any of them to get started quickly, or type your own question in the input field.
## Writing Your Question
Type your question in plain English. ClarityQ understands natural language, so ask the way you'd ask a colleague on the data team.
**Good questions to start with:**
* "How many active users did we have last week?"
* "What are the top 10 events by volume this month?"
* "Show me daily revenue for the last 30 days"
### Input Options
Beyond typing a question, you have several tools to refine your request:
| Option | How to Use | What It Does |
| - | - | - |
| **Date Range** | Select from the date picker dropdown | Scopes your question to a specific time period (last 7 days, this month, custom range, etc.) |
| **Filters** | Expand the filters panel above the input | Apply pre-configured product filters (e.g., platform, region) that constrain the data the agent queries |
| **Mentions** | Type `@` followed by a name | Reference a specific metric, feature, segment, or entity from your semantic catalog |
| **Skills** | Type `/` followed by a skill name | Invoke a reusable analysis workflow that guides the agent step by step |
Using mentions (`@`) is a great way to point the agent to the exact semantic components you want it to use. This leads to more accurate and faster answers.
Product filters are configured by the ClarityQ team during onboarding. Contact [support@clarityq.ai](mailto:support@clarityq.ai) if you need to add or modify filters.
## Understanding the Response
A typical response includes:
* **Written insight** — A natural language explanation of what the data shows and what it means.
* **Data table** — Query results in an interactive table. Sort by any column, open it fullscreen with **Enlarge**, and download the full result set as CSV.
* **Visualizations** — When the visualization toggle is on, every numeric answer includes rich visualizations — from a single chart to full visual reports with multiple charts, depending on the complexity of the answer.
* **SQL query** — Analysts and admins can click **View Code** on a result to open the generated query in a side panel. Business Users don't see this option.
Sometimes the agent will ask a **clarification question** before answering — for example, to confirm which metric you mean or what time granularity you want. This ensures accurate results rather than guessing.
## Continuing the Conversation
Ask Anything is conversational — you can ask follow-up questions in the same session. The agent retains the full context of your conversation, so you can say things like:
* "Break that down by platform"
* "What about the previous month?"
* "Filter to only premium users"
* "Show me that as a line chart"
## Sharing Conversations
Once you have a useful conversation, you can:
* **Share** — Generate a read-only link that anyone with access can view.
* **Star** — Mark conversations as favorites for easy access later.
* **Send to Analyst** — Request a data analyst to review or continue your conversation. Choose an analyst from the list, add an optional note, and it will appear in their request queue.
# Welcome to ClarityQ
Source: https://docs.clarityq.ai/index
Learn how to set up, configure, and use ClarityQ — the AI-powered data analytics platform.
ClarityQ lets anyone in your organization ask data questions in plain English and get accurate answers, insights, and visualizations. These docs cover everything from connecting your data warehouse to building the Context Layer that makes it all work.
![ClarityQ product overview showing Ask Anything, dashboards, and the Context Layer]()
## Get Started
Learn what ClarityQ does and how it works.
Set up your first data warehouse connection.
Teach ClarityQ your business context.
Ask your first question and get answers.
## Explore
Table catalog, event catalog, semantic catalog, memory, and skills.
Ask data questions in natural language.
Create, edit, share, and manage dashboards for recurring insights.
Schedule recurring questions and reports.
## API Reference
The ClarityQ API allows you to interact with ClarityQ programmatically using API keys. All endpoints require authentication via the `X-API-Key` header.
**Base URL:** `https://api.clarityq.ai`
### Endpoints
Submit a natural-language question to the ClarityQ agent.
Poll for the status of an ongoing conversation.
Retrieve the summary of a completed conversation.
Get the summary of the last completed task run.
## Support
Need help? Reach out to us at [support@clarityq.ai](mailto:support@clarityq.ai).
# Connecting AI Assistants
Source: https://docs.clarityq.ai/integrations/clarityq-mcp-server/connecting-ai-assistants
Add ClarityQ as an MCP server in Claude, Claude Code, ChatGPT, Cursor, or another MCP client and sign in with your ClarityQ account.
The **ClarityQ MCP server** lets you ask ClarityQ questions from an AI assistant such as Claude, Claude Code, ChatGPT, or Cursor. The assistant sends your question to ClarityQ, and ClarityQ answers it the same way it does in [Ask Anything](/ask-anything/how-ask-anything-works) — using your Context Layer, your data warehouse, and your permissions.
This is the opposite direction from [MCP Connectors](/integrations/mcp/connecting-mcp-servers). MCP Connectors bring outside tools into ClarityQ. The ClarityQ MCP server brings ClarityQ into your AI assistant.
## Prerequisites
* MCP access turned on for your organization. It's off by default. An Admin turns it on in **Organization Settings ❭ External Access** — see [Turning On MCP Access](#turning-on-mcp-access).
* A ClarityQ account with a role that can use Ask Anything in at least one product. Viewers can't ask questions through the MCP server.
* An AI assistant that supports remote MCP servers with sign-in (OAuth). This includes Claude (web and desktop), Claude Code, ChatGPT (web, with developer mode), and Cursor.
## Turning On MCP Access
MCP access is off for every organization until an Admin turns it on. Only Admins can change it, and it applies to the whole organization, not to a single workspace.
Open **Organization Settings** from your profile menu and select **External Access**.
On the **MCP access** card, turn the switch on. The card shows **ON**, and members can now connect their AI assistants.
To turn it off, turn the switch off and confirm. Members' existing connections stop working right away, and their assistants show this message:
> MCP access is turned off for your organization. Ask your admin to turn it on in Organization Settings ❭ External Access.
When you turn it on again, members don't need to reconnect.
If the **MCP access** card doesn't appear under **External Access**, MCP isn't available for your organization yet. Contact your ClarityQ account manager.
## Server URL
Use this URL in every client:
```
https://agent.clarityq.ai/mcp
```
## Claude (web and desktop)
In Claude, go to **Customize → Connectors** and click **Add custom connector**. On Claude Team and Enterprise plans, an organization owner adds the connector once under **Organization settings → Connectors**, and members then click **Connect**.
Name the connector `ClarityQ` and paste `https://agent.clarityq.ai/mcp` as the URL. Leave the OAuth client fields empty.
Claude opens the ClarityQ sign-in page. Sign in with the same method you use for the ClarityQ app — email and password, Google, or your company's single sign-on.
Review the consent screen and click **Accept**. Claude can now ask ClarityQ questions on your behalf.
## Claude Code
Add the server from your terminal:
```bash theme={null}
claude mcp add --transport http clarityq https://agent.clarityq.ai/mcp
```
Then run `/mcp` inside Claude Code, select `clarityq`, and choose **Authenticate**. A browser tab opens the ClarityQ sign-in page; sign in and approve access.
## ChatGPT
ChatGPT connects to MCP servers through **developer mode**, on ChatGPT on the web. Which plans include developer mode, and who can turn it on, is set by OpenAI — see [Developer mode and MCP apps in ChatGPT](https://help.openai.com/en/articles/12584461-developer-mode-and-mcp-apps-in-chatgpt). On Business, Enterprise, and Edu workspaces, a workspace admin may need to allow it first.
Turn on developer mode in your ChatGPT settings, as described in OpenAI's article above.
Go to **Plugins** ([chatgpt.com/plugins](https://chatgpt.com/plugins)), click **+**, and select **Add custom MCP server**. Name it `ClarityQ`, enter `https://agent.clarityq.ai/mcp` as the **Server URL**, and choose **OAuth** for authentication. Leave the client ID and secret empty. Review the warning, confirm, and select **Create as a plugin**.
ChatGPT opens the ClarityQ sign-in page. Sign in and click **Accept**. Then install the ClarityQ plugin from your plugins.
In the message box, type `@` and select **ClarityQ**, then ask your question — for example, *"@ClarityQ what was weekly active users for Star Surge last week?"*
Both ClarityQ tools only read data, so ChatGPT doesn't ask you to confirm each call.
## Cursor
In Cursor, open **Cursor Settings → Tools & MCP** and click **New MCP Server**. Cursor opens its `mcp.json` file. Add ClarityQ inside `mcpServers`:
```json theme={null}
{
"mcpServers": {
"clarityq": {
"url": "https://agent.clarityq.ai/mcp"
}
}
}
```
Don't add a token or headers — you sign in with your ClarityQ account in the next step.
Back in **Tools & MCP**, click **Connect** next to `clarityq`. Your browser opens the ClarityQ sign-in page. Sign in and click **Accept**.
`clarityq` shows a green status and two tools, `list_products` and `ask_question`.
## Other MCP clients
Add a remote (HTTP) MCP server with the URL `https://agent.clarityq.ai/mcp` in your client's MCP settings. The client opens the ClarityQ sign-in page the first time it connects.
## Staying Signed In
You sign in once per client. Your assistant renews access automatically in the background, so you don't need to sign in again each session. You'll be asked to sign in again only if your access is revoked or you remove and re-add the connector.
If you're already signed in to ClarityQ in the same browser, the sign-in step may be skipped and you'll go straight to the consent screen — with that account. To connect with a different ClarityQ account, sign out of ClarityQ in that browser first.
## Disconnecting
Remove the connector from your assistant's connector or MCP settings — for example **Customize → Connectors** in Claude, **Plugins** in ChatGPT, or `claude mcp remove clarityq` in Claude Code. Your ClarityQ conversations and data are not affected.
# Using ClarityQ from AI Assistants
Source: https://docs.clarityq.ai/integrations/clarityq-mcp-server/using-clarityq-from-ai-assistants
How to ask ClarityQ questions from Claude, ChatGPT, and other MCP clients — products, long answers, clarifying questions, follow-ups, and usage.
Once the ClarityQ MCP server is [connected](/integrations/clarityq-mcp-server/connecting-ai-assistants), you can ask your assistant business questions in plain language and it asks ClarityQ for the answer. ClarityQ runs the question exactly like [Ask Anything](/ask-anything/asking-questions): same Context Layer, same data warehouse, same permissions.
## Available Tools
The server gives your assistant two tools:
| Tool | What it does |
| - | - |
| `list_products` | Lists the ClarityQ products you can ask about, grouped by organization. Workspaces show their member products. |
| `ask_question` | Asks ClarityQ a question about one product and returns the answer. |
Both tools only read data. They never change your Context Layer, dashboards, or settings.
## Choosing a Product
Every question runs against one product. Name it in your message — for example, *"Using ClarityQ, what was weekly active users for Star Surge last week?"*
If you don't name a product, or the name matches more than one, your assistant can call `list_products` and ask you which one you mean.
## What an Answer Contains
Each answer includes:
* **The answer** as text, with tables where the data calls for them.
* **The SQL ClarityQ ran**, so you or your assistant can check how the numbers were calculated.
* **An Open in ClarityQ link** to the same conversation in the ClarityQ app.
Answers through the MCP server don't include ClarityQ visualizations or dashboards. Your assistant can build its own charts from the data in the answer, or you can open the conversation in ClarityQ.
## Long Answers
Some questions take several minutes to answer. If an answer isn't ready yet, ClarityQ tells your assistant it's still working, and the assistant asks again to pick up the result. ClarityQ keeps working on the question in the meantime, so nothing is lost.
While a question runs, some assistants (for example Claude Code) show progress updates such as *Running SQL on the warehouse…*.
## Clarifying Questions
When a question is ambiguous, ClarityQ may ask for clarification — for example, which revenue definition to use. Your assistant shows you ClarityQ's question and options. Your reply goes back to ClarityQ and the answer continues from there.
## Follow-up Questions
Follow-up questions continue the same ClarityQ conversation, so ClarityQ keeps the earlier context. To make sure your assistant continues the conversation rather than starting a new one, say so — for example, *"Follow-up in the same ClarityQ conversation: how does that compare to the week before?"*
Every conversation started from an assistant also appears in your Ask Anything history in ClarityQ.
## Workspaces
A workspace answers across several member products. When you ask a workspace, ClarityQ uses its default member products unless you name the ones you want — for example, *"Ask the Lobby workspace, using Coffee Mania and Tea Time only"*. The answer lists which members it used and which others you can include. The members you choose stay fixed for that conversation.
## Usage
Questions asked through the MCP server count toward your organization's usage the same way as questions asked in the ClarityQ app, in prompts or CQT depending on your plan. On the Usage page they appear under their own **MCP** type, so you can see how much of your usage comes from AI assistants.
## Permissions
Your assistant acts as you. It can only reach the products you can access in ClarityQ, and your role decides what it can do — the same rules as in the ClarityQ app. Only products of organizations where an Admin turned on MCP access ([Organization Settings ❭ External Access](/integrations/clarityq-mcp-server/connecting-ai-assistants#turning-on-mcp-access)) are available.
# Embedding ClarityQ
Source: https://docs.clarityq.ai/integrations/embed/embedding-clarityq
Put ClarityQ inside your own product with an iframe. Your users ask questions of their data without leaving your app and without a ClarityQ login — your backend vouches for each one with a short-lived token.
## Quick test
Create an API key at **Organization Settings → API → Generate Key** — it is shown once. Mint yourself a token with it:
```bash theme={null}
curl -X POST https://app.clarityq.ai/api/v1/products/PRODUCT_ID/embed/token \
-H "X-API-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"email": "you@yourcompany.com"}'
```
`email` is the only required field; `name` and `ttl_seconds` are in Step 1.
Take the `token` from the response and open:
```
https://app.clarityq.ai/PRODUCT_ID/embed#token=YOUR_TOKEN
```
Once that works, drop it into any page:
```html theme={null}
```
The frame fills whatever you give it, so the container needs a real height. Aim for 1000px of width or more — answers open a side panel for charts and SQL, and it gets cramped below that.
That token is tied to you and expires in an hour, so the chat will stop working — the three steps below replace it with one token per user, refreshed automatically.
## How it works
Your user logs into your app → your frontend asks your backend for a token → your backend calls ClarityQ with your API key → your page loads the iframe, and hands it a fresh token whenever it asks for one.
| | Lives | Can do |
| - | - | - |
| **API key** | Your server, in a secret manager | Mint a token for any address |
| **Token** | Your page — in the iframe URL at boot, then swapped in by postMessage | Act as one user, until it expires |
## Identity
**The email you send is the identity.** Same address next visit, same conversation history. It becomes an ordinary ClarityQ user — so if that address already uses ClarityQ, it *is* that person, with the history and remembered preferences they already have.
### What the embedded user can do
A brand-new address gets the **Business User** role. That's the analyst answering questions with charts and tables: they can ask and follow up, edit their own messages, use filters, export results to CSV, rate answers, turn an answer into a dashboard, and schedule a task — all inside the iframe.
Two things are **analyst-tier** and therefore hidden from a Business User: **viewing the generated SQL** (the View Code button) and **editing** it. If the address belongs to someone who already holds an analyst role in ClarityQ, they keep it and see both.
The Context Layer is **read-only** in the embed, and the Builder isn't part of it at all — no embedded user can edit your catalog, whatever their role. Navigation out of the iframe is removed too, so there's no route from the embed into the main app.
Embedded usage counts toward your organization's usage limits just like usage in the app, and shows up as **Embedded** in the [Usage dashboard](/admin/usage-dashboard). Limit warnings and blocks apply to embedded users the same way they do in the app.
## Step 1 — Mint tokens on your server
Using the same API key, expose one endpoint behind your own login:
```js theme={null}
app.post('/api/clarityq-token', requireLogin, async (req, res) => {
const response = await fetch(
`https://app.clarityq.ai/api/v1/products/${PRODUCT_ID}/embed/token`,
{
method: 'POST',
headers: {
'X-API-Key': process.env.CLARITYQ_API_KEY,
'Content-Type': 'application/json'
},
body: JSON.stringify({
email: req.user.email, // from the session, never the body
name: req.user.name,
ttl_seconds: 3600
})
}
);
res.json(await response.json()); // { token, expires_in }
});
```
Every field, error and a playground to try it: [Mint an embed token](/integrations/embed/mint-embed-token).
## Step 2 — Render the iframe and answer one message
The embed manages its own lifetime. At about 80% of its token's lifetime it posts
`clarityq:token-expiring` to your page. You reply with a fresh token and it swaps it in
place: no reload, no timer, nothing lost. An answer that is streaming keeps streaming;
text your user has typed stays typed.
| Message | Direction | Meaning |
| - | - | - |
| `clarityq:token-expiring` | embed → your page | The token is at \~80% of its lifetime. Reply soon. |
| `clarityq:token` `{ token }` | your page → embed | Your reply. Swapped in place, invisibly. |
| `clarityq:token-expired` | embed → your page | Backstop — the token died before a swap arrived (slept laptop, throttled tab). Reply the same way: the expired screen clears and the session resumes without a reload. |
| `clarityq:conversation-created` | embed → your page | A brand-new conversation got its id (Step 3). |
| `clarityq:title` `{ conversationId, title }` | embed → your page | The conversation got its title, a few seconds into the first answer. Rename your panel row in place — no re-fetch, no polling (Step 3). |
| `clarityq:open-conversation` `{ conversationId }` | your page → embed | Show another conversation, or `null` for a new chat. Internal navigation — instant, no reload, no token needed (Step 3). |
```js theme={null}
const CLARITYQ = 'https://app.clarityq.ai';
const PRODUCT_ID = 'your-product-id';
let token = null; // the current one — Step 3 reuses it
// No timers, no expiry math — the embed asks when it wants a new one.
async function mintToken() {
token = (await (await fetch('/api/clarityq-token')).json()).token;
return token;
}
const frame = document.createElement('iframe');
async function boot() {
frame.src = `${CLARITYQ}/${PRODUCT_ID}/embed#token=${encodeURIComponent(await mintToken())}`;
frame.title = 'ClarityQ';
frame.allow = 'clipboard-write';
frame.style.cssText = 'width:100%;height:100%;border:0;display:block';
document.getElementById('clarityq').appendChild(frame);
}
// One handler for both events — the warning and the backstop want the same reply.
window.addEventListener('message', async (event) => {
if (event.origin !== CLARITYQ) return; // always check the origin
const type = event.data?.type;
if (type === 'clarityq:token-expiring' || type === 'clarityq:token-expired') {
event.source.postMessage(
{ type: 'clarityq:token', token: await mintToken() },
CLARITYQ // never '*' — this message carries the token
);
}
});
boot();
```
If your mint endpoint fails once, do nothing special — the embed falls through to
`clarityq:token-expired`, and the same listener answers it when your endpoint recovers.
The request that failed in between is lost; the conversation is not.
## Step 3 — Add a history panel (optional)
The iframe is chat only — one conversation at a time, no list, no new-chat button — so it drops into your product without bringing a second navigation with it.
Skip this step and every visit starts a fresh chat. Nothing is lost: ClarityQ stores each conversation either way, and they are all still there under that user's identity. Your users just have no route back to them.
If you want that route, the split is: **ClarityQ stores the conversations, you render the list.** Two things to wire.
**1. List them** — [List conversations](/integrations/embed/list-conversations), authenticated
with the token you already hold (`Authorization: Bearer`, not the API key). It returns only
that user's conversations, scoped server-side. Sort on `last_updated`; `description` is the
title — a new conversation starts as `New Chat` and gets its real title a few seconds into
the first answer, announced by `clarityq:title` so you never re-fetch for it.
**2. Open them, and keep them named** — switching is a postMessage into the running iframe: internal navigation, so it is instant and needs no token. A brand-new chat announces its id the same way — catch it or that conversation is orphaned — and its title arrives moments later:
```js theme={null}
// Clicking a row in your panel
function openConversation(id) {
frame.contentWindow.postMessage(
{ type: 'clarityq:open-conversation', conversationId: id }, CLARITYQ
);
}
// Your "New chat" button
function newConversation() {
frame.contentWindow.postMessage(
{ type: 'clarityq:open-conversation', conversationId: null }, CLARITYQ
);
}
window.addEventListener('message', (event) => {
if (event.origin !== CLARITYQ) return;
// A brand-new conversation gets its id here. Add the row optimistically —
// it will not come back from the API until the first message is stored.
if (event.data?.type === 'clarityq:conversation-created') {
addRowToPanel({ id: event.data.conversationId, description: 'New chat' });
}
// Its title lands here a few seconds later. Rename the row in place.
if (event.data?.type === 'clarityq:title') {
renameRowInPanel(event.data.conversationId, event.data.title);
}
});
```
## Open in ClarityQ (optional)
A button in your own header that hands the conversation to the full ClarityQ app — same
conversation, same user, nothing to set up:
```html theme={null}
Open in ClarityQ
```
The button works for anyone who can sign into ClarityQ. Embed users are ordinary
ClarityQ users under the same email, so there are two ways in: your organization's SSO,
or an invitation sent from ClarityQ user management — either one lands them in the
conversation, with their history. Users with neither hit a login screen they cannot
pass, so only show the button to people who have a way in.
## Troubleshooting
| What you see | Where | Cause and fix |
| - | - | - |
| `401` | Mint | Despite the wording, this is the API key: `X-API-Key` missing, malformed, revoked or expired. |
| Session has expired | iframe | The token died and no `clarityq:token` arrived — your listener is missing, or your mint endpoint failed. Reply with a fresh token and it clears without a reload. |
| Expired screen in a loop, swaps don't help | iframe | The `productId` in the URL does not match the token's product. Requests are refused in a way that reads as expiry, so every fresh token fails identically. |
# List conversations
Source: https://docs.clarityq.ai/integrations/embed/list-conversations
post /api/v1/products/{product_id}/conversations
List the conversations belonging to the token's user in this product — for rendering a history panel next to the embedded analyst. Authenticated with the embed token itself, not the API key. Returns a flat array in no particular order.
# Mint an embed token
Source: https://docs.clarityq.ai/integrations/embed/mint-embed-token
post /api/v1/products/{product_id}/embed/token
Mint a short-lived token for one end user of your embedded analyst. Call this from your server — the API key must never reach the browser; only the returned token does. The email you send becomes the ClarityQ identity, so send only addresses your application has authenticated.
# Connecting Atlassian (Jira)
Source: https://docs.clarityq.ai/integrations/mcp/connecting-atlassian
Connect Jira and other Atlassian products to ClarityQ through the Atlassian Rovo MCP Server, including the Atlassian admin step required before anyone can authorize.
Connecting Atlassian lets ClarityQ's agent search, read, and work with Jira issues (and other Atlassian products your account can access, such as Confluence) while answering your questions. The connection runs through the **Atlassian Rovo MCP Server** (Atlassian also calls it the Atlassian MCP server), added to ClarityQ as an OAuth [MCP connector](/integrations/mcp/connecting-mcp-servers). Each teammate signs in with their own Atlassian account, so ClarityQ only sees what that person can see in Jira.
Three parts, in this order:
1. An **Atlassian organization admin** allows the ClarityQ domain in Atlassian. Required. Without it, authorization fails for everyone.
2. A **ClarityQ admin** adds the Atlassian MCP connector once for the organization.
3. **Each teammate** connects their own Atlassian account.
## Before connecting Atlassian: allow ClarityQ in Atlassian
Atlassian only lets an AI tool complete OAuth if the tool's domain is on the organization's allowed list. An **Atlassian organization admin** must add this exact entry before anyone tries to connect:
```
https://app.clarityq.ai/**
```
If you aren't an Atlassian organization admin, send this section to the person who manages your company's Atlassian account.
The `/**` at the end is Atlassian's path wildcard. It allows any path under `app.clarityq.ai`, which covers the ClarityQ callback URLs Atlassian redirects to after you approve access. Enter the value exactly as shown, including `https://` and the trailing `/**`.
### Steps for the Atlassian organization admin
Go to [admin.atlassian.com](https://admin.atlassian.com/) and sign in. If you manage more than one organization, select the one that owns the Jira site you want to connect.
In the left menu, select **Rovo**, then **Rovo MCP server**. The domain list is under **Domain settings**.
Below the **Allow Atlassian supported domains** setting, in your organization's own domain list, click **Add domain**.
Add exactly:
```
https://app.clarityq.ai/**
```
Save. The entry appears in the list alongside any other domains your organization has allowed.
Start the Atlassian connection again from the beginning (see below).
That's the only Atlassian change needed. ClarityQ uses Atlassian's default OAuth 2.1, so leave **API token** authentication and enterprise-managed authentication as they are.
## Add the Atlassian MCP connector in ClarityQ
A ClarityQ admin does this once. Open **Organization Settings → MCP** and click **Add Connector**.
For example, `Jira` or `Atlassian`.
In **Remote MCP server URL**, enter:
```
https://mcp.atlassian.com/v2/mcp
```
No headers makes the connector **OAuth**, so each teammate signs in with their own Atlassian account rather than sharing one token.
ClarityQ validates the connection and the connector appears in the list. Teammates now see it as an **App**.
## Connect your Atlassian account
Each teammate connects individually.
Find the Jira / Atlassian app in the list. See [Using Apps](/integrations/mcp/using-mcp-servers) for the full Apps page.
A new window opens on Atlassian's authorization screen. Sign in if prompted, review the access ClarityQ requests, and approve it. Atlassian redirects back to ClarityQ and the app shows **Connected**.
In a chat, click **+ → Apps** and switch on Jira. ClarityQ can now use Jira tools in that conversation.
## Troubleshooting
### "Access to this domain is restricted"
If the Atlassian authorization screen shows **"Access to this domain is restricted"** or **"Your organization admin must authorize access from this redirect URL"** and mentions a ClarityQ URL, the ClarityQ domain has not been allowed in the organization's Rovo MCP server settings.
Ask your Atlassian organization admin to confirm this exact entry exists under **Rovo → Rovo MCP server → Domain settings**:
```
https://app.clarityq.ai/**
```
Common mistakes: missing `https://`, missing `/**`, a typo in the domain, or the domain added to a different Atlassian organization than the one that owns the Jira site.
### The domain was just added and authorization still fails
Atlassian can take a moment to apply a new domain, and your browser may be holding on to the earlier failed attempt.
1. Close the existing Atlassian authorization window.
2. In ClarityQ, start again from **Settings → Apps → Connect**.
3. Still failing? Retry in a private / incognito browser window.
4. Still failing? Have the Atlassian admin delete the domain entry, add the identical entry again, save, and retry. Atlassian recommends this when the entry looks correct in the admin UI but is still being blocked.
### The Jira app doesn't appear on the Apps page
A ClarityQ admin hasn't added the Atlassian MCP connector yet, or it's disabled. See [Add the Atlassian MCP connector](#add-the-atlassian-mcp-connector-in-clarityq) and check the connector's enable/disable toggle under **Organization Settings → MCP**.
### Connected, but Jira tools fail in chat
ClarityQ acts with your own Jira permissions. If you can't see a project or issue in Jira, ClarityQ can't either.
If Atlassian shows *"You don't have permission to connect from this IP address"*, your organization's Atlassian IP allowlist is blocking the connection. The Atlassian admin needs to adjust the IP allowlist, not the domain list.
# MCP Connectors
Source: https://docs.clarityq.ai/integrations/mcp/connecting-mcp-servers
Add and manage MCP (Model Context Protocol) connectors for your organization so ClarityQ's agent can use external tools.
MCP connectors let ClarityQ's agent use tools from external services — pulling data from another product, searching a connected knowledge base, calling an internal API, and so on. An admin adds a connector once under **Organization Settings → MCP**; it then shows up for your team as an **App** (see [Using Apps](/integrations/mcp/using-mcp-servers)).
## Two kinds of connectors
A connector authenticates one of two ways. ClarityQ picks the type from what you enter — **add a header and it's token-based; leave headers empty and it's OAuth.**
You paste one API token. It's sent as a header on every request and shared by the **whole team** — no personal sign-in. Teammates see it marked **Controlled by admin**.
You add the URL with **no headers**. Each teammate clicks **Connect** and signs in with their **own account**, so access follows their own permissions.
## Before you start
* Admin rights in your ClarityQ organization.
* The MCP server's URL (HTTP / streamable HTTP).
* For a token-based server, an API token issued by that service. (OAuth servers need nothing here — each teammate signs in themselves.)
**Adding Jira / Atlassian?** The Atlassian Rovo MCP server is OAuth, and Atlassian additionally requires an **Atlassian organization admin** to allow `https://app.clarityq.ai/**` in the organization's Rovo MCP server domain settings before any teammate can connect. See [Connecting Atlassian](/integrations/mcp/connecting-atlassian) for the full walkthrough, including that admin step.
## Add a connector
Open **Organization Settings → MCP** and click **Add Connector**.
Give the connector a **Name** and paste the **Remote MCP server URL**.
* **Token-based:** under **Headers**, add the header the server expects — for example an `Authorization` header with value `Bearer `, or `X-API-Key`. Use **Add header** for more rows.
* **OAuth:** leave **Headers** empty. Teammates connect their own account later.
ClarityQ validates the connection against the server. For token-based servers, a rejected token surfaces the error right in the dialog so you can fix it before saving — no silently broken connectors.
## Edit a connector
Open the **⋯** menu on a connector and choose **Edit**. You can change the name, URL, or headers.
Saved secret header values show **masked** (`••••••`). Leave one masked to keep the stored value; click the edit icon to type a **new** value to replace it (the eye icon reveals what you type). Click **Save** — the connection is re-validated.
## Enable, disable, or remove
* Each connector has an **enable/disable toggle**. Turn it **off** to stop teammates using it without deleting it; **on** to make it available again.
* To remove one entirely, open **⋯ → Delete → Remove**. It's removed for everyone on your team. Past chats that referenced its content stay readable, but the connector can't be used going forward.
Header values (tokens) are stored encrypted in a secrets manager — never in the database in plain text, and only ever shown masked in the UI. A token-based connector inherits the permissions of the account that issued the token, so scope it to what your team needs.
# Using Apps
Source: https://docs.clarityq.ai/integrations/mcp/using-mcp-servers
Connect to the MCP apps your organization provides and turn them on in a chat so ClarityQ can use their tools.
Once an admin has added [MCP connectors](/integrations/mcp/connecting-mcp-servers) for your organization, they appear to you as **Apps**. Connect any personal ones, then turn an app on inside a chat so ClarityQ can use its tools.
## The Apps page
Open **Settings → Apps** to see the apps your organization has made available. Each shows how it's connected:
* **Controlled by admin** — an organization-managed app. It's connected for everyone; there's nothing for you to do.
* **Connect** — a personal app. Click **Connect** and sign in with your own account. Only you can access the data and permissions tied to that account. Once connected it shows **Connected**, with **Reconnect** and **Disconnect** options.
**Connecting Jira / Atlassian?** Before anyone clicks **Connect**, an Atlassian organization admin must allow `https://app.clarityq.ai/**` in the organization's Rovo MCP server **Domain settings**. Without it, Atlassian blocks the sign-in with *"Access to this domain is restricted"* or *"Your organization admin must authorize access from this redirect URL."* Follow [Connecting Atlassian](/integrations/mcp/connecting-atlassian) first.
If you see *"Contact your admin to receive access to apps,"* no apps have been enabled for your organization yet.
## Turning an app on in a chat
In a chat, click **+** in the composer and choose **Apps**.
Switch on the app(s) for this chat. Organization-managed apps are always available; personal apps appear here once you've connected them on the Apps page.
ClarityQ can now call that app's tools as part of answering you.
If an app isn't listed in the chat's **Apps** menu, use **Manage apps** to open the Apps page and connect it (personal apps). Organization-managed apps show up automatically once your admin enables them. Disconnecting or reconnecting a personal app is also done from the Apps page — organization-managed apps are controlled by your admin.
# Connecting Slack
Source: https://docs.clarityq.ai/integrations/slack/connecting-slack
Install the ClarityQ Slack app for your workspace so tasks, reports, and answers can flow into Slack.
Connecting Slack lets ClarityQ post task results, scheduled reports, and answers into your team's Slack workspace, and lets your team ask questions of ClarityQ without leaving Slack. The connection is at the workspace level — one ClarityQ organization links to one Slack workspace, and every workspace in that organization can use it.
## Prerequisites
* A Slack workspace where you have permission to install apps.
* Admin rights in your ClarityQ organization.
## Installing the App
Navigate to **Organization Settings → Slack** in ClarityQ.
The **Add ClarityQ to Slack** button kicks off the OAuth flow and redirects you to Slack.
Review the permissions ClarityQ requests — posting messages, reading channel and member lists, uploading files, responding to mentions, and handling slash commands — and click **Allow**.
Slack redirects you back to ClarityQ once the install is complete. The Slack page now shows **Connected to *your workspace name***.
## Choosing Channels
ClarityQ can post to:
* **Public channels** in your Slack workspace, automatically.
* **Private channels**, but only after you invite the ClarityQ bot to that channel with `/invite @ClarityQ`.
The channel picker appears when you create or edit a [task](/automations/creating-and-scheduling-tasks). It lists every channel the bot can see — public, plus any private channels it's been invited to.
## Disconnecting and Reconnecting
Two actions are available on the Slack page once Slack is connected:
* **Disconnect** removes ClarityQ's access to your Slack workspace. After confirmation, the bot stops posting and pending task deliveries fail. Existing tasks keep their channel selections but won't deliver until Slack is reconnected.
* **Reconnect** re-runs the OAuth flow with the latest permission set. Use this when:
* Slack reports a permissions error
* ClarityQ tells you the integration needs to be reauthorized (for example, when ClarityQ adds new scopes in a release)
* You want to enable new capabilities — for example, if you originally connected Slack only for task deliveries and now want to ask ClarityQ questions directly in Slack, reconnecting grants the additional permissions needed
Slack connections are per organization, not per user. Once an admin connects Slack, every workspace in your organization can deliver to that Slack workspace. If a user's Slack email matches their ClarityQ login email, they're recognized automatically. Otherwise, they'll need to run `/clarityq connect` once — see [Using ClarityQ in Slack](/integrations/slack/using-clarityq-in-slack).
# Using ClarityQ in Slack
Source: https://docs.clarityq.ai/integrations/slack/using-clarityq-in-slack
Receive task deliveries, ask questions, and manage your identity without leaving Slack.
Once your Slack workspace is [connected](/integrations/slack/connecting-slack), the ClarityQ bot can post task deliveries to channels and answer questions in DMs and in any channel it's been invited to. This page covers the day-to-day interactions.
## Receiving Task Deliveries
When a [task](/automations/what-are-tasks) runs, its results are posted to the Slack channel you selected when scheduling it. Each delivery includes:
* The workspace name and a header showing which task ran.
* The task's output, formatted as Slack-friendly Markdown — tables come through as code blocks, and larger outputs land as file attachments.
* A **View Chat** button linking back to the conversation in ClarityQ.
Each scheduled run is a separate top-level message in the channel — replying in-thread doesn't trigger a new agent run. Use **View Chat** to continue the conversation in ClarityQ.
## Asking ClarityQ in Slack
If your Slack integration was originally connected only for task deliveries, an admin will need to reconnect the app from **Settings → Integrations** to grant the additional permissions required for asking questions in Slack.
You can talk to ClarityQ without leaving Slack in two ways.
### @Mention in a channel
Mention the bot in any channel it's a member of, with your question as the text — for example, `@ClarityQ what's revenue by region this quarter?`. The bot acknowledges in a thread and updates the same message once the answer is ready. Each Slack thread maps to a single ClarityQ chat.
### Direct message the bot
DM the bot the same way. The conversation lives in the DM thread; the agent responds inline.
### Workspace resolution
When you ask a question in Slack, ClarityQ needs to know which workspace to query. It resolves this automatically:
* **Single workspace** — If you only have access to one workspace, it's used automatically.
* **Multiple workspaces** — ClarityQ uses your **default product**, which is initially set based on your most-used workspace in ClarityQ. You can change it at any time with the `/clarityq set-default-product` command.
* **Explicit selection** — You can specify a workspace by wrapping its name in brackets in your message, for example: `@ClarityQ [MyProduct] what was revenue last week?`
* **Disambiguation** — If ClarityQ can't determine which workspace to use, it shows buttons letting you pick one.
Once a conversation thread is started on a workspace, all follow-up messages in that thread stay on the same workspace — you can't switch workspaces mid-conversation.
ClarityQ does not check whether everyone in a Slack channel has access to the workspace being queried. Anyone in the channel can see the bot's answers, even if they don't have a ClarityQ account or access to that workspace (they won't be able to open the View Chat link or continue the conversation in ClarityQ). Be mindful of which channels you ask questions in if your data is sensitive.
### How Slack chats work
In both cases, the bot's reply includes a **View Chat** button that opens the conversation in ClarityQ.
A few important things to know about Slack chats:
* **One owner per thread** — Only the user who started the thread can continue the conversation. Messages from other users in the same thread are ignored by the bot.
* **Chat history** — The conversation is created in the starting user's chat history in ClarityQ, marked with a Slack icon so you can find it easily.
* **Opening in ClarityQ ends the Slack thread** — Once you open the chat in ClarityQ (via the link or from your chat history), the Slack thread stops accepting new messages. From that point on, the conversation can only be continued in Ask Anything.
* **Other users in the channel** — Other users can click the View Chat link to see a read-only version of the conversation. They can fork the chat and continue it independently in their own ClarityQ account.
## Slash Commands
Run these from any channel or DM in Slack:
| Command | What it does |
| - | - |
| `/clarityq help` | Show all available commands. |
| `/clarityq connect` | Link your Slack identity to your ClarityQ account so the bot recognises you and ties chats back to your ClarityQ user. |
| `/clarityq products` | List the ClarityQ workspaces available in this Slack workspace. |
| `/clarityq set-default-product ` | Choose which workspace the bot uses when you @mention it without context. |
If your Slack email matches the email you used to sign in to ClarityQ, the bot recognizes you automatically — no setup needed. If your Slack email is different, run `/clarityq connect` once to link your Slack identity to your ClarityQ account. Until linked, the bot won't be able to answer your questions.
## When Things Go Wrong
A few common failure modes:
* **The bot was removed from a channel.** Task deliveries to that channel start failing silently. Re-invite the bot with `/invite @ClarityQ` and the next delivery resumes.
* **The Slack token expired or new scopes are required.** Posts stop and the Integrations page shows a reconnect prompt. Click **Reconnect** to re-run OAuth.
* **The bot doesn't recognize you.** If your Slack email differs from your ClarityQ login email, run `/clarityq connect` to link your identity. Without it, chats can't be attributed to your ClarityQ account.