BigQuery Dataset Access

What you need before writing a single model

Looky runs Malloy queries against BigQuery on your behalf. For that to work, your workspace needs a GCP service account with enough permission to read data and run jobs. This page walks you through creating one, downloading the credentials file, and placing it where the runtime expects it.

You will need two things from GCP:

• A billing project: the GCP project that pays for query costs. This is your own project.
• A dataset location: the project and dataset where the data actually lives. This can be a different project — including public datasets like bigquery-public-data.

These two are often confused. Your service account lives in the billing project, but it can be granted read access to datasets in any other project.

Step 1: Create a service account in GCP

1. Open the GCP Service Accounts console and select your billing project.
2. Click Create service account.
3. Give it a clear name, for example: looky-workspace-reader.
4. Click Create and continue.

Step 2: Grant minimum required roles

In the "Grant this service account access to project" step, add these two roles:

• BigQuery Data Viewer — allows reading table data and schema.
• BigQuery Job User — allows running query jobs (required even for read-only queries).

That is the minimum. Do not add Owner, Editor, or any broader role. Click Done.

Step 3: Download the JSON key

1. In the service accounts list, click the account you just created.
2. Open the Keys tab.
3. Click Add key → Create new key.
4. Select JSON and click Create. The file downloads immediately.

Rename the file to something readable, for example: my-workspace-bq.json.

Step 4: Place the key in your workspace

Copy the JSON key into the secrets/ folder of your workspace:

<local_root>/
  <billing_account_id>/
    <workspace_slug>/
      secrets/
        my-workspace-bq.json   ← place it here

Make sure .gitignore excludes secrets/ before committing anything:

cat .gitignore

You should see secrets/ or secrets/* listed. If it is not there, add it before pushing to any remote — Looky does not enforce this; it is your responsibility.

Step 5: Reference the key in sources.runtime.yml

Open runtime/sources.runtime.yml and set credentials_file to the JSON filename you dropped into secrets/. It is a plain filename — no path, no slashes; the platform resolves it against the workspace's secrets/ folder.

sources:
  ecommerce:
    name: The Look Ecommerce
    type: bigquery
    project_id: my-gcp-billing-project
    credentials_file: my-workspace-bq.json
    datasets:
      - bigquery-public-data.thelook_ecommerce

• project_id: the GCP project that pays for query costs — your billing project.
• credentials_file: filename of the service-account JSON inside secrets/. Pattern ^[A-Za-z0-9_-][A-Za-z0-9._-]*$ — no path separators, no leading dot.
• datasets: one or more dataset references the runtime is allowed to query. These can be in a different GCP project than project_id.

Step 6: Validate the connection

From the workspace root, run:

looky sources list
looky validate

If sources list returns your alias with no errors and validate shows no blocking issues, the source declaration is structurally valid and the runtime can reach BigQuery with the supplied credentials. The default validation pass does not verify that the service account has read access to every dataset you reference — those errors surface when a real query touches the dataset. Use looky validate --strict to upgrade validation to per-visualization live-source checks (BigQuery's estimateQueryCost is free and catches permission issues before push).

If validation fails, the most common causes are:

• Credentials filename in sources.runtime.yml does not match the file in secrets/, or contains path separators (it must be a plain filename).
• Service account is missing BigQuery Job User — queries are blocked even if data is readable.
• Dataset reference uses wrong project or dataset name — verify exact names in the BigQuery console.