BigQuery Dataset Access

What you need before writing a single model

Looky runs Malloy queries against BigQuery on your behalf. For that to work, your workspace needs a GCP service account with enough permission to read data and run jobs. This page walks you through creating one, downloading the credentials file, and placing it where the runtime expects it.

You will need two things from GCP:

• A billing project: the GCP project that pays for query costs. This is your own project.
• A dataset location: the project and dataset where the data actually lives. This can be a different project — including public datasets like bigquery-public-data.

These two are often confused. Your service account lives in the billing project, but it can be granted read access to datasets in any other project.

Step 1: Create a service account in GCP

1. Open the GCP Service Accounts console and select your billing project.
2. Click Create service account.
3. Give it a clear name, for example: looky-workspace-reader.
4. Click Create and continue.

Step 2: Grant minimum required roles

In the "Grant this service account access to project" step, add these two roles:

• BigQuery Data Viewer — allows reading table data and schema.
• BigQuery Job User — allows running query jobs (required even for read-only queries).

That is the minimum. Do not add Owner, Editor, or any broader role. Click Done.

Step 3: Download the JSON key

1. In the service accounts list, click the account you just created.
2. Open the Keys tab.
3. Click Add key → Create new key.
4. Select JSON and click Create. The file downloads immediately.

Rename the file to something readable, for example: my-workspace-bq.json.

Step 4: Place the key in your workspace

Copy the JSON key into the secrets/ folder of your workspace:

<local_root>/
  <billing_account_id>/
    <workspace_slug>/
      secrets/
        my-workspace-bq.json   ← place it here

The secrets/ folder is already listed in .gitignore by default. Confirm that before committing anything:

cat .gitignore

You should see secrets/ or secrets/* listed. If it is not there, add it before pushing to any remote.

Step 5: Reference the key in sources.runtime.yml

Open runtime/sources.runtime.yml and point credentials_file to the key. Use the absolute path as the runtime sees it inside the workspace context:

sources:
  ecommerce:
    name: The Look Ecommerce
    type: bigquery
    project_id: my-gcp-billing-project
    credentials_file: /workspace/secrets/my-workspace-bq.json
    datasets:
      - bigquery-public-data.thelook_ecommerce

• project_id: the GCP project that pays for query costs — your billing project.
• credentials_file: absolute path starting with /workspace/, pointing into secrets/.
• datasets: one or more dataset references the runtime is allowed to query. These can be in a different GCP project than project_id.

Step 6: Validate the connection

From the workspace root, run:

looky sources list
looky validate

If sources list returns your alias with no errors and validate shows no blocking issues, the connection is working. You are ready to write models.

If validation fails, the most common causes are:

• Path to credentials file is wrong — double-check the /workspace/secrets/ prefix.
• Service account is missing BigQuery Job User — queries are blocked even if data is readable.
• Dataset reference uses wrong project or dataset name — verify exact names in the BigQuery console.