BigQuery
To connect BigQuery to Kaldea, you will need Admin access to your BigQuery instance.
Complete all of the following steps to see BigQuery metadata in Kaldea:
- Create a service account for Kaldea
- Create a new JSON key
- Connect BigQuery to Kaldea
- Choose databases and schemas
High Level Scope on Kaldea Integration
Kaldea operates on BigQuery using the Google Cloud SDK. Hence it is necessary to grant Kaldea an adequate access to your Google Cloud Accounts.
There are 3 types of credentials that Kaldea may need :
kaldea-system
: The credential used by Kaldea systemkaldea-default
: The default credential used by Kaldea for the user action. Can be overriden by personal.personal
: The personalised credential to be registered at the personal settings page. To be used by personnel that requires additional access (optional)
Required Permission : kaldea-system
-
Create
kaldea-system.pii-management
role on GCP. (Project Level) The following is a list of required permissions for managing PII tags:bigquery.tables.setCategory datacatalog.taxonomies.get datacatalog.taxonomies.list
-
Create
kaldea-system.bigquery-jobs
role on GCP. The following is a list of required permissions for managing BigQuery jobs (ex Query):bigquery.jobs.create bigquery.jobs.get bigquery.jobs.list bigquery.jobs.listAll bigquery.jobs.update
-
Create serviceaccount for kaldea system and attach the roles :
kaldea-system.pii-management
kaldea-system.bigquery-jobs
bigquery.dataEditor
- (This pre-defined role from GCP is also a requirement!)
- Optional (External Table)
storage.objectViewer
- (This pre-defined role from GCP is also a requirement if you are using an external table with gcs)
bigtable.reader
- (This pre-defined role from GCP is also a requirementIf you are using an external table with bigtable)
Required Permission : kaldea-default
-
Create
kaldea.bigquery-jobs
role on GCP. The following is a list of required permissions for managing BigQuery jobs:bigquery.jobs.create bigquery.jobs.get bigquery.jobs.list bigquery.jobs.listAll bigquery.jobs.update
-
Create serviceaccount for kaldea system and attach the roles :
kaldea-system.bigquery-jobs
bigquery.dataEditor
- (This pre-defined role from GCP is also a requirement!)
- Optional (External Table)
storage.objectViewer
- (This pre-defined role from GCP is also a requirement if you are using an external table with gcs)
bigtable.reader
- (This pre-defined role from GCP is also a requirementIf you are using an external table with bigtable)
Required Permission for : personal
You need a superset of the permission provided for kaldea-default
.
For those who needs access to columns access-controlled by policy tags, Fine-Grained Reader role is required. Please refer to the background information above.
Integrating within Kaldea Interface
Given the above steps are completed, you now have :
- Credential key(.json) for
kaldea-system
- Credential key(.json) for
kaldea-default