Skip to main content

Google Cloud Storage (GCS)

How to configure credentials in Rill​

How you configure access to GCS depends on whether you are developing a project locally using rill start or are setting up a deployment using rill deploy.

Configure credentials for local development​

When developing a project locally, Rill uses the credentials configured in your local environment using the Google Cloud CLI (gcloud). Follow these steps to configure it:

  1. Open a terminal window and run gcloud auth list to check if you already have the Google Cloud CLI installed and authenticated.

  2. If it did not print information about your user, follow the steps on Install the Google Cloud CLI. Make sure to run gcloud init after installation as described in the tutorial.

You have now configured Google Cloud access from your local environment. Rill will detect and use your credentials next time you try to ingest a source.

Configure credentials for deployments on Rill Cloud​

When deploying a project to Rill Cloud, Rill requires you to explicitly provide a JSON key file for a Google Cloud service account with access to the buckets used in your project.

When you first deploy a project using rill deploy, you will be prompted to provide credentials for the remote sources in your project that require authentication.

If you subsequently add sources that require new credentials (or if you input the wrong credentials during the initial deploy), you can update the credentials used by Rill Cloud by running:

rill env configure

Note that you must cd into the Git repository that your project was deployed from before running rill env configure.

How to create a service account using the Google Cloud Console​

Here is a step-by-step guide on how to create a Google Cloud service account with read-only access to GCS:

  1. Navigate to the Service Accounts page under "IAM & Admin" in the Google Cloud Console.

  2. Click the "Create Service Account" button at the top of the page.

  3. In the "Create Service Account" window, enter a name for the service account, then click "Create and continue".

  4. In the "Role" field, search for and select the "Storage Object Viewer" role. Click "Continue", then click "Done".

    • This grants the service account access to data in all buckets. To only grant access to data in a specific bucket, leave the "Role" field blank, click "Done", then follow the steps described in Add a principal to a bucket-level policy.

  5. On the "Service Accounts" page, locate the service account you just created and click on the three dots on the right-hand side. Select "Manage Keys" from the dropdown menu.

  6. On the "Keys" page, click the "Add key" button and select "Create new key".

  7. Choose the "JSON" key type and click "Create".

  8. Download and save the JSON key file to a secure location on your computer.

How to create a service account using the gcloud CLI​

  1. Open a terminal window and follow the steps on Install the Google Cloud CLI if you haven't already done so.

  2. You will need your Google Cloud project ID to complete this tutorial. Run the following command to show it:

    gcloud config get project

  3. Replace [PROJECT_ID] with your project ID in the following command, and run it to create a new service account (optionally also replace rill-service-account with a name of your choice):

    gcloud iam service-accounts create rill-service-account --project [PROJECT_ID]

  4. Grant the service account access to data in Google Cloud Storage:

    • To grant access to data in all buckets, replace [PROJECT_ID] with your project ID in the following command, and run it:
      gcloud projects add-iam-policy-binding [PROJECT_ID] \
          --member="serviceAccount:rill-service-account@[PROJECT_ID].iam.gserviceaccount.com" \
          --role="roles/storage.objectViewer"
    • To only grant access to data in a specific bucket, replace [BUCKET_NAME] and [PROJECT_ID] with your details in the following command, and run it:
      gcloud storage buckets add-iam-policy-binding gs://[BUCKET_NAME] \
          --member="serviceAccount:rill-service-account@[PROJECT_ID].iam.gserviceaccount.com" \
          --role="roles/storage.objectViewer"

  5. Replace [PROJECT_ID] with your project ID in the following command, and run it to create a key file for the service account:

    gcloud iam service-accounts keys create rill-service-account.json \
        --iam-account rill-service-account@[PROJECT_ID].iam.gserviceaccount.com

  6. You have now created a JSON key file named rill-service-account.json in your current working directory.