Skip to main content

Connector YAML

When you add olap_connector to your rill.yaml file, you will need to set up a <connector_name>.yaml file in the 'connectors' directory. This file requires the following parameters,type and driver (see below for more parameter options). Rill will automatically test the connectivity to the OLAP engine upon saving the file. This can be viewed in the connectors tab in the UI.

Did you know?

Starting from Rill 0.46, you can directly create OLAP engines from the UI! Select + Add -> Data -> Connect an OLAP engine

Properties​

type​

[string] - Refers to the resource type and must be connector (required)

Common Properties​

name​

[string] - Name is usually inferred from the filename, but can be specified manually.

refs​

[array of string] - List of resource references

dev​

[object] - Overrides any properties in development environment.

prod​

[object] - Overrides any properties in production environment.

One of Properties Options​

athena​

Configuration properties specific to the athena

driver​

[string] - Refers to the driver type and must be driver athena (required)

aws_access_key_id​

[string] - AWS Access Key ID for Athena access

aws_secret_access_key​

[string] - AWS Secret Access Key for Athena access

aws_access_token​

[string] - Optional AWS session token for temporary credentials

external_id​

[string] - Optional External ID for assuming a role

role_arn​

[string] - Optional AWS Role ARN to assume when accessing Athena

role_session_name​

[string] - Optional Session name when assuming the role

allow_host_access​

[boolean] - Allow access to host environment configuration

azure​

Configuration properties specific to the azure

driver​

[string] - Refers to the driver type and must be driver azure (required)

azure_storage_account​

[string] - Azure storage account name

azure_storage_key​

[string] - Azure storage access key

azure_storage_sas_token​

[string] - Optional azure SAS token for authentication

azure_storage_connection_string​

[string] - Optional azure connection string for storage account

allow_host_access​

[boolean] - Allow access to host environment configuration

bigquery​

Configuration properties specific to the bigquery

driver​

[string] - Refers to the driver type and must be driver bigquery (required)

google_application_credentials​

[string] - Path to the Google Cloud credentials JSON file

allow_host_access​

[boolean] - Allow access to host environment configuration

clickhouse​

Configuration properties specific to the clickhouse

driver​

[string] - Refers to the driver type and must be driver clickhouse (required)

managed​

[boolean] - true means Rill will provision the connector using the default provisioner. false disables automatic provisioning.

dsn​

[string] - DSN(Data Source Name) for the ClickHouse connection

username​

[string] - Username for authentication

password​

[string] - Password for authentication

host​

[string] - Host where the ClickHouse instance is running

port​

[integer] - Port where the ClickHouse instance is accessible

database​

[string] - Name of the ClickHouse database within the cluster

ssl​

[boolean] - Indicates whether a secured SSL connection is required

cluster​

[string] - Cluster name, required for running distributed queries

log_queries​

[boolean] - Controls whether to log raw SQL queries

settings_override​

[string] - override the default settings used in queries. example readonly = 1, session_timezone = 'UTC'

embed_port​

[integer] - Port to run ClickHouse locally (0 for random port)

can_scale_to_zero​

[boolean] - Indicates if the database can scale to zero

max_open_conns​

[integer] - Maximum number of open connections to the database

max_idle_conns​

[integer] - Maximum number of idle connections in the pool

dial_timeout​

[string] - Timeout for dialing the ClickHouse server

conn_max_lifetime​

[string] - Maximum time a connection may be reused

read_timeout​

[string] - Maximum time for a connection to read data

druid​

Configuration properties specific to the druid

driver​

[string] - Refers to the driver type and must be driver druid (required)

dsn​

[string] - Data Source Name (DSN) for connecting to Druid (required)

username​

[string] - Username for authenticating with Druid

password​

[string] - Password for authenticating with Druid

host​

[string] - Hostname of the Druid coordinator or broker

port​

[integer] - Port number of the Druid service

ssl​

[boolean] - Enable SSL for secure connection

log_queries​

[boolean] - Log raw SQL queries sent to Druid

max_open_conns​

[integer] - Maximum number of open database connections (0 = default, -1 = unlimited)

skip_version_check​

[boolean] - Skip checking Druid version compatibility

duckdb​

Configuration properties specific to the duckdb

driver​

[string] - Refers to the driver type and must be driver duckdb (required)

pool_size​

[integer] - Number of concurrent connections and queries allowed

allow_host_access​

[boolean] - Whether access to the local environment and file system is allowed

cpu​

[integer] - Number of CPU cores available to the database

memory_limit_gb​

[integer] - Amount of memory in GB available to the database

read_write_ratio​

[number] - Ratio of resources allocated to the read database; used to divide CPU and memory

boot_queries​

[string] - SQL to run when initializing a new connection, before extensions and defaults

init_sql​

[string] - SQL to run when initializing a new connection, after extensions and defaults

log_queries​

[boolean] - Whether to log raw SQL queries executed through OLAP

gcs​

Configuration properties specific to the gcs

driver​

[string] - Refers to the driver type and must be driver gcs (required)

google_application_credentials​

[string] - Google Cloud credentials JSON string

allow_host_access​

[boolean] - Allow access to host environment configuration

key_id​

[string] - Optional S3-compatible Key ID when used in compatibility mode

secret​

[string] - Optional S3-compatible Secret when used in compatibility mode

https​

Configuration properties specific to the https

driver​

[string] - Refers to the driver type and must be driver https (required)

path​

[string] - The full HTTPS URI to fetch data from (required)

headers​

[object] - HTTP headers to include in the request

local_file​

Configuration properties specific to the local_file

driver​

[string] - Refers to the driver type and must be driver local_file (required)

dsn​

[string] - Data Source Name (DSN) indicating the file path or location of the local file (required)

allow_host_access​

[boolean] - Flag to indicate if access to host-level file paths is permitted

motherduck​

Configuration properties specific to the motherduck

driver​

[string] - Refers to the driver type and must be driver motherduck (required)

dsn​

[string] - Data Source Name (DSN) specifying the MotherDuck connection endpoint (required)

token​

[string] - Authentication token for accessing MotherDuck (secret) (required)

mysql​

Configuration properties specific to the mysql

driver​

[string] - Refers to the driver type and must be driver mysql (required)

dsn​

[string] - DSN(Data Source Name) for the mysql connection (required)

pinot​

Configuration properties specific to the pinot

driver​

[string] - Refers to the driver type and must be driver pinot (required)

dsn​

[string] - DSN(Data Source Name) for the Pinot connection (required)

username​

[string] - Username for authenticating with Pinot

password​

[string] - Password for authenticating with Pinot

broker_host​

[string] - Hostname of the Pinot broker (required)

broker_port​

[integer] - Port number for the Pinot broker

controller_host​

[string] - Hostname of the Pinot controller (required)

controller_port​

[integer] - Port number for the Pinot controller

ssl​

[boolean] - Enable SSL connection to Pinot

log_queries​

[boolean] - Log raw SQL queries executed through Pinot

max_open_conns​

[integer] - Maximum number of open connections to the Pinot database

postgres​

Configuration properties specific to the postgres

driver​

[string] - Refers to the driver type and must be driver postgres (required)

dsn​

[string] - DSN(Data Source Name) for the postgres connection (required)

redshift​

Configuration properties specific to the redshift

driver​

[string] - Refers to the driver type and must be driver redshift (required)

aws_access_key_id​

[string] - AWS access key ID for authentication

aws_secret_access_key​

[string] - AWS secret access key for authentication

aws_access_token​

[string] - AWS session token for temporary credentials (optional)

allow_host_access​

[boolean] - Allow access to host environment configuration

s3​

Configuration properties specific to the s3

driver​

[string] - Refers to the driver type and must be driver s3 (required)

aws_access_key_id​

[string] - AWS Access Key ID used for authentication

aws_secret_access_key​

[string] - AWS Secret Access Key used for authentication

aws_access_token​

[string] - Optional AWS session token for temporary credentials

endpoint​

[string] - Optional custom endpoint URL for S3-compatible storage

region​

[string] - AWS region of the S3 bucket

allow_host_access​

[boolean] - Allow access to host environment configuration

retain_files​

[boolean] - Whether to retain intermediate files after processing

salesforce​

Configuration properties specific to the salesforce

driver​

[string] - Refers to the driver type and must be driver salesforce (required)

username​

[string] - Salesforce account username (required)

password​

[string] - Salesforce account password (secret)

key​

[string] - Authentication key for Salesforce (secret)

endpoint​

[string] - Salesforce API endpoint URL (required)

client_id​

[string] - Client ID used for Salesforce OAuth authentication

slack​

Configuration properties specific to the slack

driver​

[string] - Refers to the driver type and must be driver slack (required)

bot_token​

[string] - Bot token used for authenticating Slack API requests (required)

snowflake​

Configuration properties specific to the snowflake

driver​

[string] - Refers to the driver type and must be driver snowflake (required)

dsn​

[string] - DSN (Data Source Name) for the Snowflake connection (required)

parallel_fetch_limit​

[integer] - Maximum number of concurrent fetches during query execution

sqlite​

Configuration properties specific to the sqlite

driver​

[string] - Refers to the driver type and must be driver sqlite (required)

dsn​

[string] - DSN(Data Source Name) for the sqlite connection (required)