Data Sources
By default, Rill will use a managed embedded analytics engine (DuckDB or ClickHouse) to support data ingestion. Whether you're working with cloud data warehouses, databases, file storage, or streaming data sources, Rill provides seamless connectivity and data ingestion capabilities. Once you have connected to your data source you can create downstream models, metrics views and visualize your data.
Don't forget to create a managed ClickHouse server before getting started!
# Connector YAML
# Reference documentation: https://docs.rilldata.com/reference/project-files/connectors
type: connector
driver: clickhouse
managed: true
In order to connect and browse through your data, you'll need to create a connector file. Browse through the options below for our supported connectors. Each connector is designed to handle the specific authentication and configuration requirements of your data source.
Rill supports connecting your data to both DuckDB and ClickHouse. However, there are still some features in development for managed ClickHouse. For more information see our managed ClickHouse docs. If you've still got questions, contact our team for more information and scheduled feature releases!
Data Warehouses
Athena
BigQuery
Redshift
Snowflake
Databases
MySQL
PostgreSQL
SQLite
Object Storage
Amazon S3
Google Cloud Storage
Microsoft Azure Blob Storage
Other Data Connectors
Google Sheets
HTTPS
Local File
Salesforce
Externally Hosted Services
If you have a firewall in front of your externally hosted service, you will need to whitelist the IP addresses below. This will allow you to connect to/from your service once your project is deployed to Rill Cloud.
35.196.245.100
34.74.117.37
35.196.153.31
34.75.22.143
34.148.167.51
35.237.60.193
Managed OLAP Engine Caveats
When deciding on which managed OLAP engine to use with Rill, you'll need to decide based on the following factors:
- Size of data: Consider the volume and growth rate of your datasets
- Familiarity with respective OLAP engine features: Assess your team's expertise with each engine's capabilities
- Integration complexity: Consider how well each engine integrates with your existing data infrastructure
In the case of sub 100GB of data, we recommend keeping the default engine, DuckDB, in order to minimize the integration complexity. The reason for this is that DuckDB has built-in functions to support the connectors listed on this page.
On the other hand, if you need to analyze 100s of GB of data, we would recommend using Managed ClickHouse. This will add some complexity (staging tables), but will in turn provide better dashboard performance.
If data leans either way, a good deciding factor for which OLAP engine to use is your familiarity with their SQL syntax. Whether you're creating models or using arithmetic functions in the metrics view, you'll need to utilize the engine's built-in functions.
If there's a connector that you're interested in or you're looking for the list of currently supported ClickHouse connectors, contact us!