Skip to main content

Pinot

Overview​

Apache Pinot is a real-time distributed OLAP datastore purpose-built for low-latency, high-throughput analytics, and perfect for user-facing analytical workloads.

Rill supports connecting to an existing Pinot cluster and using it as an OLAP engine to power Rill dashboards built against external tables.

Connection string (DSN)​

Rill connects to Pinot using the Pinot Golang Client and requires a connection string of the following format: http://<user>:<password>@<host>:<port>. hostand port should be of the Pinot Controller server. If user or password contain special characters they should be URL encoded (i.e. p@ssword -> p%40ssword). This should be set in the connector.pinot.dsn property in Rill.

As an example, this typically looks something like:

connector.pinot.dsn="https://username:password@localhost:9000"
Need help connecting to Pinot?

If you would like to connect Rill to an existing Pinot instance, please don't hesitate to contact us. We'd love to help!

Setting the default OLAP connection​

You'll also need to update the olap_connector property in your project's rill.yaml to change the default OLAP engine to Pinot:


olap_connector: pinot

note

For more information about available properties in rill.yaml, see our project YAML documentation.

Interested in using multiple OLAP engines in the same project?

Please see our Using Multiple OLAP Engines page.

Configuring Rill Developer​

When using Rill for local development, there are two options to configure Rill to enable Pinot as an OLAP engine:

  • You can set connector.pinot.dsn in your project's .env file or try pulling existing credentials locally using rill env pull if the project has already been deployed to Rill Cloud
  • You can pass in connector.pinot.dsn as a variable to rill start directly (e.g. rill start --var connector.pinot.dsn=...)
Getting DSN errors in dashboards after setting .env?

If you are facing issues related to DSN connection errors in your dashboards even after setting the connection string via the project's .env file, try restarting Rill using the rill start --reset command.

Configuring Rill Cloud​

When deploying a Pinot-backed project to Rill Cloud, you have the following options to pass the appropriate connection string to Rill Cloud:

  • Use the rill env configure command to set connector.pinot.dsn after deploying the project
  • If connector.pinot.dsn has already been set in your project .env, you can push and update these variables directly in your cloud deployment by using the rill env push command
info

Note that you must cd into the Git repository that your project was deployed from before running rill env configure.

Support for Multi-Valued dimensions​

Multi-Valued dimensions needed to be defined in the dashboard yaml as expressions using arrayToMv function. For example if RandomAirports is a multi-valued column in a Pinot table then the dimension definition will look like:

- label: RandomAirports
  expression: arrayToMv(RandomAirports)
  name: RandomAirports
  description: "Random Airports"
  ignore: false

Refer to the Dashboard YAML reference page for all dimension properties detail.

note

Pinot does not support unnest function so don't set unnest property to true in the dimension definition of dashboard yaml.

Additional Notes​

  • At the moment, we do not support modeling with Pinot. If this is something you're interested in, please contact us.
  • For dashboards powered by Pinot, measure definitions are required to follow Pinot SQL syntax.