Configure Data Refresh

When creating or updating a model in Rill Cloud, you also have the option to configure how often the underlying model is refreshed (and thus ingested into the underlying OLAP layer powering Rill models and dashboards). By default, models are refreshed manually, but this can also be automated to a predefined schedule. This is handled through the underlying model settings and/or project YAML using standard cron or Go duration syntax.

Configuring model refreshes for Cloud deployments

It is generally strongly recommended to configure model refreshes when deploying a project to Rill Cloud to ensure that your production data (and dashboards) remains up-to-date. The interval that you should set really depends on how often your own data is being refreshed. Furthermore, while it is technically possible to configure model refreshes for Rill Developer as well, Rill Developer is primarily used for local development and thus typically does not require working with the most up-to-date data (local model refreshes that occur too often could also lead to resource constraints on your local machine). For more details, please see our pages on environments, templating, and performance optimization.

Configuring model refresh individually

To specify a model refresh schedule for a particular model, this can be handled using the refresh property in the underlying YAML file. For example, to set a daily refresh for a model, you can do the following:

refresh:
  every: 24h

Similarly, if you would like to utilize cron syntax, the following example would update a model every 15 minutes:

refresh:
  cron: '*/15 * * * *'

model settings

For more details about available model configurations and properties, check our model YAML reference page.

Configuring a project-wide default

You can also specify a project-wide refresh schedule that will apply to all models by default. This can be done through the rill.yaml file. More details can be found here.

Using the same example as above, the following sets a project-wide default of refreshing models every 24 hours:

models:
  refresh:
    every: 24h

Similarly, the following would use cron syntax to set a project-wide configuration of refreshing models by default every 15 minutes (unless overridden at the individual model level):

models:
  refresh:
    cron: '*/15 * * * *'

Did you know?

If you have both a project-wide default and model specific refresh schedule configured in the same project, the model specific refresh will override the project default based on how inheritance works in Rill. Otherwise, if not specified, the project-wide default will be used instead!

Running scheduled refreshes in development

By default, scheduled refreshes are not applied in Rill Developer. If you want to run or test scheduled refreshes in local development, you can override this behavior using the run_in_dev property:

refresh:
  cron: 0 * * * *
  run_in_dev: true