BlogRelease 0.3 - environments

August 31, 2024

Release 0.3 - environments

Joe Freeman

Version 0.3 has been released. This reworks how environments are implemented. Previously, environments were mostly isolated from each other, but this release allows interaction between them. Environments are now arranged into a hierarchy, where an environment may specifiy a 'base' environment. The base environment (and any ancestors) will be used for looking up cached results.

For example, a 'development' environment could have a 'production' environment as its base, which means that cached production results will be available when running a workflow in the development environment. The flow of data is limited in this direction - results from the development environment won't be available in the production environment.

Another feature this unlocks is the ability to re-run a step within a workflow in any 'child' environment. Continuing the example from above: when investigating a failed production run, you can repatedly re-run the problematic step in a development environment while making changes to the code, and without having to re-compute upstream dependencies.

Memoisation follows the same inheritance logic: steps that are memoised become available to the development environment.

Join the mailing list

Get notified of new product features.