How it works

This page gives a brief explanation of how Coflux works. More detail is available in the documentation.

Repositories

A repository is a Python module, which contains one or more functions annotated with @workflow or @task. In both cases, these indicate targets that Coflux can call. The difference between the two is simply that workflow functions will become available as entry points for a workflow run, whereas taskfunctions are called by workflows (or other tasks).

# example1/repo.py

@task()
def build_greeting(name):
    return f"Hello, {name}!"

@workflow()
def my_workflow(name):
    print build_greeting(name)

Agents

An agent is started with references to one or more repositories. For each repository, the agent will discover all of the targets (the annotated workflows and tasks). The function names and parameters all contribute to a manifest. The agent connects to the Coflux server, sends the manifest, and then waits to be assignedexecutions to run.

$ coflux agent.run --reload \
    example1.repo \
    example2.repo

Runs

Now a workflow can be triggered. This can be done manually from the Coflux UI, or via the CLI; or programmatically, using the API. Doing so will create a run, an initial step, and a corresponding execution. The execution is placed in a queue to be sent to a connected agent.

$ coflux workflow.run \
    example1.repo my_workflow '"Joe"'

Executions

The execution will be received by an agent, along with any arguments. The agent will then create a new sub-process, and run the function in this process. Whilst executing the function, a couple of things might happen:

The function completes - either by (explicitly or implicitly) returning a value, or by raising an exception. In either case, the result will be reported back to the server, and the process will be shut down.
The function calls another task. In this case, the task won’t be immediately executed. Instead, execution is paused, and the call details are sent back to the server. These details include the target name and the arguments being passed to it. If the call is being made synchronously (which is the default), the agent will also make a request to the server for the result of this function, and the process will wait for the response. Meanwhile, the server takes care of recording and then sending that execution to an agent for processing (possibly back to the same agent). Once the result is available, the server will send it back to the process, which can then continue executing.

Join the mailing list

Get notified of new product features.