Pipeline Manager (PM)

Works in cooperation with the Workflow Manager (WFM) to ensure ingestion executions are healthy.

The workflow manager will assign tasks to the Pipeline Manager that require to scale.

For instance, after a seed execution is finished we need to make sure there are no leftover contents in the cache or the binary data server. So the pipeline manager kicks in and does that for each job / record in an execution.