Funnel

Funnel is a toolkit for distributed task execution with a simple API. A task records its metadata, state, input and output files, resource requests, commands, and logs. Funnel queues tasks, schedules them to workers, and tracks their progress.

A worker downloads task inputs, runs the requested containerized commands, uploads outputs, and emits events and logs. Funnel supports a range of storage and scheduling back ends, including cloud services and HPC schedulers such as HTCondor and Slurm. Read the Funnel documentation for details.