Simple Pipeline

Type: Automation architecture pattern

Definition

The Simple Pipeline is the most fundamental automation pattern: trigger → process → output. A defined event (time, file change, webhook) initiates work that runs sequentially through processing steps and produces a final output. Examples include daily sales report generation (cron triggers CSV fetch → Python transform → markdown summary → save to folder) or backup automation (time trigger → compress directory → upload to storage). It's the building block from which all other patterns are composed.

How It Works

A pipeline starts with a trigger — something that initiates the run. Triggers can be time-based (cron job at 7am), event-based (file appears in folder, webhook received), or manual (user runs a command). The trigger provides any necessary input context (which file to process, which URL to fetch).

The process step runs the actual work: fetch data, transform it, apply business logic, generate output. In OpenClaw, this is typically a shell command or script invoked via exec, possibly with multiple stages chained via pipes.

The output step delivers results: saving a file, sending a message, updating a dashboard. For OpenClaw pipelines, this often means writing to a reports folder and sending a WhatsApp summary via the message tool.

The key discipline: test the full pipeline manually before scheduling it with cron. Run each step, verify the output, then automate. Skipping manual testing means debugging blindly in a cron context, which is much harder.

Pipelines compose with other patterns. The Kelly Router software factory uses Simple Pipeline for individual tasks within a larger Pipeline with Gates structure. Cron triggers pipelines; message tool delivers outputs.

Key Properties

  • Three stages — trigger initiates, process does the work, output delivers results
  • Sequential execution — each step runs after the previous completes; no parallelism
  • Test before automate — always run the full pipeline manually before scheduling
  • Common trigger types — cron (time), file watcher (event), webhook (external event)
  • Composes with other patterns — used within larger Hub and Spoke or Pipeline with Gates structures
  • Log every run — cron-history.log with timestamps and success/failure status
  • pipeline-with-gates — extends Simple Pipeline with quality validation between stages
  • event-driven — event-based variant of the pipeline trigger pattern
  • hub-and-spoke — multiple pipelines routed through a central dispatcher

Source Chapters