About
A pipeline is a finite or infinite automata known as a stream) where:
- the data
- transition (pipe)
- from one state to another
- via a series of transformations (work)
A pipeline creates a composition relationship.
A pipeline is also known as:
- Compose
- Chain (for instance Chain of command) - Daisy Chain ;)
Pipeline follows a compositional structure known as cascade of operations.
Flow vs Pipeline
A dataflow (data workflow) is:
- a directed graph structure (loop may be present)
- of pipeline (ie the nodes are pipelines)
Model
A pipeline is a combination of:
- an input source
- an output destination.
- and a sequence of data (that may be a byte of an object). In the general sense, it's called a message.
- that may be chained to form a pipeline
Type
Imperative
The pipeline is executed step by step
Declarative
The pipeline is executed only when the terminal operation is called.
All steps are building a composite type known as algebraic data type.
Example
Shell
In an OS Shell (Dos, bash), a serie of command connected by the pipe operators forms a pipeline. See Pipeline
Code
By returning the calling object from a function, you can compose (or chain) functions. See Builder. When we compose (chain) an operation, the output of one operation becomes the input for the next operation, and operations are applied from left to right.