Skip to main content
A pipeline is a connected set of nodes that label, transform, or route your data on the way from a fixed Start to a fixed End, with the nodes you add in between. Every node output, including each Match branch, must connect onward toward End. An unconnected output blocks the deploy. Most nodes read input fields and write output fields. The exceptions are the Start, the End, and the Match node, noted below.

​
Start

Every pipeline has exactly one Start node, and you cannot add or remove it. It reads no fields from other nodes. It binds to a dataset and exposes the dataset’s fields as the pipeline’s input fields. An item filter and a cap on items to launch narrow what a run processes. No run launches more than 10,000 items. The Start node also carries the built-in Dataset Items lint, which must finish before the deploy is allowed.

​
Generation

Generation turns raw items into labeled data: it reads its input fields and returns the labels the labeler produces, plus a verdict for each quality criterion you define. By default expert labelers do the work, and you set their audience, per-task price, and an optional onboarding course. On request an LLM labels instead (synthetic labeling). Quality control runs as LLM review, human review, or both. Under synthetic labeling it is LLM review only.

​
Quorum

Quorum has the same item labeled independently by one or more experts. You choose how many, 1 to 100. It returns the full set of their labels, one per expert. The reduction to a single agreed answer happens in a following Code node. Majority agreement only removes experts who persistently disagree with the group. Quality control is LLM review or no review. There is no human review, and experts are always ID-verified on this node.

​
Code

Code runs your Python code in a sandbox to transform data by reshaping fields, computing values, or calling an external service. Your code is an async main function that reads the input fields you declare, as one dictionary, and returns a dictionary keyed by the declared output field names. Environment variables (fixed values or secrets) carry configuration. Check cases run the code on sample inputs while you edit. A run may take up to 60 seconds per item and return up to 10 MB of output.

​
Match

Match routes each item down one of several branches based on conditions you define. An item that meets no condition follows the default branch. It reads the input fields its conditions test and produces no new fields. Each condition combines field tests (equal or not equal, compared as text) with AND or OR, and the first matching condition wins.

​
End

Every pipeline has exactly one End node, and you cannot add or remove it. It produces no new fields. You choose which of the fields the pipeline produced are returned as the run’s output.