Skip to main content
Self-check combines automatic validation of node settings, automatically started lints, and test cases you or the agent run. Review test results on the node’s check cards and validation findings in the Issues window before you deploy. Configuration validation, automatic lints, and sample-based tests feed the Issues window. Errors block deployment; warnings can be overridden before reviewing the cost forecast.

​
Configuration validation and checks

Configuration validation checks settings whenever you or the agent edit a node. Missing or malformed settings, such as empty code or an invalid node name, block deployment until you fix them. Test checks run on sample values you provide. You or the agent start them and inspect the result. Lint checks inspect the node’s configuration or selected data without a sample case. They start automatically after you save changes. If a lint has not started or has failed, use Run lints in the Issues window or ask the agent to rerun it. Deployment waits for the required lints to finish.

​
The kinds of check

You or the agent can run all three test kinds. Annotation Quality Assurance requires quality criteria and LLM review: LLM Only or Hybrid on Generation, LLM Only on Quorum. Without LLM review, use View on UI for Experts to preview the form. Lint findings appear on the affected settings and in Issues. Match and End nodes have no checks.

​
Running a check

Ask the agent to add sample cases and run the node’s tests, or open a test-check card on the node. Click +, name the case, enter the sample JSON, and click Run. For Annotation Quality Assurance, fill in Task Input and Task Output. To test the form itself, enter Task Input, click Annotate with UI, complete the form, and submit. This validates the submission against the node’s output fields and fills Task Output. Then click Run to review the annotation against the quality criteria. Use Reannotate with UI to try another answer. See Review the labeling interface.

​
The states a check shows

Each case on a test-check card shows a state icon: Check not run, Check running, Check has run, or Check run failed. A run fails when it errors or returns an invalid result. A completed run shows its Check result; a failed run shows an Error. A run may also include Logs. Annotation Quality Assurance also shows how many verdicts passed, such as 2/3 passed. Failing verdicts are part of a completed result: they do not fail the check or block deployment. If a lint is unfinished or fails, Issues identifies it. Wait for a running lint, or click Run lints to retry one that failed. A completed lint can report findings that still need attention.

​
The deploy gate

A check case can become invalid if its input names a field removed from the node. Fix or delete the case before deploying. Only nodes on a path from Start to End affect deployment. While a node has a configuration error, its Checks tab shows Checks are disabled because the node has errors. Fix the error to run the node’s checks.
  • With Errors present, resolve the errors before deploying. Use Fix all with agent, click an issue to open the setting it points to, or use Run lints when it appears.
  • With Warnings only, review them and select I want to deploy with unresolved warnings if you choose to proceed.
  • With No issues, Deploy opens Cost breakdown (forecast) to confirm the spend. See Pipeline pricing.
The public API uses the same deployment gate. A rejected deployment request starts eligible pending lints. Wait for them to finish, then retry. Restart a failed lint before retrying deployment.

​
When a passing check goes stale

Changing a setting used by a check invalidates its previous result. Test cases must be run again; lints start automatically after saved edits. Editing a test case’s sample values also resets its result. After changing dataset item values, ask the agent to rerun the dataset lints: those edits do not invalidate a completed result, so Run lints does not appear for them. An interface edit does not reset an Annotation Quality Assurance result. After editing the form, use Reannotate with UI and rerun the check to verify a new submission.

​
What the self-check is not

Checks help find configuration problems before a paid run. They do not replace a pilot on representative data.
  • Test checks use the samples you provide. Dataset lints inspect the selected data.
  • Running checks adds nothing to your platform bill.
  • Annotation Quality Assurance reviews one sample; a passing sample does not establish the quality of every label in a run.
  • Items can still fail during a run because of invalid outputs, timeouts, or size limits. See Code node limits.
A Code Test runs your real code with your real environment variables and secrets. Calls to outside services can write records or incur charges on those services.