TypeScript SDK

import { createClient } from "@toloka-platform/sdk";

const client = createClient({
  baseUrl: "https://platform.toloka.ai",
  apiKey: process.env.TOLOKA_API_KEY!,
});

const review = await client.reviews.reject({
  reviewId: "trr.019a0000-0000-7000-8000-000000000001",
  criteria: [
    "example"
  ],
  comment: "example"
});
console.log(review);

curl --request POST \
  --url https://platform.toloka.ai/api/v2-beta/reviews/{reviewId}/reject \
  --header 'Authorization: <api-key>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "comment": "<string>",
  "criteria": [
    "<string>"
  ]
}
'

import requests

url = "https://platform.toloka.ai/api/v2-beta/reviews/{reviewId}/reject"

payload = {
    "comment": "<string>",
    "criteria": ["<string>"]
}
headers = {
    "Authorization": "<api-key>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

{
  "checkedRejectsLeft": 4503599627370495,
  "closedAt": "2023-11-07T05:31:56Z",
  "deadlineAt": "2023-11-07T05:31:56Z",
  "id": "<string>",
  "itemId": "<string>",
  "nodeId": "<string>",
  "nodeName": "<string>",
  "performer": {
    "expertId": "<string>",
    "type": "expert"
  },
  "qualityCriteria": [
    {
      "fail": "<string>",
      "id": "<string>",
      "pass": "<string>",
      "qualityElement": "<string>"
    }
  ],
  "runId": "<string>",
  "status": "needs_review",
  "submittedAt": "2023-11-07T05:31:56Z",
  "taskContent": {
    "input": {},
    "output": {}
  },
  "verdicts": [
    {
      "comment": "<string>",
      "createdAt": "2023-11-07T05:31:56Z",
      "criteria": [
        "<string>"
      ],
      "decision": "accept",
      "forced": true,
      "id": "<string>",
      "llmQaCheck": {
        "checkedAt": "2023-11-07T05:31:56Z",
        "reason": "<string>",
        "verdict": "agree"
      },
      "scopedTokenId": "<string>",
      "source": "ui",
      "userId": "<string>"
    }
  ]
}

Reviews

Reject a task

Only teams with customer review enabled: other teams get 403 The "customerReview" feature is not enabled for this team. Reject the task under review, naming the quality criteria it fails and explaining what is wrong. The LLM QA check then checks this evidence: if it agrees, the reject is final, the item fails at this node, and nothing is billed for the task; otherwise the review returns with the check’s reason for a new decision, or expires if its deadline has passed. With force, the check is skipped and the reject is final at once; a forced reject is accepted only after the LLM QA check has returned a reject of this review, so reject without force first.

POST

reviews

{reviewId}

reject

TypeScript SDK

import { createClient } from "@toloka-platform/sdk";

const client = createClient({
  baseUrl: "https://platform.toloka.ai",
  apiKey: process.env.TOLOKA_API_KEY!,
});

const review = await client.reviews.reject({
  reviewId: "trr.019a0000-0000-7000-8000-000000000001",
  criteria: [
    "example"
  ],
  comment: "example"
});
console.log(review);

curl --request POST \
  --url https://platform.toloka.ai/api/v2-beta/reviews/{reviewId}/reject \
  --header 'Authorization: <api-key>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "comment": "<string>",
  "criteria": [
    "<string>"
  ]
}
'

import requests

url = "https://platform.toloka.ai/api/v2-beta/reviews/{reviewId}/reject"

payload = {
    "comment": "<string>",
    "criteria": ["<string>"]
}
headers = {
    "Authorization": "<api-key>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

{
  "checkedRejectsLeft": 4503599627370495,
  "closedAt": "2023-11-07T05:31:56Z",
  "deadlineAt": "2023-11-07T05:31:56Z",
  "id": "<string>",
  "itemId": "<string>",
  "nodeId": "<string>",
  "nodeName": "<string>",
  "performer": {
    "expertId": "<string>",
    "type": "expert"
  },
  "qualityCriteria": [
    {
      "fail": "<string>",
      "id": "<string>",
      "pass": "<string>",
      "qualityElement": "<string>"
    }
  ],
  "runId": "<string>",
  "status": "needs_review",
  "submittedAt": "2023-11-07T05:31:56Z",
  "taskContent": {
    "input": {},
    "output": {}
  },
  "verdicts": [
    {
      "comment": "<string>",
      "createdAt": "2023-11-07T05:31:56Z",
      "criteria": [
        "<string>"
      ],
      "decision": "accept",
      "forced": true,
      "id": "<string>",
      "llmQaCheck": {
        "checkedAt": "2023-11-07T05:31:56Z",
        "reason": "<string>",
        "verdict": "agree"
      },
      "scopedTokenId": "<string>",
      "source": "ui",
      "userId": "<string>"
    }
  ]
}

Authorizations

Authorization

string

header

required

Enter ApiKey <your-api-key> in Authorization, including the ApiKey prefix. In cURL and Python samples, <api-key> means this full value. The TypeScript SDK adds the prefix for you.

Path Parameters

reviewId

string

required

Review ID with prefix trr..

Pattern: ^trr\.[0-9a-f]{8}-[0-9a-f]{4}-[7][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$

Body

application/json

A reject with its evidence.

comment

string

required

What is wrong with the task, at most 10000 characters.

Maximum string length: 10000

criteria

string[]

required

IDs of the quality criteria the task fails: at least one, each named once.

force

boolean

Skip the LLM QA check and reject at once. The reject is final and does not count toward the checked rejects. Allowed only after the LLM QA check has returned a reject of this review; otherwise the response is 409. Defaults to false.

Response

The review after the decision

A customer review of one task, with its content, quality criteria and decisions.

checkedRejectsLeft

integer

required

How many more rejects the LLM QA check will check, out of 10. At 0, accept the task or send a forced reject.

Required range: 0 <= x <= 9007199254740991

closedAt

string<date-time> | null

required

Time the review closed, as an ISO 8601 timestamp, or null while it is open.

Pattern:

^(?:(?:\d\d[2468][048]|\d\d[13579][26]|\d\d0[48]|[02468][048]00|[13579][26]00)-02-29|\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\d|30)|(?:02)-(?:0[1-9]|1\d|2[0-8])))T(?:(?:[01]\d|2[0-3]):[0-5]\d:[0-5]\d(?:\.\d+)?(?:Z))$

deadlineAt

string<date-time>

required

Deadline for a decision, as an ISO 8601 timestamp. From then on the review takes no decisions, but it closes a little later: a review waiting for a decision expires shortly after the deadline, and its task is accepted automatically (the expert is paid and the task is billed); a review checking a reject closes when the LLM QA check answers.

Pattern:

^(?:(?:\d\d[2468][048]|\d\d[13579][26]|\d\d0[48]|[02468][048]00|[13579][26]00)-02-29|\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\d|30)|(?:02)-(?:0[1-9]|1\d|2[0-8])))T(?:(?:[01]\d|2[0-3]):[0-5]\d:[0-5]\d(?:\.\d+)?(?:Z))$

string

required

Review ID with prefix trr..

Pattern: ^trr\.[0-9a-f]{8}-[0-9a-f]{4}-[7][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$

itemId

string

required

Run item ID with prefix plri..

Pattern: ^plri\.[0-9a-f]{8}-[0-9a-f]{4}-[7][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$

nodeId

string

required

ID of the generation node whose task is under review.

nodeName

string

required

Name of that generation node.

performer

object

required

Who did the task.

Show child attributes

qualityCriteria

object[]

required

The quality criteria a reject can name, as they were when the task reached review.

Show child attributes

runId

string

required

Pipeline run ID with prefix plr..

Pattern: ^plr\.[0-9a-f]{8}-[0-9a-f]{4}-[7][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$

status

enum<string>

required

Review status. needs_review: waiting for the first decision. returned: the LLM QA check did not confirm the last reject, so the review waits for a new decision. checking: the LLM QA check is checking a reject, and decisions are refused until it answers. accepted: the team accepted the task. rejected: the reject is final, either forced or confirmed by the LLM QA check. expired: the deadline passed while the review was open, whether it waited for a decision or was checking a reject the check then did not confirm, so the task was accepted automatically.

Available options:

needs_review,

returned,

checking,

accepted,

rejected,

expired

submittedAt

string<date-time>

required

Time the task was submitted, as an ISO 8601 timestamp.

Pattern:

^(?:(?:\d\d[2468][048]|\d\d[13579][26]|\d\d0[48]|[02468][048]00|[13579][26]00)-02-29|\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\d|30)|(?:02)-(?:0[1-9]|1\d|2[0-8])))T(?:(?:[01]\d|2[0-3]):[0-5]\d:[0-5]\d(?:\.\d+)?(?:Z))$

taskContent

object

required

What the task showed and what was submitted. Project files appear as file proxy URLs.

Show child attributes

verdicts

object[]

required

Decisions on this review, oldest first.

Show child attributes

Reference

TypeScript SDK

CLI

HTTP API

Reject a task

Authorizations

Path Parameters

Body

Response