Compress a prompt with gisting

Learn a compact prompt representation from conversations and a prompt text file. A Training is an editable project draft. A Training Run is one billed execution. A completed gisting Training Run publishes a full Model Version. Always pass --method gisting and --prompt-file. For weight adaptation, use Fine-tune a model.

To work with a coding agent, run claude -p "Let's use gisting. Follow https://platform.toloka.ai/agents/gisting.md" or give the same runbook URL to another agent.

Copy IDs from command output into later commands.

Before you start

You need:

macOS or Linux on ARM64 or x64. The installer uses compatible Bun or a standalone executable; standalone Linux downloads require glibc.
A Toloka account on a team. Browser sign-in needs no API key. Noninteractive automation can use an API key from the Business Console.
A funded team balance. Cost estimation is free; starting a Training Run places a hold up to the printed maximum charge.
Example conversations and a non-blank UTF-8 prompt file no larger than 256 KiB.

curl -fsSL https://platform.toloka.ai/cli/install.sh | sh
toloka version --json
toloka login
toloka whoami

Use the latest CLI release. This workflow does not import the TypeScript SDK and needs no project runtime. Rerun the installer for an immediate update. See CLI for installation modes, PATH fixes, and noninteractive authentication.

Step 2 — Create or select a project

The Training, datasets, prompt, billing, and output Model belong to one project.

toloka project list
toloka project create gisting --name "Support ticket gisting"
toloka project current

If you belong to more than one team, pass --team-id. To reuse an existing project:

toloka project add support-gisting pp.YOUR_PROJECT_ID
toloka project switch support-gisting

Step 3 — Put example conversations in a dataset

Each JSONL row is one chat. At least one turn must be user, and the final turn must be assistant with the expected output. Save these rows as support-tickets.jsonl:

{"messages":[{"role":"user","content":"Hi, how do I reset my password? I can't find the option anywhere."},{"role":"assistant","content":"question"}]}
{"messages":[{"role":"user","content":"This is the third time my order arrived damaged. I'm furious."},{"role":"assistant","content":"complaint"}]}
{"messages":[{"role":"user","content":"Just wanted to say your support team was incredibly helpful today, thank you!"},{"role":"assistant","content":"praise"}]}
{"messages":[{"role":"user","content":"What are your business hours on weekends?"},{"role":"assistant","content":"question"}]}
{"messages":[{"role":"user","content":"The app keeps crashing every time I open the checkout page. Please fix this."},{"role":"assistant","content":"complaint"}]}
{"messages":[{"role":"user","content":"Amazing product, it has saved me hours every week. Great job!"},{"role":"assistant","content":"praise"}]}
{"messages":[{"role":"user","content":"Can I change the shipping address after placing an order?"},{"role":"assistant","content":"question"}]}
{"messages":[{"role":"user","content":"I was charged twice for the same subscription and nobody is responding."},{"role":"assistant","content":"complaint"}]}
{"messages":[{"role":"user","content":"Do you offer student discounts?"},{"role":"assistant","content":"question"}]}
{"messages":[{"role":"user","content":"The new dashboard update is fantastic, much easier to use now."},{"role":"assistant","content":"praise"}]}

The conversation should contain the dynamic task input and expected output. Keep the static instructions you want to compress in the prompt file in the next step. The Training command will give this new dataset an 80% train, 10% validation, 10% test split and freeze it in a slice. These 10 rows yield 8 training rows, 1 validation row, and 1 test row. Create an empty dataset, upload the rows once, and list its ID:

toloka dataset create support-tickets
toloka dataset upload support-tickets ./support-tickets.jsonl
toloka dataset list

Uploads append, so repeating the upload adds the same rows again. Copy the ds.… ID.

Step 4 — Write the prompt file

Save this as prompt.txt:

Classify each support ticket as question, complaint, or praise.
Reply with one of those three labels and nothing else.

The file must contain non-whitespace UTF-8 text and be no larger than 256 KiB.

Step 5 — Choose a Base model

toloka base-model list

Gisting supports qwen3-4b and qwen3-8b. Larger catalog tiers exceed Model Version size limits because gisting publishes full weights. This guide uses qwen3-8b. You can also start from an SFT LoRA adapter Model Version based on one of the supported tiers. Pass --model-version instead of --base-model. A full-weight Model Version cannot be a source.

Step 6 — Create the gisting Training draft

toloka training create "Support ticket gisting" \
  --method gisting \
  --prompt-file ./prompt.txt \
  --base-model qwen3-8b \
  --dataset ds.YOUR_DATASET_ID

The command freezes a slice named after the Training and prints Created training trg.…. To use an existing ready slice, pass --slice <id> instead of --dataset; see Dataset selections. Copy that Training ID and inspect the draft:

toloka training get trg.YOUR_TRAINING_ID

Method is Prompt gisting. Output Model is - until the first Training Run is created, even if that run later blocks or fails before publishing a version. Optional gisting settings are --gist-tokens, --epochs, --learning-rate, --batch-size, and --max-sequence-length. --gist-tokens takes 8, 32, or 64, or the prompt fraction 1/4, 1/8, 1/16, or 1/32. The default is 1/8. See Training settings for accepted values and defaults.

Step 7 — Review the cost

toloka training cost trg.YOUR_TRAINING_ID --json > estimate.json

Open estimate.json, require status: "ready", and review the training rows, tokens, maximum GPU seconds, maximumCharge, and fingerprint. With the 10-row example and default split, expect 8 training rows and 1 validation row; 1 test row stays out of learning. Keep this file for the run command. Gisting counts each training token twice when calculating work. If validation blocks the estimate, the command exits with code 1 and lists every issue.

Starting the Training Run places a hold for the saved estimate’s maximum charge. Do not continue until you accept that amount. If the saved estimate is rejected or expires, calculate and review a new one before retrying.

See Training cost for the estimate, hold, settlement, and no-charge cases.

Step 8 — Start the Training Run

toloka training run trg.YOUR_TRAINING_ID --estimate-file estimate.json

The command uses the saved estimate and Support ticket gisting as the Model Version name. Copy the trn.… Training Run ID. Omitting --estimate-file would calculate a new estimate instead of using the one you reviewed.

toloka training status trn.YOUR_RUN_ID
toloka training logs trn.YOUR_RUN_ID
toloka training metrics trn.YOUR_RUN_ID

Status normally moves from queued to running to completed. Progress queued or pending means the job is waiting for GPU capacity. A Training Run can train longer than estimated. Its charge remains capped at the approved maximum, and it publishes when complete. Do not start a duplicate because the first Training Run is slower than estimated. If status is blocked with reason funds, add funds and run toloka training resume trn.YOUR_RUN_ID. See Training errors for other blocked states.

Step 9 — Use the trained model

When status is completed, inspect the Training and list its Model Versions:

toloka training get trg.YOUR_TRAINING_ID
toloka model versions mdl.YOUR_MODEL_ID

Copy the mdv.… Model Version ID and download it:

toloka model version download mdv.YOUR_VERSION_ID --out ./support-ticket-gisting

The directory contains a full model: config.json and either model.safetensors or a complete numbered set of model-*.safetensors weight shards. Tokenizer files, merges.txt, and chat_template.jinja can be included but are not guaranteed. Inspect the download manifest before choosing serving tooling. The CLI verifies each file’s size and SHA-256. The platform does not host or serve the downloaded Model Version.

If something fails

Use Training errors to search exact prompt, method, estimate, and Training Run messages.

Start

Data Labeling

Train

Workflows

Admin

Compress a prompt with gisting

Before you start

Step 2 — Create or select a project

Step 3 — Put example conversations in a dataset

Step 4 — Write the prompt file

Step 5 — Choose a Base model

Step 6 — Create the gisting Training draft

Step 7 — Review the cost

Step 8 — Start the Training Run

Step 9 — Use the trained model

If something fails

​Before you start

​Step 1 — Install and sign in

​Step 2 — Create or select a project

​Step 3 — Put example conversations in a dataset

​Step 4 — Write the prompt file

​Step 5 — Choose a Base model

​Step 6 — Create the gisting Training draft

​Step 7 — Review the cost

​Step 8 — Start the Training Run

​Step 9 — Use the trained model

​If something fails

​Related

Before you start

Step 1 — Install and sign in

Step 2 — Create or select a project

Step 3 — Put example conversations in a dataset

Step 4 — Write the prompt file

Step 5 — Choose a Base model

Step 6 — Create the gisting Training draft

Step 7 — Review the cost

Step 8 — Start the Training Run

Step 9 — Use the trained model

If something fails

Related