HeyGen Decision

Classify text, images, and video, evaluate yes/no questions, and score content against your own criteria with HeyGen Decision 1.

HeyGen Decision 1 turns text, images, and video into decisions your application can use. Supply text or JSON in state, add optional media in assets, and ask named questions about them. The API returns an answer for each question, with probabilities you can use to route a request, rank a result, or decide when a person should review it.

Choose the question type that matches the answer you need. You can combine all three in one request.

Question type	Use it for	Answer
noul	A yes/no question or a statement to evaluate	An estimated probability of true, from 0 to 1
choice	Picking one category from a set you define	The selected category, a probability for every category, and confidence
score	Rating content against an ordered scale	A score from 0 to the last level’s index, probabilities, a level legend, and confidence

Every request starts from text. Add media in assets when the decision depends on what something looks like:

Text

Messages, transcripts, records, and JSON.

Image

Up to 4 images with your text.

Video

One video, read from sampled frames.

URLs inside state are text; the model reads the URL, and fetches media only from assets.

​

The API

Models	heygen-decision-1, heygen-decision-1-turbo (preview)
Endpoint	POST /v3/models/decisions
Completion	Synchronous HTTP 200: answers are in the response. No polling or webhook.
Input	One state, optional image and video assets, and 1–400 named questions
Auth	X-Api-Key from the API keys page, with the assets:write scope. Trial API keys and OAuth are not supported.

Set model to heygen-decision-1 on every request. heygen-decision-1-turbo is in preview and opens to all API keys soon.

​

Make a decision

This example classifies a support request, checks whether work is blocked, and rates its impact in one call.

curl -X POST https://api.heygen.com/v3/models/decisions \
  -H "X-Api-Key: $HEYGEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "heygen-decision-1",
    "state": "Video export fails for everyone on our team after today’s update. We have no workaround and cannot deliver our project.",
    "questions": {
      "team": {
        "type": "choice",
        "instructions": "Which support team should handle this request?",
        "criteria": {
          "video": "Video creation, rendering, or export problems",
          "account": "Sign-in, access, or workspace administration",
          "billing": "Payments, invoices, or subscription charges"
        }
      },
      "blocked": {
        "type": "noul",
        "instructions": "Is the customer unable to complete their work without a workaround?"
      },
      "impact": {
        "type": "score",
        "instructions": "How much does this issue affect the customer’s work?",
        "criteria": [
          "Cosmetic issue; work can continue normally",
          "Work is slower, but a usable workaround exists",
          "Work cannot continue and no workaround is available"
        ]
      }
    }
  }'

Illustrative response; IDs, probabilities, and token counts vary:

Response

{
  "data": {
    "id": "decision-0123456789abcdef",
    "request_id": "0123456789abcdef0123456789abcdef",
    "model": "heygen-decision-1",
    "answers": {
      "team": {
        "type": "choice",
        "choice": "video",
        "probabilities": { "video": 0.8, "account": 0.15, "billing": 0.05 },
        "confidence": 0.7
      },
      "blocked": { "type": "noul", "noul": 0.96 },
      "impact": {
        "type": "score",
        "score": 1.75,
        "probabilities": { "0": 0.05, "1": 0.15, "2": 0.8 },
        "legend": {
          "0": "Cosmetic issue; work can continue normally",
          "1": "Work is slower, but a usable workaround exists",
          "2": "Work cannot continue and no workaround is available"
        },
        "confidence": 0.625
      }
    },
    "usage": { "input_tokens": 1200, "output_tokens": 0 }
  }
}

Read data.answers.team.choice to select a support queue. Use the other answers in your own application logic. One question’s answer does not become input to another question in the same request; use a later request when a question depends on an earlier answer.

​

Parameters

Field	Required	Description
model	No	Set heygen-decision-1. heygen-decision-1-turbo is in preview. A null or unknown value is rejected.
state	Yes	The material to evaluate: a string, JSON object, or JSON array. Objects and arrays are serialized as text. A root number, boolean, or null is not accepted.
assets	No	Media shared by all questions: optional images and videos maps. Omit it or use null for text only. See Media inputs.
assets.images	No	An object mapping names to image inputs. Each input uses url, asset_id, or base64.
assets.videos	No	An object mapping names to video inputs, using the same source formats.
questions	Yes	An object with 1–400 entries. Each key names a question and becomes the matching key in data.answers. Names must be nonempty, contain no colon or newline, and have no leading or trailing whitespace.
questions.{name}.type	Yes	noul, choice, or score. Determines the shape of criteria and the answer.
questions.{name}.instructions	No	The question to answer or the rule to apply. Accepts a string, JSON object, array, or null. If omitted or blank, the question name is used, with underscores replaced by spaces. Explicit instructions are clearer.
questions.{name}.criteria	Depends on type	Optional true/false descriptions for noul; required named options for choice; required ordered levels for score. See below.

Unknown request and question fields are rejected. There is no top-level instructions, prompt, temperature, or max_tokens parameter.

For how to shape state, see Text.

​

Media inputs

Add media under assets.images or assets.videos. Each is an object keyed by names you choose, 1–1,024 characters long. All questions share all supplied media, and any question type works with any input.

The model reads images first, then videos, then state; within each group, inputs follow your request order. Names organize your request and are not shown to the model, so describe each input’s role in state by its order.

Each input page has a worked example: Image and Video.

​

Choose an asset source

Each named input accepts one of these source objects. You can mix source types in one request.

Uploaded asset

{ "type": "asset_id", "asset_id": "ASSET_ID" }

• Uploaded asset: use an asset_id from the asset upload API in the calling workspace. The asset must still exist, match the media group, and be readable by the caller. A generated output’s image_id or video_id is not an asset_id.
• URL: use a public HTTPS URL that serves the media directly, without authentication or redirects, with a matching image/* or video/* Content-Type. Local file paths and web page URLs are not media inputs.
• Base64: provide only the base64-encoded bytes in data, without a data: prefix. Set media_type to match the file and group, for example image/png under images or video/mp4 under videos.

​

Media limits

Limit	Images	Video
Per request	4 images	1 video
Each URL download	32 MiB	50 MiB
Each inline base64 file, decoded	5 MiB	16 MiB

The entire JSON request body must fit within 16 MiB, including state, questions, and encoded media. Base64 adds roughly one third to the file size, so use an uploaded asset or a URL for larger files. Media also consumes the model’s context budget and contributes to usage.input_tokens.

​

Question types

​

Noul: evaluate a yes/no question

Use noul to test a condition. The answer’s noul value is the model’s estimated probability of true: near 1 favors yes, near 0 favors no, and near 0.5 is uncertain. It does not include a separate confidence field.

You can optionally define what each outcome means:

Noul question

{
  "type": "noul",
  "instructions": "Does the request meet the refund policy?",
  "criteria": {
    "true": "All requirements for a refund are met by the supplied evidence.",
    "false": "At least one requirement is not met or lacks supporting evidence."
  }
}

criteria accepts only the keys true and false. Either can be omitted or null to keep its default meaning. A description can be text, a JSON object, or an array. Choose your decision threshold using examples from your application; 0.5 is not a guarantee that an answer is reliable.

​

Choice: pick a category

criteria is an object containing 1–255 options. Its keys are nonempty option names, and its values describe when to select each option. A description can be text, a JSON object, an array, or null. With null or an empty string, only the option name is shown.

The answer includes:

Field	Meaning
choice	The option with the highest probability
probabilities	A value from 0 to 1 for every option, keyed by your option names; the values sum to 1
confidence	How far the highest probability is above a uniform guess, rescaled to 0–1

Make categories distinguishable. If none of your categories may fit, add an explicit option such as other and describe when it applies. The model chooses among the options you supply; it does not invent another category.

A choice with one option returns that option with probability 1 and confidence 1, without running the model for that question. This is a fixed answer, not a prediction.

​

Score: rate against ordered levels

criteria is an array of 1–10 level descriptions, ordered from low to high. Each level can be a nonempty string, a JSON object, or an array. Describe an observable condition at each level rather than using labels such as “bad,” “okay,” and “good” without context.

Levels are numbered from zero. Three levels give a score from 0 to 2; five levels give a score from 0 to 4. The answer includes:

Field	Meaning
score	The probability-weighted average of the level indices, which may be fractional
probabilities	A probability for each level, keyed by its index as a string: "0", "1", and so on
legend	The mapping from each string index back to its level description; structured descriptions are returned as serialized text
confidence	Concentration around the most likely level, accounting for the distance between levels

In the response above, 0 × 0.05 + 1 × 0.15 + 2 × 0.8 = 1.75. This places the result near the highest-impact level. It does not mean that 87.5% of users are affected. Check the distribution as well as the average: different distributions can produce the same score.

A score with one level returns score: 0, probabilities: {"0": 1}, and confidence: 1, without running the model for that question. Its legend contains that one level.

​

Read probabilities and confidence

Confidence is derived from the returned distribution. It is not a calibrated probability that the answer is correct, and it is not interchangeable between question types.

For a choice with at least two options, let N be the option count and p the highest probability. Confidence is (p - 1/N) / (1 - 1/N), limited to 0–1. Three equal probabilities give confidence 0; a highest probability of 0.8 gives confidence 0.7.

Score confidence also considers how far probability lies from the most likely level. Probability on a neighboring level counts less against confidence than probability far away. Use both the score and its distribution when deciding whether to accept or review a result.

Test thresholds on labeled examples from your own use case. A high value can still be wrong, especially when the state lacks necessary evidence or the criteria overlap. An answer may include a warning; treat it as diagnostic information and review the result before relying on it.

​

Response fields

All success fields are inside data.

Field	Description
id	Identifier for this decision result
request_id	Request identifier to retain when troubleshooting or contacting support
model	The public model name that answered the request
answers	One answer for every question, under the same name. Each answer’s type matches the question.
answers.{name}.warning	Optional diagnostic text when a model read produces a warning
usage.input_tokens	Total input tokens reported for the model reads used to answer this request
usage.output_tokens	Always 0

​

Context limits

Limit	Quality	Turbo (preview)
Model	heygen-decision-1	heygen-decision-1-turbo
Context budget per model read	131,072 tokens (128k)	262,144 tokens (256k)
Questions per request	1–400	1–400
Choice options	1–255	1–255
Score levels	1–10	1–10
JSON request body	16 MiB	16 MiB

The context budget includes the formatted state, images, sampled video frames, question, criteria, any repeated context or other prompt content, and one reserved answer token. It is not a limit on the raw state alone. A lower serving-engine limit still applies. Requests that exceed the applicable budget return HTTP 400 with context_length_exceeded; reduce the text or media input rather than retrying the same payload.

The context limit applies to each model read. usage.input_tokens is the total across reads and can exceed that limit. Shared state can be counted more than once when several questions or repeated reads use it, so use the returned count rather than the length of state.

Questions with a single choice option or a single score level are resolved without a model read. A request containing only those fixed questions reports zero input and output tokens, and its media is not fetched or evaluated. API access checks still apply.

​

Errors and retries

Errors use the standard HeyGen error format, separate from the success data object.

HTTP status	Code or cause	What to do
400	invalid_parameter	Check the field named in error.param, the question type, and the supported shapes and counts above.
400	context_length_exceeded	The message gives the prompt token count and applicable budget. Reduce the input.
401 / 403	Authentication, scope, or API access	Use a supported API key with the required scope and access.
402	insufficient_credit	Add credits to the paying balance.
429	Request rate limit	Back off; honor Retry-After when present.
502	voice_provider_error	The model could not complete the request. Back off before retrying. This shared error code is also used by decisions.
503	service_unavailable	Model capacity is temporarily unavailable. Retry with backoff.
504	gateway_timeout	The model did not respond in time. Retry with backoff if the result is still needed.

The default request limit is 50 requests per second, shared by API keys owned by the same user in the same workspace. Keep concurrency bounded and increase it gradually. There is no supported idempotency key for this endpoint: retrying a request may run it again.

​

Build it with an agent

Paste this into your coding agent to build a small integration:

Prompt for your agent

Help me classify content with HeyGen Decision 1. Read https://developers.heygen.com/docs/models/heygen-decision.md and https://developers.heygen.com/docs/for-ai-agents.md first.

Use HEYGEN_API_KEY from my environment without printing it. If it is missing, point me to https://platform.heygen.com/api-keys; do not ask me to paste it into chat. Ask what text, images, or video I want to evaluate and what decisions I need. Put text in state and media in assets.images or assets.videos, following the documented source formats and limits. For local files, use the asset upload API and its returned asset_id; never invent an ID or URL. Write explicit instructions and clear criteria for each question, then make one POST /v3/models/decisions request with model heygen-decision-1. Show me the answers, probabilities, and input token usage. Do not assume the endpoint supports the video APIs’ polling, callbacks, or idempotency keys.