# FrameQuery API

> Send a video or audio file and get back timestamped scene captions and a speaker-labelled transcript. One asynchronous job, priced per footage-hour, charged only for stages that complete.

Source: https://www.framequery.com/docs (updated 2026-10-13)

The FrameQuery API runs the same server pipeline as the FrameQuery desktop app, behind a REST API. A job has up to two billable stages:

- **Vision**: scene detection, plus a one-sentence caption for every scene with object labels, a colour palette, shot type and shot angle.
- **Transcript**: speech-to-text with speaker labels (diarisation), as JSON or as an SRT, WebVTT or plain-text file.

It accepts anything FFmpeg can decode, audio-only files (transcript only), and camera RAW (RED R3D, Blackmagic BRAW, ARRIRAW) for the formats [`GET /formats`](https://www.framequery.com/docs/formats) reports as enabled. You create a job, upload the file or give us a URL, and read the results once the job's `state` is `completed`, either by polling or from a signed webhook.

> **Public beta.** The contract is versioned under `/v1` and changes are additive; see [Changelog and beta status](https://www.framequery.com/docs/changelog). The API does not do search, face recognition or NLE export. Those are desktop-app features.

## What you get back

One scene from `GET /jobs/{jobId}/scenes`:

```json
{
  "index": 3,
  "startTime": 41.2,
  "endTime": 48.9,
  "description": "A woman pours coffee at a kitchen table while a man reads a newspaper in morning light.",
  "objects": ["woman", "man", "coffee-pot", "mug", "newspaper", "table"],
  "colorPalette": ["#d8c3a5", "#8e6c4a", "#3b3a36", "#e9e4d8"],
  "shotType": "medium",
  "shotAngle": "eye-level"
}
```

One segment from `GET /jobs/{jobId}/transcript`:

```json
{ "startTime": 41.6, "endTime": 44.1, "text": "Did you see the forecast for tomorrow?", "speaker": "Speaker 2" }
```

Times are seconds (float) from the start of the media. Every JSON key is camelCase. Full shapes: [Results](https://www.framequery.com/docs/results).

## Price

$1.50 per footage-hour for vision plus transcript on standard formats: $1.15 for vision only, $0.35 for transcript only. Camera RAW costs $2.50 per footage-hour. Billing is per second with a 30-second minimum per job, and a stage that fails or is skipped is never charged. Above 500 footage-hours in a UTC calendar month the volume rates apply to the extra hours. You prepay with credit packs of $10, $50 or $100; enterprise accounts can pay as they go. See [Pricing and billing](https://www.framequery.com/docs/pricing).

## Start here

- [Agent quickstart](https://www.framequery.com/docs/agent-quickstart): a short brief for an LLM agent writing code against the API.
- [Quickstart](https://www.framequery.com/docs/quickstart): your first job with curl, Python or TypeScript.
- [Authentication and keys](https://www.framequery.com/docs/authentication): get an `fq_live_` key and keep it server-side.
- [Jobs](https://www.framequery.com/docs/jobs) and [Job lifecycle](https://www.framequery.com/docs/job-lifecycle): creating jobs, uploads, `state` and stages.
- [API reference](https://www.framequery.com/docs/api): every endpoint with a copy-pasteable curl command.

## Machine-readable

- OpenAPI 3.1, the contract every page here follows: [https://api.framequery.com/v1/api/openapi.json](https://api.framequery.com/v1/api/openapi.json). No key needed.
- Keyless discovery: [`GET /pricing`](https://api.framequery.com/v1/api/pricing), [`GET /pricing/estimate`](https://api.framequery.com/v1/api/pricing/estimate?durationSeconds=3600&mode=all) and [`GET /formats`](https://api.framequery.com/v1/api/formats).
- Every docs page is also plain Markdown: append `.md` to its URL, for example [/docs/webhooks.md](https://www.framequery.com/docs/webhooks.md). The index is [/docs.md](https://www.framequery.com/docs.md); [/llms.txt](https://www.framequery.com/llms.txt) lists every page and [/llms-full.txt](https://www.framequery.com/llms-full.txt) holds them all in one file.

## Base URL and conventions

```text
https://api.framequery.com/v1/api
```

- Authenticate with `Authorization: Bearer $FRAMEQUERY_API_KEY` (or `X-API-Key`). Keys start with `fq_live_`.
- Success responses are `{"data": …, "requestId": "…"}`. Errors are `{"error": "…", "code": "…", "docUrl": "…", "param": "…", "requestId": "…"}`; branch on `code`, never on the message. See [Errors](https://www.framequery.com/docs/errors).
- Every response carries `X-Request-Id`. Quote it (or `requestId`) when you contact support.
- Job creation accepts an `Idempotency-Key` so retries never create a second job. See [Jobs](https://www.framequery.com/docs/jobs#idempotency).

## All pages

Every page is also plain Markdown at the URLs below. All of them in one file: https://www.framequery.com/llms-full.txt. The machine-readable contract is https://api.framequery.com/v1/api/openapi.json (no key needed).

### Get started

- [FrameQuery API](https://www.framequery.com/docs.md): Send a video or audio file and get back timestamped scene captions and a speaker-labelled transcript. One asynchronous job, priced per footage-hour, charged only for stages that complete.
- [Agent quickstart](https://www.framequery.com/docs/agent-quickstart.md): The brief for an LLM agent writing code against the FrameQuery API. Auth, the three calls, exact JSON shapes and which errors to retry.
- [Quickstart](https://www.framequery.com/docs/quickstart.md): Process your first video in a few minutes. Get a key, create a job from a URL or a local file, wait for it, and read the transcript and scenes.
- [Authentication and keys](https://www.framequery.com/docs/authentication.md): API keys are Polar license keys that start with fq_live_. How to get one, send it, keep it server-side and rotate it.

### Build

- [Jobs](https://www.framequery.com/docs/jobs.md): Create jobs from a URL, a direct upload or a batch of clips. Processing modes, idempotency, metadata, multi-track audio, listing and cancelling.
- [Job lifecycle and state](https://www.framequery.com/docs/job-lifecycle.md): How a job moves from creation to a terminal state, what each stage reports, how to poll, and every field of the Job object.
- [Results](https://www.framequery.com/docs/results.md): Read transcripts as JSON, SRT, WebVTT or plain text, and scenes with captions, objects, palettes and shot types. What "not ready" and "unavailable" mean.
- [Webhooks](https://www.framequery.com/docs/webhooks.md): Get a signed HTTPS callback when a job completes, fails or is cancelled. Payload, Standard Webhooks signature verification, retries, secret rotation, testing and the delivery log.

### Reference

- [Formats and limits](https://www.framequery.com/docs/formats.md): Anything FFmpeg can decode, audio-only files, and camera RAW where enabled. Size and duration limits per tier, and the live GET /formats endpoint.
- [Pricing and billing](https://www.framequery.com/docs/pricing.md): Per-stage prices per footage-hour, charged per second and only for stages that complete. Prepaid credits through Polar, the estimate endpoint, worked examples and an honest comparison.
- [Errors](https://www.framequery.com/docs/errors.md): Every API error code with its HTTP status, what it means, what to do and whether to retry. Plus the job and stage failure codes.
- [Rate limits](https://www.framequery.com/docs/rate-limits.md): Per-account request limits per minute, the active-job cap, the rate-limit headers and how to back off.
- [API reference](https://www.framequery.com/docs/api.md): Every endpoint of the FrameQuery API v1 with its parameters, a copy-pasteable curl command and the shape it returns. The OpenAPI 3.1 document is the machine-readable version.

### Tools

- [SDKs](https://www.framequery.com/docs/sdks.md): Official Python and TypeScript SDKs (0.2) with uploads, polling, retries, typed errors and webhook verification. Go, Rust and Ruby SDKs at 0.1.
- [MCP server](https://www.framequery.com/docs/mcp.md): A Model Context Protocol server so Claude, Cursor and other MCP clients can process video with FrameQuery. Coming with SDK 0.2.
- [Changelog and beta status](https://www.framequery.com/docs/changelog.md): What the public beta includes, what it does not, known limits, deprecations, and dated changes to the API contract.