Getting Started With a Video Search API: A Developer Guide

You are building an application that needs to search inside video content. Maybe it is a media management platform, an e-learning tool, a compliance review system, or an internal knowledge base. Your users need to find specific moments in videos, not just specific videos.

A video search API gives you endpoints to upload footage, process it through AI analysis, and search the resulting index programmatically. This guide walks through how FrameQuery's API works and what integrating it looks like in practice.

What a video search API provides

At its core, a video search API turns video files into searchable data and lets you query that data. The typical workflow has three stages: upload a video file (or a URL to one), process it through AI analysis, and search the resulting index for timestamped results.

What makes a video search API different from a generic transcription or object detection API is that it combines multiple analysis modalities into a single searchable index. You do not need to stitch together separate services. One API handles all of it and returns unified search results.

Four search modalities

FrameQuery's API processes each video through four analysis passes, and the search endpoint queries across all of them:

Transcript search. Full-text search across everything spoken in the video. Returns word-level timestamps. Supports exact phrase matching and speaker filtering.

Object search. Search for objects detected in the video frames: vehicles, laptops, whiteboards, animals, signage. Each detection is timestamped to the frame where it appears.

Scene search. Natural-language search against AI-generated scene descriptions. Query with phrases like "two people shaking hands in a lobby" and get matching moments.

Face search. Search by person across your indexed library. Once faces are clustered and optionally named, you can find every appearance of a specific individual.

Each modality is independently queryable, or you can search across all of them simultaneously for combined results ranked by relevance.

Authentication

The API uses key-based authentication. Include your key in the Authorization header: Bearer fq_your_api_key_here. Rate limits depend on your plan tier. Processing requests are limited by your monthly processing quota. Search requests are more generous since they run against pre-built indexes.

The upload-process-search workflow

Here is the typical integration flow with example requests.

Upload. Send a file or URL to the upload endpoint. The API returns a video_id for subsequent requests.

curl -X POST https://api.framequery.com/v1/videos \
  -H "Authorization: Bearer fq_your_api_key_here" \
  -F "file=@interview-march-14.mp4"

Process. Trigger analysis on the uploaded video. Choose which modalities to run. Transcript-only and scene-only processing each cost 50% of the full rate.

curl -X POST https://api.framequery.com/v1/videos/vid_abc123/process \
  -H "Authorization: Bearer fq_your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{"modalities": ["transcript", "objects", "scenes", "faces"]}'

Processing takes roughly five minutes per hour of video. Poll the status endpoint to check progress, or register a webhook callback URL for production integrations.

Search. Once processing completes, query the indexed content. Omit video_id to search across your entire library.

curl "https://api.framequery.com/v1/search?q=budget+projections&video_id=vid_abc123" \
  -H "Authorization: Bearer fq_your_api_key_here"

Results include timestamps, the matching modality, the matched text, a relevance score, and speaker attribution where applicable. You can filter by modality, speaker, date range, or video ID.

{
  "results": [
    {
      "video_id": "vid_abc123",
      "timestamp_start": 1232.4,
      "timestamp_end": 1238.1,
      "modality": "transcript",
      "text": "The budget projections for Q3 need to be revised.",
      "speaker": "Speaker 1",
      "score": 0.94
    }
  ]
}

Searching across your entire library

Omit the video_id parameter to search every video you have processed. Results come from all indexed videos, ranked by relevance. Your users search once and get results spanning their entire collection.

When to use an API vs the desktop app

Use the desktop app when individuals or small teams need to search their own footage library with a visual interface. No code required.

Use the API when you are building video search into your own product, automating processing pipelines, or integrating with existing systems. Both use the same processing pipeline and search engine.

Response format and pagination

Search responses are paginated with a default page size of 20 results. Use offset and limit parameters for pagination. Results are sorted by relevance score by default, with options to sort by timestamp or date.

Join the waitlist to get API access when FrameQuery launches.