Workflows
How to Search Video by Scene Description Instead of Filename
Filenames tell you nothing about what is in a clip. Scene descriptions generated by AI let you search your footage the way you think about it: by what is happening, how it looks, and what mood it conveys.
You are looking for a wide shot of the office lobby. You know you shot it. You were there. But your file browser shows you A001_C003.MOV, A001_C004.MOV, A001_C005.MOV, and 400 more files just like them. Nothing about lobbies, wide shots, or offices.
This is the fundamental mismatch between how cameras name files and how editors think about footage. Cameras generate filenames based on card slots, clip counters, and reel numbers. Editors think in terms of what is happening on screen: the wide establishing shot, the close-up of hands, the interview against the bookshelf, the drone pullback over the building.
Every other step in the post-production pipeline has adapted to how people think. Color grading tools let you select by visual characteristics. Audio tools let you search by waveform properties. But finding footage still depends on filenames, folder structures, and human memory.
Why existing organization methods fall short
Most teams develop some system for organizing footage. Folder hierarchies that encode shoot date, location, or camera. Spreadsheets with clip notes. Naming conventions that append a keyword to the original filename. Bins in the NLE with labels like "Exteriors" or "Interview B-Cam."
These systems are better than nothing, but they share a common limitation: they capture the context around the footage rather than the content within it. A folder called "Office Shoot Day 2" tells you when and where something was filmed. It says nothing about what each clip contains, how it was framed, or what mood it conveys.
The effort required to bridge that gap manually is enormous. Detailed clip logging, where someone watches each clip and writes a description, takes roughly two to three times the footage duration. A ten-hour shoot takes 20 to 30 hours to log properly. Most teams cannot justify that time. So they rely on memory, thumbnails, and fast scrubbing, which works until the editor who remembers the footage moves on to another project.
What AI-generated scene descriptions contain
Scene description takes a fundamentally different approach. Instead of asking a human to describe each clip, an AI vision model watches the footage and generates descriptions automatically during processing.
These descriptions are not simple labels. Each detected scene gets a natural language description of what is happening, along with structured metadata. A single scene might produce:
- A description like "two people seated at a conference table in a modern office, speaking, with floor-to-ceiling windows behind them showing a city skyline"
- Shot type: medium
- Camera angle: eye level
- Dominant color: cool blue-grey
- Objects detected: table, chairs, laptop, windows, buildings
- People count: 2
All of this information is generated automatically and indexed for search. No human logging required.
Interview subject seated against dark background, warm key light from camera left
Conference stage with presenter at podium, blue and white lighting, audience visible
Establishing shot of harbor at golden hour, boats moored along dock, warm light across water
Searching the way you think
With scene descriptions indexed, you search your footage using natural language. Type what you are looking for the way you would describe it to a colleague, and the search engine matches against the generated descriptions.
Here are examples of searches that work:
"interview with bookshelves in background" finds every interview setup where bookshelves are visible, regardless of who the subject is or what they were discussing.
"aerial shot of city" surfaces drone footage and helicopter shots of urban environments, even if those clips live in a folder called "B-Roll Misc" with no further organization.
"close-up product on white table" finds product photography and tabletop setups across your entire library, pulling from corporate shoots, commercial productions, and social media content alike.
"crowd at outdoor event" returns festival footage, concert B-roll, sports events, and any other scene with groups of people in outdoor settings.
"person walking through corridor" finds hallway tracking shots, which might live in completely different project folders but share the same visual content.
These searches work because the underlying system uses both traditional text matching (BM25 scoring via Tantivy, a Rust-based search engine) and semantic similarity (MiniLM embeddings). The text matching finds descriptions containing your exact words. The semantic matching finds descriptions that mean the same thing even when worded differently. Searching "golden hour exterior" can surface a scene described as "outdoor shot at sunset with warm orange light" because the semantic model understands the relationship.
Where this helps most
Scene description search is most valuable for footage that has no dialogue, because that is the footage where no other search method works.
B-roll is the clearest example. Establishing shots, cutaways, atmospheric footage, product shots, lifestyle imagery. None of it contains speech, so transcript search returns nothing. Nobody mentions these clips by name, so dialogue search is useless. But scene description search finds them by what they show.
Stock footage libraries are another strong case. If you maintain a library of purchased or self-shot stock footage, it probably lives in a loosely organized folder structure. Scene descriptions make the entire library searchable by visual content without any manual tagging.
Archival footage presents the same challenge at a larger scale. Productions that have accumulated years of footage across many projects often have poor metadata for older material. Processing that footage through FrameQuery generates descriptions retroactively, making a decade of footage searchable in the time it takes to process it.
How FrameQuery generates descriptions
During cloud processing, FrameQuery first runs scene detection to identify distinct shots within each clip. Each detected scene then gets analyzed by a vision model that generates the natural language description and structured metadata.
The structured fields (shot type, camera angle, dominant color, object list) are extracted alongside the free-text description. This means you can combine natural language searches with structured filters. Search for "interview" and filter by shot type "close-up" to find only tight interview framing, excluding wide two-shots.
Processing runs at roughly five minutes per hour of footage. Your original files never leave your machine; only extracted frames are sent for analysis, and the generated metadata is stored locally in your search index.
Closing the gap between thinking and finding
The bottleneck in post-production has shifted. Cameras are faster, storage is cheaper, shooting ratios keep climbing. But the tools for finding footage within that growing volume have barely changed. Filename search, folder browsing, and manual memory are the same methods editors used twenty years ago.
Scene description search does not replace editorial judgement or creative intuition. It replaces the hours of scrubbing and guessing that precede them. When you can search your footage the same way you think about it, finding the right clip stops being the hard part.
Join the waitlist to search your footage by scene description when FrameQuery launches.