Workflows
Dual-System Audio Sync: Auto-Aligning Boom and Camera Tracks
How FrameQuery automatically syncs boom and camera tracks using BWF timecode and acoustic fingerprinting, even when AGC mangles the camera signal.
Almost every professional shoot records sound twice. The boom or lavalier feeds a field recorder for the audio you use in the edit. The camera records its own scratch track from the on-board mic so the editor has something to sync against. By the time the cards land on a drive, you have hundreds of clips on each side and the job of matching them up.
This is dual-system sound.
What is dual-system audio sync?
Dual-system audio sync is the process of aligning a separately recorded audio file (typically from a field recorder, boom, or lavalier) with the corresponding video clip from a camera, so the clean audio replaces the camera's on-board scratch track on the timeline. It can be done by hand, by timecode, by waveform analysis, or by a combination of all three. Automatic sync tools like PluralEyes, Resolve's audio sync, and FrameQuery's audio sync remove the manual step.
The two ways to sync, and where each one breaks
There are two reliable signals for aligning a camera clip with a separate audio file: timecode and the audio waveform. Neither is sufficient on its own.
Timecode works when both devices were jammed from the same source and the recorder embedded BWF (Broadcast Wave Format) timecode in its files. When the chain holds, sync is exact. When it does not (different timecode islands, drift over a long day, a recorder set to free-run that nobody jammed at lunch, a camera that does not write timecode at all), you get clips that claim to share a clock but are seconds apart on the timeline.
Waveform sync sidesteps the clock. It compares what the two devices heard and aligns them by acoustic content. This is what tools like PluralEyes popularised and what Resolve has built into its inspector. It works when both mics captured similar audio, and fails most often when the camera mic was running automatic gain control (AGC).
Most sync time goes on clips where timecode is wrong and the waveform looks too different to match.
How FrameQuery handles it
The pipeline runs in two phases. Both run in the background while your library indexes.
Phase one: timecode, but only if it is trustworthy
When a file lands in your library, FrameQuery reads any embedded BWF timecode and the camera clip's start timecode. If the two windows overlap within a one-second tolerance, the pair is treated as a high-confidence timecode match and linked automatically.
A tighter window rejects valid pairs when long takes drift. A looser one links clips that share a clock but came from different setups.
If timecode is missing, ambiguous, or contradicted by other files in the same window, the match is not made. The pair drops to phase two.
Phase two: acoustic fingerprinting, then waveform
Phase two runs on every file, whether or not phase one succeeded.
The first step is an audio fingerprint. FrameQuery extracts a compact acoustic signature from every clip and indexes it in a local database. The algorithm is a Rust implementation of spectral landmark fingerprinting, the technique behind the original Shazam paper. Chromaprint and similar libraries are designed for matching a song against a catalogue of millions of songs. Here the task is matching a field recorder's clean signal against the same audio captured by a camera mic two metres away, with automatic gain riding the level.
Camera AGC breaks waveform sync. The camera is constantly adjusting its input level to keep things audible, so the loud peaks on the field recorder do not look loud on the camera track, and a plain correlation fails. FrameQuery's fingerprinter applies PCEN (per-channel energy normalisation), spectral whitening, and pre-emphasis before extracting peaks. The fingerprints come from the parts of the signal that survive AGC and cheap-mic frequency response.
Once fingerprints exist on both sides, FrameQuery looks up each query file's hashes in an inverted index across your entire library, including RAW camera formats, and builds a histogram of time offsets. Hashes that show up everywhere (room tone, hum, generic transients) are weighted down using inverse document frequency, as search engines do with common words. This gives a coarse offset between two files, accurate to about 200 milliseconds.
That candidate is then refined with two passes of envelope cross-correlation. The first pass uses a spectral onset envelope, which fires sharply on transients like consonants, claps, or a door closing. This pins down the lag to within a few samples. The second pass uses a multi-scale amplitude envelope to score the alignment over the overlapping region, which shows whether the match is real or coincidental. The peak position is interpolated parabolically between samples, so the final offset is sub-sample accurate. At 48 kHz that is well under a frame.
Confidence tiers
Most of the work in a sync tool is deciding which matches to trust.
FrameQuery sorts every candidate into one of three tiers based on the cross-correlation score and the number of supporting hash matches:
- High confidence (correlation at least 0.85 with twenty or more matching hashes): linked automatically. The audio is paired with the video when you next open the clip.
- Medium confidence (correlation at least 0.6 with ten or more matching hashes): staged for review in the audio match dialog. You see the pair, the offset, the score, and a waveform preview. You confirm or reject with a click.
- Low confidence: discarded. Nothing surfaces.
The thresholds were set by running the system across production audio with known ground truth and finding the point where false positives went to zero. Ambiguous pairs go to the medium tier for a human decision.
Where this still falls back to you
The system has limits.
- The two files have to share audio. If the boom was off or the camera was rolling on something the field recorder never heard, there is nothing to align against.
- Heavy continuous noise lowers confidence. Wind, traffic, ocean, generators. The fingerprints still work, but more pairs will land in the review tier rather than auto-linking.
- Very short clips are harder. Below about three seconds of overlapping audio, the cross-correlation score has too few independent peaks to discriminate confidently.
- Sync claps still help on a chaotic set. They give the algorithm an unambiguous transient that survives any mic or AGC. If they are present, the system uses them.
What it looks like in practice
Drop your camera cards and your audio cards into the same FrameQuery library. Indexing runs in the background while you start logging. Within a few minutes, the obvious pairs (timecode-jammed, clean audio, strong transients) are already linked. The medium-confidence stack lands in the AudioMatchReviewDialog with the offsets, scores, and a quick preview, and you confirm or reject each one.
By the time you are ready to cut, the only pairs left in the queue are the ones the algorithm was not sure about. From there, your synced selects can move straight onto a timeline using FrameQuery's FCPXML export, or be filtered down further with speaker-aware search once the clean audio is attached.
Frequently asked questions
Can I sync separate audio without timecode?
Yes. FrameQuery's second matching phase uses acoustic fingerprinting and waveform cross-correlation, which does not require timecode. As long as the camera scratch track and the field recorder share some overlapping audio, the system can recover an accurate offset, typically to sub-frame precision at 48 kHz.
How does FrameQuery handle camera AGC?
Most off-the-shelf waveform sync tools fail when the camera was running automatic gain, because peaks no longer line up. FrameQuery applies per-channel energy normalisation (PCEN), spectral whitening, and pre-emphasis before extracting fingerprint peaks, so the features being matched survive AGC and cheap-mic frequency response.
What happens to pairs the algorithm is not sure about?
Low-confidence candidates are discarded. Medium-confidence candidates (correlation around 0.6 to 0.85) are surfaced in a review dialog with the offset, score, and a waveform preview, so you can confirm or reject them in a single click. High-confidence pairs are linked automatically.
Download FrameQuery to try dual-system audio sync against your own production footage.