Dual-System Audio Sync: Auto-Aligning Boom and Camera Tracks

Almost every professional shoot records sound twice. The boom or lavalier feeds a field recorder for the audio you use in the edit. The camera records its own scratch track from the on-board mic so the editor has something to sync against. By the time the cards land on a drive, you have hundreds of clips on each side and the job of matching them up.

This is dual-system sound.

What is dual-system audio sync?

Dual-system audio sync is the process of aligning a separately recorded audio file (typically from a field recorder, boom, or lavalier) with the corresponding video clip from a camera, so the clean audio replaces the camera's on-board scratch track on the timeline. It can be done by hand, by timecode, by waveform analysis, or by a combination of all three. Automatic sync tools like PluralEyes, Syncaila, Resolve's audio sync, and FrameQuery's audio sync remove the manual step.

The two ways to sync, and where each one breaks

There are two reliable signals for aligning a camera clip with a separate audio file: timecode and the audio waveform. Neither is sufficient on its own.

Timecode works when both devices were jammed from the same source and the recorder embedded BWF (Broadcast Wave Format) timecode in its files. When the chain holds, sync is exact. When it does not (different timecode islands, drift over a long day, a recorder set to free-run that nobody jammed at lunch, a camera that does not write timecode at all), you get clips that claim to share a clock but are seconds apart on the timeline.

Waveform sync sidesteps the clock. It compares what the two devices heard and aligns them by acoustic content. This is what tools like PluralEyes popularised and what Resolve has built into its inspector. It works when both mics captured similar audio, and fails most often when the camera mic was far away, noisy, or running automatic gain control (AGC).

Most sync time goes on clips where timecode is wrong and the waveform looks too different to match.

How FrameQuery handles it

Add your recorder folders to the same library as your camera cards. Matching runs in the background, and newly imported files are checked first. There are two routes to a match, and the acoustic one runs whether or not timecode found anything.

Timecode: a suggestion, not a decision

FrameQuery reads the recorder's BWF time reference (an exact sample count) and its iXML speed and timestamp fields, and compares them with the camera clip's start timecode. A recording is proposed when its time range overlaps the clip and the two recording dates are no more than a day apart.

Timecode matches are listed for review rather than applied. Matching numbers do not prove that the two devices were jammed from the same source, and a recorder on free-run or record-run timecode produces plausible numbers that are wrong.

Acoustic fingerprinting

Every recording is fingerprinted in full, one channel at a time, at 11,025 Hz. Channels carrying timecode (LTC) are skipped. The fingerprint is a set of spectral landmarks, the technique from the original Shazam paper: peaks in time and frequency, picked relative to the level of their own frequency band, paired up and stored as hashes in a local index.

Each camera clip is checked against that index starting with its first 30 seconds. If one recording stands out and the clip is at least 40 seconds long, a second window later in the clip is checked as well. If the opening finds nothing, six short scouts across the clip rank where the sound is most distinctive, and up to two more windows are tried there.

Lookups build a histogram of time offsets between the clip and each recording. Hashes that turn up everywhere (room tone, hum, generic transients) are weighted down using inverse document frequency, as search engines do with common words, and the most common ones are skipped. A candidate needs at least five agreeing hashes from three anchor points spread over at least 250 ms. This gives a coarse offset, accurate to about 200 milliseconds.

Timing: offset and clock rate

Recorders and cameras run on separate clocks, so the offset at the start of a clip is not always the offset at the end. FrameQuery fits two numbers for each pair: the offset, and the clock rate of the recording against the camera.

It matches short spectral patches between the two files, fits offset and rate to one set of patches while rejecting outliers, then checks the fit against a separate set of patches it did not use. A whitened waveform cross-correlation refines the result with sub-sample interpolation. Excerpts under eight seconds use a simpler onset-envelope alignment at a fixed rate.

The rate is applied wherever the recording is used: playback, transcription, proxies and NLE exports.

When a match is applied automatically

A match is applied without asking only when all of these hold:

• The clip is at least 10 seconds long.
• Exactly one stretch of the recording fits, and the recording covers the whole clip.
• Timing anchors sit within three seconds of both ends of the clip and span at least 80% of it.
• The fit predicts held-out anchors to within 2.5 ms.
• The recorder's clock rate is within 0.1% of the camera's.
• Clips over a minute were confirmed in two separate windows.
• Every recording in the library has finished indexing, and the recording file has not changed since it was indexed.
• You have not chosen audio for the clip yourself.

When several recordings pass (a boom and a lav on the same take), they are mixed, each at -6 dB with a limiter, and you can change the gains. Every automatic choice has an Undo on the clip's page, and an undone match is not applied again.

Everything else is listed for review with its offset and score. You can listen to the camera and the recording side by side, mark the same clap or word on both waveforms, and check the aligned start and end before accepting. Rejections are remembered.

Where this still falls back to you

• The two files have to share audio. If the boom was off or the camera was rolling on something the field recorder never heard, there is nothing to align against.
• Short clips. Under 10 seconds nothing is applied automatically, and a few seconds of shared sound is the minimum for a precise fit.
• Repeated sound. The same line in several takes, or music played back on set, can fit more than one stretch of a recording. Those matches are marked ambiguous and listed with each alternative.
• Heavy continuous noise. Wind, traffic, ocean and generators leave fewer distinctive peaks, so more clips end up in review.
• Distance. Sync follows the camera's microphone. A camera ten metres from the speaker hears them about 29 ms late, so the recording is placed that much late too, which is under a frame at 24 fps.
• Sync claps still help. FrameQuery does not look for claps specifically, but a clap gives both microphones a short, loud event at the same instant, which helps on short or noisy clips.

What it looks like in practice

Drop your camera cards and your audio cards into the same FrameQuery library. Settings shows how many hours of recordings are indexed and how many remain. As indexing completes, clear matches are applied and the rest collect in the review list.

On each clip's page, the Audio Sync panel shows the selected recording, its offset and channel, and any other angles: clips from other cameras that matched the same recording at the same time. Opening one starts it at the same moment.

From there, your synced clips can move onto a timeline using FrameQuery's FCPXML export. FCPXML and Premiere XML exports include the selected recording as a WAV with offset and clock rate applied. Transcripts and speaker-aware search use the selected recording once the clip is processed.

Frequently asked questions

Can I sync separate audio without timecode?

Yes. Acoustic fingerprinting and waveform alignment do not need timecode. As long as the camera scratch track and the field recorder share some overlapping sound, FrameQuery can recover the offset and the clock rate between them.

How does FrameQuery handle camera AGC?

Fingerprint peaks are picked relative to the level of their own frequency band, and the final alignment uses whitened signals, so slow level changes from AGC affect them less than a raw waveform comparison. Clips where AGC pumps hard against noise are more likely to land in review than to be applied automatically.

Does it correct drift between the recorder and the camera?

Yes. The clock rate is fitted for every match and applied in playback, processing, proxies and exports. Automatic matching only accepts rates within 0.1% of the camera's. Larger differences are listed for review with the measured rate.

What happens to pairs the algorithm is not sure about?

They are listed for review with the offset, the score and any competing alternatives. You can listen to both, adjust the alignment by marking the same sound on each waveform, then accept or reject.

---

Download FrameQuery to try dual-system audio sync against your own production footage.