Why Our Thumbnails Were Secretly Lossless (and How We Fixed It)

We had a constant defined in image_utils.rs that looked perfectly reasonable:

pub const THUMBNAIL_WEBP_QUALITY: u8 = 82;
pub const THUMBNAIL_MAX_WIDTH: u32 = 640;

Quality 82 is a normal choice for thumbnails at 640px wide. Our thumbnails were not quality 82. They were lossless.

The discovery

Thumbnail storage was larger than our estimates. A library of a few thousand clips was producing gigabytes of thumbnail data where we expected hundreds of megabytes. The thumbnails looked fine. They were too big.

The image crate (version 0.24) has a lossless-only default WebP encoder path. It accepts a quality parameter and ignores it, with no warning.

The fix: use the deprecated API

The image crate does have a lossy WebP encoder. It is not the default path, and it is marked deprecated with removal planned for a future major version.

// Use the lossy VP8 encoder with the caller's quality setting.
// Note: lossy encoding is deprecated in the image crate (planned
// removal in a future major version). When that happens, migrate to
// ravif (AVIF) or another lossy codec.
#[expect(deprecated, reason = "image 0.24 deprecates lossy WebP; will migrate when removed")]
let encoder = image::codecs::webp::WebPEncoder::new_with_quality(
    &mut writer,
    image::codecs::webp::WebPQuality::lossy(quality),
);
encoder.encode(rgb.as_raw(), w, h, image::ColorType::Rgb8)?;

#[expect(deprecated)] suppresses the warning and records why. When the crate removes the path, the attribute itself warns.

The planned replacement is AVIF via the ravif crate. AVIF achieves 20 to 50 percent smaller file sizes than WebP at equivalent perceptual quality. We will switch when the lossy WebP path is removed.

Rebuilding the pipeline

Fixing the encoder meant opening the thumbnail code. We rebuilt the rest of it in the same week: resizing, placeholders, deduplication, tone mapping and memory allocation.

SIMD-accelerated resizing

The image crate resize is slow. We replaced it with fast_image_resize, which uses SIMD.

pub fn fast_resize(
    img: &image::DynamicImage,
    dst_width: u32,
    dst_height: u32,
) -> Result<image::DynamicImage, Box<dyn std::error::Error + Send + Sync>> {
    use fast_image_resize as fir;

    let src_image = fir::images::Image::from_vec_u8(
        src_w, src_h, rgb.into_raw(), fir::PixelType::U8x3,
    )?;

    let mut resizer = fir::Resizer::new();
    resizer.resize(
        &src_image,
        &mut dst_image,
        &fir::ResizeOptions::new()
            .resize_alg(fir::ResizeAlg::Convolution(fir::FilterType::Lanczos3)),
    )?;
}

The speedup is about 14x over the image crate resize on the same hardware. The library dispatches at runtime to AVX2 on x86-64 machines, NEON on ARM, or a scalar fallback when neither is available. We use Lanczos3 filtering. It is slower than bilinear and sharper.

Face detection input (640x640) and face embedding input (112x112) go through the same function.

ThumbHash placeholders

When you open a large library in FrameQuery, there is a moment before the real thumbnails load from disk. Without placeholders the grid is blank rectangles that fill in one by one. We use ThumbHash placeholders.

pub fn compute_thumb_hash(
    img: &image::DynamicImage,
) -> Result<Vec<u8>, Box<dyn std::error::Error + Send + Sync>> {
    let small = if w > tw { fast_resize(img, tw, th)? } else { img.clone() };
    let rgba = small.to_rgba8();
    let (sw, sh) = (rgba.width() as usize, rgba.height() as usize);
    Ok(thumbhash::rgba_to_thumb_hash(sw, sh, rgba.as_raw()))
}

A ThumbHash is approximately 28 bytes. It encodes the aspect ratio, average color, and a blurred approximation of the image structure. We downscale to 100 pixels wide before hashing and store the result as a BLOB in SQLite alongside each clip's metadata.

ThumbHash decodes in the frontend in under a millisecond. The grid fills with blurred colour blocks and the real thumbnails replace them as they load.

Perceptual hashing for deduplication

Video editors accumulate duplicates. Different exports of the same clip, copies across drives, re-encoded versions with slightly different compression. We detect near-duplicates without comparing pixels across the library.

pub fn compute_perceptual_hash(img: &image::DynamicImage) -> Vec<u8> {
    let hash: blockhash::Blockhash256 = blockhash::blockhash256(&Img(img));
    let bytes: [u8; 32] = hash.into();
    bytes.to_vec()
}

Blockhash produces a 256-bit hash that is deterministic, integer-only (no floating point), and robust to compression artifacts. Two visually identical images that differ only in encoding will produce hashes with a small Hamming distance. We store the hash as a 32-byte BLOB in SQLite and flag pairs with a Hamming distance under about 25 of 256 bits as near-duplicates.

We chose blockhash over pHash and dHash because it is integer-only and deterministic. It does not survive crops or aspect ratio changes. It does survive re-encoding at different codecs and bitrates, which is the case we care about.

HDR tone mapping for RAW footage

Professional cinema cameras shoot in wide color gamuts (Rec.2020, DCI-P3) and high dynamic range. Thumbnails need to be sRGB for display. Clamping destroys highlights and shifts colours.

const HABLE_A: f32 = 0.15;  // Shoulder strength
const HABLE_B: f32 = 0.50;  // Linear strength
const HABLE_C: f32 = 0.10;  // Linear angle
const HABLE_D: f32 = 0.20;  // Toe strength
const HABLE_E: f32 = 0.02;  // Toe numerator
const HABLE_F: f32 = 0.30;  // Toe denominator
const WHITE_POINT: f32 = 11.2;

pub fn tonemap_rgb16_to_srgb8(src: &[u16]) -> Vec<u8> {
    // 1. Normalize u16 to [0, WHITE_POINT] linear
    // 2. Hable filmic tone map to [0, 1]
    // 3. OKLAB gamut map to sRGB [0, 1]
    // 4. Apply sRGB transfer function to [0, 255]
}

We use the Hable filmic curve, originally developed for Uncharted 2. It compresses highlights instead of clipping them, so bright skies and specular reflections keep detail. The constants are the standard Hable parameters.

After tone mapping, colors may still fall outside the sRGB gamut. We handle this with OKLAB gamut mapping: a binary search (8 iterations, less than 0.4 percent chroma error) that reduces chroma until the color fits within sRGB while preserving lightness and hue. An out-of-gamut red becomes a less saturated red. The palette crate handles the OKLAB and OKLCH color space conversions.

FFmpeg seeking optimisation

FrameQuery uses FFmpeg to extract frames from video files before indexing them through the Rust pipeline. The order of arguments matters.

# Fast (input seek, keyframe-based):
ffmpeg -ss <time> -i input.mp4 -vframes 1 thumb.jpg

# Slow (output seek, frame-by-frame decode):
ffmpeg -i input.mp4 -ss <time> -vframes 1 thumb.jpg

Placing -ss before -i tells FFmpeg to seek to the nearest keyframe before opening the input, then decode only a few frames to reach the exact timestamp. Placing it after -i means FFmpeg decodes every frame from the start of the file until it reaches the target. For a timestamp 30 minutes into a file, that is the difference between milliseconds and minutes.

We have FFmpeg output PNG rather than WebP because FFmpeg builds do not always include a WebP encoder. FFmpeg writes a temporary PNG and Rust encodes it to lossy WebP. That works on every FFmpeg build.

Arena allocation for batch processing

Thumbnail generation processes clips in batches. Each clip involves multiple allocations: decoded frames, resized buffers, intermediate color conversions. With the standard allocator each is allocated and freed on its own.

pub fn rgb16_to_rgb8_arena<'a>(arena: &'a bumpalo::Bump, src: &[u16]) -> &'a [u8] {
    let dst = arena.alloc_slice_fill_default(src.len());
    for (d, &s) in dst.iter_mut().zip(src.iter()) {
        *d = (s >> 8) as u8;
    }
    dst
}

We use bumpalo, an arena allocator, to batch all temporary allocations for a single clip into one contiguous region. When the clip is done, the entire arena resets in one operation. There is no per-allocation free and no fragmentation. It is faster than the default allocator for this workload.

Result

Thumbnails are smaller, load faster, handle HDR footage and flag duplicates. We are still on the deprecated WebP API. When the image crate removes it, we will migrate to AVIF.

Download FrameQuery to try FrameQuery.