GPU video transcoding

Use mdr.video.transcode_videos to transcode whole video files, or mdr.video.encode_image_sequences to turn ordered JPEG or PNG sequences into videos. Both are async map blocks that always encode with NVIDIA NVENC.

import refiner as mdr

pipeline = mdr.read_videos("/data/source/*.mp4", file_path_column="video").map_async(
    mdr.video.transcode_videos(
        video_key="video",
        output_folder="/data/encoded",
        output_key="training_video",
        config=mdr.video.NVENCConfig(),
    ),
    max_in_flight=4,
    preserve_order=False,
    dtypes={"training_video": mdr.datatype.video_path()},
)

Inputs and outputs must be local files. Input values can be local paths, mdr.io.DataFile objects, or whole mdr.video.VideoFile objects. Each output row keeps its original fields and adds the new local MP4 path. Remote URLs are rejected before accessing their storage backend, including S3, GCS, and HTTP. The copying job owns downloading, uploading, and checkpoints: download to a local path, call the encoder, then upload its completed output. output_folder is local storage; retain or transfer results as required. Clipped views are rejected; materialize the desired clip before using this block.

Request one L4 per worker on the launch call and begin with three or four in-flight rows. Each row uses one FFmpeg process. max_in_flight bounds concurrent local encodes; it is per worker, not a global limit. Start with 8 CPUs and 8 GiB RAM per worker and provision scratch disk for the local inputs and concurrent outputs. GPU allocation belongs on the launcher, not the block.

import refiner as mdr

pipeline = mdr.from_items([
    {"episode_id": "episode-1", "images": ["/data/0001.jpg", "/data/0002.jpg"]},
]).map_async(
    mdr.video.encode_image_sequences(
        images_key="images",
        output_folder="/data/encoded",
        fps=30,
    ),
    max_in_flight=4,
    dtypes={"encoded_video": mdr.datatype.video_path()},
)

Supply a nonempty list of uniformly sized local files, all JPEG or all PNG. List order is frame order; the block does not sort filenames. Image decode and conversion to limited-range YUV happen on the CPU; encoding happens on the GPU. No Python RGB frame arrays are created. The output frame count must match the list length.

Video frame timing passes through without an FPS conversion. Source timestamps are normalized by FFmpeg's normal input handling; do not use output timestamps as an absolute source clock. Image sequences receive a regular timeline at fps. Automatic rotation is disabled so pixels remain in source annotation coordinates. Resize associated keypoints, boxes, and intrinsics to match output dimensions; these blocks do not modify annotation columns. HDR inputs require tone mapping before this SDR block.

Resizing uses Lanczos on both the CUDA and CPU paths.

On the CUDA path, decode surfaces remain on the GPU through scale_cuda and NVENC. Full-range inputs use CPU range conversion in auto mode. decode="cpu" still uses NVENC for encoding. decode="cuda" forces hardware decoding and reports unsupported input/build errors; it does not silently retry on the CPU.

The defaults use p4/HQ, VBR CQ29, and Lanczos scaling. The earlier dataset quality calibration used p6 and does not validate this exact configuration. This block disables lookahead, multipass, and B-frames and fixes the GOP at 17. CQ 29 is an encoder quality target, not an equivalence to x264 CRF 29.

Measure on your own videos before scaling worker count:

The benchmark compares p1 and p6 with one through four concurrent lanes. It reports aggregate frames per second including probing and local publication, after a warmup. It excludes remote transfer and does not measure perceptual quality. Short clips, image decoding, storage, and scene complexity can change which configuration wins.

An L4 validation run on September 8, 2026 used 8 CPUs, 8 GiB RAM, NVIDIA driver 580.95.05, and FFmpeg 8.1. It encoded eight copies of a 600-frame, 1920×1080 testsrc2 H.264 video for each configuration. This run predates explicit Lanczos scaling and does not measure the current defaults:

Three and four lanes performed similarly in that run. These are single-run synthetic local-file measurements, not a throughput guarantee for datasets or remote storage. Earlier runs varied, so benchmark both settings on your worker image and representative input.

Install a recent FFmpeg build (8.x is used for validation) and ffprobe on the worker's PATH, or set NVENCConfig(ffmpeg=..., ffprobe=...). The build must include h264_nvenc and, for CUDA decoding/scaling, scale_cuda. Install a compatible NVIDIA driver and expose the GPU's video capabilities to the container. Installing Refiner's Python video extra alone does not install these executables or the NVIDIA driver.

Run the hardware tests on the actual worker image:

The CUDA and CPU-decode tests check frame count, a variable-rate timeline, keyframe positions, and faststart MP4. The image test also checks frame order and color conversion. The encoding path was validated on an L4. Ordinary unit tests run without a GPU and verify that remote inputs and outputs are rejected.

Each encode checks codec, pixel format, dimensions, and positive output frame count before publishing. It also compares frame counts when the source container reports one; image sequence counts are always checked. This fast path does not fully decode every source/output for a timestamp audit. Use the hardware tests and dataset-specific validation when strict annotation alignment is required.

import refiner as mdr

video = await mdr.video.transcode_video("source.mp4", "new.mp4")
video = await mdr.video.encode_image_sequence(
    ["0001.jpg", "0002.jpg"], "images.mp4", fps=30,
)

Direct calls return mdr.video.VideoFile. Existing destinations are rejected. Completed local files publish atomically. The map blocks generate unique names automatically. Use direct calls with job-controlled destinations for checkpointed copying jobs. The caller owns retries and cleanup of earlier outputs; these helpers do not implement dataset checkpoints or garbage collection.

Very small images or extreme resize bounds can fall below the GPU encoder's minimum supported dimensions. The block reports FFmpeg's error rather than upscaling or switching to a software encoder.

Videos are read directly from local paths; image sequences use numbered links. Completed MP4s are validated in a temporary directory beside the destination, then published with an atomic hard link. Timeout and cancellation kill and reap FFmpeg; outstanding local filesystem operations finish before scratch cleanup. The async block uses the existing pipeline window, without another pool of GPU workers. The H.264 bitstream filter explicitly signals limited range even when NVENC omits that tag. Callable descriptions live in a lightweight module so video blocks can be imported before the pipeline planner.

Spark's partition tasks and Beam/Dataflow's element processing could host an external encoder, but would require their own scheduling integration. Hugging Face Datasets offers process-based mapping. Daft and Ray Data provide GPU task or actor scheduling; adding either scheduler here would duplicate Refiner's worker and bounded async execution. Refiner keeps those controls and delegates codec execution to FFmpeg. This also avoids transferring decoded frame arrays through Python merely to feed an encoder.

The device-resident path and concurrent-session design follow NVIDIA's FFmpeg guide.