Streaming Compression Artifacts

Mechanism of Compression Artifacts

Compression artifacts are visible errors introduced when a video is encoded with lossy codecs that discard redundant or perceptually less important data to reduce file size and transmission bitrate. The core lossy step is quantization, the process of rounding transform coefficients (typically from discrete cosine transform, or DCT) to discrete levels. Higher quantizer values discard more high-frequency data, creating both blur and blocking. At low bitrates, when aggressive quantization is forced to meet bandwidth constraints, these errors become visually apparent across multiple artifact classes.

Primary Artifact Types

Blocking (Macroblocking). Block-based codecs like H.264 process frames on a grid of discrete blocks. H.264 uses a 16×16 pixel macroblock as its basic unit. When quantization is too coarse, neighboring blocks no longer align at their shared edges, creating a tiled appearance. H.264 includes an optional in-loop deblocking filter (adjustable in strength and can be disabled by the encoder) that can reduce but does not eliminate these boundary artifacts.

Banding (Contouring). Banding appears in smooth gradients like skies or fade transitions as visible bands of flat color separated by sharp steps, rather than smooth transitions. This occurs because standard 8-bit video provides only 256 brightness levels per channel. During quantization, gradual transitions get rounded to identical code values, creating visible plateaus. This is a known risk factor in dark and low-light content where subtle gradations are common.

Ringing and Halos. Sharp, high-contrast edges like text or building outlines develop halos and oscillating distortions. Ringing results from truncated high-frequency coefficients in the transform process, causing the reconstructed edge to overshoot and oscillate.

Mosquito Noise. A flickering or ringing-like artifact appearing in smooth, homogeneous regions near strong high-contrast edges, mosquito noise results from loss of high-frequency transform coefficients during quantization. It becomes more prominent as bitrate decreases.

Temporal Artifacts. Motion-related distortions include ghosting, floaters, smeared motion, and broken edges on fast-moving objects, resulting from aggressive motion estimation and compensation when bitrate is constrained. Motion smearing occurs when moving objects leave trails because insufficient data is allocated to motion changes.

Vulnerability Factors: Dark Scenes and Fine Textures

Dark scenes are particularly vulnerable to compression artifacts including banding because limited color precision in 8-bit encoding creates quantization errors in shadow regions where subtle gradations are common. Fine textures like fabric patterns and foliage are also at risk of artifact formation.

H.265/HEVC handles both dark scenes and fine textures better than H.264 through its use of Coding Tree Units (CTUs) ranging from 16×16 to 64×64 pixels, which allow the codec to adapt to different content types. This produces cleaner edges, less banding in gradients, and fewer compression artifacts in detailed areas compared to H.264's fixed macroblock approach.

Codec Comparison: H.264 vs. H.265/HEVC

H.264/AVC uses a fixed 16×16 pixel macroblock as its basic processing unit, subdividing each macroblock into smaller partitions for prediction. H.265/HEVC replaced the fixed macroblock approach with Coding Tree Units, enabling more flexible block sizes and more sophisticated in-loop filtering to reduce blockiness artifacts.

H.265/HEVC is commonly cited by industry benchmarks as achieving similar visual quality to H.264 at roughly half the bitrate, a 40–50% reduction in file size for equivalent perceived quality, though real-world gains vary by content and encoder implementation. Netflix uses H.265/HEVC for select 4K content; its 1080p 'Super HD' tier is encoded in H.264 in part because many devices certified for Super HD predate widespread HEVC decoder support.

Bitrate Thresholds and Measurement

Low bitrate forces aggressive compression, with visible quality problems becoming apparent in dark scenes, fine textures, and fast-motion content. Industry guidance suggests 10–15 Mbps as the minimum for 1080p video at 30 fps and 35–50 Mbps for 4K to avoid visible compression artifacts, though actual requirements vary by codec and content complexity.

Objective metrics for measuring video quality include PSNR, SSIM, and VMAF, but these are blind to banding artifacts and temporal distortions like ghosting or flickering. CAMBI (Contrast Aware Multiscale Banding Index), developed by Netflix, is specifically designed to detect banding artifacts using insights from human contrast sensitivity functions to predict banding visibility at multiple scales. CAMBI scoring starts at 0 (no banding detected), with higher scores indicating more visible banding; scores in the low single digits mark the onset of visible banding.