Change #269258

Comments

swscale/x86/yuv2rgb: fix planar GBRP boundary check in ssse3 assembly
    In yuv420_gbrp_ssse3, the boundary safeguard check "h_size * 3 >
    FFABS(dstStride[0])" was erroneously set based on probably packed RGB24
    formats (where each pixel spans 3 bytes per row).

    For GBRP (planar GBR), each plane contains only 1 component per pixel
    per row, meaning dstStride[0] corresponds exactly to width.
    Multiplying h_size by 3 mistakenly triggers the condition for normal
    widths, decreasing h_size by 8. This leaves the rightmost 8 pixels
    of every row completely uninitialized (black).

    Fix this by checking "h_size > FFABS(dstStride[0])" instead.

    How to Reproduce the error:
    1. Generate buggy and fixed outputs as PNGs using the 600x600 pipeline:

       buggy output without the fix
       $ ffmpeg -f lavfi -i color=c=red:size=600x600:rate=1 \
           -vf format=yuv420p,format=gbrp \
           -frames:v 1 -y buggy_red_600.png
       fixed output with the fix
       $ ffmpeg -f lavfi -i color=c=red:size=600x600:rate=1 \
           -vf format=yuv420p,format=gbrp \
           -frames:v 1 -y fixed_red_600.png

    2. Verify buggy_red_600.png in an image viewer:
       A strict, 8-pixel wide vertical black stripe (columns 592 to 599) is
       clearly visible running top-to-bottom down the rightmost edge of the image.

    3. Verify fixed_red_600.png in an image viewer as well:
       The output renders a perfect, uniform, fully solid red square across
       the entire 600x600 canvas, indicating the boundary bug is successfully resolved.

Changed files

• libswscale/x86/yuv2rgb.c