Change #264293

Comments

avfilter/vf_ssim360: fix integer overflow in tape_length allocation
tape_length * 8 overflows 32-bit int for large input widths. Then
av_malloc_array() allocates a tiny buffer while the subsequent
loop writes tape_length*8 BilinearMap entries, causing
heap-buffer-overflow.

Validate the value in float before converting to int and left
shifting, to avoid both float-to-int and signed left shift
overflow UB. Also split av_malloc_array() arguments to avoid
the multiplication overflow.

Fixes: #21511

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>

Changed files

• libavfilter/vf_ssim360.c