Change #252853

Comments

swscale/ops: correctly truncate on ff_sws_apply_op_q(SWS_OP_RSHIFT)
Instead of using a "precise" division, simulate the actual truncation.

Note that the division by `den` is unneeded in principle because the
denominator *should* always be 1 for an integer, but this way we don't
explode if the user should happen to pass `4/2` or something.

Fixes a lot of unnecessary clamps w.r.t. xv36, e.g.:

 xv36be -> yuv444p12be:
   [u16 XXXX -> ++++] SWS_OP_READ         : 4 elem(s) packed >> 0
   [u16 ...X -> ++++] SWS_OP_SWAP_BYTES
   [u16 ...X -> ++++] SWS_OP_SWIZZLE      : 1023
   [u16 ...X -> ++++] SWS_OP_RSHIFT       : >> 4
-  [u16 ...X -> ++++] SWS_OP_CONVERT      : u16 -> f32
-  [f32 ...X -> ++++] SWS_OP_MIN          : x <= {4095 4095 4095 _}
-  [f32 ...X -> ++++] SWS_OP_CONVERT      : f32 -> u16
   [u16 ...X -> ++++] SWS_OP_SWAP_BYTES
   [u16 ...X -> ++++] SWS_OP_WRITE        : 3 elem(s) planar >> 0
     (X = unused, + = exact, 0 = zero)

Changed files

• libswscale/ops.c
• tests/ref/fate/sws-ops-list