avutil/common: Fix undefined behavior in av_clip_uintp2_c()

Fixes: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself
Fixes: 8521/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5639024952737792

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <[email protected]>
(cherry picked from commit aa41d322be71106ce147445f2b42bb763f1eff86)
Signed-off-by: Michael Niedermayer <[email protected]>

diff --git a/libavutil/common.h b/libavutil/common.h
index 53e43feb3aa4114822f6747574118e80e1edf6a0..58ead80514463a3daae5a5f2aa75d37eb172cd0b 100644 (file)
--- a/libavutil/common.h
+++ b/libavutil/common.h
@@ -228,7 +228,7 @@ static av_always_inline av_const int av_clip_intp2_c(int a, int p)
  */
 static av_always_inline av_const unsigned av_clip_uintp2_c(int a, int p)
 {
-    if (a & ~((1<<p) - 1)) return -a >> 31 & ((1<<p) - 1);
+    if (a & ~((1<<p) - 1)) return (~a) >> 31 & ((1<<p) - 1);
     else                   return  a;
 }