Skip to content
Snippets Groups Projects

Fix SIMD overflow in 12-bit ALF classification

Merged Fix SIMD overflow in 12-bit ALF classification
XiangLi/VVCSoftware_VTM:fix_alf_simd into master
Merged Xiang Li requested to merge XiangLi/VVCSoftware_VTM:fix_alf_simd into master

When internal bit depth is 12-bit, the dynamic range of colSums is 16-bit. In this case, directly using _mm_add_epi16 on top of colSums may lead to overflow.

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Loading
  • Loading
Please register or sign in to reply