Skip to content

Fix #1408: Replace int with TCoeff to fix zeroing out of coefficients.

Adrian Browne requested to merge adybrowne/VVCSoftware_VTM:fix_ticket_1408 into master

The output array from TrQuant::fwdLfnstNxN is a TCoeff but the zero out at the end of the method is computed in terms of int. If high bit depth processing is enabled TCoeff is a 64 bit int and hence the zero out calculation is incorrect.

Merge request reports