Skip to content

Fix #1408: Replace int with TCoeff to fix zeroing out of coefficients.

The output array from TrQuant::fwdLfnstNxN is a TCoeff but the zero out at the end of the method is computed in terms of int. If high bit depth processing is enabled TCoeff is a 64 bit int and hence the zero out calculation is incorrect.

Merge request reports

Loading