Skip to content

Fix: Prevents usage of wrong contexts in DepQuant

Heiko Schwarz requested to merge schwarz/VVCSoftware_VTM:fix-ctx-usage-in-depQuant into master

The suggested changes fix the following two issues:

[1] For blocks with width>32 or height>32, the scan created in initRom() (file Rom.cpp) are only valid for the non-zero-out region. However, in Rom::xInitScanArrays() (file DepQuant.cpp), the zero-out region is also used. This has the effect that the array raster2id[] contains wrong values. As a consequence, wrong neighboring positions cpos[] are derived.

Even though a previous fix (fix #161) ensures that the positions are clipped to a value inside the subblock, these positions could still be wrong. It is suggested to fix the issue by correctly considering the zero-out regions for larger blocks (which is the actual source of the problem).

[2] In State::updateStateEOS(..), the array m_absLevelAndCtxInit should be set equal to zero if a skipped subblock is selected. Otherwise, it could be possible lead to a usage of wrongs contexts for following transform coefficients.

Merge request reports