Fix for Ticket #311 Scaling List Bugs
- Wrong scaling list sizes used.
-> bug fix by changing
- Scaling factor incorrect for dependent quantization.
-> bug fix by removing to change
- 32-bit variable overflow in dependent quantization. -> it may not be happened in my understanding, because qIdx and invQScale are well clipped in a predefined range "qIdx" is related to "TransCoeffLevel" in VVC working draft
It is a requirement of bitstream conformance that the value of dec_abs_level[ n ] shall be constrained such that the corresponding value of TransCoeffLevel[ x0 ][ y0 ][ cIdx ][ xC ][ yC ] is in the range of CoeffMin to CoeffMax, inclusive.
The variables CoeffMin and CoeffMax specifying the minimum and maximum transform coefficient values are derived as follows: CoeffMin = −( 1 << 15 ) (7 117) CoeffMax = ( 1 << 15 ) − 1 (7 118)
and invQScale is a 8-bits integer
nextCoef = ( nextCoef + scaling_list_delta_coef + 256 ) % 256
so the result "nomTCoeff" won't be overflow. Could you provide an example for the overflow case?
- Transform skip only applies to luma.
-> bug fix by modifying useTransformSkip to
useTransformSkip = tu.mtsIdx==MTS_SKIP && isLuma(compID);
- Pred matrix ID delta adjusted for non-existent 2x2 luma lists. -> we have a encoder normative constraint defined in the proposed text
If sizeId is equal to 1, the value of refMatrixId shall not be equal to 0 or 3. Otherwise, iIf sizeId is less than or equal to 5, the value of scaling_list_pred_matrix_id_delta[ sizeId ][ matrixId ] shall be in the range of 0 to matrixId, inclusive. Otherwise (sizeId is equal to 6), the value of scaling_list_pred_matrix_id_delta[ sizeId ][ matrixId ] shall be in the range of 0 to matrixId / 3, inclusive.