Fix for Ticket #311 Scaling List Bugs

#311

• Wrong scaling list sizes used. -> bug fix by changing uiLog2TrWidth-1 to uiLog2TrWidth, and uiLog2TrHeight-1 to uiLog2TrHeight
• Scaling factor incorrect for dependent quantization. -> bug fix by removing to change shift value
• 32-bit variable overflow in dependent quantization. -> it may not be happened in my understanding, because qIdx and invQScale are well clipped in a predefined range "qIdx" is related to "TransCoeffLevel" in VVC working draft

It is a requirement of bitstream conformance that the value of dec_abs_level[ n ] shall be constrained such that the corresponding value of TransCoeffLevel[ x0 ][ y0 ][ cIdx ][ xC ][ yC ] is in the range of CoeffMin to CoeffMax, inclusive.

The variables CoeffMin and CoeffMax specifying the minimum and maximum transform coefficient values are derived as follows: CoeffMin = −( 1 << 15 ) (7 117) CoeffMax = ( 1 << 15 ) − 1 (7 118)

and invQScale is a 8-bits integer

nextCoef = ( nextCoef + scaling_list_delta_coef + 256 ) % 256

so the result "nomTCoeff" won't be overflow. Could you provide an example for the overflow case?

• Transform skip only applies to luma. -> bug fix by modifying useTransformSkip to useTransformSkip = tu.mtsIdx==MTS_SKIP && isLuma(compID);
• Pred matrix ID delta adjusted for non-existent 2x2 luma lists. -> we have a encoder normative constraint defined in the proposed text

If sizeId is equal to 1, the value of refMatrixId shall not be equal to 0 or 3. Otherwise, iIf sizeId is less than or equal to 5, the value of scaling_list_pred_matrix_id_delta[ sizeId ][ matrixId ] shall be in the range of 0 to matrixId, inclusive. Otherwise (sizeId is equal to 6), the value of scaling_list_pred_matrix_id_delta[ sizeId ][ matrixId ] shall be in the range of 0 to matrixId / 3, inclusive.