Skip to content
Snippets Groups Projects
Commit bf5f356c authored by Frank Bossen's avatar Frank Bossen
Browse files

Merge branch 'fix_1541' into 'master'

Fix #1541: remRegBins calculation

See merge request jvet/VVCSoftware_VTM!2197
parents 53490ea3 0f1b551f
No related branches found
No related tags found
No related merge requests found
...@@ -662,7 +662,7 @@ void QuantRDOQ::xRateDistOptQuant(TransformUnit &tu, const ComponentID &compID, ...@@ -662,7 +662,7 @@ void QuantRDOQ::xRateDistOptQuant(TransformUnit &tu, const ComponentID &compID,
int iLastScanPos = -1; int iLastScanPos = -1;
int ctxBinSampleRatio = (compID == COMPONENT_Y) ? MAX_TU_LEVEL_CTX_CODED_BIN_CONSTRAINT_LUMA : MAX_TU_LEVEL_CTX_CODED_BIN_CONSTRAINT_CHROMA; int ctxBinSampleRatio = (compID == COMPONENT_Y) ? MAX_TU_LEVEL_CTX_CODED_BIN_CONSTRAINT_LUMA : MAX_TU_LEVEL_CTX_CODED_BIN_CONSTRAINT_CHROMA;
int remRegBins = (uiWidth * uiHeight * ctxBinSampleRatio) >> 4; int remRegBins = (tu.getTbAreaAfterCoefZeroOut(compID) * ctxBinSampleRatio) >> 4;
uint32_t goRiceParam = 0; uint32_t goRiceParam = 0;
double *pdCostCoeffGroupSig = m_pdCostCoeffGroupSig; double *pdCostCoeffGroupSig = m_pdCostCoeffGroupSig;
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment