Skip to content

Fix #1498: Fix decoder mismatch for RGB content when LFNST is disabled

Adrian Browne requested to merge adybrowne/VVCSoftware_VTM:FIX_1498_RGB_LFNST_TS_MISMATCH into master

Though Fix#1497 removed the encoder crash, it also inadvertently changed the behaviour when LFNST is disabled as the original code the loop upper bound can change during the processing of the loop. This did not happen in the fix and this generated decoder mismatches.

This fix replaces the simpler #1497 fix but maintains the existing behaviour when LFNST is disabled.

This fix has been tested with the 5 RGB sequences in the HBD CTC (single frame AI at 6 QP) and the 8 RGB TGM sequences in the non-420 CTC (single frame AI at 4 QP).

For the following configurations identical results were achieved with and without the fix (i.e. the code prior to #1497) LFNST=1, TS=1, BDPCM=1 LFNST=1, TS=1, BDPCM=0 LFNST=1, TS=0, BDPCM=0 LFNST=0, TS=1, BDPCM=0 LFNST=0, TS=1, BDPCM=1

For the following configuration results were obtained with the fix. Without the fix all encoder simulations resulted in SEGV. LFNST=0, TS=0, BDPCM=0

My apologies for the erroneous fix #1497.

Merge request reports