dev_notes_hop.md

## Schedule 
### 1- extract VTM data for intra training. (1 week)
   - can be started now
   - extract json, YUV, md5 etc.
    
### 2- train intra model (2 weeks)
   - need a first working training script

### 3- validate intra model
   - need a first working NNVC-5.0 VTM (only for intra frame)
   - 

## Software structure
### Main classes

- NNFilterHOP
```C
class NNFilterHOP {
  init: call once
  filter: call to filter the whole picture
  filterBlock: call per block. storage own by the class NNFilterHOP. No scaling done, output on 14 bits (log2OutputScale)
  scaleResidualBlock: perform residual scaling
  resizeInputs: resize the inputs to target shape (take size of extended block as input)
};
```
- EncNNFilterHOP
```C
class EncNNFilterHOP {
  chooseParameters: perform the RDO (by calling parameterSearch) to get the best parameters, e.g. qp offsets, scaling factors
  parameterSearch: search to find out the best QP offsets
  scaleFactorDerivation: derive residual scaling factors with least square methods
  scalePicture: call at encoder to scale the whole picture with the above derived scaling factors
  setNnlfHopInferGranularity: call at encoder to adaptively determine the inference size
};
```

- encoder algorithm
```C
compressGOP() {
 ...
#if NN_HOP_UNIFIED
      if (pcSlice->getSPS()->getNnlfHopEnabledFlag())
      {
        m_nnfilterHOP.initCabac( m_pcEncLib->getCABACEncoder(), m_pcEncLib->getCtxCache(), *pcSlice);
        m_nnfilterHOP.setPicprms(&pcPic->m_picprm);
        m_nnfilterHOP.chooseParameters(*pcPic);
        pcSlice->setNnlfHopParameters(m_nnfilterHOP.getSliceprms());
      }
#endif
```

- NNFilterHOP::FilterParameters
```C
// parameters needed to filter the picture
struct FilterParameters
{
  int              block_size;
  int              extension;
  int              nb_blocks_width;
  int              nb_blocks_height;
  int              prmNum;
  std::vector<int> prmId; // -1: off
  SliceParameters  sprm;
};
```

- NNFilterHOP::SliceParameters
```C
// parameters signaled at slice level
struct SliceParameters {
  int mode; // -1: off
  int scaleFlag;
  int scale[MAX_NUM_COMPONENT][2];
  
  SliceParameters()
  {
    reset();
  }
  void reset()
  {
    mode = -1;
    scaleFlag = -1;
    std::memset( scale, 0, sizeof( scale ) );
  }
};
```

- CABAC write/readNnlfHopParameters: signal parameters of each block
- VLC wirte/readHopScalingMode: signal scaling mode of each slice 
 static const CtxSet   nnlfHopParams; prepare for signaling contexts.

### Algorithms
- rdo: QP adaptation, residual scaling (separate scaling factors to be used for Cb and Cr)
- signaling: data in FilterParameters
  -block_size: signalled in SPS
  -extension: signalled in SPS
  -prmNum: signalled in SPS
  -prmId: signalled in coding_tree_unit
  -sprm.mode: signalled at slice header
  -sprm.scaleFlag: signalled at slice header
  -sprm.scale: signalled at slice header


### Model
- model inputs: see NNFilterHOP::filterBlock to see model inputs and inputs quantizers. 
 - assume input in SADL as quantizer >=10 (because perform a left shift)
 - assume output in SADL as quantizer <=14 (because perform a  right shift)
- model output: the 6 data plan (4Y and UV) are unshuffled in C++ side (see NNFilterHOP.cpp:extractOutputs) 
 

### Other changes
 - pass to C++-17 (aligned with VTM, simplify generic code for int16/float handling)
 - added some const in prepareInputs