Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
V
VVCSoftware_VTM
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Custom Issue Tracker
    • Custom Issue Tracker
  • Labels
  • Merge Requests 11
    • Merge Requests 11
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Environments
  • Analytics
    • Analytics
    • CI / CD
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Jobs
  • Commits
  • jvet
  • VVCSoftware_VTM
  • Merge Requests
  • !1959

Merged
Opened Nov 19, 2020 by Adrian Browne@adybrowneContributor

Fix #1431: Avoid clipping after inverse transform when RExt__HIGH_BIT_DEPTH_SUPPORT is enabled

  • Overview 0
  • Commits 1
  • Pipelines 1
  • Changes 1

The specification defines no clipping after the inverse transform but in practice the output of the inverse transform may be clipped to the range -(2^bitdepth - 1)..(2^bitdepth - 1) for memory reduction purposes without affecting the end result. When the bitdepth is set to 16 and ExtendedPrecision is disabled, clipping to the range -(2^15)..(2^15 - 1) occurs, which is incorrect. The clipping range is modified to the numeric limits of the Pel data type, i.e. -(2^15)..(2^15 - 1) when RExt__HIGH_BIT_DEPTH_SUPPORT is disabled (default, range is unchanged) and -(2^31)..(2^31 - 1) otherwise.

Edited Nov 19, 2020 by Frank Bossen
Assignee
Assign to
Reviewer
Request review from
None
Milestone
None
Assign milestone
Time tracking
Reference: jvet/VVCSoftware_VTM!1959
Source branch: fix_ticket_1431

Revert this merge request

This will create a new commit in order to revert the existing changes.

Switch branch
Cancel
A new branch will be created in your fork and a new merge request will be started.

Cherry-pick this merge request

Switch branch
Cancel
A new branch will be created in your fork and a new merge request will be started.