CT_00_056 -- Improve x264 vectorization

CT_00_056 -- Improve x264 vectorization

About

x264 is a critical benchmark for vectorization in the spec suite showing a roughly 2X improvement across many archtiectures once vector is enabled.  This work item is mean to track further improvements that may be possible in the benchmark through compiler improvements.

 

  1. SAD optimization

    1. Shorter SADs (sad_x3_8x8) can benefit from strided loads

  2. SATD

    1. Use strided loads to avoid permutations in the first SATD loop

    2. Use wider vectors in the 2nd loop.  Smart unrolling seems to be the key here

  3. vaaddu

    1. Designs which flush pipeline on VXRM assignment may be better off using (a + b + 1) >> 1

    2. Implies expander should probably be conditional on a suitable uarch flag

  4. Use vnclip instruction more aggressively

  5. Upstream costing adjustments to allow designs to steer compiler away from indexed loads, particularly in subdct

 

Stakeholders/Partners

RISE:

Ventana: Robin Dapp – Cost model, permutation improvements, etc.  Overall lead

Ventana: Jeff Law – everything scheduling related

 

External:

 

 

 

Dependencies

 

Status

Development

IN PROGRESS

 

Development Timeline

2H2025

 

Upstreaming

IN PROGRESS

 

Upstream Version

gcc-16

Spring 2026

 

 

 

Contacts

Jeff Law (Ventana)

 

Dependencies

 

 

 

Updates

Jul 16, 2025 

  • Strided loads to help SAD/SATD will break off into its own work item for 2025 2H

    • POC has shown they can really help SATD by avoiding the costly permutes.  Assumes strong uarch support

    • Needs to be made first class citizen in the vectorizer

    • Depends on first tackling unaligned issues, which Robin is currently working through upstream

    • Then we can add strided loads to the vectorizer

  • Code to aggressively hoist VXRM assignments to reduce pipeline flush penalty has been upstreamed for some time

  • Code to use the vaaddu and related instructions more often has been upstreamed (by Intel IIRC)

  • vsetvl elimination patch was recently re-evaluated and does not seem to be helping anymore.  So dropping that from consideration.