CVE-2026-56211
Description
A remote code execution vulnerability was found in libaom, the reference AV1 codec implementation. Insufficient bounds validation in the AV1 encoder's SVC (Scalable Video Coding) layer ID control allows an attacker to supply crafted video frame pixels that overlap with internal encoder layer context structures. In fork-based video processing services, an attacker can use this to hijack the cyclic refresh map pointer, brute-force the process base address via a crash oracle, and redirect control flow to achieve arbitrary command execution. Exploitation requires the target service to use libaom with SVC encoding enabled and accept attacker-supplied video frames.
Statement
This vulnerability is rated as Critical severity because the researcher demonstrated successful remote code execution against a fork-based video processing service. The exploit chain leverages attacker-controlled pixel values to hijack internal encoder pointers, uses a crash oracle to brute-force ASLR, and ultimately achieves arbitrary command execution. However, the attack complexity is elevated: it requires a fork-based service architecture (for the crash oracle), multiple encoding attempts (for ASLR brute-force), and knowledge of the target binary layout. In Red Hat products, libaom ships bundled within Firefox and Thunderbird. Firefox does not use a fork-based architecture for WebRTC encoding, and SVC layer parameters are managed internally, making the demonstrated exploit chain not directly applicable to the browser context. RHEL-AI 3.4 (aom 3.12.0) and Hummingbird 1 (aom 3.13.3) ship standalone libaom packages within the affected version range. Fork-based transcoding or video conferencing services that use libaom with SVC encoding and accept attacker-supplied frames are at highest risk for this specific exploit chain.
Mitigation
There is no complete mitigation for this vulnerability. The following measures can reduce risk:
1. If using libaom as a standalone encoder in a fork-based service, validate all SVC layer parameters (spatial_layer_id, temporal_layer_id) against configured bounds before passing them to the encoder API.
2. Avoid fork-based architectures for encoding services that accept untrusted input. Use thread-based or container-isolated workers instead, which prevent crash oracle attacks.
3. Restrict access to encoding services to trusted clients only. Do not expose SVC encoder configuration or frame submission to untrusted network input.
4. For Firefox and Thunderbird, ensure browsers are updated to versions that include the patched libaom (v3.14.0 or later).
5. Enable all available exploit mitigations (ASLR, PIE, stack canaries, CFI) on encoding service binaries.
Common Vulnerability Scoring System (CVSS) Score Details
Info alert:Important note
CVSS scores for open source components depend on vendor-specific factors (e.g. version or build chain). Therefore, Red Hat's score and impact rating can be different from NVD and other vendors. Red Hat remains the authoritative CVE Naming Authority (CNA) source for its products and services (see Red Hat classifications).
CVSS v3 Score Breakdown
| Red Hat | NVD | cve.org | |
|---|---|---|---|
| Base Score | 7.1 | N/A | 7.1 |
| Attack Vector | Network | N/A | Network |
| Attack Complexity | High | N/A | High |
| Privileges Required | None | N/A | None |
| User Interaction | Required | N/A | Required |
| Scope | Unchanged | N/A | Unchanged |
| Confidentiality | Low | N/A | Low |
| Integrity Impact | High | N/A | High |
| Availability Impact | High | N/A | High |
Vector
Red Hat: CVSS:3.1/AV:N/AC:H/PR:N/UI:R/S:U/C:L/I:H/A:H
cve.org: CVSS:3.1/AV:N/AC:H/PR:N/UI:R/S:U/C:L/I:H/A:H
Understanding the Weakness (CWE)
Integrity
Technical Impact: Modify Memory; Execute Unauthorized Code or Commands
Write operations could cause memory corruption. In some cases, an adversary can modify control data such as return addresses in order to execute unexpected code.
Availability
Technical Impact: DoS: Crash, Exit, or Restart
Attempting to access out-of-range, invalid, or unauthorized memory could cause the product to crash.
Other
Technical Impact: Unexpected State
Subsequent write operations can produce undefined or unexpected results.
Acknowledgements
Red Hat would like to thank The FuzzAnything Team (FuzzAnything) for reporting this issue.
Frequently Asked Questions
Not sure what something means? Check out our Security Glossary.
Want to get errata notifications? Sign up here.
