"The vulnerability is a one-row heap buffer overflow in the MagicYUV decoder’s slice handling, caused by an inconsistency between how the frame allocator and the decoder compute chroma plane heights," JFrog explains.
CVE-2026-8461: a heap out-of-bounds write in MagicYUV
Security researchers at JFrog disclosed a newly tracked flaw, CVE-2026-8461, in FFmpeg’s MagicYUV decoder. The weakness is a heap out-of-bounds write — described by JFrog as a one-row heap buffer overflow — that can be triggered by crafted AVI, MKV, or MOV video files. The vulnerability carries a high-severity score of 8.8 and affects any application that uses libavcodec, FFmpeg’s core video decoding and encoding library, when the MagicYUV decoder is enabled.
How the bug is triggered and why it matters
JFrog reports the root cause as an inconsistency in how the frame allocator and decoder compute chroma plane heights when processing slices — independent regions of a video frame that can be decoded separately. PixelSmash can fire when an application opens a malicious video, generates thumbnails while browsing a directory, or runs automated media ingestion workflows that probe files for metadata. Even in cases where remote code execution (RCE) is not achieved, the flaw reliably produces a denial-of-service condition.
Real-world demonstrations: Jellyfin and Nextcloud
JFrog lead researcher Yuval Moravchick demonstrated practical exploits against real deployments. The team achieved full RCE against a Jellyfin 10.11.9 media server by dropping a crafted MagicYUV AVI into the media library. Jellyfin’s normal media library scan pipeline automatically triggers ffprobe for metadata extraction, the out-of-bounds write fires, and AVBuffer.free is hijacked to system(), allowing arbitrary commands to execute as the jellyfin service user.
Moravchick showed a similar effect against Nextcloud when the Movie preview feature was enabled. The researcher also described a no-interaction attack vector using torrents: an attacker who seeds a malicious video and persuades or tricks a target to have the download placed into the media library can rely on Jellyfin’s real-time file system monitor to trigger an ffprobe scan and execute the same exploit chain.
JFrog notes an important constraint: exploitation for RCE requires Address Space Layout Randomization (ASLR) to be disabled, or the attacker must chain PixelSmash with another vulnerability that defeats ASLR. In theory, the researchers say, an information-disclosure bug in FFmpeg’s FlashSV decoder could be paired with PixelSmash to bypass ASLR.
Who is exposed — and who has mitigations
JFrog’s testing identified a broad set of popular media applications that use FFmpeg with the MagicYUV decoder enabled, including Kodi, OBS Studio, PhotoPrism, and GNOME/KDE/XFCE thumbnail generators. The advisory also points out that messaging platforms such as Slack, Discord, Telegram, and WhatsApp may be susceptible because they use FFmpeg to generate server-side video previews, though those services were not tested by JFrog.
Not all projects are equally exposed: Plex uses a custom FFmpeg build with decoders disabled and a minimal allowlist, a configuration JFrog says effectively mitigates the PixelSmash risk for that product. In response to the disclosure, FFmpeg released version 8.1.2 on June 17, which fixes the MagicYUV issue. Jellyfin updated its bundled FFmpeg version, and PhotoPrism is working to add a file format blocklist to reduce the potential attack surface. The Nextcloud team received the report via HackerOne but declined to address the flaw because it exists outside of Nextcloud.
What this means for media server operators, messaging platforms, and open-source maintainers
- Media server operators (Jellyfin, Plex, Nextcloud): Jellyfin has updated its bundled FFmpeg; operators running versions that include vulnerable FFmpeg builds should be aware that library scans and automated ingestion can trigger the issue. Plex’s custom FFmpeg configuration shows one mitigation approach: disabling unnecessary decoders or using an allowlist.
- Messaging platforms (Slack, Discord, Telegram, WhatsApp): These services use FFmpeg for server-side preview generation and, while not tested, may be at risk when FFmpeg is configured with MagicYUV enabled. The vendor status in the report is "not tested," which leaves practical exposure dependent on each platform’s build and configuration choices.
- Open-source maintainers and downstream projects: JFrog emphasizes a supply-chain dimension — MagicYUV is present in hundreds of projects that rely on FFmpeg to handle untrusted input. The presence of a single decoder vulnerability in a widely trusted library can cascade across many products and workflows.
JFrog reported the bug to the FFmpeg security team on May 13; FFmpeg published the fix in release 8.1.2 on June 17. The researchers warn that PixelSmash enlarges the attack surface because many projects "trust FFmpeg to handle untrusted input safely," making the flaw more than a single-binary problem — it is a supply-chain issue that reaches into media servers, desktop environments, thumbnail generators, cloud previews, and automated ingestion pipelines.
Original reporting: https://www.bleepingcomputer.com/news/security/ffmpeg-fixes-pixelsmash-flaw-in-widely-used-video-decoder/
