vLLM: Request-selected PyNvVideoCodec GPU decode bypasses static VRAM reservation
Summary
Current vLLM main lets an inference request choose the PyNvVideoCodec GPU video decoder through media_io_kwargs.video.video_backend, but engine GPU memory reservation is computed only from static startup configuration and VLLM_VIDEO_LOADER_BACKEND. If the server starts with the default OpenCV/software backend and no --mm-ipc-gpu-memory-gb budget, a client can still route a video request into the PyNvVideoCodec path after startup, causing frontend CUDA-context, decoder-surface, and decoded-frame GPU allocations that were not carved out of the engine KV-cache budget.
Technical Details
The vulnerable boundary is the split between request-time media decoding choices in the API server and startup-time memory budgeting in the engine worker. Request bodies for Chat Completions and Responses expose media_io_kwargs, and those values are forwarded to the shared media connector. For video inputs, MediaConnector.fetch_video() copies self.media_io_kwargs["video"] into video_io_kwargs, only setting a model-derived backend when video_backend is absent. VideoMediaIO.__init__() then consumes video_backend from those kwargs and loads that backend from VIDEO_LOADER_REGISTRY.
The relevant request-side source path is:
``python
video_io_kwargs = dict(self.media_io_kwargs.get("video", {}))
if "video_backend" not in video_io_kwargs and (
video_backend := get_video_loader_backend_for_processor(video_processor)
):
video_io_kwargs["video_backend"] = video_backend
video_io = VideoMediaIO(image_io, *video_io_kwargs)
`
`python
video_loader_backend = (
kwargs.pop("video_backend", None) or envs.VLLM_VIDEO_LOADER_BACKEND
)
self.video_loader = VIDEO_LOADER_REGISTRY.load(video_loader_backend)
`
VideoBackend.load_bytes() then dispatches backend == "pynvvideocodec" into decode_frames_pynvvideocodec(), which constructs a PyNvVideoCodec decoder, creates or uses a CUDA stream, reads stream metadata, decodes selected frames on the GPU, and copies those frames into pinned host memory. The new frontend GPU memory pool accounts only for raw decoded frame bytes when a pool exists; it does not make request-time backend selection safe when no startup reservation was made.
The engine-side reservation code makes its decision from static model config and environment only:
`python
def _uses_pynvvideocodec_video_backend(mm_config) -> bool:
video_kwargs = mm_config.media_io_kwargs.get("video", {})
video_loader_backend = (
video_kwargs.get("video_backend") or envs.VLLM_VIDEO_LOADER_BACKEND
)
codec_backend = video_kwargs.get("backend")
return (
video_loader_backend == PYNVVIDEOCODEC_VIDEO_BACKEND
or codec_backend == PYNVVIDEOCODEC_VIDEO_BACKEND
)
`
`python
decoder_reserved_bytes = (
num_api_servers per_server_decoder_bytes
if self._uses_pynvvideocodec_video_backend(mm_config)
else 0
)
reserved_bytes = raw_frame_reserved_bytes + decoder_reserved_bytes
if reserved_bytes <= 0:
return available_kv_cache_memory_bytes
`
With default static video configuration, mm_config.media_io_kwargs["video"] does not name PyNvVideoCodec and VLLM_VIDEO_LOADER_BACKEND defaults to OpenCV/software decoding. The worker therefore reserves no PyNv decoder/CUDA-context bytes. A later request can still set media_io_kwargs.video.video_backend="pynvvideocodec" and reach the GPU decoder path because that runtime field is intentionally honored by VideoMediaIO.
PoV
An ordinary multimodal inference request can carry the backend override in the request body:
`json
{
"model": "served-vlm",
"messages": [
{
"role": "user",
"content": [
{"type": "text", "text": "summarize this clip"},
{"type": "video_url", "video_url": {"url": "data:video/mp4;base64,<small-mp4>"}}
]
}
],
"media_io_kwargs": {
"video": {
"video_backend": "pynvvideocodec"
}
}
}
`
The following bounded source-level check confirms the code path without allocating GPU memory:
`bash
git clone --filter=blob:none https://github.com/vllm-project/vllm.git
cd vllm
git checkout ddd3855a28a561a5bb54d380c6e6b8b1e883cc4a
python3 check_pynv_backend_reservation.py --repo .
`
PoC
The bounded check validates current source markers, simulates the exact static reservation predicate, and compares vulnerable and negative-control configurations. Key output:
`json
{
"vulnerable": true,
"head": "ddd3855a28a561a5bb54d380c6e6b8b1e883cc4a",
"reservation_simulation": {
"env_video_loader_backend": "opencv",
"request_selects_pynv_after_startup": true,
"vulnerable_static_reserved_bytes": 0,
"negative_control_static_pynv_reserved_bytes": 2066953011,
"raw_frame_only_control_reserved_bytes": 268435456,
"unreserved_decoder_bytes_when_only_request_selects_pynv": 2066953011
}
}
`
The negative control is important: when PyNvVideoCodec is selected statically, the worker reserves 2066953011 bytes per API process for decoder surfaces plus CUDA context. The vulnerable case reserves 0 bytes for the same decoder overhead because PyNvVideoCodec is selected only by the later request. A second control with static OpenCV plus mm_ipc_gpu_memory_gb=0.25 reserves only the raw-frame semaphore budget and still does not reserve PyNv decoder/CUDA-context bytes.
Impact
An attacker who can submit video requests to a vLLM deployment with PyNvVideoCodec available can force frontend GPU decoding even when the engine did not reserve memory for that decoder during startup. On high-utilization serving deployments, the unreserved CUDA context, retained decoder surfaces, and decoded-frame allocations can reduce or exhaust GPU memory that the engine assumed was available for weights, activations, or KV cache, causing request failures, worker crashes, or service-level denial of service.
Suggested severity is Medium with conservative CVSS v3.1 CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H (6.5). If a deployment exposes the affected API without authentication, PR:N would raise the deployment-specific score. Suggested weaknesses are CWE-770 (Allocation of Resources Without Limits or Throttling) and CWE-400 (Uncontrolled Resource Consumption). This should not be rated Low because the affected resource is shared GPU memory in the serving path and the code already treats the PyNv decoder/CUDA-context footprint as large enough to reserve at startup when statically configured.
Limitations: exploitation requires a GPU deployment where PyNvVideoCodec is installed and usable, and the request must reach a video-capable model/path. The issue does not claim code execution, data disclosure, or SSRF.
Suggested Fix
Do not allow untrusted request fields to select a GPU decoder that was not included in startup memory reservation. The simplest fix is to reject request-level media_io_kwargs.video.video_backend="pynvvideocodec" unless the static server configuration already selected PyNvVideoCodec and reserved its decoder/CUDA-context budget.
If dynamic backend selection remains supported, split software and GPU decoder policies: allow request selection among CPU/software decoders only, require an explicit operator allowlist for GPU decoders, and include every request-selectable GPU decoder in the startup reservation predicate. Add regression coverage for static OpenCV startup config plus request-level PyNvVideoCodec override, and preserve the negative control where static PyNvVideoCodec configuration reserves decoder/CUDA-context bytes.
Affected Package/Versions
Package: vllm from vllm-project/vllm.
Confirmed affected: current main at ddd3855a28a561a5bb54d380c6e6b8b1e883cc4a.
Introduced by: af16446bf39de047ab57649c933063cf1cbf1e50, Vram semaphore infra (#44465), committed 2026-06-26T17:32:51-07:00.
Release status checked: git tag --contains af16446bf returned no release tags in the fresh checkout. GitHub repository metadata reported latest published release v0.23.0 published 2026-06-15T05:27:20Z; the local v0.24.0 tag also does not contain the introducing commit. The affected range should therefore be current main builds containing af16446bf until fixed, rather than a confirmed released-version range.
Advisory History
Public vLLM advisories checked included audio decompression-bomb DoS, unbounded video/jpeg frame-count DoS, MediaConnector SSRF, video processing RCE, multimodal embedding DoS/RCE, GGUF GPU memory exposure, multimodal hashing, and other request-parameter DoS classes. None matched request-selected PyNvVideoCodec or the static VRAM reservation mismatch.
Prior local/private vLLM report families checked included request-level media_io_kwargs reopening video/jpeg frame fanout, GLM video metadata amplification, and audio media decode duration-limit bypass. Those reports share the request-level media kwargs boundary, but they target CPU/media decode limits or model metadata amplification. This report targets a different privileged asset and fix surface: GPU decoder selection after engine startup memory reservation.
Focused GitHub issue/PR searches for pynvvideocodec, mm_ipc_gpu_memory, video_backend media_io_kwargs, Vram semaphore infra, and frontend multimodal GPU decoding found the PyNvVideoCodec zero-copy RFC, an old do-not-review prototype, merged PR #44465, and an unrelated TorchCodec backend PR. No public issue or PR described this security boundary.
Appendix: Bounded Source-Level Check
`python
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import json
import re
import subprocess
from pathlib import Path
MIB = 1024 1024
GIB = 1024 MIB
def read(repo: Path, rel: str) -> str:
return (repo / rel).read_text(encoding="utf-8")
def const_int(source: str, name: str) -> int:
expr = re.search(rf"^{name}\s=\s(.+)$", source, flags=re.MULTILINE).group(1).strip()
if expr == "128 MiB_bytes":
return 128 MIB
if expr == "int(1.8 1024 MiB_bytes)":
return int(1.8 1024 MIB)
if expr == "1":
return 1
raise AssertionError(expr)
def uses_pynv_static(static_media_io_kwargs: dict[str, dict[str, str]], env_backend: str) -> bool:
video_kwargs = static_media_io_kwargs.get("video", {})
video_loader_backend = video_kwargs.get("video_backend") or env_backend
codec_backend = video_kwargs.get("backend")
return video_loader_backend == "pynvvideocodec" or codec_backend == "pynvvideocodec"
def reserve_bytes(static_media_io_kwargs, env_backend, mm_ipc_gpu_memory_gb, decoder_bytes, cuda_context_bytes, retained_decoders):
raw_frame_reserved_bytes = int(mm_ipc_gpu_memory_gb GIB)
per_server_decoder_bytes = decoder_bytes retained_decoders + cuda_context_bytes
decoder_reserved_bytes = per_server_decoder_bytes if uses_pynv_static(static_media_io_kwargs, env_backend) else 0
return raw_frame_reserved_bytes + decoder_reserved_bytes
parser = argparse.ArgumentParser()
parser.add_argument("--repo", required=True, type=Path)
repo = parser.parse_args().repo.resolve()
media_video = read(repo, "vllm/multimodal/media/video.py")
connector = read(repo, "vllm/multimodal/media/connector.py")
chat_protocol = read(repo, "vllm/entrypoints/openai/chat_completion/protocol.py")
responses_protocol = read(repo, "vllm/entrypoints/openai/responses/protocol.py")
gpu_worker = read(repo, "vllm/v1/worker/gpu_worker.py")
video_core = read(repo, "vllm/multimodal/video.py")
assert "media_io_kwargs: dict[str, dict[str, Any]] | None = Field(" in chat_protocol
assert "media_io_kwargs: dict[str, dict[str, Any]] | None = Field(" in responses_protocol
assert 'video_io_kwargs = dict(self.media_io_kwargs.get("video", {}))' in connector
assert 'if "video_backend" not in video_io_kwargs and (' in connector
assert 'kwargs.pop("video_backend", None) or envs.VLLM_VIDEO_LOADER_BACKEND' in media_video
assert "elif backend == PYNVVIDEOCODEC_VIDEO_BACKEND:" in video_core
assert 'video_kwargs = mm_config.media_io_kwargs.get("video", {})' in gpu_worker
decoder_bytes = const_int(video_core, "PYNVVIDEOCODEC_DECODER_GPU_MEMORY_BYTES")
retained_decoders = const_int(video_core, "PYNVVIDEOCODEC_MAX_RETAINED_DECODERS")
cuda_context_bytes = const_int(video_core, "PYNVVIDEOCODEC_CUDA_CONTEXT_BYTES")
per_server_decoder_bytes = decoder_bytes * retained_decoders + cuda_context_bytes
vulnerable_static_reserved = reserve_bytes({}, "opencv", 0.0, decoder_bytes, cuda_context_bytes, retained_decoders)
negative_control_reserved = reserve_bytes({"video": {"video_backend": "pynvvideocodec"}}, "opencv", 0.0, decoder_bytes, cuda_context_bytes, retained_decoders)
raw_frame_only_control = reserve_bytes({}, "opencv", 0.25, decoder_bytes, cuda_context_bytes, retained_decoders)
head = subprocess.check_output(["git", "-C", str(repo), "rev-parse", "HEAD"], text=True).strip()
print(json.dumps({
"head": head,
"vulnerable": vulnerable_static_reserved == 0 and negative_control_reserved == per_server_decoder_bytes,
"reservation_simulation": {
"env_video_loader_backend": "opencv",
"request_selects_pynv_after_startup": True,
"vulnerable_static_reserved_bytes": vulnerable_static_reserved,
"negative_control_static_pynv_reserved_bytes": negative_control_reserved,
"raw_frame_only_control_reserved_bytes": raw_frame_only_control,
"unreserved_decoder_bytes_when_only_request_selects_pynv": per_server_decoder_bytes,
},
}, indent=2, sort_keys=True))
``
vLLM: Unauthenticated audio decompression-bomb DoS in /v1/chat/completions
Summary
The audio decode-duration guard (max_duration_s, env VLLM_MAX_AUDIO_DECODE_DURATION_S, default 600s) that protects against audio decompression-bomb DoS is wired into only the speech-to-text path (/v1/audio/transcriptions). The chat audio path (/v1/chat/completions, input_audio content parts) calls the same decoder with no limit, so an unauthenticated client can submit a few-KB compressed audio file that expands to multiple GB of float32 PCM at decode time, OOM-killing the worker. This is a distinct sibling of CVE-2026-5497 (video frame-count bomb, VideoMediaIO.load_base64) and GHSA-pq5c-rjhq-qp7p (image) in the same media subsystem.
Verified against main at HEAD d78650c (2026-06-16); applicable to the latest release v0.23.0.
Details
The guard rejects long audio during decode (before allocation), implemented in vllm/multimodal/media/audio.py:
load_audio_pyav — metadata reject (~82-98) and live sample-count reject (~129-136)
load_audio_soundfile — frames reject (~165-174)
All are gated on if max_duration_s is not None.
It is passed in exactly one place — the transcription serving layer:
``python
.../speech_to_text/base/serving.py:~170-174
load_audio(buf, sr=..., max_duration_s=self.max_audio_decode_duration_s)
self.max_audio_decode_duration_s = envs.VLLM_MAX_AUDIO_DECODE_DURATION_S (default 600)
`
The chat path never threads it:
`python
vllm/multimodal/media/audio.py:237-238
def load_bytes(self, data: bytes) -> tuple[npt.NDArray, float]:
return load_audio(BytesIO(data), sr=None) # no max_duration_s -> every guard above is skipped
`
Unauthenticated reachability chain (chat):
parse_input_audio (chat_utils.py) -> parse_audio -> connector.fetch_audio -> AudioMediaIO._load_data_url -> load_base64 -> load_bytes -> load_audio(..., sr=None). The connector never passes max_duration_s, and inline data: URLs need no HTTP fetch (so VLLM_AUDIO_FETCH_TIMEOUT does not bound them). The OpenAI-compatible server has no auth by default (auth only when --api-key / VLLM_API_KEY` is set).
Impact
Unauthenticated remote denial of service (availability) via memory amplification on a default-no-auth endpoint, on any deployment serving an audio-capable model. Same class and impact as the sibling CVE-2026-5497 (video). CWE-770 / CWE-409.
Fix
A fix was introduced in this MR: https://github.com/vllm-project/vllm/pull/45908
vLLM: SSRF + arbitrary local file read in MiMoV2OmniMultiModalProcessor `_fetch_image` and audio loader bypass MediaConnector protections
Summary
vllm/transformers_utils/processors/mimo_v2_omni.py — the multimodal processor for MiMoV2OmniForCausalLM — issues requests.get(...) directly on user-supplied image and audio URL strings and Image.open(...) on user-supplied local paths, without the SSRF / allowed_local_media_path checks that vllm.multimodal.utils.MediaConnector was hardened with in GHSA-qh4c-xf7m-gxfc, GHSA-v359-jj2v-j536, and GHSA-pf3h-qjgv-vcpr.
This is the same bug class as those three published advisories, in a code path the patches missed. When a user passes a URL or local-file string through multi_modal_data (e.g. LLM.generate(multi_modal_data={"image": "http://..."})), the processor takes the unsanitized string and dispatches it without any URL-scheme allowlist, network-target allowlist, size cap, or local-path allowlist.
Details
File: vllm/transformers_utils/processors/mimo_v2_omni.py (current main)
Sink 1 — image SSRF + local-file read (_fetch_image, lines 231–249):
``python
def _fetch_image(src: Any) -> Image.Image:
if isinstance(src, Image.Image):
return _to_rgb(src)
if isinstance(src, bytes):
return _to_rgb(copy.deepcopy(Image.open(BytesIO(src))))
if isinstance(src, str):
if src.startswith(("http://", "https://")):
r = requests.get(src, timeout=30) # SSRF: no allowlist, follows redirects
r.raise_for_status()
return _to_rgb(copy.deepcopy(Image.open(BytesIO(r.content))))
if src.startswith("file://"):
return _to_rgb(Image.open(src[7:])) # arbitrary local file read
if src.startswith("data:image"):
...
return _to_rgb(Image.open(src)) # fallback also opens local files
raise ValueError(f"Unrecognized image source: {type(src)}")
`
Sink 2 — audio SSRF (around line 471):
`python
elif audio.startswith(("http://", "https://")):
r = requests.get(audio, timeout=30) # SSRF: same pattern
r.raise_for_status()
file_obj = io.BytesIO(r.content)
`
Reachability. _fetch_image is invoked from MiMoVLProcessor.process_image:
`python
def process_image(self, image: ImageInput) -> torch.Tensor:
kw = self._resolve_img_kw(image)
src = image.image
if isinstance(src, (str, bytes)):
src = _fetch_image(src)
...
`
MiMoVLProcessor is wrapped by MiMoV2OmniMultiModalProcessor and registered for the MiMoV2OmniForCausalLM model architecture (vllm/model_executor/models/mimo_v2_omni.py:1169). Whenever a user passes a string into multi_modal_data["image"] (or ["audio"]) for this model, the unsanitized URL/path reaches the sink.
Comparison to the recent fixes. The remediation pattern adopted in the three earlier advisories was to route every external resource fetch through MediaConnector, which checks allowed_local_media_path and applies SSRF protection before issuing the network request. chat_utils.py (lines 838, 902, 924, 963, 1053, 1081) already uses self._connector.fetch_image / fetch_audio / fetch_video. The model processor in mimo_v2_omni.py was added later and skipped the connector — it calls requests.get and Image.open directly. Result: the public OpenAI chat-completion path is protected, but library use (LLM.generate(multi_modal_data=...)), batch processing, and any other path that lets a string reach the processor receive no protection.
Impact
1. SSRF — internal-network probing / cloud-metadata theft. Standard requests.get follows redirects and accepts any URL. An attacker who controls a multi_modal_data value can:
read AWS / GCP / Azure instance metadata (e.g. http://169.254.169.254/latest/meta-data/iam/security-credentials/),
probe internal services on the vLLM host (http://127.0.0.1:<port>, http://10.x.y.z),
exfiltrate via DNS / HTTP timing oracles even when the body is rejected by Image.open.
2. Arbitrary local file read via file://path (line 242) and the unguarded fallback Image.open(src) (line 248). Any file readable by the vLLM process is reachable through the model pipeline; with suitable formats this exposes /etc/passwd, ~/.aws/credentials, etc.
3. Server-side traffic generation / amplification by hammering arbitrary URLs from the vLLM host, with a 30-second timeout per request.
Suggested remediation
Replace direct requests.get and bare Image.open paths with MediaConnector.fetch_image / fetch_audio_async (or pass the inputs through MediaConnector before they reach the processor):
`python
vllm/transformers_utils/processors/mimo_v2_omni.py
from vllm.multimodal.utils import MediaConnector
_connector = MediaConnector()
def _fetch_image(src):
if isinstance(src, Image.Image):
return _to_rgb(src)
if isinstance(src, bytes):
return _to_rgb(copy.deepcopy(Image.open(BytesIO(src))))
if isinstance(src, str):
return _to_rgb(_connector.fetch_image(src)) # delegates to the hardened path
raise ValueError(f"Unrecognized image source: {type(src)}")
`
Same change for the audio loader at line 471. This re-uses the SSRF allowlist, allowed_local_media_path policy, and size caps that the previous patches added.
Alternative: forbid str src from reaching the processor and require all multi-modal pre-processing to go through chat_utils.py / MediaConnector before hitting the model. Larger surface change, but completes the architectural fix.
Discovery
Static review on vllm@main (HEAD as of 2026-04-30) — found by triaging the file list against the three recent SSRF advisories: the mimo_v2_omni.py processor, added after those fixes, reintroduced the same bypass class.
Reporter
Ievgen Bondarenko — sactransport2000@gmail.com — GitHub @ibondarenko1`
vLLM: Cross-User Data Leak Vulnerability
Summary
An integer overflow in the act_and_mul_kernel kernel can cause the output of one user request to be incorporated into the response of another request within the same inference batch. Under certain conditions, the last request in a batch can receive a partial or complete copy of the first user's inference result, resulting in cross-user data leakage.
Details
The root cause is an integer overflow in the expression blockIdx.x 2 d at https://github.com/vllm-project/vllm/blob/ff712f6447093d07747c88680b9d006b119f5890/csrc/activation_kernels.cu#L82.
As a result, the computation for one user (User A) can incorrectly consume input data from another user (User B). In particular, when 2^32 is divisible by d, the overflow can cause User A's output to contain portions of User B's inference result. In some cases, User B's response may be copied entirely into User A's response.
This constitutes a severe cross-user information disclosure vulnerability and is straightforward to trigger.
PoC
We reproduced the issue using meta-llama/Llama-3.2-1B-Instruct, for which d = 8192.
Using the following configuration:
Batch size: 17
Sequence length: 16384
The final response in the batch becomes an exact copy of the first response in the batch, demonstrating complete cross-user data leakage.
Impact
This vulnerability enables cross-user information disclosure. An attacker can intentionally craft requests that are processed within the same inference batch as a victim's request and cause the victim's inference output to be copied into the attacker's response.
As a result, sensitive information contained in another user's model response may be exposed to an unauthorized party.
Versions
For versions prior and equal to 0.21.0, the bug is in csrc/activation_kernels.cu, and for versions later than 0.21.0, the bug is in csrc/libtorch_stable/activation_kernels.cu.
vLLM: Incomplete CVE-2025-62164 remediation can be bypassed by concurrent prompt parts
Executive Summary
The follow-up protection for CVE-2025-62164 is incomplete at vLLM revision 26587f9519e22a5c4549ead7595ad9ca3229c4fd. It wraps serialized prompt-embedding reconstruction and dense conversion in torch.sparse.check_sparse_tensor_invariants(), but PyTorch 2.11.0 implements that context with save/enable/restore operations over process-global state. Two prompt-embedding parts in one /v1/chat/completions request are gathered concurrently on the event loop's default executor. When one context exits before the other loads its tensor, it can restore the global flag to False while the second part remains inside its guard.
In a deterministic run against hash-verified source from the affected revision, the actual target loader rejected an invalid sparse payload as a negative control. The frozen chat tracker then scheduled benign and malicious parts on distinct asyncio_0 and asyncio_1 threads. The benign context exited, the malicious loader observed the invariant flag disabled, and torch.load(weights_only=True) reconstructed indices [[10], [10]] for a declared shape of [3, 3]. The run intercepted the target's to_dense() call before it operated on the invalid tensor.
This primary trigger requires --enable-prompt-embeds, which is default-off, but it does not require renderer_num_workers > 1, a multimodal model, or --enable-mm-embeds. API authentication is optional in the stock server: middleware is installed only when CLI or environment API keys are supplied.
The lab proves bypass of the follow-up guard, invalid sparse reconstruction, and guarded-sink reachability. Crash and memory-corruption consequences are conditional on the behavior documented by the published CVE.
Background
CVE-2025-62164 / GHSA-mrw7-hf4f-83pf concerns client-controlled serialized prompt_embeds reaching torch.load(weights_only=True) and an invalid sparse tensor reaching to_dense(). The advisory attributes memory corruption, denial of service, and potential code execution to that historical unsafe operation.
The remediation chronology matters for duplicate handling:
PR #27204, merge commit 58fab50d82838d5014f4a14d991fdb9352c9c84b on 2025-10-22, introduced the default-off enable_prompt_embeds gate. It did not add the sparse-invariant context.
Commit 84e23d103d3483f944780d0d42bcf0993fd27e3a on 2025-12-15, titled additional protection for CVE-2025-62164 (#30649), added the process-global sparse-invariant context around load, type check, and dense conversion.
Refactor commit f0a1c8453ad1c664c8a04c83fe545195fcd556eb on 2026-01-31 moved the guarded loader into vllm/renderers/embed_utils.py while preserving the same context.
Chat content-part commit 14043dfecd35dd2f12b4d51eb9fa166184a0ca0f on 2026-05-01 introduced prompt_embeds chat parts and the concurrent one-request schedule described here.
This report therefore does not present the malformed sparse payload or to_dense() sink as new. It reports a distinct concurrency root cause and trigger: unsynchronized save/enable/restore of the process-global follow-up guard, reachable through the later multi-part chat scheduler.
The affected revision pins PyTorch 2.11.0 in pyproject.toml:10.
Vulnerability Details
The target's safe_load_prompt_embeds performs the guarded operation in vllm/renderers/embed_utils.py:16-39:
``python
with torch.sparse.check_sparse_tensor_invariants():
tensor = torch.load(
BytesIO(pybase64.b64decode(embed, validate=True)),
weights_only=True,
map_location=torch.device("cpu"),
)
if not isinstance(tensor, torch.Tensor):
raise VLLMValidationError(...)
tensor = tensor.to_dense()
`
The context is not request-local. With the global flag initially disabled, we can describe the verified interleaving:
1. Benign part A enters, saves False, and enables the flag.
2. Malicious part B enters, saves True, and leaves the flag enabled.
3. A completes its load and exits, restoring its saved False value.
4. B remains lexically inside its context but observes the actual global flag as False.
5. B's torch.load(..., weights_only=True) reconstructs the malformed sparse tensor.
6. The target reaches tensor.to_dense() before later rank, hidden-size, and dtype checks.
weights_only=True constrains deserialization types; it does not compensate for a sparse invariant check that another request has disabled.
The complete stock actor-to-sink chain, traced in the affected source, is:
POST /v1/chat/completions (vllm/entrypoints/openai/chat_completion/api_router.py:41-61) -> OpenAIServingChat.create_chat_completion -> _create_chat_completion -> render_chat_request (vllm/entrypoints/openai/chat_completion/serving.py:206-280) -> OnlineRenderer.render_chat (vllm/renderers/online_renderer.py:95-190) -> preprocess_chat (vllm/renderers/online_renderer.py:335-380) -> BaseRenderer.render_chat_async (vllm/renderers/base.py:1070-1105) -> HfRenderer.render_messages_async (vllm/renderers/hf.py:1049-1085) -> parse_chat_messages_async (vllm/entrypoints/chat_utils.py:1911-1945) -> content-part parse_prompt_embeds and _load_prompt_embeds_async (vllm/entrypoints/chat_utils.py:1099-1120) -> AsyncMultiModalItemTracker.resolve_items (vllm/entrypoints/chat_utils.py:818-835) -> asyncio.gather of both prompt parts -> safe_load_prompt_embeds_async -> make_async -> loop.run_in_executor(executor=None, ...) (vllm/utils/async_utils.py:28-45) -> guarded torch.load -> to_dense().
The prompt async helper is created without an explicit executor, so it uses the event loop's default executor. This path is separate from the renderer's configurable pool. The deterministic scheduler run observed the two parts on distinct default-executor threads while leaving renderer_num_workers at its default of one.
prompt_embeds bypasses multimodal processing, and the tracker explicitly permits it when is_multimodal_model=False (vllm/entrypoints/chat_utils.py:793-837). Consequently, the primary trigger needs neither a multimodal model nor enable_mm_embeds.
The source also states that async wrappers must be thread-safe (vllm/utils/async_utils.py:28-38), while a target test acknowledges that the sparse flag is not thread-local and concurrent users can leak state (tests/renderers/test_sparse_tensor_validation.py:58-61).
Exploitability Analysis
The following evidence labels separate what was demonstrated from what remains conditional:
| Label | Claim |
| --- | --- |
| Verified by run | PyTorch 2.11.0 rejects the identical invalid payload through the actual target loader without the race. |
| Verified by run | The hash-verified frozen tracker schedules two prompt parts on distinct default-executor threads, races the flag to False, reconstructs the invalid sparse tensor, and reaches the target to_dense() call while the interception prevents execution. |
| Traced in source | A client can supply multiple prompt_embeds content parts through the stock /v1/chat/completions route and the function chain above. |
| Traced in source | enable_prompt_embeds defaults to False (vllm/config/model.py:255-260), so the operator must opt in. enable_mm_embeds and non-default renderer workers are not preconditions for this path. |
| Traced in source | api_key defaults to None (vllm/entrypoints/openai/cli_args.py:264), and authentication middleware is installed only when a CLI or environment key is present (vllm/entrypoints/openai/api_server.py:306-310). With a configured key, the attacker must authenticate; without one, the stock route has no API-key middleware. |
| Unrun | A live HTTP/GPU server, real-world race win rate, unsafe dense conversion, process crash, memory corruption, and reliable code execution. |
The feature is documented for trusted users, which narrows intended exposure. It is not a memory-safety boundary: a user authorized to submit embedding inputs should not be able to disable a process-wide invariant for concurrent work.
The current run proves the same invalid sparse object can cross the guard and reach the historical sink. If executing that sink retains the behavior described in CVE-2025-62164 for the deployed PyTorch build, denial of service or memory corruption may follow. This is a conditional impact statement, not a reproduced outcome. Reliable RCE is not claimed.
The opt-in feature, scheduling requirement, and absence of a measured live win rate support Medium/P2 despite the serious historical sink class. No additional deployment assumptions are required for the one-request scheduler beyond stock default-executor concurrency being available.
Remediation
The immediate fix is one shared process-wide lock around every use of this process-global sparse guard. The lock must cover invariant enabling, deserialization, tensor type validation, and dense conversion:
`python
with shared_sparse_load_lock:
with torch.sparse.check_sparse_tensor_invariants():
tensor = torch.load(..., weights_only=True, map_location="cpu")
validate_tensor_type(tensor)
tensor = tensor.to_dense()
`
Every prompt, image, and audio loader that manipulates the same global flag must use the same lock. A lock only around torch.load, separate per-loader locks, or a lock omitted from the chat helper would leave overlapping save/restore sequences possible.
The stronger design is to avoid mutable process-global validation state in concurrent request code. Prefer a PyTorch per-call invariant check if one is available, or reconstruct and validate serialized embeddings inside a deliberately serialized boundary before any sparse operation.
Regression coverage should:
Preserve the actual-target negative control using the identical malformed payload.
Force A-enter, B-enter, A-exit, B-load and assert B remains protected.
Execute the multi-part chat tracker with the event loop's default executor and renderer_num_workers=1.
Cover cross-loader overlap so later prompt, image, or audio changes cannot bypass a shared fix.
Assert the global flag is restored after success and exceptions.
Reject invalid tensors before any dense conversion.
Until a fix is deployed, leaving enable_prompt_embeds disabled removes this stock source path.
Summary
The affected vLLM revision uses a process-global PyTorch context as the follow-up protection for CVE-2025-62164. A later chat feature causes two prompt-embedding parts from one request to run concurrently on the default executor. One context can restore the flag to False while the other is still guarded, allowing the historical malformed sparse payload class to reach the historical to_dense()` sink. The new issue is the concurrent guard bypass and shipped trigger, not the payload or sink. Runtime validation proves the bypass and safe sink reachability on PyTorch 2.11.0; historical crash and memory-corruption effects remain conditional, and RCE was not tested or claimed.
vLLM: ReDoS via structured_outputs.regex in the lm-format-enforcer backend (no compile timeout) — missed sibling of GHSA-rwxx-mrjm-wc2m
Summary
The fix for GHSA-rwxx-mrjm-wc2m ("ReDoS via structured_outputs.regex compiled without timeout") wrapped the regex compile in the xgrammar and outlines backends with compile_regex_with_timeout (and, for outlines, validate_regex_is_buildable). The lm-format-enforcer backend was left unguarded: it compiles the attacker-supplied regex with no timeout and no buildability check. A single request with a catastrophic regex hangs the structured-output compile step and stalls the engine worker (denial of service).
Affected code (HEAD d6d39c1)
vllm/v1/structured_output/backend_lm_format_enforcer.py:
line 110: character_level_parser = lmformatenforcer.RegexParser(grammar_spec) — builds an interegular FSM from the attacker regex synchronously, no timeout.
line 155: validate_structured_output_request_lm_format_enforcer returns immediately on if so_params.regex: — no validation.
Sibling backends that WERE patched by GHSA-rwxx:
backend_xgrammar.py:92 → compile_regex_with_timeout(...).
backend_outlines.py:65 → compile_regex_with_timeout(...) (plus validate_regex_is_buildable).
lm-format-enforcer uses the same interegular DFA-construction primitive the advisory cites for the outlines backend.
Reproduction (runtime-verified against the sink)
The sink lmformatenforcer.RegexParser(<regex>) was exercised directly (this is exactly what the backend calls):
``
baseline '[0-9]{3}' -> 0.0002 s
attacker '(a{1,300}){300}' -> DID NOT COMPLETE in 20 s (one core pegged at 100% in interegular FSM construction)
`
End-to-end: start vllm serve <model> --structured-outputs-config '{"backend":"lm-format-enforcer"}', then POST /v1/completions with {"structured_outputs":{"regex":"(a{1,300}){300}"}, ...}. The request never returns; because grammar compile runs in the engine's structured-output path, concurrent requests stall = worker-level DoS. The identical request against the outlines backend is bounded by compile_regex_with_timeout and returns a clean error.
Impact
Unauthenticated denial of service (vLLM ships with no authentication by default). One request pegs a CPU core and blocks the structured-output engine path.
Reachability precondition: the operator must have selected backend=lm-format-enforcer via --structured-outputs-config (the default is auto → xgrammar). This is the same opt-in tier as the outlines backend that GHSA-rwxx already covered.
Suggested remediation
Route the lm-format-enforcer regex compile (backend_lm_format_enforcer.py:110) through the same compile_regex_with_timeout guard already applied to the xgrammar and outlines backends, and reject un-buildable / oversized patterns in validate_structured_output_request_lm_format_enforcer`.
vLLM: Unauthenticated Internal Path and Username Disclosure via Validation Error Messages
Summary
When the vLLM API receives a malformed request (e.g., invalid JSON or missing required fields), FastAPI raises a Pydantic RequestValidationError. The validation_exception_handler in vllm/entrypoints/openai/server_utils.py converts this exception to a string via str(exc), which includes the internal file path and line number of the handler function. The existing sanitize_message() function in vllm/entrypoints/utils.py strips memory addresses (e.g., 0x7f...) but does not strip File "...", line X patterns. The result is a user-facing HTTP response that leaks internal system information.
Impact
An unauthenticated attacker can extract the following with a single malformed request:
OS username running the vLLM process (e.g., ubuntu)
Home directory path (e.g., /home/ubuntu/)
Virtual environment path (e.g., vllm-env/)
Python version (e.g., 3.12)
Internal package structure and line numbers (e.g., vllm/entrypoints/openai/chat_completion/api_router.py)
Handler function names per endpoint, enabling precise version fingerprinting
This information aids attackers in constructing targeted exploits: environment paths narrow the attack surface, and handler function names + line numbers enable exact version identification even when the /version endpoint is disabled.
All POST endpoints that accept JSON bodies are affected, including /v1/chat/completions, /v1/completions, /tokenize, and /detokenize.
Workarounds
Deploying vLLM behind a reverse proxy that rewrites error response bodies to strip file paths would mitigate this, though it is fragile.
Remediation Recommendation
Two possible fixes (either suffices):
Option A — Fix validation_exception_handler: Construct the error message from exc.errors() (the structured Pydantic error list) rather than str(exc). This avoids the traceback-style string entirely.
Option B — Fix sanitize_message: Add a regex to strip File "...", line \d+ patterns, similar to how memory addresses are already stripped:
``python
import re
msg = re.sub(r'File ".*?", line \d+, in \w+', '[internal]', msg)
``
Option A is preferred as it addresses the root cause rather than filtering symptoms.
Environment Tested
vLLM 0.20.1 (pip install, latest stable as of May 2026)
Python 3.12
Ubuntu 22.04
Model: Qwen/Qwen2-0.5B (text-only; bug is model-independent)
This was fixed here: https://github.com/vllm-project/vllm/commit/e87521626f
vLLM: Derender endpoints decode caller-supplied GenerateResponse token IDs without output bounds
Summary
The /v1/completions/derender and /v1/chat/completions/derender endpoints accept caller-supplied GenerateResponse objects and postprocess every nested choices[].token_ids list directly. Unlike the normal render/generate path, derender does not enforce model context length, resolved max_tokens, max_num_seqs, choice-count, or response-size bounds before detokenizing and returning the supplied token IDs. An authenticated API client can therefore make the CPU-only render frontend, or any server exposing these /v1 derender routes, spend CPU and memory proportional to attacker-chosen generated-output-shaped JSON rather than to a bounded generation result.
Technical Details
The render router registers /v1/chat/completions/derender and /v1/completions/derender in vllm/entrypoints/serve/render/api_router.py, and the OpenAI API server attaches this router whenever "generate" or "render" is in supported_tasks (vllm/entrypoints/openai/api_server.py). The routes are under /v1, so they are part of the OpenAI-compatible HTTP API surface and are protected by the API-key middleware when --api-key is configured.
The request types trust generated-output-shaped data from the client. In vllm/entrypoints/serve/disagg/protocol.py, GenerateResponseChoice accepts token_ids: list[int] | None = None, GenerateResponse accepts choices: list[GenerateResponseChoice], and DerenderCompletionRequest accepts generate_responses: list[GenerateResponse]. These fields have no max length, max item count, or relationship to a prior GenerateRequest.
The sink is OnlineDerenderer. derender_completion() iterates every supplied generate_responses entry and every nested choice, calls tokenizer.decode(choice.token_ids, skip_special_tokens=True), appends the decoded text to the response choices, and increments total_completion_tokens from the same supplied list length. derender_chat() has the same shape for a single supplied generate_response, and can also feed the decoded text into tool/reasoning parsers when a parser and chat_request are present. ServingRender.derender_completion_response() calls online_derenderer.derender_completion(request.generate_responses, request.prompt_tokens) before applying any completion-level validation beyond the model check.
Normal render and generation paths derive output limits from max_model_len, the rendered prompt length, request max_tokens / max_completion_tokens, and scheduler limits. Derender bypasses that invariant because it accepts the already-generated output shape directly from the HTTP caller. The missing invariant is: derender should only postprocess bounded generated output, and client-supplied derender payloads must be rejected if their nested generated token/logprob structures exceed the same limits that generation would have enforced.
PoV
The following bounded PoV can be run from a current vLLM checkout containing PR #43606. It asserts the current source facts for the derender routes, unchecked request fields, and decode sink, then simulates the same derender loop with a counting tokenizer. The negative control is a one-choice, 32-token response. The amplified payload keeps the test bounded but demonstrates that all decoded work and returned text scale directly with caller-supplied GenerateResponse contents.
``python
#!/usr/bin/env python3
import subprocess
from dataclasses import dataclass
from pathlib import Path
SOURCE = Path(".")
def require_source_fact(path: str, needles: list[str]) -> None:
text = (SOURCE / path).read_text()
missing = [needle for needle in needles if needle not in text]
if missing:
raise AssertionError(f"{path} missing expected facts: {missing}")
def source_head() -> str:
return subprocess.check_output(["git", "rev-parse", "HEAD"], cwd=SOURCE, text=True).strip()
@dataclass
class Choice:
index: int
token_ids: list[int]
@dataclass
class GenerateResponse:
request_id: str
choices: list[Choice]
class CountingTokenizer:
def __init__(self) -> None:
self.decode_calls = 0
self.decoded_ids = 0
def decode(self, token_ids: list[int], , skip_special_tokens: bool = True) -> str:
self.decode_calls += 1
self.decoded_ids += len(token_ids)
return "x" len(token_ids)
def derender_completion_like_current_head(generate_responses: list[GenerateResponse], tokenizer: CountingTokenizer) -> tuple[int, int, int]:
output_chars = 0
choices = 0
total_completion_tokens = 0
for gen in generate_responses:
for choice in gen.choices:
if not choice.token_ids:
raise ValueError("choice has empty or null token_ids")
decoded_text = tokenizer.decode(choice.token_ids, skip_special_tokens=True)
output_chars += len(decoded_text)
total_completion_tokens += len(choice.token_ids)
choices += 1
return choices, total_completion_tokens, output_chars
def make_payload(responses: int, choices_per_response: int, tokens_per_choice: int) -> list[GenerateResponse]:
token_ids = [42] tokens_per_choice
return [GenerateResponse(request_id=f"gen-{r}", choices=[Choice(index=c, token_ids=list(token_ids)) for c in range(choices_per_response)]) for r in range(responses)]
def run_case(name: str, payload: list[GenerateResponse]) -> None:
tokenizer = CountingTokenizer()
choices, completion_tokens, output_chars = derender_completion_like_current_head(payload, tokenizer)
print(f"{name}: responses={len(payload)} choices={choices} decode_calls={tokenizer.decode_calls} decoded_token_ids={tokenizer.decoded_ids} completion_tokens={completion_tokens} output_chars={output_chars}")
require_source_fact("vllm/entrypoints/serve/render/api_router.py", ['"/v1/completions/derender"', '"/v1/chat/completions/derender"', "app.include_router(router)"])
require_source_fact("vllm/entrypoints/serve/disagg/protocol.py", ["class GenerateResponseChoice(BaseModel):", "token_ids: list[int] | None = None", "class GenerateResponse(BaseModel):", "choices: list[GenerateResponseChoice]", "class DerenderCompletionRequest(BaseModel):", "generate_responses: list[GenerateResponse]"])
require_source_fact("vllm/renderers/online_derenderer.py", ["async def derender_completion(", "for gen, pt in zip(generate_responses, prompt_tokens_list):", "for choice in gen.choices:", "decoded_text = tokenizer.decode(", "total_completion_tokens += len(choice.token_ids)"])
print("source_checks=ok")
print(f"source_head={source_head()}")
run_case("negative_control", make_payload(responses=1, choices_per_response=1, tokens_per_choice=32))
run_case("amplified_payload", make_payload(responses=16, choices_per_response=4, tokens_per_choice=8192))
print("observation=derender decodes every caller-supplied token id before any max_model_len, max_tokens, max_num_seqs, or response-size check")
`
Impact
An attacker with access to the /v1 API can send derender requests that consume CPU and memory in the frontend/postprocessing process and can cause large responses unrelated to any bounded generation. In disaggregated deployments, this affects the CPU-only render frontend; in servers where the render router is attached alongside generation, it affects the same OpenAI-compatible server process that handles normal client traffic. This can degrade availability for other clients sharing the process.
Likely CWE: CWE-400 (Uncontrolled Resource Consumption) / CWE-770 (Allocation of Resources Without Limits or Throttling). Conservative CVSS v3.1: CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:L (4.3). This is not Low severity because a regular network API client can induce availability impact in a shared service without local access, invalid model artifacts, or special runtime privileges. If the server is deployed without API-key enforcement for /v1, the privileges component becomes PR:N.
Suggested Fix
Validate derender payloads before any detokenization or parser invocation. Apply bounded limits to generate_response(s), choices, token_ids, prompt_logprobs, logprobs.content, top_logprobs, and routed_experts that are at least as strict as the corresponding generation-side limits. For completions, reject generate_responses counts above the number of prompts that /v1/completions/render would have produced, and reject total nested choice counts above max_num_seqs / n limits. For each choice, reject token_ids longer than the resolved output-token budget, or require derender callers to submit the original bounded GenerateRequest / sampling metadata and validate the GenerateResponse against it before decoding.
Add regression tests for both derender endpoints. The tests should show that a normal bounded derender payload succeeds, while oversized generate_responses, oversized choices, oversized token_ids, and oversized logprob/top-logprob structures are rejected before tokenizer.decode() or parser execution.
Affected Package/Versions
Confirmed affected: current main at ddd3855a28a561a5bb54d380c6e6b8b1e883cc4a and downstream/nightly builds that include the derender endpoints introduced by PR #43606. The derender router, request models, decode sink, render serving bridge, and OpenAI API router attachment have no relevant diff from 00e045b7c7b82599f626779e111233abd4d0a64e to ddd3855a28a561a5bb54d380c6e6b8b1e883cc4a.
Latest release checked: v0.23.0, published on 2026-06-15. Its vllm/entrypoints/serve/render/api_router.py does not expose /v1/completions/derender or /v1/chat/completions/derender, so v0.23.0 was not confirmed affected.
Advisory History
PR #43606 ("[Render] Add /derender endpoints for disaggregated postprocessing") introduced the derender endpoints on main. PR #44285 later refactored the render serving code, and current head still contains the unchecked derender flow.
Public issue search for derender GenerateResponse token_ids returned no reports. Public search for "/v1/completions/derender" returned the derender feature RFC #42729 and unrelated bugs, but no size-bound, DoS, or generated-output postprocessing issue.
Related public request-fanout and resource-bound advisories are distinct:
GHSA-3mwp-wvh9-7528 covers an unbounded n parameter on the normal OpenAI completion/chat generation routes. Its root cause is missing upper-bound validation for generated sequence count, its sink is request fanout and request-object copying into the async engine path before scheduling, its precondition is a caller-controlled n, and its fix surface is a cap on generated sequence count. This report reaches /v1/completions/derender and /v1/chat/completions/derender, not the normal generate routes; its root cause is unchecked caller-supplied GenerateResponse / choices / token_ids structures, its sink is OnlineDerenderer detokenization and response construction after generation, its precondition is access to the derender API with generated-output-shaped JSON, and its fix surface is derender payload validation before decode.
PR #45390 includes the GHSA-83mh-6mwq-3hg9 batch-message fanout fix class: it bounds the outer BatchChatCompletionRequest.messages conversation list to prevent one request from creating many conversation/request objects before normal generation. This report has no batch conversation list and does not rely on n; one derender request can instead supply oversized nested GenerateResponse choices and token IDs that are detokenized and returned directly. A batch-message max_length limit would not bound derender generate_response(s) or per-choice token/logprob structures.
The completed local report titled "Explicit truncation_side disables tokenizer-level prompt truncation" is also distinct. That report used /v1/completions and /v1/chat/completions with ordinary prompt text plus truncate_prompt_tokens and explicit truncation_side; its root cause was the renderer omitting tokenizer-level max_length and the pre-tokenization character guard before post-token slicing; its sink was prompt tokenization; and its fix surface was preserving tokenizer-level truncation or rejecting over-budget prompts before tokenization. This derender report uses /v1 derender routes, has no prompt text tokenization or truncation-side control, starts from caller-supplied generated-output token IDs, and needs aggregate bounds on derender generate_response(s), choices, token IDs, logprobs, parser inputs, and response construction before detokenization.
Other adjacent vLLM advisories for Rust/gRPC token-id and logprob bounds, structured-output grammar amplification, repetition-detection windows, and pooling/rerank batch fanout are distinct. Those issues affect Rust/gRPC request conversion, grammar compilation, scheduler loops, or engine fanout. This issue affects /v1 derender postprocessing of caller-supplied generated-output objects and requires derender-specific request validation before detokenization.
Resources
vllm/entrypoints/serve/render/api_router.py
vllm/entrypoints/serve/disagg/protocol.py
vllm/renderers/online_derenderer.py
vllm/entrypoints/serve/render/serving.py
vllm/entrypoints/openai/api_server.py
PR #43606: https://github.com/vllm-project/vllm/pull/43606
PR #44285: https://github.com/vllm-project/vllm/pull/44285
GHSA-3mwp-wvh9-7528: https://github.com/vllm-project/vllm/security/advisories/GHSA-3mwp-wvh9-7528
PR #45390: https://github.com/vllm-project/vllm/pull/45390
Release v0.23.0: https://github.com/vllm-project/vllm/releases/tag/v0.23.0`
vLLM: Completion prompt lists fan out into unbounded engine requests
Summary
The /v1/completions request model accepts prompt as a list of text prompts or a list of token-id prompts without any outer prompt-count bound. The serving path turns each element into a separate engine input, creates one engine generator per element, merges all generators, and allocates a response slot per prompt. An authenticated API client can therefore turn one request into an attacker-chosen number of backend subrequests before any aggregate request-count budget is enforced.
Technical Details
CompletionRequest.prompt allows both list-shaped prompt inputs and scalar prompts:
``python
vllm/entrypoints/openai/completion/protocol.py
prompt: (
list[Annotated[int, Field(ge=0)]]
| list[list[Annotated[int, Field(ge=0)]]]
| str
| list[str]
| None
) = None
`
The validator only requires some prompt-like input to be present:
`python
def validate_prompt_and_prompt_embeds(cls, data):
prompt = data.get("prompt")
prompt_embeds = data.get("prompt_embeds")
...
if prompt_is_empty and embeds_is_empty:
raise VLLMValidationError(...)
`
The renderer then expands list-shaped prompts as a sequence. prompt_to_seq() wraps a scalar string or a single token-id list, but returns a list[str] or list[list[int]] unchanged:
`python
vllm/renderers/inputs/preprocess.py
def prompt_to_seq(prompt_or_prompts):
if isinstance(prompt_or_prompts, (dict, str, bytes)) or (
len(prompt_or_prompts) > 0 and is_list_of(prompt_or_prompts, int)
):
return [prompt_or_prompts]
return prompt_or_prompts
`
OnlineRenderer.preprocess_completion() appends that whole sequence, and the renderer processes every element:
`python
vllm/renderers/online_renderer.py
prompts = list[SingletonPrompt | bytes]()
if prompt_input is not None:
prompts.extend(prompt_to_seq(prompt_input))
...
parsed_prompts = [
prompt if isinstance(prompt, bytes) else parse_model_prompt(model_config, prompt)
for prompt in prompts
]
return await renderer.render_cmpl_async(parsed_prompts, tok_params, ...)
`
Finally, completion serving creates one backend generator and one response slot per rendered prompt:
`python
vllm/entrypoints/openai/completion/serving.py
generators: list[AsyncGenerator[RequestOutput, None]] = []
for i, engine_input in enumerate(engine_inputs):
...
generator = self.engine_client.generate(...)
generators.append(generator)
result_generator = merge_async_iterators(generators)
num_prompts = len(engine_inputs)
...
final_res_batch: list[RequestOutput | None] = [None] num_prompts
`
The violated invariant is that one HTTP request should have a bounded backend request count. Current code enforces per-prompt token and sampling limits, but not the number of prompts in the outer completion request.
PoV
A minimal oversized request keeps normal generation parameters small but supplies a large outer prompt list:
`json
{
"model": "served-model",
"prompt": ["x", "x", "x"],
"max_tokens": 1,
"n": 1
}
`
Scaling the prompt array to tens or hundreds of thousands of short entries makes the server allocate, preprocess, schedule, merge, and buffer one subrequest per entry. The same applies to token-id prompt lists:
`json
{
"model": "served-model",
"prompt": [[1], [1], [1]],
"max_tokens": 1,
"n": 1
}
`
The intended negative control is a scalar prompt:
`json
{
"model": "served-model",
"prompt": "x",
"max_tokens": 1,
"n": 1
}
`
The scalar string is wrapped as one prompt; the list form is not bounded and fans out by list length.
Impact
An authenticated API client can make one /v1/completions request consume CPU, memory, async task scheduling, engine request slots, and response buffering proportional to an attacker-chosen outer prompt list. This can starve or disrupt other tenants sharing the same vLLM server. The report does not claim unauthenticated access, confidentiality impact, integrity impact, code execution, or impact where /v1/completions is not reachable by untrusted or semi-trusted clients.
Suggested Fix
Reject oversized prompt lists before renderer preprocessing. Add an outer prompt-count limit to CompletionRequest.prompt when the prompt is list[str] or list[list[int]], and consider making the limit configurable in the same style as the batch-chat and sampling-list bounds. The check should run before OnlineRenderer.preprocess_completion() expands the prompt sequence, so oversized requests do not allocate parsed prompt lists, async render/tokenization tasks, engine generators, or response result slots.
Regression coverage should include a scalar prompt, a bounded prompt list, an oversized list[str], and an oversized list[list[int]]. The oversized requests should fail with a controlled validation error before any backend generator is created.
Affected Package/Versions
Package ecosystem: pip
Package name: vllm
Affected range confirmed by source proof: >=0.19.0, <=0.24.0; current main at cbe9c40f998f13975b967773ac7e7920e115387f remains affected.
Patched versions: unknown.
Latest release checked: v0.24.0, published on 2026-06-29.
GitHub Advisory Metadata
Package ecosystem: pip
Package name: vllm
Vulnerable version range: >=0.19.0, <=0.24.0
Patched versions: unknown
Advisory History
Public issue and PR searches for CompletionRequest prompt list, "prompt" "list[str]" "completion", and "CompletionRequest" "max_length" did not find an existing report or fix for this exact path.
The closest published advisory is GHSA-3mwp-wvh9-7528, "OOM Denial of Service via Unbounded n Parameter in OpenAI API Server", patched in 0.19.0. This report is distinct because it keeps n=1 and uses the /v1/completions prompt outer list to create one engine request per prompt. The fix invariant is an outer prompt-count and aggregate request budget, not only a generated-sequence-count cap.
The closest public PR is vllm-project/vllm#45390, which covers multiple DoS fixes including GHSA-83mh-6mwq-3hg9 for BatchChatCompletionRequest.messages. That PR adds an outer bound to batch chat conversations, but its diff does not touch vllm/entrypoints/openai/completion/protocol.py or vllm/entrypoints/openai/completion/serving.py.
Prior local/private report families checked included pooling/rerank batch fanout, derender token-id postprocessing, explicit truncation_side` tokenizer-limit bypass, Python disaggregated generate prompt-length bypass, and priority scheduling. Those reports differ by endpoint, attacker-controlled field, sink, and fix surface.
Pydantic AI: Unvalidated UploadedFile references in UI adapters allow server-side file access using the application's credentials
Summary
A client that can submit message history to a Pydantic AI UI adapter can reference arbitrary files in the application's model-provider or cloud-storage account. The server forwards the reference to the model provider, which fetches it using the server's own credentials, allowing the client to read files it should not have access to.
Details
UI adapters reconstruct file parts from client-submitted message history and forward them to the model provider. File URL parts are validated against a scheme allowlist before being forwarded, but UploadedFile references — which point to a file by provider file ID or cloud-storage URI (e.g. s3://…, gs://…) — were forwarded without validation.
Because the provider resolves an UploadedFile using the server-side identity (IAM role, service account, or provider API key) rather than the client's, a client that crafts message history containing an attacker-chosen UploadedFile can cause the server to read objects belonging to its own account or to other tenants, given a referenceable identifier.
Impact
Applications that pass untrusted client-submitted message history to an agent through a UI adapter (such as the Vercel AI adapter). Exploitation requires the attacker to reference a valid file identifier; depending on how the application names objects, such identifiers are not always unguessable.
Patches
Upgrade to 1.106.0 (1.x) or 2.0.0b6 (the 2.x beta line), which validate UploadedFile references on client-submitted messages the same way file URLs are validated.
Workarounds
If users cannot upgrade, do not pass untrusted client-submitted message history to the agent, or strip UploadedFile parts from incoming messages before running the agent.
Transformers save_pretrained path traversal allows arbitrary file writes through chat template names
A vulnerability in huggingface/transformers versions < 5.10.0 allows an attacker to perform arbitrary file writes via path traversal. The issue resides in the save_pretrained() methods of PreTrainedTokenizerBase and ProcessorMixin, where keys from the chat_template dictionary are used directly as filenames without proper validation. An attacker can exploit this by publishing a malicious Hugging Face Hub repository with a crafted tokenizer_config.json file. When a victim downloads and saves the tokenizer or processor, the attacker-controlled keys can escape the intended save directory, enabling arbitrary file writes with attacker-controlled content. This vulnerability affects multiple processors inheriting from ProcessorMixin, including Idefics, Florence, Gemma, Phi, and Qwen-VL.
vLLM denial of service via prompt embeds on M-RoPE models
Summary
_Short summary of the problem. Make the impact and severity as clear as possible. For example: An unsafe deserialization vulnerability allows any unauthenticated user to execute arbitrary code on the server._
Sending a pure prompt embeds payload in a /v1/completions request with a model using M-RoPE causes the EngineCore to fail an assertion and fatally crash, shutting down the entire server application.
Any remote user who is authorized to make a /v1/completions endpoint can trivially make such a request and induce a crash.
Details
_Give all details on the vulnerability. Pointing to the incriminated source code is very helpful for the maintainer._
In commit 56669c1, a simple assert intended to be a type-narrowing assert was added to the _init_mrope_positions method in GPUModelRunner (the offending line on main at the time of writing: https://github.com/vllm-project/vllm/blob/2d481f8a946ee0521872af0f098674a8ee01ce4a/vllm/v1/worker/gpu_model_runner.py#L1588-L1607).
``python
assert req_state.prompt_token_ids is not None, (
"M-RoPE requires prompt_token_ids to be available."
)
`
This type narrowing assert is to prevent mypy errors later in the function because None is not a valid type for mrope_model.get_mrope_input_positions. Unfortunately, this assertion is not always true. /v1/completions requests that specify prompt=None and prompt_embeds=<not none> will indeed create a CachedRequestState where prompt_token_ids is None. This triggers the assertion, which in turn crashes the EngineCore and the Server application.
`
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 3997, in execute_model
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] deferred_state_corrections_fn = self._update_states(scheduler_output)
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 1239, in _update_states
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] self._init_mrope_positions(req_state)
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 1582, in _init_mrope_positions
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] assert req_state.prompt_token_ids is not None, (
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=351) ERROR 06-11 00:48:03 [core.py:1167] AssertionError: M-RoPE requires prompt_token_ids to be available.
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] AsyncLLM output_handler failed.
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] Traceback (most recent call last):
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async_llm.py", line 660, in output_handler
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] outputs = await engine_core.get_output_async()
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py", line 1030, in get_output_async
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] raise self._format_exception(outputs) from None
(APIServer pid=1) ERROR 06-11 00:48:03 [async_llm.py:704] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
`
All requests using the /v1/chat/completions endpoints will have text/prompt_token_ids parts (corresponding to the chat template), and prompt_embeds parts are handled as mm_features. This method (rightly) filters out those prompt_embeds content parts as they are treated as text positions.
A sufficient solution to type narrowing here without raising a fatal assertion is to instead replace the assertion with a using dummy token ids:
`python
def _init_mrope_positions(self, req_state: CachedRequestState):
model = self.get_model()
assert supports_mrope(model), "M-RoPE support is not implemented."
mrope_model = cast(SupportsMRoPE, model)
# Filter out prompt_embeds modality (text-only position info)
mrope_features = [
f for f in req_state.mm_features if f.modality != "prompt_embeds"
]
# Handle both token_ids and embeddings-only inputs
if req_state.prompt_token_ids is not None:
input_tokens = req_state.prompt_token_ids
elif req_state.prompt_embeds is not None:
# For text-only embeddings, dummy token IDs are safe since
# get_mrope_input_positions only uses len(input_tokens) when mm_features is empty
seq_len = req_state.prompt_embeds.shape[0]
input_tokens = list(range(seq_len))
# Verify no mm_features remain (should be true after prompt_embeds filter)
assert len(mrope_features) == 0, (
"M-RoPE with prompt_embeds-only input should have no multimodal features"
)
else:
raise ValueError(
"M-RoPE requires either prompt_token_ids or prompt_embeds."
)
req_state.mrope_positions, req_state.mrope_position_delta = (
mrope_model.get_mrope_input_positions(
input_tokens,
mrope_features,
)
)
`
Technically, in isolation, this method still crashes in the case where req_state.prompt_token_ids is None and req_state.mm_features, so the solution above still leaves that potential vector open. As far as can be determined, however, such a req_state is impossible in the first place in online mode, because it would require a /v1/completions request with prompt_embeds AND multimodal features, but the /v1/completions request schema does not expose multimodal inputs in any discernible way. Today, those are the only two endpoints with prompt_embeds support.
When in offline mode, it is technically possible to directly create an EngineCoreRequest that has prompt_embeds and not prompt_token_ids and mm_features, and pass that to LLM.generate. That would trigger this same assertion, and no validation would prevent that combination. It is strongly suspected, though, that this combination would be undefined in any model that support M-RoPE, because it would not be possible to determine which token positions correspond to mm_features. The proposed solution above would end up not setting req_state.mrope_positions and req_state.mrope_position_delta in this scenario, which could result in undefined behavior.
prompt_embeds is far more familiar here than M-RoPE, and it is understood that each model that supports it is responsible for defining its own get_mrope_input_positions which have varying implementations. There is insufficient knowledge to be prescriptive in how the two features should interact in the offline case, other than possibly raising a validation error earlier on preventing that combination (which would emulate the current assertion behavior). Regardless, in offline mode, the chances of a remote user being able to exploit this are slim-to-nil compared to the online case which is incredibly straightforward.
Impact
_What kind of vulnerability is it? Who is impacted?_
Denial of Service caused by an incorrect assertion inside of the GPUModelRunner which causes a fatal EngineCore exception
Any configuration with --enable-prompt-embeds and M-RoPE-supported model is vulnerable
The attack is extremely easy from the remote attacker's perspective (copying the official prompt_embeds` online mode docs examples almost-verbatim, accounting for model-name and connection details, of course, will induce a guaranteed shutdown)
vLLM: Speech-to-text upload size limit is enforced after full UploadFile read
Summary
Current-head vLLM documents VLLM_MAX_AUDIO_CLIP_FILESIZE_MB as the maximum audio file size accepted by the speech-to-text APIs. The default is 25 MB. vllm/envs.py also describes files larger than this value as rejected.
The /v1/audio/transcriptions and /v1/audio/translations routes call await request.file.read() before vLLM checks that limit. In FastAPI and Starlette, UploadFile.read() returns bytes from the uploaded file object; when called without a size argument, the route materializes the remaining file contents. vLLM then performs the compressed file-size check later in _preprocess_speech_to_text() against the already-created bytes object.
As a result, the documented compressed audio file-size limit does not bound the memory allocated by vLLM endpoint code before validation. An oversized multipart upload can cause vLLM to allocate memory proportional to the uploaded file size before rejecting the request as too large.
This is distinct from GHSA-6pr9-rp53-2pmc, which covered decoded PCM expansion after compressed input was accepted. This report covers compressed upload materialization before compressed-size validation.
Technical Details
The upload routes perform an unbounded read before vLLM checks the documented compressed audio file-size limit:
vllm/entrypoints/speech_to_text/transcription/api_router.py: audio_data = await request.file.read()
vllm/entrypoints/speech_to_text/translation/api_router.py: audio_data = await request.file.read()
vllm/entrypoints/speech_to_text/base/serving.py later checks: if len(audio_data) / 1024**2 > self.max_audio_filesize_mb
There is no route-level check of request.file.size, Content-Length, a bounded read(max_bytes + 1), or a streaming copy that stops at the configured limit before the full file is materialized.
This does not appear to be intended behavior. vLLM's security guide treats request-controlled resource use as a security boundary: for example, requests that exceed VLLM_MAX_N_SEQUENCES are rejected before reaching the engine. The speech-to-text upload limit is documented in the same spirit as an API enforced limit, but the first vLLM check happens after the over-limit upload has already been copied into a bytes object.
Impact
Attack requirements:
the deployment exposes /v1/audio/transcriptions or /v1/audio/translations;
a speech-to-text capable model/task is configured; and
the caller can submit requests to the endpoint, including any API key the deployment requires.
An API caller who meets those requirements can send an oversized audio file. vLLM reads the full uploaded file into memory before applying the configured compressed audio file-size limit. This can create memory pressure or, depending on process/container limits and concurrency, terminate the process before the request is rejected.
Suggested severity: Moderate, CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H (6.5), CWE-770/CWE-400.
The impact is availability-only. This is not a claim of code execution, data access, cross-tenant data exposure, parser-level multipart memory exhaustion, or persistence after process restart. Deployment body-size limits at a reverse proxy or ASGI layer can mitigate the issue before vLLM sees the request, but vLLM's own documented file-size limit does not currently provide that memory boundary.
Suggested Fix
Enforce the compressed audio upload limit before the unbounded read:
Check reliable upload size metadata before reading when available.
Read at most max_bytes + 1 bytes in chunks as a defense-in-depth guard against missing or unreliable metadata.
Share the helper between transcription and translation routes.
Add regression tests that prove an over-limit UploadFile is rejected without calling an unbounded read().
The important property is that over-limit compressed uploads are rejected before vLLM allocates the full uploaded file as bytes.
Resources
vLLM security policy: https://github.com/vllm-project/vllm/security/policy
vLLM speech-to-text docs: https://docs.vllm.ai/en/latest/serving/online_serving/speech_to_text/
vLLM security guide, request parameter resource limits: https://docs.vllm.ai/en/latest/usage/security/
vLLM vulnerability management docs: https://docs.vllm.ai/en/latest/contributing/vulnerability_management/
FastAPI file uploads: https://fastapi.tiangolo.com/reference/uploadfile/
Starlette uploaded files: https://www.starlette.io/requests/
Adjacent published audio advisory: https://github.com/vllm-project/vllm/security/advisories/GHSA-6pr9-rp53-2pmc
Request-parameter resource DoS precedent: https://github.com/vllm-project/vllm/security/advisories/GHSA-3mwp-wvh9-7528
Appendix: Affected Version
Validated against current head:
commit: 1033ffac2eccf986fdd880f4dee64ca3b22c63c9
described version: v0.22.1rc0-491-g1033ffac2e
Known affected range: current head. It has not been determined the introducing commit or release range.
Appendix: Proof Of Vulnerability
The attached proof is a non-destructive static probe. It does not upload a large file or contact a running vLLM server:
``bash
python3 attached-evidence/poc/audio_upload_size_precheck_probe.py
`
Observed result:
`json
{
"pov": "this report",
"validated": true,
"default_limit_mb": 25,
"documented_api_limit": true,
"routes": {
"transcription_route": {
"unbounded_upload_read": true,
"early_size_guard_before_read": false,
"chunked_bounded_read": false
},
"translation_route": {
"unbounded_upload_read": true,
"early_size_guard_before_read": false,
"chunked_bounded_read": false
}
},
"late_size_check": {
"present": true
}
}
`
Expected behavior: vLLM rejects over-limit audio files before materializing the entire upload into memory in vLLM endpoint code.
Actual behavior: the route materializes the upload into memory first, and only then does vLLM reject the request as exceeding VLLM_MAX_AUDIO_CLIP_FILESIZE_MB`.
vLLM: ReDoS via structured_outputs.regex compiled without timeout in xgrammar and outlines backends
Summary
The structured_outputs.regex API parameter passes a user-supplied regex string directly to grammar compiler backends with no compilation timeout. In the xgrammar backend, the string reaches compile_regex() with no guard. In the outlines backend, validate_regex_is_buildable() blocks structural issues (lookarounds, backreferences) but provides zero protection against exponential DFA state-space explosion. Patterns like (a+)+b pass all checks and hang the inference worker.
Root Cause
backend_xgrammar.py:91 — no timeout:
``python
ctx = self.compiler.compile_regex(grammar_spec)
`
backend_outlines.py:299–330 — structural checks only, no complexity analysis:
`python
def validate_regex_is_buildable(regex: str) -> None:
sre_parse.parse(regex) # AST parse only — does not detect exponential patterns
_check_unsupported(...) # blocks lookarounds/backrefs, not nested quantifiers
`
backend_outlines.py:64 — no timeout:
`python
oc.Index(regex_string, vocabulary.inner)
`
Impact
Denial of service — one request with an adversarial regex pattern hangs an inference worker indefinitely.
Remediation
Wrap compile_regex() and oc.Index() calls in a thread with a deadline (e.g., 5 seconds). Add complexity analysis to validate_regex_is_buildable()` to detect nested quantifier patterns before compilation.
vLLM has Remote DoS via Invalid Recovered Token Reinjection
Summary
A frontend-legal multi-request speculative workload can make vLLM produce an out-of-vocabulary recovered token equal to vocab_size, convert that value to -1 when choosing the next live token for a request, and then feed that -1 back into the next drafter input ids. On Qwen3 GPTQ this reaches the worker-side drafting / attention path and crashes the engine with a GPU device-side assert.
The same issue is reachable through the public gRPC request surface by sending a specific overlapping Generate / Abort sequence.
Impact
A remote client that can send public gRPC generation requests can crash the
shared vLLM engine worker
The triggering request sequence aborts concurrent requests and prevents later
requests from completing until the worker is restarted
In shared deployments, this is a service-wide denial of service for other
clients, not just a failure isolated to the attacking requests
The failure is reproducible, so repeated request sequences can sustain the
outage
Affected version
Confirmed on vLLM 0.17.1
Earlier and later versions have not been checked yet in this report
Repro model
Official Hugging Face repo:
Qwen/Qwen3-0.6B-GPTQ-Int8
Anyone wants to reproduce the bug with my PoC scripts should download Qwen3-0.6B-GPTQ-Int8 first
Trigger chain
1. A legal multi-request speculative workload keeps structured-output state,
speculative decoding, overlap, and request cancellation active in the same
live engine.
2. During rejection sampling, vLLM produces a recovered token equal to the
model vocab_size boundary value.
3. That recovered token appears in position 0 of the sampled speculative row
for a live request. The same row also contains trailing padding entries
equal to -1, but those padding entries are not the key fault by
themselves.
4. The next-token preparation step treats the position-0 recovered token as the
real next token for that request and converts that out-of-vocabulary value
to -1.
5. The drafter writes that converted -1 back into the live next-step input-id
row for the request.
6. The drafting / embedding / attention path later consumes that live invalid
token and the worker crashes on GPU.
Details
Simple example
The important distinction is:
trailing -1 values in a speculative row can be ordinary padding
the bug appears when the first live token for a request becomes
151936 == vocab_size, and that live token is then converted into -1
In simplified form, the bad transition looks like this:
``text
sampled speculative row:
[151936, -1, -1, -1, ...]
`
At this point, the trailing -1 values are only padding. The critical problem
is that the first position holds 151936, which is out of vocabulary and is
being treated as the request's real next token.
Then vLLM prepares the next-token buffer:
`text
next_token_ids:
[-1, ...]
`
Finally, that converted -1 is written back into the live model input ids:
`text
input_ids_after:
[-1, 0, 0, 0, ...]
`
The crash happens because the live next token became -1 and was later consumed by the drafting / embedding / attention path, not merely because the speculative row contained padded -1 entries.
Trigger path in code
1. The workload is frontend-legal. The requests use normal SamplingParams
features such as structured outputs, stop, bad_words, min_tokens, and
streaming overlap. No malformed token-id list is required at the request
boundary.
2. In speculative decoding, the rejection sampler can generate recovered tokens
when drafted tokens are rejected.
`python
# vllm/v1/sample/rejection_sampler.py
def sample_recovered_tokens(...):
recovered_token_ids = torch.empty_like(draft_token_ids)
sample_recovered_tokens_kernel(batch_size, max_spec_len)
return recovered_token_ids
`
On the verified Qwen3 run, the recovered-token trace shows
recovered_token_ids[0] = 151936, which is exactly vocab_size for this
checkpoint.
3. The speculative proposer then prepares the next-token row from the sampled
speculative row.
`python
# vllm/v1/spec_decode/eagle.py
def prepare_next_token_ids_padded(...):
...
eagle_prepare_next_token_padded_kernelgrid
return next_token_ids, valid_sampled_tokens_count
`
In the verified trace, this step receives a sampled row beginning with
151936, followed by -1 padding. The important point is that 151936
occupies the first live token position for the request. This step then
produces next_token_ids[0] = -1, meaning the live next token for the
request has been converted to -1.
4. The drafter then rotates the draft input ids and inserts those
next_token_ids back into the live input-id buffer.
`python
# vllm/v1/spec_decode/eagle.py
def set_inputs_first_pass(...):
...
self.input_ids[token_indices_to_sample] = next_token_ids
`
In the verified trace, this produces input_ids_after[0] = -1.
5. The model-side embed path later consumes those input ids.
`python
# vllm/model_executor/models/qwen2.py
def embed_input_ids(self, input_ids: torch.Tensor) -> torch.Tensor:
return self.embed_tokens(input_ids)
`
In the verified trace, this is the first point where the converted -1
becomes visible as a real model input. The bug is not merely that the
sampled speculative row contained padding -1; the bug is that the live
next token for the request became -1 and was written back into input ids.
6. After that point, the visible sink depends on timing and backend state. On
the attached Qwen3 reproducer, the engine commonly dies later in the
drafting / attention path with CUDA error: device-side assert triggered,
for example under flash_attn_varlen_func(...).
Local script breakdown
repro_g4_recovered_minus1_local.py is a standalone local reproducer.
It reads the Qwen3 checkpoint path from VLLM_POC_G4_MODEL or the built-in
/path/to/qwen3 placeholder
It creates EngineCore directly without any external helper dependency
It submits one fixed multi-request workload that preserves the same overlap
and speculative-decoding state needed for the bug
It writes:
request_payloads.json
repro_config.json
timeline.json
responses.json
error.txt
recovered_chain_trace.jsonl
recovered_chain_trace.jsonl is the key attribution artifact. It records the
recovered-token chain directly from the standalone reproducer
gRPC script breakdown
repro_g4_recovered_minus1_grpc.py is a standalone public gRPC reproducer.
It reads the Qwen3 checkpoint path from VLLM_POC_G4_MODEL or the built-in
/path/to/qwen3 placeholder
It starts a temporary vllm.entrypoints.grpc_server process
It sends only public Generate and Abort RPCs
It submits one fixed overlapping request sequence that preserves the same
speculative-decoding state needed for the bug
After the crash window, it sends one more public Generate probe request to
confirm that later gRPC requests also fail after the worker dies
It writes:
request_payloads.json
timeline.json
server_command.json
responses.json
post_crash_probe.json
server.stdout.log
server.stderr.log
Observed result
Local repro typically ends with:
a recovered-token trace showing:
sample_recovered_tokens_return -> recovered_token_ids[0] = 151936
prepare_next_token_ids_padded -> next_token_ids[0] = -1
set_inputs_first_pass -> input_ids_after[0] = -1
embed_input_ids_out_of_range -> input_ids[0] = -1
CUDA error: device-side assert triggered
a fatal engine-side failure
gRPC repro typically ends with:
the triggering gRPC requests failing with
INTERNAL: EngineCore encountered an issue. See stack trace (above) for the root cause.
server logs showing the worker dies with
CUDA error: device-side assert triggered
a later public probe request also failing after the worker is dead
This demonstrates that the issue is reachable through the public gRPC request surface, not only through a local reproducer.
Log snippets
Local recovered-chain trace
`text
sample_recovered_tokens_return:
recovered_token_ids = [151936, ...]
vocab_size = 151936
prepare_next_token_ids_padded:
sampled_token_ids_head = [[151936, -1, -1, ...], ...]
next_token_ids = [-1, ...]
set_inputs_first_pass:
input_ids_after = [-1, 0, 0, 0, ...]
embed_input_ids_out_of_range:
input_ids = [-1, 0, 0, 0, ...]
`
gRPC server log
`text
torch.AcceleratorError: CUDA error: device-side assert triggered
...
vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
...
Error in Generate for request post_crash_probe
vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
`
Root cause
This is a speculative-decoding state-handling bug, not an invalid frontend token-id input bug.
The root cause is that a recovered speculative token can become equal to vocab_size, then be selected as the live next token for a request, then be converted to -1, and that converted -1 is still written back into live drafter input ids and later consumed by the drafting / embedding / attention path.
For the Qwen3 checkpoint used here:
151936 == vocab_size
This value should be described as the model vocab_size boundary value, not as a legal token id.
Attachments
The attached bundle for this report should contain:
repro_g4_recovered_minus1_local.py
repro_g4_recovered_minus1_grpc.py`
These two standalone scripts are sufficient to reproduce the issue and its public gRPC reachability.
Fix
A fix for this vulnerability has been merged in: https://github.com/vllm-project/vllm/pull/44744
vLLM: Processing differential in multi-channel audio downmixing enables hidden-input/moderation bypass for audio models
Issue Description
Librosa defaults to using numpy.mean for mono downmixing (to_mono), while the international standard ITU-R BS.775-4 specifies a weighted downmixing algorithm. This discrepancy results in:
Inconsistency between audio heard by humans (e.g., through headphones/regular speakers) and audio processed by AI models (Which infra via Librosa, such as vllm, transformer).
https://github.com/librosa/librosa/blob/af8c839fb15317fa2712ea66e7a22da6a9267b32/librosa/core/audio.py#L478
Attack Scenario and Impact
LFE (Low-Frequency Effects) Channel Exploit
Attackers can craft special multichannel audio files containing:
1. Normal content in front channels (L/R)
2. Either interference signals or hidden content in the LFE channel
Notice: It is worth noting that not only the LFE channel is excluded, but in fact, channels beyond the 6th (such as rear surround channels, overhead channels, height speakers, etc.) are also not supported.
Attack Methodology:
Attackers can create specially engineered multichannel audio with LFE interference, where front channels (L/R) contain normal content while the LFE channel carries interference signals or hidden content. When played on consumer devices that ignore LFE channels, only the normal content is heard. However, when processed by AI systems using Librosa (which mixes all channels), the LFE interference affects speech recognition feature extraction or masks critical detection features. This enables malicious content to bypass AI detection while still reaching end users, potentially compromising voice authentication systems, evading content moderation, or disrupting speech recognition accuracy.
Potential Exploitation Scenarios:
Voice authentication systems may be tricked into accepting anomalous audio
Content moderation systems may fail to detect prohibited content hidden in LFE channels
Speech recognition systems may produce incorrect transcriptions
Note: torch.audio implements this correctly. Failure to do so may lead to inconsistencies between training and test audio, resulting in performance degradation.
Resources
ITU-R BS.775-4 Standard
Librosa Source Code
Librosa securty report
Fixes
https://github.com/vllm-project/vllm/pull/37058, which removes the librosa dependency from vLLM.
pydantic-ai: SSRF blocklist bypass via IPv4-compatible, SIIT/IVI, and local NAT64 IPv6 addresses (incomplete fix of CVE-2026-46678)
Summary
When an application using Pydantic AI opts a URL into force_download='allow-local' (which disables the default block on private/internal IPs) and runs on a network that routes the affected IPv6 transition forms (NAT64- or ISATAP-configured networks), the cloud-metadata blocklist could be bypassed by encoding the metadata IP in an IPv6 transition form that the previous fix did not decode — IPv4-compatible IPv6 (::a.b.c.d), the NAT64 RFC 8215 local-use prefix (64:ff9b:1::/48), operator-chosen NAT64 prefixes, or ISATAP. The IPv6 wrapper is then delivered to the underlying IPv4 metadata endpoint, exposing cloud IAM short-term credentials.
The bypass is exploitable only in environments whose network actually routes these forms — NAT64-configured networks (IPv6-only or dual-stack-with-NAT64 deployments, including some Kubernetes setups) for the NAT64 variants, or networks with an ISATAP tunnel for ISATAP. A standard dual-stack cloud VM or container does not route them and is not affected in practice. The IPv4-compatible and Teredo variants are deprecated and addressed as defense-in-depth.
This is an incomplete fix of GHSA-cqp8-fcvh-x7r3 / CVE-2026-46678 (itself a follow-up to CVE-2026-25580). The prior remediation decoded only IPv4-mapped IPv6, 6to4, and the NAT64 well-known prefix; the metadata guarantee did not hold for the remaining transition forms.
Severity
MEDIUM — CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:C/C:H/I:N/A:N = 6.8
Same impact metrics and narrow attack surface as the parent advisory (AC:H): exploitation requires the application to have opted into allow-local on a URL influenced by untrusted input, and the NAT64/ISATAP variants additionally require the deployment network to route those forms.
CWE-918: Server-Side Request Forgery (SSRF)
Affected Versions
| Package | Vulnerable | Patched |
|---|---|---|
| pydantic-ai | >= 1.56.0, < 1.102.0; >= 2.0.0b1, < 2.0.0b3 | 1.102.0; 2.0.0b3 |
| pydantic-ai-slim | >= 1.56.0, < 1.102.0; >= 2.0.0b1, < 2.0.0b3 | 1.102.0; 2.0.0b3 |
These transition forms have not been decoded since SSRF protection was introduced in 1.56.0.
Who Is Affected
Users are affected only if their application explicitly opts a FileUrl (ImageUrl, AudioUrl, VideoUrl, DocumentUrl) into force_download='allow-local' on a URL that is, or could be, influenced by untrusted input.
Beyond that precondition, the affected encodings only reach a metadata endpoint in environments whose network actually routes them. The broadly-routable IPv4-mapped form was addressed in 1.99.0 (CVE-2026-46678); the additional forms addressed here require a NAT64-configured network (IPv6-only or dual-stack-with-NAT64 deployments, including some Kubernetes setups) for the NAT64 variants, or an ISATAP tunnel for the ISATAP variant. The IPv4-compatible and Teredo forms are deprecated and not routed by modern stacks; they are addressed as defense-in-depth. Most deployments on a standard dual-stack cloud VM or container are therefore not exploitable in practice, but the fix restores the "always blocked" guarantee for the environments that are.
Users are not affected if they use any of the bundled integrations to ingest user input, because they do not propagate force_download from external data:
Agent.to_web / clai web
VercelAIAdapter
AGUIAdapter / Agent.to_ag_ui
Applications that only download from developer-controlled URLs are not affected.
Remediation
Upgrade to 1.102.0 or later (or 2.0.0b3 or later on the 2.0 pre-release line). The cloud-metadata and private-IP blocklists now decode the embedded IPv4 of every standardized IPv6 transition form before evaluating it — IPv4-mapped, IPv4-compatible, 6to4, NAT64 across all prefix lengths (including the RFC 8215 local-use prefix and operator-chosen prefixes), ISATAP, and Teredo. The set of always-blocked cloud metadata/credential endpoints has also been expanded across providers.
Workaround for Unpatched Versions
Avoid passing force_download='allow-local' on any URL that could be influenced by untrusted input. If developers must, resolve the hostname themselves and validate the result against their own metadata blocklist — including IPv6 transition forms — before constructing the FileUrl.
Credits
Reported by @SnailSploit.
vLLM: OOM Denial of Service via Audio Decompression Bomb
Summary
vLLM's /v1/audio/transcriptions endpoint limits compressed upload size but not decoded PCM output. A 25MB OPUS file expands to ~14.9GB of float32 PCM at decode time. Tested on vLLM v0.19.0.
Details
SpeechToTextProcessor rejects uploads over VLLM_MAX_AUDIO_CLIP_FILESIZE_MB (default 25MB) based on compressed byte length, but the audio decoder in audio.py accumulates all decoded frames into memory with no size limit before returning:
``python
speech_to_text.py L184-189
if len(audio_data) / 1024 ** 2 > self.max_audio_filesize_mb:
raise VLLMValidationError(...)
y, sr = load_audio(buf, sr=self.asr_config.sample_rate) # decoded size unchecked
audio.py L77-107
chunks: list[npt.NDArray] = []
for frame in container.decode(stream):
chunks.append(frame.to_ndarray())
audio = np.concatenate(chunks, axis=-1).astype(np.float32) # single contiguous allocation
`
A 25MB OPUS file at 6kbps encodes ~8.7 hours of audio. Decoding produces ~5.7GB of float32 PCM (232x amplification), and np.concatenate then allocates a second contiguous array, bringing peak RSS to ~14.9GB from a single request. SpeechToTextConfig.max_audio_clip_s` (default 30s) applies only after the full decode and does not prevent the allocation.
Impact
An unauthenticated attacker can exhaust server memory with a small number of concurrent requests, each a valid upload within the documented size limit. Severity was assessed with reference to prior OOM vulnerability reports in vLLM.
Fix
A fix for this vulnerability was merged here: https://github.com/vllm-project/vllm/pull/44970
vLLM: incomplete CVE-2026-22778 fix leaks PIL repr addresses via Anthropic router
vLLM: incomplete CVE-2026-22778 fix leaks PIL repr addresses via the Anthropic API router
Researcher: Kai Aizen — SnailSploit (@SnailSploit), Adversarial & Offensive Security Research
Severity: CVSS 3.1 5.3 (Medium) AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:N/A:N
Target: https://github.com/vllm-project/vllm
---
Summary
The fix for CVE-2026-22778 / GHSA-4r2x-xpjr-7cvv (PRs #31987 and #32319) introduced sanitize_message and applied it at four FastAPI exception-handling sites in the OpenAI router. The sanitizer strips object-repr memory addresses (<_io.BytesIO object at 0x7a95e299e750> → <_io.BytesIO object>) before error messages reach the client, defeating the ASLR-bypass primitive that CVE-2026-22778 chained with a libopenjp2 heap overflow for RCE.
The fix is incomplete: response paths added to vLLM at or after the same time as the fix continue to echo str(exc) directly to clients without sanitize_message. The original Stage 1 primitive — sending malformed image bytes so PIL raises UnidentifiedImageError whose message contains the BytesIO object repr — reaches all of them unmodified and leaks the heap address verbatim in the response body.
All five lines below are present in main HEAD (771e1e48b, 2026-05-26).
Affected sites
Current main HEAD (771e1e48b, 2026-05-26):
| # | File | Line | Code |
|---|---|---|---|
| 1 | vllm/entrypoints/anthropic/api_router.py | 78 | message=str(e), (inside POST /v1/messages exception handler) |
| 2 | vllm/entrypoints/anthropic/api_router.py | 124 | message=str(e), (inside POST /v1/messages/count_tokens) |
| 3 | vllm/entrypoints/anthropic/serving.py | 808 | error=AnthropicError(type="internal_error", message=str(e)), (SSE streaming converter) |
| 4 | vllm/entrypoints/speech_to_text/realtime/connection.py | 75 | await self.send_error(str(e), "processing_error") (WebSocket event loop) |
| 5 | vllm/entrypoints/speech_to_text/realtime/connection.py | 265 | await self.send_error(str(e), "processing_error") (WebSocket generation loop) |
Why the global exception handler does not save these paths
api_server.py registers a catch-all app.exception_handler(Exception)(exception_handler) at line 262, and that handler calls create_error_response(exc) which DOES apply sanitize_message. However, FastAPI exception handlers fire only on unhandled exceptions that propagate out of a route function.
All affected HTTP paths catch Exception inside the route coroutine and construct the response themselves:
``python
vllm/entrypoints/anthropic/api_router.py:71-81 (POST /v1/messages)
try:
generator = await handler.create_messages(request, raw_request)
except Exception as e:
logger.exception("Error in create_messages: %s", e)
return JSONResponse(
status_code=HTTPStatus.INTERNAL_SERVER_ERROR.value,
content=AnthropicErrorResponse(
error=AnthropicError(
type="internal_error",
message=str(e), # <-- unsanitized
)
).model_dump(),
)
`
Because the exception is caught and a JSONResponse is returned in-route, every registered FastAPI exception handler — including the sanitizing global one — is bypassed. The WebSocket path bypasses it for a different reason: WebSocket frames don't traverse FastAPI's HTTP exception handler chain at all.
Reachability — the same primitive as the parent CVE
The Anthropic Messages API accepts image content parts in the request body (type: "image" with base64 source.data or type: "image_url"). Image bytes are passed to the same multimodal loader used by the OpenAI router. Malformed bytes cause PIL.Image.open to raise:
`
UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x7a95e299e750>
`
The exception propagates up through handler.create_messages into the except Exception as e: at api_router.py:75. str(e) returns the exception message verbatim, including the address. The address ends up in the error.message field of the JSON response body returned to the attacker. ASLR entropy on the affected process drops from ~4 billion to ~8 candidates, identically to CVE-2026-22778 Stage 1.
The same primitive is reachable on POST /v1/messages/count_tokens (route #2), inside the SSE streaming converter when an exception is raised mid-stream (route #3), and over the realtime speech-to-text WebSocket when audio decoder or generation paths raise an exception containing any object repr (routes #4, #5).
Chronology — these are scope misses, not legacy code
2026-01-09: PR #31987 (aa125ecf0) introduces sanitize_message and applies it to OpenAI router HTTP exception handlers.
2026-01-15 (six days later): PR #32369 (4c1c501a7) adds vllm/entrypoints/anthropic/api_router.py containing line 78's message=str(e). The fix was not applied to the new router.
2026-03-02 (~two months later): PR #35588 (9a87b0578) adds the Anthropic count_tokens endpoint, replicating the same message=str(e) pattern at line 124.
2026-05-12 (~four months later): PR #42370 (d37e25ffb) consolidates speech-to-text entrypoints and the realtime WebSocket uses send_error(str(e), ...) for both error paths.
2026-05-26: current main HEAD, all five lines still present.
CVSS v3.1
AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:N/A:N — Base 5.3 (MEDIUM)
The parent CVE-2026-22778 was 9.8 (CRITICAL) because it chained the Stage 1 leak with a Stage 2 libopenjp2 heap overflow. Stage 2 is patched in OpenCV ≥ 4.13.0 (PR #32668 bumped the requirement), so Stage 1 alone here is a partial ASLR-bypass info-disclosure primitive rather than a complete RCE. Deployments that ship vLLM alongside an older OpenCV/libopenjp2 (system OpenCV on long-LTS distros, custom Docker images, downstream rebuilds) re-enable the full chain via the affected endpoints.
CWE
CWE-532 (Insertion of Sensitive Information into Log File / Error Message). Same CWE assigned to the parent CVE-2026-22778.
Remediation
1. Apply sanitize_message symmetrically to the five sites
`python
vllm/entrypoints/anthropic/api_router.py — add at top:
from vllm.entrypoints.utils import sanitize_message
Line 78 (POST /v1/messages) and Line 124 (count_tokens):
message=sanitize_message(str(e)),
`
`python
vllm/entrypoints/anthropic/serving.py — add at top:
from vllm.entrypoints.utils import sanitize_message
Line 808:
error=AnthropicError(type="internal_error", message=sanitize_message(str(e))),
`
`python
vllm/entrypoints/speech_to_text/realtime/connection.py — add at top:
from vllm.entrypoints.utils import sanitize_message
Lines 75 and 265:
await self.send_error(sanitize_message(str(e)), "processing_error")
`
2. Tighten the regex (defense in depth)
The current regex r" at 0x[0-9a-f]+>" is narrow — it only matches the exact CPython builtin object-repr suffix in lowercase hex with a trailing >. Future Python versions, C extensions, or custom __repr__ methods could produce non-matching formats that re-enable the leak:
`python
vllm/entrypoints/utils.py
def sanitize_message(message: str) -> str:
# Strip any standalone hex address; downstream observers don't need them.
return re.sub(r"\b0x[0-9a-fA-F]{6,}\b", "0x?", message)
`
3. Future-proofing: consider a response middleware
Both the route-local exception handling pattern (Anthropic router) and the WebSocket path bypass FastAPI's exception handler chain. A response-level middleware that always invokes sanitize_message on outgoing error bodies would prevent this class of regression entirely.
Affected versions
All vLLM versions containing vllm/entrypoints/anthropic/api_router.py (introduced 2026-01-15 in PR #32369).
All vLLM versions containing vllm/entrypoints/speech_to_text/realtime/connection.py (introduced 2026-05-12 in PR #42370).
Confirmed present in main HEAD 771e1e48b (2026-05-26).
References
Parent advisory: https://github.com/vllm-project/vllm/security/advisories/GHSA-4r2x-xpjr-7cvv (CVE-2026-22778)
Fix PRs to model the patch on: #31987, #32319, #32668
vllm/entrypoints/utils.py:sanitize_message: https://github.com/vllm-project/vllm/blob/771e1e48b/vllm/entrypoints/utils.py#L323-L326
vllm/entrypoints/anthropic/api_router.py leak site: https://github.com/vllm-project/vllm/blob/771e1e48b/vllm/entrypoints/anthropic/api_router.py#L78
Steps to reproduce
1. Clone the target: git clone --depth 1 https://github.com/vllm-project/vllm
2. Run the proof of concept (PoC.py`) against the cloned source.
3. Observe the result shown under Verified result below.
Credit
Kai Aizen — SnailSploit (@SnailSploit). Adversarial & Offensive Security Research.
Fix
A fix for this vulnerability was added here: https://github.com/vllm-project/vllm/pull/45119
vLLM: GGUF dequantize kernel int truncation exposes uninitialized GPU memory in multi-tenant serving
Summary
Integer truncation of tensor dimensions in vLLM's GGUF dequantize kernels (csrc/quantization/gguf/gguf_kernel.cu) causes partial tensor processing. The output tensor is allocated at full size via torch::empty (uninitialized memory), but the dequantize CUDA kernel processes only a truncated number of elements. The unfilled portion of the output tensor retains whatever was previously in GPU memory. In multi-tenant inference deployments, this residual GPU memory may contain tensor data from other users' inference requests, constituting information disclosure.
Root Cause
The to_cuda_ggml_t function pointer type at ggml-common.h:1067 declares its element count parameter as int (32-bit):
``cpp
using to_cuda_ggml_t = void ()(const void __restrict__ x,
dst_t __restrict__ y,
int k, // 32-bit
cudaStream_t stream);
`
All dequantize kernel functions (dequantize_block_cuda, dequantize_row_q2_K_cuda, etc. in dequantize.cuh) inherit this int k parameter and use it as the kernel launch grid size:
`cpp
static void dequantize_block_cuda(..., const int k, cudaStream_t stream) {
const int num_blocks = (k + 2CUDA_DEQUANTIZE_BLOCK_SIZE - 1) / (2CUDA_DEQUANTIZE_BLOCK_SIZE);
dequantize_block<<<num_blocks, CUDA_DEQUANTIZE_BLOCK_SIZE, 0, stream>>>(vx, y, k);
}
`
In ggml_dequantize() at gguf_kernel.cu:85, the caller passes m n (an int64_t product) to this int k parameter:
`cpp
at::Tensor DW = torch::empty({m, n}, options); // line 80: full-size, UNINITIALIZED
// ...
to_cuda((void)W.data_ptr(), (scalar_t)DW.data_ptr(), m n, stream); // line 85: mn truncated to int
`
When m n > INT_MAX, the truncated k is smaller than the actual tensor size. The kernel processes k elements. The remaining (m n) - k elements in DW are never written and contain stale GPU memory.
This is a single root cause -- the int type on the k parameter in to_cuda_ggml_t -- with a single fix: change int k to int64_t k. All dequantize functions inherit this type through the same typedef.
Affected Functions
All in csrc/quantization/gguf/gguf_kernel.cu:
| Function | Line | Allocation | Info Disclosure? |
|----------|------|-----------|-----------------|
| ggml_dequantize | 74 | torch::empty({m, n}) at line 80 | Yes -- mn truncated to int k at line 85 |
| ggml_mul_mat_vec_a8 | 91 | torch::empty({vecs, row}) at line 99 | Yes -- int col = X.sizes()[1] at line 94 |
| ggml_mul_mat_a8 | 207 | torch::empty({batch, row}) at line 215 | Yes -- int col = X.sizes()[1] at line 210 |
| ggml_moe_a8 | 279 | torch::empty({tokenstop_k, row}) at line 289 | Yes -- int col = X.sizes()[1] at line 285 |
All four functions allocate output tensors with torch::empty (uninitialized) and then run CUDA kernels that use truncated dimension values as loop bounds. The unfilled portion of each output tensor retains stale GPU memory.
ggml_moe_a8_vec (line 382) uses torch::zeros instead of torch::empty, so it is not affected by the info disclosure variant.
Impact: Information Disclosure in Multi-Tenant Serving
vLLM is designed for multi-tenant inference serving. GPU memory is reused across requests from different users. When the dequantize kernel partially fills an output tensor:
1. The output tensor DW is allocated with torch::empty -- the buffer contains whatever was previously in that GPU memory region
2. The dequantize kernel fills only a truncated portion of the buffer
3. The unfilled portion retains residual data from prior GPU operations, which may include tensor data from other users' inference requests
4. The contaminated tensor proceeds through the model computation
5. No error or warning is generated -- the partial fill is silent
This is a confidentiality violation. In shared inference deployments (the primary vLLM use case), one user's inference data can leak into another user's model computation through residual GPU memory.
Attacker Control
The attacker crafts a GGUF model file with weight tensor dimensions whose product exceeds INT_MAX (e.g., a matrix with shape [65536, 65536] gives m * n = 4,294,967,296`). The model is hosted on HuggingFace or any model hub. The victim loads the model with vLLM for inference serving. The truncation happens automatically during model weight dequantization.
Fix
A fix for this vulnerability was added here: https://github.com/vllm-project/vllm/pull/44971
