vLLM Agentic API Demo

Open Models. Open Inference. Open Agents.

Running Codex and Claude Code CLIs with vLLM and Open Models

Francisco Javier Arceo

Francisco Javier Arceo

Sr. Principal Software Engineer

Red Hat AI
Tun Jian Tan

Tun Jian Tan

Principal Engineer

Embedded LLM
Codex · Claude Code · Agentic API · vLLM 01
vLLM Agentic API Community update

Open-source agentic inference

Agentic workloads need more than token generation

Agentic API is a high-performance Rust runtime for state, tools, and client compatibility around vLLM.

01State managementcontinuation · persistence
02Tool executioninvoke external tools
03Tool orchestrationcoordinate multi-round loops
04Provider integrationsYou.com · PostgreSQL · more coming
Infrastructure you control 02
Complementary by design

vLLM core generates the tokens.

Agentic API runs the agent loop.

Clear responsibilities. Tighter integration. 03
Agentic API MVP

MVP shipped

The MVP is working today

Codex and Claude Code can use open-weight models served by vLLM—with the agent runtime outside the inference engine.

01State managementcontinuation · PostgreSQL persistence
02Tool runtimeexecution · orchestration · MCP
03Provider ecosystemYou.com today · more tool and database providers next
RUST RUNTIME CODEX CLAUDE CODE OPEN-WEIGHT MODELS
From prototype to a first-class serving experience 04
Enterprise scale with llm-d

Kubernetes-native distributed inference

llm-d turns one runtime into a Kubernetes fleet

Agentic API manages each agent loop. vLLM runs inference. llm-d routes and scales workloads across the cluster.

01Fleet routingplace requests across model-serving pods
02Elastic scaleexpand serving pools across accelerators
03Kubernetes operationsdeploy, observe, and tune production inference
Abstract Kubernetes inference fleet managed by llm-d
Fleet layerllm-d
Kubernetes clusterAgentic API + vLLM fleet
One runtime in the demo. A fleet in production. 05
Recorded end-to-end demo

Agentic tool loop

One request. Two inference rounds.

Claude Code and Codex use the same tool-aware runtime path.

Codex /Claude Code Agentic APIServer vLLM Server You.com 01 Request 02 First inference 03 Tool-call tokens 04 Web Search execution 05 Web Search results 06 Resume inference 07 Final tokens 08 Streamed response
Agentic inference on infrastructure you control 06