DynBranch: Speculative Subgraph Reuse for Dynamic Agentic LLM Serving
DynBranch is a speculative subgraph reuse technique for dynamic agentic LLM serving. It reduces latency by up to 32% and 46-66% by reusing completed subgraph results across requests, without requiring changes to agent harnesses or model execution engines.
Save an API key to vote.