Other resources
JohnScheuer/llm-serving-sim — AI/Bot Repository
End-to-end LLM serving simulator integrating scheduling, prefix caching, tensor allocation, and KV-cache management. 168-run sweep (72 baseline + 96 pressure). Key finding: ChunkedPrefill + LFU cache achieves 41% lower TTFT p95 and 94% prefix hit rate, but hits OOM first under memory pressure.
Provider: JohnScheuer (GitHub owner). GitHub repository metadata; provider website and service not independently reviewed.
View public repository ↗- Best suited to
- Exploring public AI and bot repository documentation
- Owner next step
- Review the source, required permissions and current terms before connecting.
- Access
- Credentials or local installation may be required. Read the source documentation.
- Cost
- Check current provider terms. Provider pricing has not been independently checked. Review current terms.
- Interfaces
- Website
- Source evidence
- Official public source · Repository snapshot 2026-11-05 · GitHub ID 1296004669 · JohnScheuer/llm-serving-sim. Public, nonarchived, nonfork and nonempty repository metadata was returned during source collection; this is not a provider, pricing, safety or compatibility review.
- Metadata screening
- Metadata screened 2026-11-05. Public repository metadata available 2026-11-05. Screening scope and limits. Publisher metadata remains untrusted; this is not a security audit or compatibility test.
- Traffic and availability
- Not monitored. Bot visits, citations, uptime and successful delivery have not been measured by this directory.
Listing content is informational. Inclusion does not authorize installation, changes to your site, or payment. Check current provider terms and access policies.