Skip to main content

​
Benchmark snapshot

This page is a short reference for one public benchmark run comparing GoModel and LiteLLM on OpenAI-compatible traffic. The full article contains the complete write-up, all charts, and the original discussion: GoModel vs LiteLLM Benchmark: Speed, Throughput, and Resource Usage.
This benchmark is a point-in-time snapshot published on March 5, 2026. Treat it as data, not dogma. Gateway performance depends on workload, provider mix, deployment setup, and tuning.

​
Visual snapshot

Chart source and full context: Original benchmark post.

​
At a glance

In this benchmark run, GoModel came out ahead on the main operational signals most teams care about:
  • Added latency
  • Throughput under concurrency
  • CPU overhead
  • Memory overhead

​
Test shape

The comparison used a simple like-for-like setup:
  • OpenAI-compatible /v1/chat/completions
  • The same prompt and request shape on both sides
  • Concurrency levels of 1, 4, and 8
  • A focus on clean runs with 0% errors
  • Metrics including req/s, latency percentiles, CPU usage, and RSS memory
This docs page keeps only the primary comparison matrix from the blog post.

​
Reference table

​
Key readouts

Some useful reads from that March 5, 2026 run:
  • Lower p95 latency at every tested concurrency level.
  • Higher throughput across the benchmark matrix.
  • 45-46 MB RSS, while LiteLLM stayed near 320-321 MB.
  • Less CPU in these runs.
At the highest tested concurrency, GoModel reached 52.75 req/s versus LiteLLM at 35.81 req/s.

​
Reproduce it yourself

All the tooling used in the published benchmark is available in this repository.

​
Prerequisites

  • Go 1.26.3+
  • Python 3.10+ with matplotlib and numpy
  • jq, curl
  • A Groq API key (or any OpenAI-compatible provider — adjust the script)
  • litellm[proxy] (pip install "litellm[proxy]")

​
Scripts

The benchmark suite lives in docs/about/benchmark-tools/:

​
Quick start

The script creates a timestamped directory under benchmark-results/ containing JSON result files, gateway logs, and a REPORT.md with the results table.

​
Tuning

You can override defaults via environment variables:
See the top of compare.sh for the full list of knobs.

​
Why this page is short

This page is intentionally shorter and more operational than the blog version. It exists so docs readers can see the benchmark result quickly without reading a full article inside the product docs. If you want the full narrative, more charts, and the original context, use the source post. No single benchmark settles the question for every environment. If you are evaluating gateways seriously, reproduce the test against your own traffic and infrastructure.