Mode-Latency-Benchmark

A CLI tool for real-time comparison of Chinese LLM API response speed (TTFT and total time).

Ported into dsh-budget — part of the PerryLink DSH Plugin Family.

English · 简体中文

---

What it does

model-latency-benchmark streams the same prompt to multiple Chinese LLM providers concurrently and measures each one's time-to-first-token (TTFT) and total response time, showing live, racing-style progress bars and a ranked result table.

Features

• 🚀 Concurrent real-time comparison of multiple LLM APIs
• ⚡ Time-to-first-token (TTFT) and total-time measurement
• 🎮 Racing-style live progress bars (Rich)
• 🔧 Custom prompt, provider selection, and timeout

Quick start

Using pip:

pip install model-latency-benchmark

Using Poetry (for development):

git clone https://github.com/PerryLink/model-latency-benchmark.git
cd model-latency-benchmark
poetry install

Create a .env file with the API keys for the providers you want to test:

DEEPSEEK_API_KEY=sk-your-key-here
BAIDU_API_KEY=your-key-here
ALIBABA_API_KEY=your-key-here
BYTEDANCE_API_KEY=your-key-here

Usage

# Test all providers with an API key set
model-latency-benchmark

# Custom prompt
model-latency-benchmark --prompt "Introduce machine learning in one sentence"

# Only specific providers
model-latency-benchmark --providers deepseek,baidu

# Set request timeout (default 30 s)
model-latency-benchmark --timeout 60

# Show debug information
model-latency-benchmark --debug

Supported providers

Provider	Name	Environment variable	Get API key
deepseek	DeepSeek	DEEPSEEK_API_KEY	DeepSeek Platform
baidu	百度文心	BAIDU_API_KEY	Baidu AI Cloud
alibaba	阿里通义	ALIBABA_API_KEY	Alibaba Cloud DashScope
bytedance	字节豆包	BYTEDANCE_API_KEY	Volcano Engine

Project structure

model-latency-benchmark/
├── src/model_latency_benchmark/
│   ├── __init__.py
│   ├── __main__.py
│   ├── cli.py              # CLI interface
│   ├── core.py             # Core benchmark logic
│   └── utils.py            # Utility functions
├── tests/                  # Test files
├── pyproject.toml          # Poetry configuration
└── README.md

Tech stack

• CLI framework: Click
• HTTP client: httpx (async support)
• Terminal UI: Rich
• Environment: python-dotenv
• Package manager: Poetry

Development

poetry install
poetry run pytest --cov
poetry run black .
poetry run ruff check .
poetry run mypy src/

Contributing

See CONTRIBUTING.md for contribution guidelines. For issues or suggestions, please submit an Issue.

Related

• dsh-budget — the DSH plugin this project was ported into
• PerryLink — the PerryLink DSH Plugin Family

License

Apache License 2.0 © 2026 PerryLink