ttft

批量测试符合 OpenAI 协议和 Anthropic 协议的 AI 模型性能指标。支持 TTFT（首字节时间）、TPS（吞吐量）、网络延迟等关键性能指标的测量，提供多模型对比测试和详细的性能报告生成功能。

Zero-config local LLM optimization for Ollama, LM Studio, and Apple Silicon MLX. Reduces TTFT by 40%, wall time for local agents by 46%, and RAM usage by 3x.

A Go CLI tool to benchmark local LLMs via Ollama, measuring Time To First Token (TTFT) and throughput on your specific hardware.

LLM inference benchmarking toolkit. Measure TTFT, inter-token latency, throughput, and P50–P99 across concurrency levels.

Pi Coding Agent extension for real-time execution turns, steps, LLM/tool durations, TTFT, and TPS in the footer status bar

The only voice agent context manager with a TTFT feedback loop

Linux kernel and systems fast path for LLM inference: eBPF tracing, runtime hints, cgroups, NUMA/GPU locality, KV-cache memory policies, TTFT boost, and experimental kernel primitives.

An asynchronous, neuro-symbolic VLA (Vision-Language-Action) orchestration stack for edge autonomy. Fuses probabilistic Qwen2-VL visual reasoning and faster-whisper ASR with deterministic PX4/MAVSDK flight-control loops and HSV color guardrails.

Mesure les métriques d'inférence LLM (TTFT, TPOT, débit, coût, VRAM) sur n'importe quelle API OpenAI-compatible. Inclut infer-serve pour héberger un GGUF via llama.cpp en une commande.

How to benchmark LLM inference: 3,000 measured DigitalOcean requests, raw JSON, rebuildable charts, and a 15-point disclosure checklist (August 24, 2026).

LLM inference observability sidecar for vLLM in Python: FastAPI middleware exports TTFT/TBT/E2E, KV-cache and queue-depth metrics to Prometheus + Grafana; Docker Compose stack; 32/32 pytest passing. Demo: TTFT p50 84ms / p99 244ms, TBT p50 17ms, E2E p50 384ms.

A professional concurrent stress testing tool for Large Language Models. Test P99 latency, TTFT (Time To First Token), and token generation speed of your deployed models. 一个专业的大语言模型并发压测工具。测试部署模型的P99延迟、首字延迟(TTFT)和Token生成速度。

Prometheus exporter for LLM API monitoring — probes OpenAI, Anthropic, Google Gemini, Azure OpenAI and any OpenAI-compatible endpoint, collecting TTFT, latency, token usage and availability metrics.

A CLI tool for real-time comparison of Chinese LLM API response speeds.实时对比国产大模型 API 响应速度的 CLI 工具

实测中转平台首字节延迟TTFT，支持Anthropic/OpenAI协议

LLM inference benchmarking dashboard: Python FastAPI backend with async orchestration, WebSocket live TTFT/TBT/throughput comparison across configs (512/128 to 4096/1024 tokens), Grafana + Docker Compose stack, GitHub Actions CI; 21/21 pytest passing.

A measurement-first MLX inference engine for Apple Silicon: every performance claim is backed by a preregistered experiment with an A/A control and a confidence interval.

Dynamically benchmarks every model on NVIDIA NIM — type, context window, reasoning/cache, TTFT & token rate — and renders an interactive, dated HTML report.

Benchmark for prefill/decode interference on a single GPU, measuring TTFT inflation when new requests arrive while decode is already running.

Load generator and performance analyzer for LLM inference serving. Measures TTFT, TPOT, and throughput against vLLM/TGI/Ollama or a built-in mock, and flags serving anti-patterns with tuning advice.