test: focus PR benchmarks on client_java and measure label lookups - #2468
Merged
Merged
Conversation
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
zeitlinger
requested review from
dhoard,
fstab and
jaydeluca
as code owners
September 15, 2026 11:12
Member
Author
|
The histogram timeout was investigated separately from this benchmark work. The forced interleaving fails on the old implementation after five seconds, then passes with the This PR remains benchmark-only. Once the correctness fix lands, update/retest this branch rather |
Contributor
Benchmark resultsBenchmark run succeeded for
Prometheus Java Client BenchmarksRun Information
Comparison with base
Results for PR headCounterBenchmark
HistogramBenchmark
HistogramTextFormatBenchmark
TextFormatUtilBenchmark
New benchmarks in PR headThese benchmarks have no base counterpart; scores are descriptive only and have no regression verdict.
Allocation per operationJMH GC profiler
Raw ResultsNotes
Benchmark Descriptions
|
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
Member
Author
|
I rebased this benchmark PR additively with the focused buffer handoff fix from #2471 so its full CI can validate on the same head. Relevant core regression tests pass locally, lint and |
jaydeluca
approved these changes
Sep 15, 2026
zeitlinger
enabled auto-merge (squash)
September 15, 2026 16:06
zeitlinger
added a commit
that referenced
this pull request
Sep 15, 2026
Improve benchmark comparison reporting without changing benchmark execution or sample configuration. - Display JMH 99.9% confidence intervals beside PR and base means. - Distinguish overlapping intervals, below-threshold changes, and missing/invalid uncertainty. - Keep the verdict explicitly conservative and heuristic, not a significance test. - Add coverage for rendering, malformed data, thresholds, metadata mismatch, and the supplied overlapping example. This is separate from #2468 and does not rerun benchmarks. Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
zeitlinger
disabled auto-merge
September 15, 2026 16:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Benchmark-only follow-up for the release review; no production metric changes.
OpenTelemetry, Codahale, and legacy simpleclient comparisons remain in full/local/nightly runs.
Keep all client_java exposition benchmarks, including OpenMetrics.
with one-thread and four-thread variants. One invocation is one metric update.
base/head allocation deltas only when configurations match.
Related PR
Label lookup optimization #2442 is pending this
benchmark infrastructure. After this lands, update that branch and rerun the benchmarks so base and
head contain identical lookup benchmark code. The new methods in this PR itself have head-only
results and do not establish the optimization's benefit.
Validation
mise run lint:fix— passed; formatter changes retained.mise run test— passed.mise run build -- -DskipITs=true— passed.mise run buildcompiled the benchmark module, but Docker-backed integration tests randespite
-DskipTestsand failed because no Docker environment was available.OpenMetrics/Prometheus exposition cases.
JDK 25.0.3,
-Xms128m -Xmx256m. This was a smoke test on a shared development host, not acontrolled base/head performance comparison. Repeated lookup reported about 64 B/update;
cached increments were near zero.
git diff --check— passed.