Skip to content

perf: update otel benchmark dependencies, add otel bound instrument scenarios - #2451

Open
jack-berg wants to merge 2 commits into
prometheus:mainfrom
jack-berg:update-otel-benchmarks
Open

perf: update otel benchmark dependencies, add otel bound instrument scenarios#2451
jack-berg wants to merge 2 commits into
prometheus:mainfrom
jack-berg:update-otel-benchmarks

Conversation

@jack-berg

Copy link
Copy Markdown
Collaborator

Updating benchmarks after substantial efforts to improve otel performance under contention. Changes:

  • Add scenarios for experimental bound instruments. I understand if you want to reject this until stable.
  • Disable exemplars for apples to apples comparison. My understanding is that the prom client_java cases don't have exemplars enabled, can change if I'm wrong.
  • Update to opentelemetry-java v1.66.0. If we wait until opentelemetry-java-instrumentation version is released which bundles the v1.66.0 version, the clutter in pom.xml goes away.

Benchmark results on my machine:

Benchmark                                          Mode  Cnt      Score      Error  Units
CounterBenchmark.codahaleIncNoLabels              thrpt   25  95236.680 ±  755.718  ops/s
CounterBenchmark.openTelemetryAdd                 thrpt   25  54598.238 ±  290.633  ops/s
CounterBenchmark.openTelemetryBoundAdd            thrpt   25  82770.849 ±  342.576  ops/s
CounterBenchmark.openTelemetryBoundInc            thrpt   25  85165.317 ±    7.693  ops/s
CounterBenchmark.openTelemetryInc                 thrpt   25  75136.160 ±  552.123  ops/s
CounterBenchmark.openTelemetryIncNoLabels         thrpt   25  85517.108 ±  538.105  ops/s
CounterBenchmark.prometheusAdd                    thrpt   25  84533.092 ±  680.646  ops/s
CounterBenchmark.prometheusInc                    thrpt   25  85135.753 ±   23.184  ops/s
CounterBenchmark.prometheusNoLabelsInc            thrpt   25  85061.556 ±  194.011  ops/s
CounterBenchmark.simpleclientAdd                  thrpt   25   9174.173 ± 1542.709  ops/s
CounterBenchmark.simpleclientInc                  thrpt   25   8860.946 ± 1306.154  ops/s
CounterBenchmark.simpleclientNoLabelsInc          thrpt   25  17502.624 ±  830.879  ops/s
HistogramBenchmark.openTelemetryBoundClassic      thrpt   25  25261.802 ± 6559.742  ops/s
HistogramBenchmark.openTelemetryBoundExponential  thrpt   25   2367.637 ±  381.165  ops/s
HistogramBenchmark.openTelemetryClassic           thrpt   25  28697.261 ± 4082.434  ops/s
HistogramBenchmark.openTelemetryExponential       thrpt   25   1868.494 ±  101.732  ops/s
HistogramBenchmark.prometheusClassic              thrpt   25  23600.746 ± 7292.571  ops/s
HistogramBenchmark.prometheusClassicPerThread     thrpt   25  58674.695 ±   98.620  ops/s
HistogramBenchmark.prometheusClassicSingleThread  thrpt   25  14865.898 ±   12.295  ops/s
HistogramBenchmark.prometheusNative               thrpt   25  10726.819 ±  677.244  ops/s
HistogramBenchmark.simpleclient                   thrpt   25  17340.988 ±  266.176  ops/s

The improvements to otel explicit histogram performance came from a design inspired by prometheus client_java. The code contains attribution. I did manage to slightly improve on the design - If you agree, I'm happy to chat about it and encourage prom client_java to copy any of the bits you think are helpful.

@jack-berg jack-berg changed the title Update otel benchmark dependencies, add otel bound instrument scenarios perf: update otel benchmark dependencies, add otel bound instrument scenarios Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant