Skip to content

perf(observed): benchmark emit paths with the production clock - #724

Open
Evgenii (Vaiz) wants to merge 2 commits into
mainfrom
u/eshutov/observed-bench-production-clock
Open

perf(observed): benchmark emit paths with the production clock#724
Evgenii (Vaiz) wants to merge 2 commits into
mainfrom
u/eshutov/observed-bench-production-clock

Conversation

@Vaiz

@Vaiz Evgenii (Vaiz) commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

🤖 Clawpilot here! Posted automatically by Clawpilot (an AI agent), not by a human. Please verify before acting.

What this changes

These benchmarks exist to answer one question: what does it cost to emit an event in production? They did not measure that. Every benchmark sink was built with tick::SimpleClock::new_frozen(), the test clock. This change builds them with tick::SimpleClock::new_system(), the clock production uses. Refs AB#7757582.

Why the clock matters

A sink stamps a timestamp on every event it dispatches. The clock therefore sits in the emit hot path, and its cost lands in every emit measurement.

The two clocks reach the time differently. new_system() is a stateless enum variant that reads OS time directly. new_frozen() wraps a ClockControl, which holds its state in an Arc<Mutex<State>> (crates/tick/src/clock_control.rs). The frozen clock thus replaces a real OS time read with mutex locking. Neither cost resembles the other, so the published numbers described the test clock rather than the production emit path.

One case was plainly wrong, not merely unrepresentative. construct_processor_free_sink built a fresh frozen clock inside the measured closure. It allocated an Arc<Mutex<..>> per iteration and counted that as sink-construction cost. Production never makes that allocation.

Sink::new already documents the split: pass the application clock in production, pass a frozen clock in tests for deterministic, Miri-safe timestamps (crates/observed/src/sink/core.rs). Benchmarks are not tests. None of these reads a timestamp value, so none belonged on the test side of that line.

Details

  • 12 sink constructions now use SimpleClock::new_system(): simple_log_2_fields, log_8_fields, log_with_body, log_of_metric_event, log_2_redacted_strings, emit_with_3_enrichments, emit_with_10_nested_enrichments, enrichment_vec_collect_3, emit_depth_0, emit_depth_5, construct_processor_free_sink, and the composite children of pending_enriched_future_8_polls_composite_3.
  • construct_processor_free_sink builds the clock once outside the measured closure and clones it per iteration. A comment states why: an application owns one clock and hands it to every sink, so clock construction is not part of sink construction.
  • No benchmark keeps a frozen clock.
  • Benchmark names, groups, event shapes, and the tracking harness are unchanged.

Effects

  • Reported timings change for all 12 benchmarks. Saved Criterion baselines for these IDs no longer compare like for like and should be discarded.
  • Emit timings now include a real OS time read. That source is platform-dependent, so run-to-run variance may increase.
  • The reported allocation count for construct_processor_free_sink drops by the per-iteration Arc<Mutex<..>>.
  • No library code changes. The diff touches one bench target, so there is no public API or semver impact.
  • Cargo.toml is unchanged. The tick test-util dev-feature is still required, because unit tests under crates/observed/src still use frozen clocks.
  • Benchmark timestamps are no longer deterministic. A future benchmark that needs a fixed timestamp must opt back into a frozen clock and say why.
  • The size of the correction is not quantified. This change includes no before-and-after measurements.

Out of scope

  • AB#7757571 residue: no log_4_fields control, no pooled-allocation alternative, no depth scaling model. The existing emit_alloc commentary is untouched.
  • AB#7757574: whether observed should replace all_the_time with the workspace benchmarking crate. That decision is open.
  • crates/observed_macros* is untouched.

Validation

  • cargo bench -p observed --features test-util --bench observed_benchmarks -- --test runs every benchmark once with the system clock. CI does not execute benchmarks, so this confirms the target still runs rather than only compiles.
  • Deliberately not verified: no comparative measurement run was taken, so the corrected numbers are not reported here.

The benchmarks built every sink with `tick::SimpleClock::new_frozen()`,
including the production-path emit benchmarks. A frozen clock is backed
by `ClockControl`, which keeps its state in an `Arc<Mutex<State>>`, so
every timestamp read paid mutex locking that production never pays, and
the sink-construction benchmark additionally paid an `Arc<Mutex<..>>`
allocation inside the measured closure. The numbers therefore described
controlled-clock behaviour while being reported as production emit cost.

Switch all twelve sink constructions to `tick::SimpleClock::new_system()`,
the clock `Sink::new` documents for production, and hoist the clock out of
the `construct_processor_free_sink` closure so it measures sink
construction rather than clock construction. No benchmark here needs
deterministic timestamps, so none stays frozen. Benchmark names, shapes
and groups are unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.0%. Comparing base (1b0f3fe) to head (9c592ec).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff            @@
##             main     #724    +/-   ##
========================================
  Coverage   100.0%   100.0%            
========================================
  Files         587      587            
  Lines       63007    63131   +124     
========================================
+ Hits        63007    63131   +124     
Flag Coverage Δ
linux 64.3% <ø> (?)
linux-arm 63.7% <ø> (?)
windows 63.6% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

⚠️ Potential breaking changes detected

cargo semver-checks flagged the following on this PR. This is informational -- breaking changes between commits are expected; the major-version bump happens at release time, not on every PR.

fetch_winhttp

     Cloning origin/main
    Building fetch_winhttp v0.1.1 (current)
       Built [   0.381s] (current)
     Parsing fetch_winhttp v0.1.1 (current)
error: unsupported rustdoc format v60 for file: /home/runner/work/oxidizer/oxidizer/target/semver-checks/local-fetch_winhttp-0_1_1-default-01666ec060466c14/target/doc/fetch_winhttp.json
(supported formats are v55, v56, v57)

fetch_winhttp_impl

     Cloning origin/main
    Building fetch_winhttp_impl v0.1.1 (current)
       Built [   0.404s] (current)
     Parsing fetch_winhttp_impl v0.1.1 (current)
error: unsupported rustdoc format v60 for file: /home/runner/work/oxidizer/oxidizer/target/semver-checks/local-fetch_winhttp_impl-0_1_1-default-f6bbf157605592e2/target/doc/fetch_winhttp_impl.json
(supported formats are v55, v56, v57)

observed

     Cloning origin/main
    Building observed v0.25.0 (current)
       Built [   5.803s] (current)
     Parsing observed v0.25.0 (current)
error: unsupported rustdoc format v60 for file: /home/runner/work/oxidizer/oxidizer/target/semver-checks/local-observed-0_25_0-default-d7b8c5ec6ce39049/target/doc/observed.json
(supported formats are v55, v56, v57)

observed_testing

     Cloning origin/main
    Building observed_testing v0.0.0 (current)
       Built [   5.947s] (current)
     Parsing observed_testing v0.0.0 (current)
error: unsupported rustdoc format v60 for file: /home/runner/work/oxidizer/oxidizer/target/semver-checks/local-observed_testing-0_0_0-default-01666ec060466c14/target/doc/observed_testing.json
(supported formats are v55, v56, v57)

observed_utils

     Cloning origin/main
    Building observed_utils v0.2.0 (current)
       Built [   5.852s] (current)
     Parsing observed_utils v0.2.0 (current)
error: unsupported rustdoc format v60 for file: /home/runner/work/oxidizer/oxidizer/target/semver-checks/local-observed_utils-0_2_0-default-01666ec060466c14/target/doc/observed_utils.json
(supported formats are v55, v56, v57)

@Vaiz
Evgenii (Vaiz) marked this pull request as ready for review September 3, 2026 14:54
Copilot AI lite review requested due to automatic review settings September 3, 2026 14:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

One benchmark still includes avoidable per-iteration clock cloning overhead, which can skew the intended sink-construction measurement.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This pull request updates the observed benchmark suite to measure event emission using the production clock implementation (tick::SimpleClock::new_system()) instead of the frozen test clock, so benchmark results better reflect real-world emit-path costs.

Changes:

  • Switched multiple benchmark sinks from SimpleClock::new_frozen() to SimpleClock::new_system().
  • Adjusted the construct_processor_free_sink benchmark to construct the clock once outside the measured closure and reuse it per iteration.
File summaries
File Description
crates/observed/benches/observed_benchmarks.rs Updates benchmark sink construction to use the production clock and refines the sink-construction benchmark to avoid measuring clock creation.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread crates/observed/benches/observed_benchmarks.rs Outdated
`Sink::new` takes `impl AsRef<SimpleClock>` and clones it internally, so
`clock.clone()` at the call site made two clones per iteration inside the
measured closure. Pass `&clock` instead.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 3, 2026 15:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes are bench-only, align with Sink::new’s documented production clock usage, and introduce no evident correctness or design issues.

Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@evgenyfedorov2

Copy link
Copy Markdown
Member

Deliberately not verified: no comparative measurement run was taken, so the corrected numbers are not reported here.

ok, but how about adding benchmark results to some readme.md file for future reference?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants