Skip to content

Reproduce the artifact-contract diff CLI from a clean clone #93

Description

@stacknil

Review target

Reproduce the standalone artifact-contract diff CLI from a clean clone using tiny sanitized artifact trees. This is a focused follow-up to the implementation in #85 and #90-#92; it is narrower than the earlier five-demo operator-path reproduction in #77.

Relevant paths on the current default branch:

  • scripts/artifact_contract_diff.py
  • docs/reviewer-artifact-diff.md
  • schemas/artifact_contract_diff.schema.json
  • tests/test_artifact_contract_diff_core.py
  • tests/test_artifact_contract_diff_structured.py
  • tests/test_artifact_contract_diff_json_report.py

Reproduction

  1. Record the default-branch commit, OS, Python version, and clean-clone install command.
  2. Create two disposable, sanitized directory pairs: one byte-equivalent text/JSON case and one case with a small reviewer-visible difference.
  3. Run the documented human-readable comparison for both pairs.
  4. Repeat with --json-out located outside both input roots, then run the three focused test files above.

The documented contract is exit 0 for unchanged artifacts and exit 1 for detected differences. Repeating the same JSON-report command on the same trees should be byte-identical. Output must not disclose artifact bodies, timestamps, or absolute checkout paths.

Evidence to post

  • Exact commands and exit codes.
  • A concise description of relative paths/statuses reported for both cases.
  • Whether repeated JSON output is byte-identical and validates against artifact-contract-diff/v1.
  • Focused test results.
  • One classification: reproduced as documented, documentation ambiguity, environment-specific limitation, or probable defect.

Change threshold

Post reproduction evidence before opening a PR. Only propose a change for a concrete mismatch, and preserve the smallest affected CLI, contract, test, or instruction.

Boundaries

  • Keep fixture trees tiny, synthetic, and disposable.
  • Do not include real telemetry, secrets, usernames, absolute machine paths, or artifact bodies in the issue.
  • Do not accept regenerated outputs, change release compatibility labels, or broaden the comparator's fail-closed path semantics.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    good first issueGood for newcomershelp wantedExtra attention is neededtestingTest coverage and edge-case behavior

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions