Skip to content

ci: gate integration tests on unit-test success #1283

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ogenstad merged 1 commit into from
Aug 28, 2026
+2 −2
Merged

ci: gate integration tests on unit-test success #1283

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -317,7 +317,7 @@ jobs:
!contains(needs.*.result, 'failure') &&
!contains(needs.*.result, 'cancelled') &&
needs.files-changed.outputs.python == 'true'
needs: ["prepare-environment", "files-changed", "yaml-lint", "python-lint"]
needs: ["prepare-environment", "files-changed", "yaml-lint", "python-lint", "unit-tests"]

@cubic-dev-ai cubic-dev-ai Bot Aug 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Adding unit-tests to this job's needs now gates integration-tests-latest-infrahub (40-min huge-runner job) not just on test failures but on every step inside the unit-tests matrix leg, including the Upload coverage to Codecov and Report coverage for pytest-plugin steps. Those upload CODECOV_TOKEN to an external service, so a transient Codecov outage or upload failure marks the leg failure and silently skips integration coverage even when the actual tests are green. Previously integration ran independently of unit-tests/Codecov health. Consider making the coverage-upload steps conditional on test success or not part of the aggregate gate, so a non-test failure does not block the expensive integration job.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/ci.yml, line 320:

<comment>Adding `unit-tests` to this job's `needs` now gates `integration-tests-latest-infrahub` (40-min huge-runner job) not just on test failures but on every step inside the `unit-tests` matrix leg, including the `Upload coverage to Codecov` and `Report coverage for pytest-plugin` steps. Those upload `CODECOV_TOKEN` to an external service, so a transient Codecov outage or upload failure marks the leg `failure` and silently skips integration coverage even when the actual tests are green. Previously integration ran independently of unit-tests/Codecov health. Consider making the coverage-upload steps conditional on test success or not part of the aggregate gate, so a non-test failure does not block the expensive integration job.</comment>

<file context>
@@ -317,7 +317,7 @@ jobs:
       !contains(needs.*.result, 'cancelled') &&
       needs.files-changed.outputs.python == 'true'
-    needs: ["prepare-environment", "files-changed", "yaml-lint", "python-lint"]
+    needs: ["prepare-environment", "files-changed", "yaml-lint", "python-lint", "unit-tests"]
     runs-on:
       group: "huge-runners"
</file context>

@ogenstad ogenstad Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified against this repo's setup - this specific risk doesn't apply here, so no change is needed.

The concern assumes a failed Codecov upload marks the unit-tests leg as failure. But the repo pins the legacy codecov==2.1.13 uploader, whose upload path is non-fatal by default. From codecov/__init__.py:

except Exception as e:
    write("Error: " + str(e))
    ...
    sys.exit(1 if codecov.required else 0)  # exits 0 unless --required is passed

The steps invoke uv run codecov --flags ... with no --required/-Z, so any upload error (transient 5xx outage, network failure, 4xx) is caught and exits 0. The leg stays success and integration is not spuriously skipped.

The reasoning is sound in the general case (codecov-cli, or this uploader with --required, both fail on upload error) - just not for the current config. Noting for the record: if we ever migrate to codecov-cli or add --required, the clean fix is continue-on-error: true on the coverage-upload steps.

runs-on:
group: "huge-runners"
timeout-minutes: 40
Expand Down Expand Up @@ -355,7 +355,7 @@ jobs:
# !contains(needs.*.result, 'cancelled') &&
# needs.files-changed.outputs.python == 'true' &&
# (github.base_ref == 'stable' || github.base_ref == 'develop')
# needs: ["prepare-environment", "files-changed", "yaml-lint", "python-lint"]
# needs: ["prepare-environment", "files-changed", "yaml-lint", "python-lint", "unit-tests"]
# runs-on:
# group: "huge-runners"
# timeout-minutes: 30
Expand Down