Skip to content

Pin the ECR CVE gate to the image digest instead of the mutable tag - #6520

Open
Adityaj0 wants to merge 1 commit into
aws:mainfrom
Adityaj0:fix/ecr-scan-pin-to-digest
Open

Adityaj0 wants to merge 1 commit into
aws:mainfrom
Adityaj0:fix/ecr-scan-pin-to-digest

Conversation

@Adityaj0

Copy link
Copy Markdown

Note on CONTRIBUTING.md: I am aware this repo currently states it is not accepting external contributions. This PR is offered only as a ready-made patch for #6517 — please close it without review if that is the standing policy, and treat the issue as the real report. No offence taken.

Purpose

Fixes #6517.

test/security/scripts/ecr_scan.py resolved the image digest and then never used it. sha was read from describe_images at L206 and passed only to a log line, while both describe_image_scan_findings callers still addressed the image by its mutable tag (L53, L63).

CI tags are deterministic and overwritten: compute_ci_tag.sh derives the CI tag from config metadata, so every rebuild of a config pushes the same tag to a new digest. The gate reads findings across a window in which that tag can move:

  • SCAN_WAIT_PERIOD (40) × SCAN_WAIT_LENGTH (30) = up to 20 minutes of polling
  • plus SCAN_POST_COMPLETE_WAIT (120s) before findings are read

A re-run, a dispatch build, or any other caller pushing the same CI tag inside that window redirects the query to a different image. The pipeline concurrency key includes inputs.config-file, which serializes runs of the same config, but nothing else.

The failure direction that matters for a security gate is the bad one: an image carrying non-allowlisted CRITICAL/HIGH CVEs can pass on findings belonging to a different, cleaner digest.

Change: thread the already-resolved digest through get_scan_status() and get_scan_findings() as imageId={"imageDigest": sha} — ECR accepts a digest in place of a tag in imageId. The tag stays in log output for readability. An added comment records why the digest is used, so it does not get collapsed back to the tag later.

Deliberately minimal: this is the tag/digest defect only.

Test Plan

No behavioural test is included, and I want to be straightforward about why: exercising this needs a live ECR repository with an Inspector v2 enhanced scan and a tag re-pushed mid-scan. That is beyond what a unit test can reach, and a mock that asserts "we passed a digest" would only restate the diff. The issue documents the race in full so it can be validated against real infrastructure.

What is checkable statically, and what I ran:

# No callers outside this file — the signature change is contained
grep -rn "get_scan_status\|get_scan_findings" --include="*.py" --include="*.yml" .

# No imageTag left on any scan-findings call
grep -n "imageTag\|imageDigest" test/security/scripts/ecr_scan.py

python3 -m py_compile test/security/scripts/ecr_scan.py
ruff format --check test/security/scripts/ecr_scan.py

Test Result

$ grep -rn "get_scan_status\|get_scan_findings" --include="*.py" --include="*.yml" .
test/security/scripts/ecr_scan.py:46:def get_scan_status(ecr_client, image: ImageURI, digest: str) -> str:
test/security/scripts/ecr_scan.py:61:def get_scan_findings(ecr_client, image: ImageURI, digest: str) -> list:
test/security/scripts/ecr_scan.py:220:        get_scan_status,
test/security/scripts/ecr_scan.py:229:    findings = get_scan_findings(ecr_client, image, sha)

No callers outside the file, so the signature change is self-contained.

$ grep -n "imageTag\|imageDigest" test/security/scripts/ecr_scan.py
53:            imageId={"imageDigest": digest},
63:    image_id = {"imageDigest": digest}
204:        imageIds=[{"imageTag": image.image_tag}],

The only remaining imageTag is the describe_images lookup that resolves the digest in the first place — which is correct, that is the tag→digest resolution.

$ python3 -m py_compile test/security/scripts/ecr_scan.py   # clean
$ ruff format --check test/security/scripts/ecr_scan.py
1 file already formatted

Adjacent observations (not addressed here)

Both in wait_for_status() (test/test_utils/__init__.py), independent of this fix:

  • It sleeps before the first poll, so the first status check is always one period_length late — 30s for this caller.
  • It has no terminal-state check, so a scan reporting FAILED still polls for the full 20 minutes.

Left alone to keep this PR to one defect. Happy to file separately.


Toggle if you are merging into main Branch

PR Checklist

  • I ran pre-commit run --all-files locally before creating this PR.

Not fully run — disclosing rather than checking the box. pre-commit could not install its hook environments in my sandbox (URLError: CERTIFICATE_VERIFY_FAILED fetching the hook repos). What I ran instead, directly:

  • ruff format --check on the changed file — pass (1 file already formatted), using the repo's pyproject.toml config (line-length 100, py312)
  • python3 -m py_compile — pass

Not run: typos, gitleaks, mdformat.

ecr_scan.py resolved the image digest and then never used it — `sha` was
read from describe_images and passed only to a log line, while both
describe_image_scan_findings callers still addressed the image by its
mutable tag.

CI tags are deterministic: compute_ci_tag.sh derives them from config
metadata, so every rebuild of a config pushes the same tag to a new
digest. The gate then reads findings across a window where the tag can
move — SCAN_WAIT_PERIOD * SCAN_WAIT_LENGTH is up to 20 minutes of
polling, followed by another SCAN_POST_COMPLETE_WAIT of 120 seconds
before findings are read. A re-run, dispatch build, or any other caller
pushing the same tag inside that window redirects the query to a
different image. The pipeline concurrency key includes inputs.config-file,
which serializes runs of the same config but nothing else.

The direction that matters for a security gate is the bad one: an image
carrying non-allowlisted CRITICAL/HIGH CVEs can pass on findings that
belong to a different, cleaner digest.

Thread the resolved digest through get_scan_status() and
get_scan_findings() as imageId={"imageDigest": sha}, which ECR accepts in
place of a tag. The tag is kept in log output for readability.

Closes aws#6517

Signed-off-by: Aditya Jain <adityaj0@uci.edu>
@Adityaj0

Copy link
Copy Markdown
Author

@Jyothirmaikottu can you review this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ecr_scan.py: CVE gate queries scan findings by mutable tag while the resolved digest goes unused

1 participant