Skip to content

feat(e2e): add JSON file support for extra test cases - #727

Draft
mdryaaan wants to merge 1 commit into
urunc-dev:mainfrom
mdryaaan:feat/e2e-json-test-cases
Draft

mdryaaan wants to merge 1 commit into
urunc-dev:mainfrom
mdryaaan:feat/e2e-json-test-cases

Conversation

@mdryaaan

Copy link
Copy Markdown
Contributor

Description

Adds a JSON loading layer to the e2e test suite so operators can inject additional foreground test cases into nerdctlTestCases() and ctrTestCases() by editing JSON files under tests/e2e/testdata/ without modifying Go source.

JSON-defined cases always use runForegroundTest (output matching via ExpectOut) since TestFunc is a function
pointer that cannot be serialized to JSON. Docker and crictl are excluded because their test runners call tc.TestFunc(tool) directly and would panic on a nil TestFunc — those can be extended in a follow-up once a function registry or similar mechanism is added.

New files:

  • tests/e2e/json_test_cases.gojsonTestCase struct,
    loadTestCasesFromJSON, optionalJSONTestCases
  • tests/e2e/json_test_cases_test.go — unit tests
  • tests/e2e/testdata/nerdctl_extra_test_cases.json — empty by default
  • tests/e2e/testdata/ctr_extra_test_cases.json — empty by default

Modified:

  • tests/e2e/test_cases.gonerdctlTestCases() and
    ctrTestCases() now append optionalJSONTestCases(...)

Related issues

How was this tested?

Added TestLoadTestCasesFromJSON (6 subtests) and TestOptionalJSONTestCases (3 subtests) in tests/e2e/json_test_cases_test.go. All 9 subtests pass: go test -count=1 -timeout 20m ./tests/e2e/... -v -run "TestLoadTestCasesFromJSON|TestOptionalJSONTestCases" Linter passes (golangci-lint run, 0 issues). Build passes (make).

LLM usage

N/A

Checklist

  • I have read the contribution guide.
  • The linter passes locally (make lint).
  • The e2e tests of at least one tool pass locally (make test_ctr, make test_nerdctl, make test_docker, make test_crictl).
  • If LLMs were used: I have read the llm policy.

@netlify

netlify Bot commented May 26, 2026

Copy link
Copy Markdown

Deploy Preview for urunc canceled.

Name Link
🔨 Latest commit 3a009e6
🔍 Latest deploy log https://app.netlify.com/projects/urunc/deploys/6a14ebc65af0820009c9ac65

@mdryaaan
mdryaaan marked this pull request as draft May 26, 2026 00:42
@mdryaaan
mdryaaan marked this pull request as ready for review May 26, 2026 12:43
@mdryaaan
mdryaaan marked this pull request as draft May 27, 2026 05:15
@mdryaaan

Copy link
Copy Markdown
Contributor Author

Implementation JSON test cases work for nerdctl and ctr via the foreground path. Docker and crictl are excluded for now since their runners call TestFunc directly and a nil value would panic, Happy to extend those in a follow up if needed.

@github-actions

Copy link
Copy Markdown

This PR is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 15 days.

@github-actions github-actions Bot added the Stale label Jul 27, 2026
@cmainas cmainas removed the Stale label Jul 27, 2026
@cmainas

cmainas commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Hello @mdryaaan ,

can you bring this PR up-to-date with the main branch?

Fixes: urunc-dev#123
Signed-off-by: Md Raiyan <alikhurshid842001@gmail.com>
@mdryaaan
mdryaaan force-pushed the feat/e2e-json-test-cases branch from 3a009e6 to 997e9cb Compare September 16, 2026 14:42
@netlify

netlify Bot commented Sep 16, 2026

Copy link
Copy Markdown

Deploy Preview for urunc ready!

Name Link
🔨 Latest commit 997e9cb
🔍 Latest deploy log https://app.netlify.com/projects/urunc/deploys/6aaaaae622e6460008396f2b
😎 Deploy Preview https://deploy-preview-727--urunc.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make end-to-end testing read from a JSON file

2 participants