Skip to content

Fix AI widget rendering and automate visual quality review - #47

Merged
everettjf merged 1 commit into
mainfrom
codex/ai-quality-evaluation
Sep 22, 2026
Merged

everettjf merged 1 commit into
mainfrom
codex/ai-quality-evaluation

Conversation

@everettjf

Copy link
Copy Markdown
Owner

AI-generated widgets could pass runtime validation while showing invisible light-mode text, clipped rings or an incorrectly narrow iPad preview. This fixes those rendering issues and adds a repeatable way to inspect generated widgets before release.

  • Capture native appearance before JavaScript worker execution; macOS no longer reports dark mode unconditionally.
  • Support documented numeric/single-string square frames, rejecting boolean, negative and non-finite values; align extra-large preview dimensions with the generation prompt.
  • Improve generation guidance for contrast, spacing, chart axes and requested data. Refresh DeepSeek presets using the service's current model list.
  • Add an opt-in eight-case generation matrix, light/dark native snapshots, component/text checks, local Vision OCR, offline HTML reports and zero-API replay. Update frame completion metadata, generated docs and both editor bundles.

Validation: DeepSeek flash baseline and improved runs each produced 16/16 renderable samples. The improved samples passed expected-text/component checks; all 32 improved screenshots were manually reviewed. A separate 16-case offline replay passed. This is bounded smoke coverage, not a universal visual-quality score. Before/replay/after screenshots and reports remain local under __Release/ai-quality/2026-09-22/.

Local validation:

  • ./Scripts/release-readiness.sh passed: 29 editor tests, 164 macOS tests, 166 iPhone simulator tests; app builds passed.
  • ./Scripts/ipad-icloud-tests.sh passed: 166 iPad Pro simulator tests.
  • AI_QUALITY_REPLAY=... AI_QUALITY_OUTPUT=... ./Scripts/ai-quality.sh
  • Metadata, documentation links and git diff --check passed. The normal suites skipped 7 macOS / 6 iOS tests requiring opt-in live services or signed iCloud.

Security/compatibility: no new permissions, dependencies or credential storage. Live tests are opt-in and use synthetic offline prompts; no user widgets are sent. Credentials and generated artifacts are excluded from the PR. Square-frame behavior is additive; existing width/height forms are preserved. Physical WidgetKit/Live Activity, signed iCloud containers and Apple PCC calls were not tested in this round. No App Store submission is performed.

@everettjf
everettjf merged commit 95c4c83 into main Sep 22, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant