Skip to content

fix(frontend): stop showing a stale model catalog for a rejected key - #6472

Open
Aman-goel-04 wants to merge 3 commits into
Agenta-AI:mainfrom
Aman-goel-04:fix/listed-openai-models-for-invalid-key
Open

fix(frontend): stop showing a stale model catalog for a rejected key#6472
Aman-goel-04 wants to merge 3 commits into
Agenta-AI:mainfrom
Aman-goel-04:fix/listed-openai-models-for-invalid-key

Conversation

@Aman-goel-04

Copy link
Copy Markdown
Contributor

Summary

Fixes #6443

The provider connection drawer was silently showing a 34-model shipped catalog for a key OpenAI had just rejected, with no indication the list wasn't real, and the rejection message ran two sentences together with no punctuation between them.

Root cause: credentialStatusLine unconditionally stripped the API message's trailing period, on the assumption a · N models fetched suffix would always follow. When modelCount is null (no fetch happened, e.g. an invalid key), no suffix is appended, so the caller's separately-appended " Nothing has been saved." landed directly after the now-period-less verdict. Separately, ProviderConnectionCard silently fell back to Agenta's bundled model catalog whenever discovery didn't return a live list, with no signal to the user that the list wasn't the key's actual models, true for a rejected key, an untested key, and a transport failure alike.

This change:

  • Moves the period-strip in credentialStatusLine inside the modelCount !== null branch, so a fetch-less verdict keeps its own punctuation.
  • Adds a note in ActiveModelsSection ("Showing Agenta's shipped catalog. X's own list was not fetched.") whenever discoveryStatus isn't "fetched", so the fallback list is never presented as if it came from the key.
  • Stops painting the credential status dot green for unknown verdicts, it now only goes green on a confirmed valid status.

Testing

Verified locally

Ran the updated flow against a rejected OpenAI key and a valid one in the local app; confirmed the rejection message now reads as two properly punctuated sentences, and the fallback-catalog note appears under Active Models whenever a live fetch didn't happen. Confirmed doneState and probeFailureMessage needed no changes, both already routed through the fixed credentialStatusLine / already treated unknown correctly.

Added or updated tests

Updated the assertion in provider-card-copy.test.ts for credentialStatusLine's modelCount === null case to expect the trailing period to be preserved. Full agenta-entities unit suite passes (100 test files, 1460 tests). Integration suite skipped locally (no AGENTA_API_URL/AGENTA_AUTH_KEY), unrelated to this change.

QA follow-up

N/A

Demo

Before: rejected-key message runs together with no punctuation, and the shipped catalog appears with no indication it isn't the key's real list.

Screenshot 2026-09-01 at 9 26 00 PM

After: message reads as two clean sentences, and a note under Active Models states the list is Agenta's shipped catalog rather than a live fetch.

Screenshot 2026-09-01 at 10 39 52 PM

Checklist

  • Demo shows the real app running this branch (not a mock-up or recreated UI), or is marked N/A
  • Relevant tests pass locally
  • Relevant linting and formatting pass locally
  • I have signed the CLA, or I will sign it when the bot prompts me

Copilot AI lite review requested due to automatic review settings September 1, 2026 17:32
@vercel

vercel Bot commented Sep 1, 2026

Copy link
Copy Markdown

@Aman-goel-04 is attempting to deploy a commit to the agenta projects Team on Vercel.

A member of the Team first needs to authorize it.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes misleading UI in the provider connection drawer when a provider key is rejected or otherwise doesn’t yield a live model discovery result, ensuring users aren’t shown a bundled catalog as if it came from the provider and improving the credential status copy/indicator behavior.

Changes:

  • Fixes credentialStatusLine punctuation handling so messages keep their trailing period when no model fetch occurred.
  • Passes discovery status into the Active Models UI and adds a fallback-catalog note when discovery didn’t return a live list.
  • Updates the credential status indicator so the dot is only green for confirmed valid verdicts.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
web/packages/agenta-entity-ui/src/secretProvider/ProviderConnectionCard.tsx Passes discovery status/title into the models section and adjusts credential status indicator styling.
web/packages/agenta-entity-ui/src/secretProvider/ActiveModelsSection.tsx Adds a user-visible note when the displayed model list is the bundled fallback rather than a fetched provider list.
web/packages/agenta-entities/tests/unit/provider-card-copy.test.ts Updates unit expectations so credentialStatusLine(..., null) preserves the trailing period.
web/packages/agenta-entities/src/secret/core/cardCopy.ts Fixes credentialStatusLine to only strip trailing periods when appending the “· N models fetched” suffix.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread web/packages/agenta-entity-ui/src/secretProvider/ActiveModelsSection.tsx Outdated
Comment on lines 470 to +476
>
{credentialFailed ? (
<WarningCircle size={14} className="mt-0.5 shrink-0" />
) : (
) : credentialStatus === "valid" ? (
<span className="mt-1.5 size-1.5 shrink-0 rounded-full bg-colorSuccess" />
) : (
<span className="mt-1.5 size-1.5 shrink-0 rounded-full bg-colorWarning" />
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 1, 2026 17:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

web/packages/agenta-entity-ui/src/secretProvider/ProviderConnectionCard.tsx:477

  • The dot is now only green for credentialStatus === "valid", but the surrounding status line still uses text-colorSuccess for every non-failed probe (including unknown). That keeps unknown/indeterminate verdicts visually “successful”. Consider keying the text color off credentialStatus === "valid" as well (and using a warning/neutral tone otherwise) so the dot and text communicate the same state.
                            ) : credentialStatus === "valid" ? (
                                <span className="mt-1.5 size-1.5 shrink-0 rounded-full bg-colorSuccess" />
                            ) : (
                                <span className="mt-1.5 size-1.5 shrink-0 rounded-full bg-colorWarning" />
                            )}

Comment on lines +94 to +100
const catalogNote =
discoveryStatus === "fetched"
? null
: discoveryStatus == null
? `Showing Agenta's shipped catalog. ${title}'s own list has not been fetched yet.`
: `Showing Agenta's shipped catalog. ${title}'s own list was not fetched.`

@coderabbitai

coderabbitai Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Team

Run ID: 9b73b7ac-c301-4bc8-a88f-737bffc187e6

📥 Commits

Reviewing files that changed from the base of the PR and between 0e074db and b3fb36f.

📒 Files selected for processing (4)
  • web/packages/agenta-entities/src/secret/core/cardCopy.ts
  • web/packages/agenta-entities/tests/unit/provider-card-copy.test.ts
  • web/packages/agenta-entity-ui/src/secretProvider/ActiveModelsSection.tsx
  • web/packages/agenta-entity-ui/src/secretProvider/ProviderConnectionCard.tsx
🚧 Files skipped from review as they are similar to previous changes (4)
  • web/packages/agenta-entity-ui/src/secretProvider/ProviderConnectionCard.tsx
  • web/packages/agenta-entities/tests/unit/provider-card-copy.test.ts
  • web/packages/agenta-entity-ui/src/secretProvider/ActiveModelsSection.tsx
  • web/packages/agenta-entities/src/secret/core/cardCopy.ts

Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review.


📝 Walkthrough

Summary by CodeRabbit

  • UI Improvements
    • Clarified when displayed active models come from Agenta’s catalog rather than the provider’s fetched list.
    • Improved credential status indicators to distinguish confirmed credentials from pending or unknown statuses.
    • Preserved punctuation in credential error messages for clearer feedback.

Walkthrough

The provider card now preserves credential punctuation, labels shipped catalog fallbacks, distinguishes unknown or pending credentials from valid credentials, and passes discovery status to the active-models section.

Changes

Provider discovery clarity

Layer / File(s) Summary
Credential status line punctuation
web/packages/agenta-entities/src/secret/core/cardCopy.ts, web/packages/agenta-entities/tests/unit/provider-card-copy.test.ts
Trailing periods remain when no model count is available. The unit test expects the preserved period.
Catalog fallback note
web/packages/agenta-entity-ui/src/secretProvider/ActiveModelsSection.tsx
The section accepts title and discoveryStatus props. It displays a note when the shipped catalog replaces a provider-fetched list.
Provider card status wiring
web/packages/agenta-entity-ui/src/secretProvider/ProviderConnectionCard.tsx
The card derives discovery status, uses an amber indicator for non-valid non-failed credentials, and passes discovery data to ActiveModelsSection.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: 🟡 Moderate · up to b3fb3

The PR improves rejected-key messaging and explains when a shipped model catalog is shown, but the current behavior may still mislabel non-fetched model lists and show green success text for unknown or pending credentials, which can mislead users about available models and key validity; these bounded correctness issues should be fixed or explicitly accepted before merge.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes satisfy issue #6443 by preserving punctuation, labeling the fallback catalog when discovery is not fetched, and limiting the green credential indicator to confirmed valid credentials.
Out of Scope Changes check ✅ Passed All changes are limited to the issue requirements, related UI behavior, supporting copy logic, and the affected unit test.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 4…
Description check ✅ Passed The description clearly explains the punctuation fix, fallback catalog notice, credential status change, testing, and issue context. It is directly related to the changeset.
Title check ✅ Passed The title clearly identifies the main frontend fix: preventing a stale model catalog from appearing for a rejected key.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 4 files.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Team

Run ID: c75f4fae-772e-4ea2-996d-093f68a1d465

📥 Commits

Reviewing files that changed from the base of the PR and between 0e074db and b3fb36f.

📒 Files selected for processing (4)
  • web/packages/agenta-entities/src/secret/core/cardCopy.ts
  • web/packages/agenta-entities/tests/unit/provider-card-copy.test.ts
  • web/packages/agenta-entity-ui/src/secretProvider/ActiveModelsSection.tsx
  • web/packages/agenta-entity-ui/src/secretProvider/ProviderConnectionCard.tsx

Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review.

Comment on lines +94 to +99
const catalogNote =
discoveryStatus === "fetched"
? null
: discoveryStatus == null
? `Showing Agenta's shipped catalog. ${title}'s own list has not been fetched yet.`
: `Showing Agenta's shipped catalog. ${title}'s own list was not fetched.`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Render the catalog note only for an actual catalog fallback.

discoveryStatus !== "fetched" does not prove that the displayed options came from Agenta's shipped catalog. Credential-set providers are documented as discovery- and manual-entry-only, so this text can claim a catalog is shown when it is not. Pass an explicit fallback flag and gate this note on that flag.

Comment on lines +473 to +476
) : credentialStatus === "valid" ? (
<span className="mt-1.5 size-1.5 shrink-0 rounded-full bg-colorSuccess" />
) : (
<span className="mt-1.5 size-1.5 shrink-0 rounded-full bg-colorWarning" />

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Use non-success text styling for unconfirmed credentials.

The new amber dot is inside a parent span that still uses text-colorSuccess for every non-failed status. An unknown or pending verdict can therefore appear with green status text. Apply the same valid/failed/other classification to the parent text color.

@Aman-goel-04

Copy link
Copy Markdown
Contributor Author

@ashrafchowdury, please review this PR and let me know if any changes need to be made!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

(bug) The provider drawer lists 34 models for a key that OpenAI just rejected

2 participants