Feat: nl2agent - #3912
Feat: nl2agent#3912xuyaqist wants to merge 109 commits into
Conversation
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
(cherry picked from commit 0359088)
…us routing - Move knowledge_base and conversation_guide from advanced to basic tab - Move tools_skills, run_strategy, publish_attributes to advanced tab - Add section-to-tab mapping (BASIC_CONFIG_SECTIONS) for consistent routing - Complete all section DOM refs for focus requests (run_strategy, publish_attributes, collaborative_agents, guardrail) - Add activeConfigTab to focus effect dependencies to re-trigger scroll after tab switch - Replace nested ternary with centralized sectionRefs map for maintainability Co-authored-by: Claude Code <noreply@anthropic.com> Generated-by: claude-opus-4-7 Co-authored-by: Cursor <cursoragent@cursor.com>
- Add new security tab to separate security-related configurations - Move guardrail section from advanced tab to security tab - Reorder advanced tab sections: tools and skills, collaborative agents, run strategy, publish attributes - Update section-to-tab mapping to support security tab auto-navigation - Preserve NL2Agent focus behavior for all sections including guardrail Co-authored-by: Claude Code <noreply@anthropic.com> Generated-by: claude-opus-4-7 Co-authored-by: Cursor <cursoragent@cursor.com>
Let users revise, remove, or create Skills for multiple resource-gap requirements before one rediscovery run. Co-authored-by: Codex <noreply@openai.com> Generated-by: gpt-5-codex
Exclude generic interface fields from resource capability scoring and preserve core domain actions during requirement extraction. Co-authored-by: Codex <noreply@openai.com> Generated-by: gpt-5-codex
Require an evidence-based model verdict for backend-ranked candidates before\nallowing a resource to cover a requirement.\n\nCo-authored-by: Codex <noreply@openai.com>\nGenerated-by: gpt-5
Keep authorization arguments isolated and pass capability verification data to the resource resolver.\n\nCo-authored-by: Codex <noreply@openai.com>\nGenerated-by: gpt-5
Allow model verdicts for all displayed matches while retaining strong accepted matches as the coverage gate.\n\nCo-authored-by: Codex <noreply@openai.com>\nGenerated-by: gpt-5
Replace per-candidate verdicts and explanations with requirement-level accepted candidate allowlists. Validate complete requirement coverage and candidate references before deriving resource states. Co-authored-by: Codex <noreply@openai.com> Generated-by: gpt-5
Resolve the context-budget migration test conflict while retaining SQL and ORM contract coverage. Co-authored-by: Codex <noreply@openai.com> Generated-by: gpt-5
Add the two-row resource-gap controls and route MCP configuration through the existing tool modal before a unified resource recheck. Co-authored-by: Codex <noreply@openai.com> Generated-by: gpt-5
Let every interface field participate in similarity retrieval while leaving final capability coverage to model verification. Co-authored-by: Codex <noreply@openai.com> Generated-by: gpt-5
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical and moderate findings affect resource validation, binding, configuration, and installation flows.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds end-to-end NL2Agent resource-gap resolution, including resource discovery, installation, Skill/MCP configuration, verification, and binding.
Changes:
- Adds backend resource resolution, installation, configuration, and MCP tooling.
- Adds frontend resource-gap, installation, configuration, and binding flows.
- Updates prompts, localization, models, APIs, and regression tests.
Review findings:
- Critical (2 votes):
backend/tool_collection/mcp/nl2agent_mcp_tools.pylines 971, 1020, and 1477 allow unvalidated gap-card construction. - Moderate (2 votes): MCP cancellation, resource configuration payloads, and installation requests have state or request-handling issues.
- Moderate (1 vote): Skill binding state, resource hints, and inactive installation-card actions need correction.
- Nits (1 vote): English and Chinese prompt examples need safer candidate handling and consistent resource-task progression.
File summaries
| File | Description |
|---|---|
test/backend/services/test_nl2agent_service.py |
Backend service regression tests |
test/backend/services/test_mcp_internal_tool_search.py |
MCP search and tool tests |
test/backend/app/test_agent_app.py |
Agent application endpoint tests |
test/backend/agents/test_nl2agent_agent.py |
NL2Agent integration tests |
frontend/tests/nl2agentResourceResolution.test.ts |
Frontend resource-resolution tests |
frontend/services/api.ts |
Shared API request handling |
frontend/services/agentConfigService.ts |
Agent configuration API client |
frontend/public/locales/zh/common.json |
Chinese localization strings |
frontend/public/locales/en/common.json |
English localization strings |
frontend/lib/nl2agent-resource-resolution.ts |
Resource-resolution helpers |
frontend/lib/nl2agent-resource-gap.ts |
Resource-gap helpers |
frontend/lib/nl2agent-mcp-configuration.ts |
MCP configuration helpers |
frontend/contexts/nl2AgentFlow.tsx |
NL2Agent workflow state |
frontend/app/[locale]/newchat/ui/suggested-resource-installation-card.tsx |
Resource installation UI |
frontend/app/[locale]/newchat/ui/resource-gap-resolution-card.tsx |
Resource-gap resolution UI |
frontend/app/[locale]/newchat/ui/installed-resource-binding-card.tsx |
Resource binding UI |
frontend/app/[locale]/newchat/assistant-ui/thread.tsx |
Chat thread integration |
frontend/app/[locale]/newchat/adapter/remote-chat-model-adapter.ts |
Resource payload normalization |
frontend/app/[locale]/agents/components/agent-capability.tsx |
Skill and MCP configuration UI |
backend/tool_collection/mcp/nl2agent_mcp_tools.py |
MCP schemas, validation, and tools |
backend/tool_collection/mcp/nl2agent_mcp_service.py |
MCP service integration |
backend/services/skill_repository_service.py |
Skill repository operations |
backend/services/nl2agent_service.py |
Resource resolution and installation logic |
backend/prompts/nl2agent_zh.yaml |
Chinese NL2Agent workflow prompts |
backend/prompts/nl2agent_en.yaml |
English NL2Agent workflow prompts |
backend/consts/model.py |
Model and resource constants |
backend/apps/agent_app.py |
Installation and configuration endpoints |
backend/agents/nl2agent_agent.py |
NL2Agent agent integration |
Review details
Suppressed comments (10)
backend/prompts/nl2agent_en.yaml:81
- This transition requires the resource task to enter the same atomic Prompt-generation sequence, but the English prompt's atomic-action rule above limits
updated_fieldsprogression tofull_generation(the Chinese prompt includesmanage_resources). As written, a partial resource task can stop after binding instead of regenerating the required resource-dependent Prompt fields; align the English stage condition and numbered branches with this transition.
7. When the backend returns `next_action="BIND"`, pass the complete unified resolution and the same `agent_id` unchanged to `{{ wrapper_name }}` with subtype `installed_resource_binding`.
8. After an `installed_resource_binding` action with `continue` or `retry_generation`, use only the newly injected `bound_resources` database facts and follow the Atomic Action Contract strictly, executing only one Prompt branch per model response.
9. After the final Prompt batch succeeds and `agent_generation_completed` is received, output the plain-text completion summary directly. Do not call another tool or wrapper. For tool errors, use only `code` and `retryable`; retry at most once.
backend/prompts/nl2agent_en.yaml:127
- The example unconditionally indexes the first resource. An uncovered result may have no resources, which raises an index error; if it has only weak references, this code incorrectly accepts a weak candidate despite the workflow requiring candidates whose facts prove the capability. Build the accepted list only from strong/recommended candidates after the model confirms their declared facts, and use an empty list otherwise.
capability_verifications = [
{"requirement_id": "verified_research", "accepted_candidate_refs": [verification_result["resources"][0]["candidate_ref"]]},
]
backend/prompts/nl2agent_zh.yaml:128
- The example unconditionally indexes the first resource. An uncovered result may have no resources, which raises an index error; if it has only weak references, this code incorrectly accepts a weak candidate despite the workflow requiring candidates whose facts prove the capability. Build the accepted list only from strong/recommended candidates after the model confirms their declared facts, and use an empty list otherwise.
capability_verifications = [
{"requirement_id": "verified_research", "accepted_candidate_refs": [verification_result["resources"][0]["candidate_ref"]]},
]
backend/services/nl2agent_service.py:1485
- Every installed candidate is marked
is_bound=Falsehere, even though the binding card uses this flag as the agent-specific persisted binding state. The frontend only synchronizes existing tool bindings, not skills, so an already-bound skill is shown as pending and is submitted again instead of remaining bound; load the target agent's binding state before emitting this summary.
is_bound=False,
backend/tool_collection/mcp/nl2agent_mcp_tools.py:1482
- This re-resolution drops the
capability_verificationsselected by the model. If verification rejects every candidate, the backend can rediscover the same strong resource and returnBIND/INSTALL(or make the wrapper subtype mismatch), so the final card no longer reflects the verified result. Preserve the accepted candidate refs when re-resolving, rather than passing an unrestricted request.
verified = await resolve_resource_requirements_impl(
requirements=[item.requirement for item in supplied.requirements],
phase=supplied.phase,
exclude_refs=[],
tenant_id=tenant_id,
user_id=user_id,
backend/tool_collection/mcp/nl2agent_mcp_tools.py:1023
- The prompt requires a verification pass with
verification_required=Trueand thencapability_verifications, but thisToolConfigexposes neither optional argument to the NL2Agent model. Because this config is installed intocreate_nl2agent_agent_config, the model cannot reliably perform the mandated verification call. Add both parameters to the advertised input schema.
'{"agent_id":"int",'
'"requirements":"list[ResourceRequirement]",'
'"phase":"INITIAL | POST_INSTALL | POST_GAP",'
'"exclude_refs":"list[str]"}'
backend/tool_collection/mcp/nl2agent_mcp_tools.py:432
ResourceRequirementcarriesresource_name_hint, and the frontend's unchanged-gap action expects to preserve it, but this gap-card model omits the field._build_v2_resource_payloadtherefore cannot emit the original hint, so confirming an unchanged requirement silently drops matching metadata despite the prompt requiring unchanged requirements to be preserved. Add the hint to this payload and populate it when building the card.
requirement_id: str = Field(min_length=1, max_length=100)
query: str = Field(min_length=1, max_length=500)
search_terms: list[str] = Field(default_factory=list, max_length=8)
weak_references: list[WeakResourceReference] = Field(
frontend/app/[locale]/agents/components/agent-capability.tsx:179
- The only transition that marks this request complete is the success callback. The modal's cancel handler merely closes the modal, leaving the request pending and disabling the gap card; because the request ID is already recorded, reopening the same request is also suppressed. Add a cancellation/reset transition to the flow state and invoke it on cancel so an abandoned creation remains retryable.
const handleSkillBuildSuccess = useCallback(() => {
invalidateSkills();
if (
skillCreationRequest &&
skillCreationRequest.agentId === currentAgentId &&
!skillCreationRequest.completed
) {
completeSkillCreation(
skillCreationRequest.agentId,
skillCreationRequest.requestId
);
}
frontend/app/[locale]/newchat/ui/suggested-resource-installation-card.tsx:91
canContinueignoresinteractive, so a card that has become inactive can still enable Continue once its items reach terminal states and submit a duplicate action. Include the interaction state in this derived guard so only the active card can continue.
const canContinue = items.every(
(item) => item.status !== "idle" && item.status !== "installing"
);
frontend/app/[locale]/newchat/ui/suggested-resource-installation-card.tsx:259
- The failed-item Skip button remains enabled when
interactiveis false. Users can mutate an inactive installation card after a newer card has taken over, producing stale skipped results. Disable this action for inactive cards.
<Button
type="button"
size="icon"
variant="ghost"
className="size-9"
- Files reviewed: 28/28 changed files
- Comments generated: 4
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| elif subtype == "resource_gap_resolution": | ||
| if requirements is None: | ||
| raise ValueError("resource_gap_resolution requires requirements") | ||
| output = ResourceGapResolutionPayload( | ||
| agent_id=agent_id, | ||
| requirements=requirements, | ||
| ).model_dump(mode="json") |
| const handleCloseMcpModal = useCallback(() => { | ||
| setIsMcpModalOpen(false); | ||
| if ( | ||
| mcpConfigurationRequest && | ||
| mcpConfigurationRequest.agentId === currentAgentId && | ||
| !mcpConfigurationRequest.completed | ||
| ) { | ||
| completeMcpConfiguration( | ||
| mcpConfigurationRequest.agentId, | ||
| mcpConfigurationRequest.requestId | ||
| ); | ||
| } |
| content.resources = content.resources.map((resource: any) => ({ | ||
| candidate: resource, | ||
| recommendation: resource.recommendation, | ||
| is_bound: resource.is_bound, | ||
| form_kind: | ||
| resource.resource_type === "tool" ? "TOOL_CONFIG" : "SKILL_CONFIG", | ||
| config: [], | ||
| })); |
| method: "POST", | ||
| body: JSON.stringify({ | ||
| agent_id: payload.agent_id, | ||
| candidate_ref: ref(item), | ||
| }), |
Carry the backend-validated accepted candidate allowlists through the unified resolution result and reuse them when the wrapper revalidates current resource metadata. This keeps model-rejected strong matches uncovered. Co-authored-by: Codex <noreply@openai.com> Generated-by: gpt-5
# Conflicts: # backend/prompts/nl2agent_en.yaml # backend/prompts/nl2agent_zh.yaml # backend/tool_collection/mcp/nl2agent_mcp_tools.py # frontend/app/[locale]/newchat/adapter/remote-chat-model-adapter.ts # frontend/public/locales/en/common.json # frontend/public/locales/zh/common.json # test/backend/agents/test_nl2agent_agent.py # test/backend/app/test_agent_app.py
Replace the unsafe first-resource example and require per-requirement strong-match checks before accepting candidates. Add a prompt regression test for both languages. Co-authored-by: Codex <noreply@openai.com> Generated-by: gpt-5
Separate resource discovery from capability verification so the model can inspect declared candidate facts before building per-requirement allowlists. Remove the all-empty allowlist example that caused false resource gaps. Co-authored-by: Codex <noreply@openai.com> Generated-by: gpt-5
Clarify that capability verification calls must omit the initial candidate-request flag, preventing invalid mixed resolver inputs. Add a prompt regression assertion. Co-authored-by: Codex <noreply@openai.com> Generated-by: gpt-5
当资源不足时弹出资源不足卡片,支持用户创建skill,和配置mcp


支持绑定相关资源