Problem
Issue #24 gave parse and resolve failures a model-derived suffix. Runtime failures raised by the Starlark interpreter itself — not by a handler, resolver, or policy — still return the bare sentinel, so the model gets no position and no message for a mistake entirely contained in its own program.
Measured against github.com/meigma/codemode v0.2.1 through the MCP execute tool of a downstream server:
| Submitted source |
Tool error text |
def main():\n return [1][5] |
invalid program |
def main():\n d = {"a": 1}\n return d["b"] |
invalid program |
def main():\n return undefined_name |
invalid program: <codemode>:2:9: undefined: undefined_name |
def main()\n return 1 |
invalid program: <codemode>:2:1: got newline, want ':' |
The first two rows are the gap. The underlying *starlark.EvalError already carries index 5 out of range / key "b" not in dict plus a CallStack whose frames point into the submitted source. Both the message and the position are model-derived: no host text is involved, because no host code ran.
Why it costs turns
This is not hypothetical. Building a program against a capability whose output shape I had misremembered, I read a field the capability does not return. The response was invalid program with no field name and no line, indistinguishable from a syntax error I could not find. It took two extra execute round trips plus a describe_api re-read to localize a one-token mistake, and the eventual fix was obvious once the field name was visible. A model has strictly less context than I did and will bisect by resubmission — exactly the blind-retry loop #24 set out to end.
Attribute access on a capability result is the common shape:
def main():
guest = instance.get(sandbox="demo", name="gui")
return guest.addresses # capability returns `nics`, not `addresses`
Result: invalid program. Desired: the attribute name and the line.
Proposal
Extend the #24 selectivity table with one row, keeping its principle — echo only what the model already possesses:
| Error class |
Echo to the model |
| Runtime error raised by the interpreter with no handler, resolver, or policy in the cause chain |
innermost CallStack position in the submitted source plus the interpreter message, e.g. invalid program: <codemode>:2:12: struct has no .addresses attribute |
classifyRuntimeError already unwraps with errors.Is against each sentinel class, so the branch is the existing one: when a *starlark.EvalError unwraps to no host-authored cause, the error was produced by the interpreter and its Msg contains no host text. When it does unwrap to a handler, resolver, policy, or internal failure, the current coarse sentinel must stay — the danger note in #24 still applies unchanged.
Worth guarding explicitly in tests, since it is the one way this could leak: EvalError.Msg from a failed native call embeds the wrapped cause, so the discriminator must be the cause chain, not the error type.
Acceptance sketch
return [1][5], return {"a": 1}["b"], and a missing attribute on a capability result each return invalid program with a <codemode>:line:col: position and the interpreter's own message.
- A handler returning
errors.New("db password rejected") still yields exactly capability failed, with no host text in any content item.
- Policy diagnostics, panic values, and Go stack detail remain absent.
- The Errors table and the "Authoring and recovery" list in
docs/docs/reference/mcp-tools.md describe runtime-error detail, since they are the model-facing contract.
Environment
github.com/meigma/codemode v0.2.1, stdio MCP adapter, Go 1.26.6. All four rows above are reproducible with any registered capability set; the first two need none.
Problem
Issue #24 gave parse and resolve failures a model-derived suffix. Runtime failures raised by the Starlark interpreter itself — not by a handler, resolver, or policy — still return the bare sentinel, so the model gets no position and no message for a mistake entirely contained in its own program.
Measured against
github.com/meigma/codemode v0.2.1through the MCPexecutetool of a downstream server:def main():\n return [1][5]invalid programdef main():\n d = {"a": 1}\n return d["b"]invalid programdef main():\n return undefined_nameinvalid program: <codemode>:2:9: undefined: undefined_namedef main()\n return 1invalid program: <codemode>:2:1: got newline, want ':'The first two rows are the gap. The underlying
*starlark.EvalErroralready carriesindex 5 out of range/key "b" not in dictplus aCallStackwhose frames point into the submitted source. Both the message and the position are model-derived: no host text is involved, because no host code ran.Why it costs turns
This is not hypothetical. Building a program against a capability whose output shape I had misremembered, I read a field the capability does not return. The response was
invalid programwith no field name and no line, indistinguishable from a syntax error I could not find. It took two extraexecuteround trips plus adescribe_apire-read to localize a one-token mistake, and the eventual fix was obvious once the field name was visible. A model has strictly less context than I did and will bisect by resubmission — exactly the blind-retry loop #24 set out to end.Attribute access on a capability result is the common shape:
Result:
invalid program. Desired: the attribute name and the line.Proposal
Extend the #24 selectivity table with one row, keeping its principle — echo only what the model already possesses:
CallStackposition in the submitted source plus the interpreter message, e.g.invalid program: <codemode>:2:12: struct has no .addresses attributeclassifyRuntimeErroralready unwraps witherrors.Isagainst each sentinel class, so the branch is the existing one: when a*starlark.EvalErrorunwraps to no host-authored cause, the error was produced by the interpreter and itsMsgcontains no host text. When it does unwrap to a handler, resolver, policy, or internal failure, the current coarse sentinel must stay — the danger note in #24 still applies unchanged.Worth guarding explicitly in tests, since it is the one way this could leak:
EvalError.Msgfrom a failed native call embeds the wrapped cause, so the discriminator must be the cause chain, not the error type.Acceptance sketch
return [1][5],return {"a": 1}["b"], and a missing attribute on a capability result each returninvalid programwith a<codemode>:line:col:position and the interpreter's own message.errors.New("db password rejected")still yields exactlycapability failed, with no host text in any content item.docs/docs/reference/mcp-tools.mddescribe runtime-error detail, since they are the model-facing contract.Environment
github.com/meigma/codemode v0.2.1, stdio MCP adapter, Go 1.26.6. All four rows above are reproducible with any registered capability set; the first two need none.