Preserve structured output stop reasons - #1850
Open
sylvesterkaczmarek wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Preserve structured-output responses when generation terminates with
stop_reason="refusal"or"max_tokens"instead of replacing the response with a schema-validation exception.messages.parse()and the beta equivalent currently attempt to validate every text block againstoutput_formatunconditionally. Refusal text and max-token-truncated JSON are not guaranteed to satisfy the requested output schema, so those terminal responses can fail inside Pydantic/JSON parsing before the caller can inspect the model's actualstop_reason.For example, a refusal containing ordinary refusal text is currently fed to
TypeAdapter.validate_json(), and a response truncated at{"value":is treated as malformed structured output. In both cases the SDK loses the more important information: why generation stopped.Fix
Skip structured-output validation only for:
refusalmax_tokensThe text remains available unchanged,
parsed_outputisNone, and the originalstop_reasonis preserved.Normal completed responses continue through the existing validation path, so malformed structured output on an ordinary completed turn still raises as before.
The behavior is applied symmetrically to GA and beta parsed messages.
Regression coverage
Adds tests verifying:
parsed_output=Nonefor GA and beta;max_tokensreturns withparsed_output=None;