test(integrations): guard multiline/control-char SKILL.md frontmatter escaping - #3392
Conversation
There was a problem hiding this comment.
Pull request overview
Fixes YAML frontmatter generation for skills-based integrations by centralizing proper YAML double-quoted scalar escaping, preventing multiline descriptions and control characters from corrupting or breaking generated SKILL.md files.
Changes:
- Added
specify_cli/_yaml_string.quote_yaml_double()to escape newlines/CR/tab and other control characters into valid YAML escapes. - Switched the manual
SKILL.mdfrontmatter renderers inSkillsIntegrationandHermesIntegrationto use the shared quoting helper. - Added a regression test to ensure multiline template descriptions round-trip through YAML parsing.
Show a summary per file
| File | Description |
|---|---|
| tests/integrations/test_integration_base_skills.py | Adds regression coverage for multiline description round-trip in generated SKILL.md frontmatter. |
| src/specify_cli/integrations/hermes/init.py | Uses shared YAML quoting helper for Hermes skill frontmatter rendering. |
| src/specify_cli/integrations/base.py | Uses shared YAML quoting helper for skills frontmatter rendering in the base skills setup path. |
| src/specify_cli/_yaml_string.py | Introduces a shared, reusable YAML double-quoted scalar escaping helper. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 4/4 changed files
- Comments generated: 1
- Review effort level: Low
mnriem
left a comment
There was a problem hiding this comment.
Please address Copilot feedback and resolve conflicts
|
good call. added a second round-trip (fcdbbcd) that feeds a C0 control char (U+0008, via a yaml escape in the source template) through quote_yaml_double and asserts it survives byte for byte. confirmed it fails on the pre-fix naive quote (raw control char makes the SKILL.md unparseable, yaml ReaderError) and passes with the fix. |
|
Please resolve conflicts |
|
Closing this as superseded rather than resolving the conflicts, since the conflict is with the fix that replaced it. This PR added a Rebasing would just re-introduce a second, parallel helper for behavior that already ships, so closing is cleaner. Happy to reopen if there's an edge case |
fcdbbcd to
c31e18d
Compare
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (1)
tests/integrations/test_integration_base_skills.py:190
- This test also runs through
TestHermesIntegration, so the unisolatedsetup()call writes the generated sample skill into the user's actual Hermes home and may destroy an existing skill. RedirectPath.home()totmp_pathbefore setup to keep the regression test hermetic.
i = get_integration(self.KEY)
template = tmp_path / "sample.md"
- Files reviewed: 1/1 changed files
- Comments generated: 2
- Review effort level: Medium
|
Please address Copilot feedback |
f4a21f0 to
fe3edd6
Compare
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (5)
tests/integrations/test_integration_base_skills.py:187
- The implementation retained after the rebase is named
yaml_quote;quote_yaml_doubledoes not exist. Update the regression-test explanation to reference the actual helper.
SKILL.md unparseable / lossy. ``quote_yaml_double`` must escape it so the
tests/integrations/test_integration_base_skills.py:169
- This inherited test also runs for Hermes, whose
setup()writes toPath.home()/.hermes/skills; unlike every Hermes-specific setup test, it does not isolatePath.home()first (test_integration_hermes.py:3-10,36-39). Running the suite can therefore overwrite a developer's realspeckit-sample/SKILL.md. Redirect Hermes' global skills directory totmp_pathbefore setup.
created = i.setup(tmp_path, m)
tests/integrations/test_integration_base_skills.py:203
- This test is inherited by Hermes and invokes its global-home setup without the isolation required by
test_integration_hermes.py:3-10. It can modify the user's real~/.hermes/skills/speckit-sample/SKILL.md; redirect the Hermes destination intotmp_pathbefore calling setup.
created = i.setup(tmp_path, m)
src/specify_cli/integrations/base.py:1633
- Splitting away line endings and joining only the frontmatter slice removes the terminal newline when a literal block scalar is the final key before
---. That changesdescription: |from"first\nsecond\n"to"first\nsecond"; the existing hostile-template regression test intest_skill_frontmatter_quoting.py:95-98catches this, while the new mixin test does not because it placesscriptsafter the description. Preserve the original line endings when parsing.
fm_lines = raw.split("\n")
src/specify_cli/integrations/base.py:1671
- This drops the newline attached to the closing template delimiter, so generated skills change from
---\n\nBodyto---\nBody, contrary to the byte-parity requirement in the comment above. Keep the closing marker's trailing line ending when stripping frontmatter.
processed_body = "\n".join(body_lines[body_close + 1 :])
- Files reviewed: 2/2 changed files
- Comments generated: 5
- Review effort level: Medium
fe3edd6 to
afc57f5
Compare
There was a problem hiding this comment.
Review details
Suppressed comments (3)
src/specify_cli/integrations/base.py:39
- The PR is described as test-only, but this production-file diff rolls back unrelated behavior already on
main: event APIs/install hooks are removed, native$//skill:invocation prefixes are removed, and the line-ending-preserving frontmatter parser is reverted. This is immediately breaking:agents.py:787andshared_infra.py:369still callresolve_command_refswith three arguments, whileintegrations/copilot/__init__.py:385still callsemit_events. Please restorebase.pytomainand keep this PR limited to the intended regression tests.
"- When constructing slash commands from hook command names, "
"replace dots (`.`) with hyphens (`-`). "
"For example, `speckit.git.commit` → `/speckit-git-commit`.\n"
tests/integrations/test_integration_base_skills.py:203
- This second inherited setup call has the same Hermes side effect: it writes the generated skill into the real
~/.hermes/skillsdirectory becausePath.home()is not redirected. Keep the regression test hermetic before invoking setup.
m = IntegrationManifest(self.KEY, tmp_path)
created = i.setup(tmp_path, m)
tests/integrations/test_integration_base_skills.py:188
quote_yaml_doubledoes not exist in this codebase; the production helper added by #3399 isyaml_quote. Keeping the stale name makes the regression documentation point maintainers to the wrong implementation.
SKILL.md unparseable / lossy. ``quote_yaml_double`` must escape it so the
value survives byte-for-byte."""
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Balanced
… escaping Add regression tests for SkillsIntegration mixin that verify: - Multiline (block-scalar) description round-trips byte-for-byte - C0/DEL control characters in description survive YAML escaping Tests properly isolate Path.home() for Hermes to prevent overwriting a developer's real global skill directory. Refs: github#3392
afc57f5 to
62c8aff
Compare
|
Thank you! |
Resolved the conflicts by rebasing onto
main.When this PR was opened it added a helper to escape multiline / control characters in generated SKILL.md frontmatter. In the meantime #3399 landed
yaml_quote()inintegrations/base.pyand routes every SKILL.md frontmatter field through it, which already provides that escaping. So the production change here was fully redundant and I dropped it during the rebase (including the now-unused_yaml_string.py).What remains is the regression coverage, which
maindid not have: two tests on the sharedSkillsIntegrationTestsmixin that round-trip a multiline description and a control-character description back through the YAML parser, asserting the rendered SKILL.md frontmatter stays valid and lossless. They run across every skills integration (agy, codex, hermes, kimi, …) and lock in theyaml_quote()behavior so a future hand-rolled quote can't silently regress it.Net diff is now test-only. Happy to close instead if you'd rather not carry the extra coverage, but it guards a real corruption path that had no direct test before.