feat: use YAML for harness project configuration

[aidandaly24]

Summary

• Replace project harness.json files with commented harness.yaml configuration.
• Support YAML-relative file:// prompt references and enable managed memory for newly scaffolded harnesses.
• Keep CLI export and generated CDK reads consistent, preserve prompt contents, and reject unknown root/model settings.

Generated harness.yaml

# Optional settings are shown with example values.

name: assistant

# Inline prompt text or a file:// path relative to this YAML file.
systemPrompt: file://./system-prompt.md

# Model
model:
  provider: bedrock
  modelId: global.anthropic.claude-sonnet-4-6
  # Output tokens per model call, rather than across the whole invocation.
  # maxTokens: 4096

# Tools
# Code Interpreter and Browser use built-in resources when no ARN is specified.
# awsIam uses the harness execution role, which must allow Gateway invocation.
# https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/gateway.html
# tools:
#   - name: code_interpreter
#     type: agentcore_code_interpreter
#   - name: browser
#     type: agentcore_browser
#   - name: research
#     type: remote_mcp
#     config:
#       remoteMcp:
#         url: https://mcp.example.com/mcp
#   - name: company_tools
#     type: agentcore_gateway
#     config:
#       agentCoreGateway:
#         gatewayArn: arn:aws:bedrock-agentcore:us-west-2:123456789012:gateway/example-1234567890
#         outboundAuth:
#           awsIam: {}

# Tool patterns: @<server-name>/<tool-name> or @builtin.
# This controls agent tool selection, not IAM permissions.
# allowedTools:
#   - "@builtin"
#   - "@research/search"

# Skill path sources refer to files already present in the runtime container.
# skills:
#   - s3Uri: s3://your-skills-bucket/skills/research/
#   - gitUrl: https://github.com/your-org/agent-skills.git
#     path: skills/research
#   - path: /opt/skills/research

# Memory
# https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/memory.html
# Alternatives: existing (name or ARN), disabled.
memory:
  mode: managed
  # Managed-memory settings. Absent values use service defaults.
  # Event retention is in days.
  # strategies:
  #   - SEMANTIC
  #   - SUMMARIZATION
  # eventExpiryDuration: 30
  # Existing-memory reference: project resource name or external ARN.
  # name: ConversationMemory
  # arn: arn:aws:bedrock-agentcore:us-west-2:123456789012:memory/example-1234567890

# Execution limits apply per invocation, across all model calls.
# maxIterations: 15
# maxTokens: 20000
# timeoutSeconds: 300

# Truncation changes the context sent to the model, not the saved memory.
# truncation:
#   strategy: sliding_window
#   config:
#     slidingWindow:
#       messagesCount: 40

# dockerfile and containerUri are mutually exclusive. With neither set, the
# service-provided environment is used. Dockerfile paths are relative to this directory.
# dockerfile: Dockerfile
# containerUri: 123456789012.dkr.ecr.us-west-2.amazonaws.com/my-harness:latest

# Environment values are stored in plaintext.
# environmentVariables:
#   LOG_LEVEL: info

# Deployment creates a role when executionRoleArn is absent.
# executionRoleArn: arn:aws:iam::123456789012:role/MyHarnessRole

# Tags
# tags:
#   team: support
#   environment: development

Breaking change: Project harness configuration now uses harness.yaml; harness.json is no longer supported.

Testing

• 3,341 Bun tests passed, including 70 reader tests; typecheck, lint, formatting, and build passed.
• 13 generated-CDK tests passed. Packaged Node/Linux native workflows were verified during implementation, with independent Node probes for the BOM/CRLF follow-up.
• Earlier isolated AWS deployment, readback, and invocation passed with managed memory and file-backed prompts. All test resources were cleaned up.

Observed service limitation: AWS runtime creation rejected a summary prompt with a trailing newline. The successful smoke run used a single-line summary; the CLI does not trim prompt contents.

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

AgentCore Harness Review

Verdict: Looks good

Nice migration from harness.json to harness.yaml. The design is careful and the test coverage is thorough — I especially appreciated:

• The FIFO / symlink-race hardening in readPrompt (nonblocking O_RDONLY + post-open stat), and the subprocess-bounded test.
• The YAML alias / shared-node test in harnessConfig.test.ts ("does not read misplaced prompt keys or overwrite shared YAML aliases") — the shallow-clone-on-descent in the reader threads that needle correctly.
• The obsolete-JSON diagnostic in both readers, and the migration guidance in README.md covering the copied agentcore/cdk app.
• Running the same reader test suite against both the CLI copy (src/io/harnessConfig.ts) and the vended CDK copy (src/assets/cdk/io/harnessConfig.ts), which mitigates drift between the near-duplicate files.
• The HarnessAuthoringSchema split that keeps file:// references out of HarnessSpecSchema while still surfacing empty-file:// errors before any files are written.
• test("reports schema errors with the YAML path before creating export output", ...) asserts export output isn't created on failure.

Nothing worth blocking on. A couple of small non-blocking notes for future consideration:

• The CLI and CDK reader files are nearly line-for-line duplicates (only the error-wrapping differs). The shared cross-runner test loop makes drift detectable, but a longer-term refactor to share source (e.g., via @aws/agentcore-cdk) would be nicer.
• readFile(configPath, "utf8") has no size guard on the YAML itself (only the referenced prompt files are capped at 1 MiB). Low risk for author-controlled input, but a cheap stat guard would be consistent with the prompt-file policy.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.11%. Comparing base (f8e1495) to head (c38ab9e).
⚠️ Report is 9 commits behind head on refactor.

Additional details and impacted files

@@             Coverage Diff              @@
##           refactor    #2297      +/-   ##
============================================
+ Coverage     97.09%   97.11%   +0.02%     
============================================
  Files           578      581       +3     
  Lines         39639    39880     +241     
============================================
+ Hits          38486    38729     +243     
+ Misses         1153     1151       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)