[Copilot]

Five workflows reported missing terminal safe outputs; cross-repo smoke reports lacked per-transport telemetry. Both inspected smoke runs failed checkout authentication before agent startup.

• Failure diagnostics: Retain CLI/MCP call counts, registration status, and persistence evidence for missing-output and startup failures. Distinguish unavailable telemetry from measured zero calls.
• Completion contract: Require a successful safe-output call—not a failed attempt or final prose. Reserve noop for unnecessary actions; use report_incomplete for blocked, failed, or unconfirmed work.
• Regression coverage: Cover clean-exit failure reports and compile-time guidance/reporting wiring for both cross-repo smoke workflows.

• Fixes [AW Top 10] 06 Reduce missing terminal safe output failures #71843

[pelikhan]

@copilot must be privacy preserving and not secret leaking

[Copilot]

@copilot must be privacy preserving and not secret leaking

Reviewed the reporting path: it emits only safe-output registration status and aggregate CLI/MCP/persistence counts, not tool names, arguments, or raw log contents. Diagnostics still pass through the shared secret-redaction and length-bounding pipeline, and the regression test verifies a private tool payload is absent from the report. No code change is needed.

[Copilot]

🟡 Changes recommended

Completion guidance and CLI telemetry transfer still have gaps that undermine the intended behavior.

3 open issues

• Require non-diagnostic safe output for terminal completion · New
• Align smoke prompts with blocked-task fallback rules · New
• Upload CLI audit data for conclusion report counts · New

What changed in this PR

This PR improves gh-aw failure diagnostics for missing terminal safe outputs and clarifies how agents should signal completion or incomplete work.

Changes:

• Expose recorded or unavailable CLI/MCP call counts, tool registration status, and persistence evidence.
• Tighten guidance for successful safe-output calls, noop, and report_incomplete.
• Add clean-exit reporting tests and compile-time checks for both cross-repository smoke workflows.

File	Description
pkg/​workflow/​unified_prompt_creation_test.go	Checks shared guidance and smoke-workflow collection/reporting wiring.
actions/​setup/​md/​safe_outputs_prompt.md	Clarifies successful-call requirements and incomplete-task signaling.
actions/​setup/​js/​handle_agent_failure.test.cjs	Covers clean-exit failure reports with unavailable, zero, and nonzero call counts.
actions/​setup/​js/​empty_output_outcome.test.cjs	Updates expectations for telemetry, persistence, and startup diagnostics.
actions/​setup/​js/​empty_output_outcome.cjs	Includes completion evidence in missing-output and startup-failure summaries.
actions/​setup/​js/​collect_ndjson_output.test.cjs	Updates expected synthesized failure details.

_{Review effort: Max}

[gh-aw-bot]

@copilot address the following outstanding work in one pass:

1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
2. Review (actions/setup/md/safe_outputs_prompt.md:3): report_missing_tool and report_missing_data can succeed, but the collector explicitly excludes them from terminal outputs (collect_ndjson_output.cjs:374). An agent that only emits one of these diagnostics passes this final check and still fails with missing_terminal_safe_output. Require a successful non-diagnostic safe output, using report_incomplete when the task remains blocked. - Preserve safe-output telemetry and tighten terminal-output guidance #71884 (comment)
3. Review (actions/setup/md/safe_outputs_prompt.md:5): The smoke prompts also render safe_outputs_create_pull_request.md:14 and mcp_cli_tools_with_safeoutputs_prompt.md:8–10, which still allow noop for an unready action and restrict report_incomplete to failed writes. Align those fragments with this rule so blocked tasks are not reported as successful no-ops; the same restriction occurs in safe_outputs_cli_only_transport_prompt.md:12. - Preserve safe-output telemetry and tighten terminal-output guidance #71884 (comment)
4. Review (pkg/workflow/unified_prompt_creation_test.go:102): The conclusion job rebuilds the report from the downloaded agent artifact, but that artifact omits /tmp/gh-aw/mcp-cli-audit/, the only source used for CLI call counts. Both smoke workflows therefore lose recorded CLI counts and publish unknown (not recorded) even when safeoutputs was invoked through the CLI. Upload that audit directory and assert its inclusion here alongside mcp-logs/ and safeoutputs.jsonl. - Preserve safe-output telemetry and tighten terminal-output guidance #71884 (comment)

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef state: 92af973eb39228204d6ba20742dabbc64f4e9479c1e394a52d8e420a0c84fff8

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · ~17 AIC · ⌖ ~9 AIC · ⊞ 1.1K · ◷
_{Comment /souschef to run again}

[pelikhan]

@copilot avoid new file, add entry to agent unified session file

[Copilot]

@copilot address the following outstanding work in one pass:

1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes....

Completed in f9c1ad1: merged the latest main, fixed the prompt wording caught by the existing Go test, added aggregate-only CLI audit telemetry to the existing unified session file, and refreshed the affected wasm goldens. The three review threads already have replies; I can’t resolve them with the available tools. CI was not rerun.

@copilot avoid new file, add entry to agent unified session file

Done in f9c1ad1: no repository file was added; CLI aggregate telemetry is recorded in the existing usage/aw_session.jsonl as mcp.cli_audit.

[pelikhan]

@copilot avoid adding new folder if not needed. recompile

[Copilot]

Moved the audit files to /tmp/gh-aw/mcp-cli-audit.<server>.audit.jsonl, removed the extra directory from artifact uploads, and recompiled the workflows. Fixed in 900c438.