feat(compile): add Conclusion job for pipeline failure reporting#1076
Conversation
1720ec7 to
67e3050
Compare
🔍 Rust PR ReviewSummary: Generally solid — the Conclusion job wires up correctly and the TypeScript logic handles edge cases well. Two issues worth fixing before merge. Findings🐛 Bugs / Logic Issues
|
🔍 Rust PR ReviewSummary: Needs changes — two correctness bugs that will silently break the core feature (failure detection and per-tool opt-out), plus a data-loss codemod bug. Findings🐛 Bugs / Logic Issues1. let agent_result = format!("$[dependencies.{}.result]", agent_id.as_str());
// ...
conclusion_step = conclusion_step
.with_env("AW_AGENT_RESULT", EnvValue::Literal(agent_result))
.with_env("AW_DETECTION_RESULT", EnvValue::Literal(detection_result))
.with_env("AW_SAFEOUTPUTS_RESULT", EnvValue::Literal(safeoutputs_result));ADO only evaluates The fix is to hoist these to job-level 2. Per-tool configs are emitted by Rust but never consumed by The Rust compiler passes for tool_key in &["noop", "missing-tool", "missing-data"] {
if let Some(tool_config) = front_matter.safe_outputs.get(*tool_key) {
let env_key = format!("AW_{}_CONFIG", tool_key.to_uppercase().replace('-', "_"));
// ...
}
}But
The conclusion test ( 3. Codemod silently drops non-mapping let Value::Mapping(wi_map) = wi_val else {
// work-item was present but not a mapping — put it back as-is
continue; // <── wi_val was already removed; this drops it
};
let Value::Mapping(wi_map) = wi_val else {
tool_map.insert(wi_key, wi_val); // restore
continue;
};4. Default work-item type is workItemType: readOptionalEnv(env, "AW_WORK_ITEM_TYPE") ?? "Bug",
✅ What Looks Good
|
🔍 Rust PR ReviewSummary: Solid architecture with one wiring gap and a few logic nuances worth addressing before merge. Findings🐛 Bugs / Logic Issues
Users can configure per-tool settings via
if (!config.reportFailureAsWorkItem) {
logInfo("Conclusion work-item filing disabled via AW_REPORT_FAILURE_AS_WORK_ITEM=false");
return 0; // exits before filing noop/missing-tool/missing-data WIs too
}The name and the PR description ("Global opt-out — no failure work items at all") suggest this flag is intentionally global. But return {
enabled: config.reportFailureAsWorkItem, // always true here because of the early return above
...
};This is dead code — No per-tool opt-out path for
This should be documented; it's a footgun for users who want "no pipeline-failure WIs but still want noop WIs."
|
🔍 Rust PR ReviewSummary: needs changes — two correctness bugs and a documentation/front-matter mismatch that would produce hard parse errors for users following the docs. Findings🐛 Bugs / Logic Issues
if has_write_sc {
conclusion_step = conclusion_step.with_env(
"SYSTEM_ACCESSTOKEN",
EnvValue::AdoMacro("$(SC_WRITE_TOKEN)"),
);
}
The correct form is
conclusion:
work-item-type: Bug
area-path: "MyProject\\MyTeam"But The PR description itself says "Lives entirely under
The table in
The test fixture uses 🔒 Security Concerns
|
🔍 Rust PR ReviewSummary: Looks good overall — clean architecture shift, solid test coverage, and the migration path (codemod + backward-compat) is well-handled. A few issues worth addressing before merge. Findings🐛 Bugs / Logic Issues
|
🔍 Rust PR ReviewSummary: Solid implementation with one real resilience bug worth fixing before merge; everything else is high quality. Findings🐛 Bugs / Logic Issues
|
🔍 Rust PR ReviewSummary: Good architectural direction — moving WI filing from Stage 3 into an always-running Conclusion job is the right separation of concerns. One real bug will silently neutralise the manifest-based signal reporting (noop/missing_tool/missing_data) on every pipeline run. Findings🐛 Bugs / Logic Issues
The Conclusion job downloads the // build_conclusion_job – download step
.with_input("artifact", "safe_outputs")
.with_input("path", "$(Pipeline.Workspace)/conclusion_inputs");
// ...and the env var:
EnvValue::Literal("$(Pipeline.Workspace)/conclusion_inputs".to_string()),But the "ado-aw execute ... --safe-output-dir \"$(Pipeline.Workspace)/analyzed_outputs_$(Build.BuildId)\" ..."The ...and skip noop/missing_tool/missing_data signal reporting entirely. Pipeline-failure reporting (driven by upstream job results, not the manifest) is unaffected and will still work. Fix — add a copy into cp "$(Pipeline.Workspace)/analyzed_outputs_$(Build.BuildId)/safe-outputs-executed.ndjson" \
"$(Agent.TempDirectory)/staging/safe-outputs-executed.ndjson" 2>/dev/null || true
|
🔍 Rust PR ReviewSummary: Solid feature with good test coverage — one correctness bug needs fixing before merge. Findings🐛 Bugs / Logic Issues
|
🔍 Rust PR ReviewSummary: Solid architecture, but there's a blocking bug: the documented Findings🐛 Bugs / Logic Issues
🔒 Security Concerns
|
Add an always-running Conclusion job to the canonical pipeline shape that reports pipeline failures and diagnostic signals (noop, missing-tool, missing-data) to Azure DevOps work items. Key changes: - New ConclusionConfig front-matter type (conclusion: block) - New conclusion.js ado-script bundle (TypeScript, ncc-bundled) - build_conclusion_job() in agentic_pipeline.rs with condition: always() - Work-item write helpers in shared/wit.ts (create, comment, WIQL dedup) - Removed work-item filing from noop/missing-tool safe-output executors (moved to conclusion job - separation of concerns) - Renamed src/safeoutputs/ to src/safe_outputs/ (snake_case convention) Pipeline shape: Setup -> Agent -> Detection -> SafeOutputs -> Teardown -> Conclusion Co-authored-by: Copilot <[email protected]>
…oss-platform path in conclusion test - noop/missing-tool now return plain success (not warning) since work-item filing moved to the Conclusion job - conclusion test uses stringContaining for cross-platform path separators Co-authored-by: Copilot <[email protected]>
- Use anyhow::Context instead of unwrap_or_default for tags serialization (propagates errors instead of silently dropping tags) - Change DownloadPipelineArtifact step to condition: always() with continueOnError: true (avoids red failure noise when artifact is unavailable due to early Agent failure) Co-authored-by: Copilot <[email protected]>
…tten codemod Non-breaking change: conclusion job config now lives under safe-outputs: (aligned with gh-aw's pattern) instead of a separate conclusion: block. Three opt-out levels: - Global: safe-outputs: report-failure-as-work-item: false - Per-tool: safe-outputs: noop: report-as-work-item: false - Disable tool: safe-outputs: noop: false The Conclusion job is always emitted when safe-outputs: exists (gh-aw pattern where noop is always-on). Per-tool config (title-prefix, work-item-type, area-path, tags) is passed as JSON env vars (AW_NOOP_CONFIG, AW_MISSING_TOOL_CONFIG, AW_MISSING_DATA_CONFIG). Adds codemod 0003_flatten_work_item_config that migrates the old nested work-item: sub-block to the flat form (title -> title-prefix, enabled -> report-as-work-item). Co-authored-by: Copilot <[email protected]>
…onfigured When permissions.write is set, the Conclusion job must use the service-connection-minted token (SC_WRITE_TOKEN) instead of System.AccessToken — matching the SafeOutputs job's pattern. Co-authored-by: Copilot <[email protected]>
1. Hoist $[dependencies.*.result] to job-level variables (ADO only evaluates runtime expressions in variables:/condition:, NOT step env). Step env now uses $(AW_AGENT_RESULT) macro references. 2. conclusion.js now reads per-tool configs from AW_NOOP_CONFIG, AW_MISSING_TOOL_CONFIG, AW_MISSING_DATA_CONFIG env vars. Per-tool opt-out (report-as-work-item: false) and per-tool overrides (title-prefix, work-item-type, area-path, tags) now actually work. 3. Codemod 0003 no longer drops non-mapping work-item: values — restores the value to the mapping if it is not a Mapping (e.g. work-item: true). 4. Default work-item type changed from Bug to Task (matching the documented default and the old WorkItemReportConfig convention). Filed #1081 for the type-level fix (EnvValue::RuntimeExpression variant). Co-authored-by: Copilot <[email protected]>
Matches gh-aw pattern: each per-tool config field gets its own env var (AW_NOOP_TITLE_PREFIX, AW_NOOP_WORK_ITEM_TYPE, etc.) instead of a single AW_NOOP_CONFIG JSON blob. Avoids ADO variable expansion corrupting JSON payloads. Also removes dead global env var reads from conclusion.js (AW_WORK_ITEM_TYPE, AW_WORK_ITEM_AREA_PATH, etc.) that the compiler never emitted. Co-authored-by: Copilot <[email protected]>
…kout 1. SC_WRITE_TOKEN: use EnvValue::secret (not AdoMacro) to avoid $($(SC_WRITE_TOKEN)) double expansion and allowlist bypass 2. Remove checkout_self_step from Conclusion job — only needs downloaded bundle and pipeline artifact, saves ~30s 3. Rename shadowed prefix → env_prefix in per-tool loop 4. Fix docs/conclusion.md: config lives under safe-outputs: not a stale conclusion: block; fix field name (title-prefix not work-item-title) and default (Task not Bug) 5. Fix docs/front-matter.md: remove conclusion: field reference Co-authored-by: Copilot <[email protected]>
AdoMacro values must be bare variable names (e.g. "Build.Reason"); the $() wrapper is added by lowering. Direct construction bypassing the ado_macro() factory could pass pre-wrapped names like "$(SC_WRITE_TOKEN)" which would emit $($(SC_WRITE_TOKEN)) — a double expansion that ADO silently drops to an empty string. The guard rejects names containing $, (, or ) in both lower_env_value and lower_env_value_as_expr_atom, with an actionable error message pointing to PipelineVar/secret. Co-authored-by: Copilot <[email protected]>
…r fix
1. titlePrefix now supports {pipeline_name}/{signal} placeholders via
renderTitle (was dead code — always called with undefined template)
2. report-as-work-item env var uses as_bool()/as_str() instead of
v.to_string() which JSON-encodes strings as "\"false\"" — silently
inverting the opt-out
3. Document pipeline_failure has no per-tool config (deliberate)
4. Document enabled field is always true from conclusion.js (guarded
by main() early return and per-tool opt-out in fileSignal)
Co-authored-by: Copilot <[email protected]>
…fy pipeline_failure 1. conclusion.js step now guards against missing node binary — logs a warning and exits 0 instead of failing with exit 127. This ensures the Conclusion job never fails the pipeline on infra issues (UseNode failure, missing runner tooling). 2. Remove duplicated comment block in build_conclusion_job. 3. Add comment in fileSignal explaining why pipeline_failure has no per-tool config entry (intentional, matches gh-aw). Co-authored-by: Copilot <[email protected]>
…bVariable import 1. Copy safe-outputs-executed.ndjson from analyzed_outputs to the safe_outputs artifact staging directory. Without this, the Conclusion job could never find the manifest — noop/missing-tool/ missing-data signal reporting was silently broken. 2. Move JobVariable import to file-level (consistent with other IR imports). Co-authored-by: Copilot <[email protected]>
…align title-prefix
1. Copy safe-outputs-executed.ndjson into the safe_outputs artifact
so the Conclusion job can find noop/missing-tool/missing-data
entries. Without this, diagnostic signal reporting was silently
skipped (pipeline-failure reporting was unaffected).
2. Extend bash guard to also check conclusion.js exists on disk
before running node — handles download failures gracefully.
3. Align title-prefix with gh-aw: it is a prefix prepended to the
pipeline name (${titlePrefix} ${pipelineName}), not a template
with placeholder substitution. Matches gh-aw missing_issue_helpers.
4. Move JobVariable import to file-level (consistency with other IR
imports).
Co-authored-by: Copilot <[email protected]>
validate_safe_outputs_keys rejected report-failure-as-work-item as an unrecognised tool name, breaking the documented global opt-out. Add SAFE_OUTPUT_CONFIG_KEYS for global config-level keys (not tool names) and allow them in the validation pass. Co-authored-by: Copilot <[email protected]>
…ion job Addresses review feedback plus two latent bugs found while re-evaluating against main (which now has the supply-chain mirror feature, #1080): - Conclusion job now reuses ado_script::install_and_download_steps_typed, which (a) respects the supply-chain feed mirror and (b) unzips to /tmp/ado-aw-scripts/ (the hand-rolled copy double-nested to /tmp/ado-aw-scripts/ado-script/ado-script/, so conclusion.js was never at the referenced path). - release.yml now auto-globs ado-script/*.js instead of a manual file list, so every built bundle ships. conclusion.js was missing from the manual list and would never have reached a release. - New bundle-coverage vitest guards that every src/<name>/ bundle dir is wired into the npm build chain (the safety net for auto-globbing). - buildTitle truncates to ADO's 255-char System.Title limit. - Ported the WIQL trust-boundary comment to wit.ts. Co-authored-by: Copilot <[email protected]>
c3188ff to
623d410
Compare
🔍 Rust PR ReviewSummary: Solid implementation of the Conclusion job feature — architecture is well-thought-out and follows existing conventions. Three items worth addressing before merge. Findings🐛 Bugs / Logic Issues
When
|
…ep-report-failures # Conflicts: # .github/workflows/release.yml # AGENTS.md # docs/ado-script.md # scripts/ado-script/package.json # src/compile/agentic_pipeline.rs # src/compile/ir/tasks.rs # src/runtimes/node/extension.rs # src/safe_outputs/add_pr_comment.rs
🔍 Rust PR ReviewSummary: Solid implementation with one correctness gap in the split-review case and a sanitization regression to address. Findings🐛 Bugs / Logic Issues
When a pipeline has
Concrete scenario: all tools are The analogous Teardown gap is documented and intentional (you don't want Teardown blocked behind human approval). But the reporting gap is different — Conclusion is specifically meant to detect failures. Either add Codemod doesn't handle
The codemod targets 🔒 Security Concerns
The old if let Some(v) = obj.get("title-prefix").and_then(|v| v.as_str()) {
conclusion_step = conclusion_step
.with_env(format!("{env_prefix}_TITLE_PREFIX"), EnvValue::Literal(v.to_string()));
}These values come from operator-controlled front matter (not agent-generated content), so the risk is lower than agent content — but the project's explicit convention ( The minimal fix is to call
|
…usion WI config Address PR review feedback: - Conclusion job now depends on and surfaces AW_SAFEOUTPUTS_REVIEWED_RESULT in the mixed manual-review split, so a reviewer rejection is reported. - Sanitize title-prefix/work-item-type/area-path/iteration-path config fields before embedding in pipeline env vars. - Clarify AW_AGENT_RESULT literal and DownloadPipelineArtifact continue_on_error. Co-authored-by: Copilot <[email protected]>
…st env Co-authored-by: Copilot <[email protected]>
🔍 Rust PR ReviewSummary: Solid implementation with a few issues worth addressing before merge. Findings🐛 Bugs / Logic Issues
|
- Sanitize front_matter.name before AW_PIPELINE_NAME for consistency with the other operator-controlled config fields (review: sanitization gap). - Guard the conclusion node invocation with '|| true' so the always-run Conclusion job never fails on a non-zero node exit (OOM/SIGKILL), honoring the 'always runs, never fails' contract. - Make the codemod hoist loop skip non-string YAML keys defensively instead of coercing them to an empty-string key via unwrap_or_default. - Clarify docs that report-failure-as-work-item is a global master kill-switch that suppresses all signals (the global-vs-per-tool scope is intentional and covered by an existing conclusion unit test). Co-authored-by: Copilot <[email protected]>
… step A blanket '|| true' in the bash body swallows every non-zero node exit with zero visibility. Use ADO's continueOnError on the conclusion.js step instead: the pipeline still stays green (honoring the 'always runs, never fails' contract) but a genuine failure (node OOM/SIGKILL, unhandled rejection) is surfaced in the timeline as a warning rather than silently hidden. Add a test asserting the step emits continueOnError: true. Co-authored-by: Copilot <[email protected]>
🔍 Rust PR ReviewSummary: Well-structured feature with solid test coverage. Two findings worth addressing before merge — one is a silent semantic break in the codemod, one is a minor sanitization inconsistency. Findings🐛 Bugs / Logic Issues
|
- Reject ADO runtime expressions ('$[ ... ]') on provider keys: ADO does not
evaluate runtime expressions inside a step env: block (the literal is passed
verbatim, ref #1076), so only macros '$(...)' are now permitted. Previously
'$[...]' was accepted at validation but would fail IR lowering / ship broken.
- Make provider-key matching case-sensitive (exact) across is_provider_expr_env_key,
copilot_byom_active, copilot_byom_credential_keys, the base-URL host lookup, and
the copilot_provider_env prefix filter. The Copilot CLI only reads uppercase
COPILOT_PROVIDER_*; a lowercase lookalike is never usable, so exact-case matching
fails closed instead of half-activating BYOM and breaking silently at runtime.
- Replace the magic number 17 with a COPILOT_PROVIDER_PREFIX const + starts_with.
- Document that WIRE_API is deliberately excluded from the credential set, and that
AWF injects placeholder COPILOT_PROVIDER_* values into the agent container
(api-proxy-sidecar.md) so --exclude-env + sidecar keep the real credential out.
- Update tests/docs accordingly; add case-sensitivity and $[...]-rejection tests.
Co-authored-by: Copilot <[email protected]>
* feat(engine): allow ADO expressions on Copilot BYOM provider env keys Closes #1261. Aligns engine.env validation with gh-aw's BYOM/BYOK model: an allowlist of COPILOT_PROVIDER_* keys may carry ADO macro/runtime expressions so Foundry provider credentials can be sourced from Setup-job outputs; all other keys remain literal-only. A literal COPILOT_PROVIDER_BASE_URL host is auto-added to the AWF network allowlist. Co-authored-by: Copilot <[email protected]> * feat(engine): isolate Copilot BYOM provider credentials via AWF api-proxy sidecar When engine.env activates Copilot BYOM/BYOK mode (a COPILOT_PROVIDER_* credential key is present), the compiler now enables the AWF api-proxy sidecar (--enable-api-proxy) on the agent step and pre-pulls its container image. The sidecar holds the real provider credential and injects it only at the proxy layer, so the secret never reaches the agent container or the Copilot CLI process. The credential keys are also passed as --exclude-env flags (defense-in-depth). Verified AWF v0.27.9 (the pinned version) exposes --enable-api-proxy and ships the copilot-byok provider. Co-authored-by: Copilot <[email protected]> * feat(engine): extend Copilot BYOM credential isolation to the Detection stage The Detection (threat-analysis) stage now inherits the same COPILOT_PROVIDER_* routing and AWF api-proxy credential isolation as the main agent, matching gh-aw (whose detection engine config inherits the main engine's env). Without this, Foundry-only workflows — which have no GitHub Copilot token — could not run threat detection at all. - engine.rs: add copilot_provider_env() (renders the COPILOT_PROVIDER_* subset) and extract render_engine_env_entry() shared by copilot_env/copilot_provider_env. - agentic_pipeline.rs: thread detection_provider_env; run_threat_analysis_step applies the provider env + --enable-api-proxy/--exclude-env; detection job pre-pulls the api-proxy image when BYOM is active. Extract awf_api_proxy_flags() shared by the agent and detection AWF invocations. Co-authored-by: Copilot <[email protected]> * fix(engine): address Rust PR review feedback on BYOM isolation - Validate the BYOM COPILOT_PROVIDER_BASE_URL host with is_valid_hostname before adding it to the AWF allow-domains list, consistent with the api-target path (rejects IPv6 literals / non-DNS-safe hosts). [security] - Derive AWF --exclude-env from the actual provider credential keys present in engine.env (case-preserving) instead of a hardcoded uppercase set, so a user-written lowercase key cannot activate the sidecar yet still leak into the agent container. [isolation bug] - Gate BYOM detection on the Copilot engine type to prevent a future non-Copilot engine from erroneously activating the sidecar. [footgun] - Reword the engine.env expression validation errors to render clearly ('$(...)', '${{ }}', '$[...]') and drop the stray unbalanced paren. - Tighten the integration assertion to the exact emitted (unquoted) provider env line; add unit tests for host rejection and case-preserving key derivation. Co-authored-by: Copilot <[email protected]> * fix(engine): tighten BYOM provider env validation per PR review - Reject ADO runtime expressions ('$[ ... ]') on provider keys: ADO does not evaluate runtime expressions inside a step env: block (the literal is passed verbatim, ref #1076), so only macros '$(...)' are now permitted. Previously '$[...]' was accepted at validation but would fail IR lowering / ship broken. - Make provider-key matching case-sensitive (exact) across is_provider_expr_env_key, copilot_byom_active, copilot_byom_credential_keys, the base-URL host lookup, and the copilot_provider_env prefix filter. The Copilot CLI only reads uppercase COPILOT_PROVIDER_*; a lowercase lookalike is never usable, so exact-case matching fails closed instead of half-activating BYOM and breaking silently at runtime. - Replace the magic number 17 with a COPILOT_PROVIDER_PREFIX const + starts_with. - Document that WIRE_API is deliberately excluded from the credential set, and that AWF injects placeholder COPILOT_PROVIDER_* values into the agent container (api-proxy-sidecar.md) so --exclude-env + sidecar keep the real credential out. - Update tests/docs accordingly; add case-sensitivity and $[...]-rejection tests. Co-authored-by: Copilot <[email protected]> * refactor(engine): drop detection provider-env string round-trip Address the remaining (non-blocking) design note from PR review: the detection step's provider env no longer flows as a rendered YAML string that is re-wrapped and re-parsed via parse_env_block. - copilot_provider_env now returns validated raw Vec<(String, String)> pairs instead of a KEY: "VALUE" YAML string. - Split validation out of render_engine_env_entry into validate_engine_env_entry, shared by the agent (string) path and the detection (typed) path so both validate identically. - Extract env_value_from_str as the single macro-vs-literal classifier, shared by parse_env_block and the detection path; run_threat_analysis_step now applies the typed EnvValues directly (no env:\n wrap + reparse). Behavior-preserving: emitted YAML is byte-identical (macro $(Setup.FOUNDRY_TOKEN) still lowers to an unquoted PipelineVar). The agent path keeps its string form because it aggregates multiple env sources (engine + awf-path + agent-extension). Co-authored-by: Copilot <[email protected]> * docs(engine): clarify BYOM env classification + provider-key scopes per review Address the two minor (non-blocking) suggestions from the latest PR review: - Document env_value_from_str's classification boundary: only an exact single $(NAME) wrapper becomes PipelineVar; compound/partial macros (e.g. $(A)$(B), prefix-$(X)) fall through to Literal but still expand correctly at runtime (ADO expands $( ) references inside quoted step-env values). Add unit tests pinning both the single-macro and compound/partial-literal paths. - Add a scope note on copilot_provider_env distinguishing the two provider-key sets: ALL COPILOT_PROVIDER_* keys are forwarded to the detection env (routing), whereas only the narrower COPILOT_BYOM_CREDENTIAL_ENV_KEYS drive api-proxy --exclude-env isolation. - Fix a doc-comment placement slip so parse_env_block's docs sit on parse_env_block again (were left above env_value_from_str by the prior refactor). No behavior change. Co-authored-by: Copilot <[email protected]> * fix(engine): warn on unresolvable literal COPILOT_PROVIDER_BASE_URL Previously, a literal but malformed COPILOT_PROVIDER_BASE_URL (e.g. a scheme-less `my-foundry/openai/v1`, or an IPv6 literal) caused provider_base_url_host to return None and the &&-chain in required_hosts to silently short-circuit — the host was never added to the AWF allowlist and the agent failed at runtime with a firewall-blocked network error, contradicting the documented promise that a literal base URL is added automatically. Introduce resolve_provider_base_url_host -> ProviderBaseUrlHost (None / Host / Unresolved), shared by required_hosts (consumes Host) and a new Engine::network_host_warnings (surfaces Unresolved). generate_allowed_domains now prints the non-fatal warning telling the operator to add the host via network.allowed. Compile stays non-fatal. Add unit tests (malformed + IPv6 warn; valid/expression/absent do not) and document the warning in docs/engine.md. Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: Copilot <[email protected]>
Every ado-script bundle is a build artifact shipped via the release ado-script.zip and downloaded to /tmp at pipeline runtime; the in-repo copies are gitignored. conclusion.js was the sole exception — PR #1076 added the bundle but omitted it from scripts/ado-script/.gitignore, so the built file was committed and tracked by accident. Nothing consumes the repo copy (only the /tmp runtime path is referenced), so untrack it and ignore it alongside the other bundles. Co-authored-by: Copilot <[email protected]>
* fix(ado-script): read auto-injected collection URI in getWebApi conclusion.js could not file the pipeline_failure work item because the Conclusion step never injected ADO_COLLECTION_URI, which the shared getWebApi() auth helper required. ADO does not auto-inject that renamed var, and the Conclusion step builder only set SYSTEM_ACCESSTOKEN. Align ado-script with the Rust `ado-aw execute` path by reading ADO's auto-injected collection URI directly: SYSTEM_COLLECTIONURI, falling back to SYSTEM_TEAMFOUNDATIONCOLLECTIONURI. This removes the collection URI from the per-step injection contract entirely, so no compiler-emitted step can forget it again (fixes already-compiled locks too). Also drop the now-redundant ADO_COLLECTION_URI exports from the gate and synthetic-PR step builders, update gate_e2e to set SYSTEM_COLLECTIONURI, refresh docs, and add auth.test.ts precedence coverage. Fixes #1307 Co-authored-by: Copilot <[email protected]> * chore(ado-script): stop tracking built conclusion.js bundle Every ado-script bundle is a build artifact shipped via the release ado-script.zip and downloaded to /tmp at pipeline runtime; the in-repo copies are gitignored. conclusion.js was the sole exception — PR #1076 added the bundle but omitted it from scripts/ado-script/.gitignore, so the built file was committed and tracked by accident. Nothing consumes the repo copy (only the /tmp runtime path is referenced), so untrack it and ignore it alongside the other bundles. Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: Copilot <[email protected]>
Summary
Adds an always-running Conclusion job to the canonical pipeline shape, mirroring gh-aw's conclusion job pattern. The Conclusion job reports pipeline failures and diagnostic signals (noop, missing-tool, missing-data) to Azure DevOps work items.
Pipeline shape
The Conclusion job is always emitted when
safe-outputs:exists (gh-aw pattern — noop is always-on).Configuration (under
safe-outputs:)Three opt-out levels:
Per-tool config (aligned with gh-aw's flat field style):
Key changes
scripts/ado-script/src/conclusion/index.ts→conclusion.js(ncc-bundled)build_conclusion_job()inagentic_pipeline.rs, emitted whensafe_outputsis non-emptysafe-outputs:— no separateconclusion:block0003_flatten_work_item_configmigrates oldnoop: work-item: {title, ...}→ flatnoop: {title-prefix, ...}createWorkItem,addWorkItemComment,findWorkItemByTitle,fileOrAppendWorkIteminshared/wit.tsnoop.rs/missing_tool.rs(now handled by conclusion job)src/safeoutputs/→src/safe_outputs/(snake_case consistency)How it works
safe-outputs:is configuredsafe_outputsartifact (containssafe-outputs-executed.ndjson)conclusion.jsreads upstream job results + the NDJSON manifest + per-tool config (passed asAW_NOOP_CONFIGetc. JSON env vars)Test plan
cargo build✅cargo test conclusion— 4 Rust tests pass (1 types + 3 compiler integration)cargo test flatten_work_item— 6 codemod tests passnpx vitest run src/conclusion— 8 TypeScript tests pass (conclusion.js logic)npx vitest run src/shared/__tests__/wit.test.ts— WI write helper tests passnpm run typecheck— TypeScript type-checks clean