Skip to content

feat(compile): add Conclusion job for pipeline failure reporting#1076

Merged
jamesadevine merged 20 commits into
mainfrom
copilot/always-run-step-report-failures
Jun 29, 2026
Merged

feat(compile): add Conclusion job for pipeline failure reporting#1076
jamesadevine merged 20 commits into
mainfrom
copilot/always-run-step-report-failures

Conversation

@jamesadevine

@jamesadevine jamesadevine commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds an always-running Conclusion job to the canonical pipeline shape, mirroring gh-aw's conclusion job pattern. The Conclusion job reports pipeline failures and diagnostic signals (noop, missing-tool, missing-data) to Azure DevOps work items.

Pipeline shape

Setup → Agent → Detection → SafeOutputs → Teardown → Conclusion
                                                        ↑
                                              condition: always()

The Conclusion job is always emitted when safe-outputs: exists (gh-aw pattern — noop is always-on).

Configuration (under safe-outputs:)

Three opt-out levels:

# Global opt-out — no failure work items at all
safe-outputs:
  report-failure-as-work-item: false

# Per-tool opt-out
safe-outputs:
  noop:
    report-as-work-item: false
  missing-tool:
    report-as-work-item: false

# Disable tool entirely (agent can't call it)
safe-outputs:
  noop: false

Per-tool config (aligned with gh-aw's flat field style):

safe-outputs:
  noop:
    title-prefix: "[ado-aw] Agent reported no operation"
    work-item-type: Task
    area-path: "MyProject\\MyTeam"
    tags:
      - agent-noop
  missing-tool:
    title-prefix: "[ado-aw] Missing tool"
    work-item-type: Bug
    report-as-work-item: true

Key changes

Area What
ado-script bundle scripts/ado-script/src/conclusion/index.tsconclusion.js (ncc-bundled)
Pipeline compilation build_conclusion_job() in agentic_pipeline.rs, emitted when safe_outputs is non-empty
Config surface Lives entirely under safe-outputs: — no separate conclusion: block
Codemod 0003_flatten_work_item_config migrates old noop: work-item: {title, ...} → flat noop: {title-prefix, ...}
WI write helpers createWorkItem, addWorkItemComment, findWorkItemByTitle, fileOrAppendWorkItem in shared/wit.ts
Separation of concerns Work-item filing removed from noop.rs/missing_tool.rs (now handled by conclusion job)
Module rename src/safeoutputs/src/safe_outputs/ (snake_case consistency)

How it works

  1. Compiler emits a Conclusion job whenever safe-outputs: is configured
  2. The job downloads the safe_outputs artifact (contains safe-outputs-executed.ndjson)
  3. conclusion.js reads upstream job results + the NDJSON manifest + per-tool config (passed as AW_NOOP_CONFIG etc. JSON env vars)
  4. Files/comments ADO work items for: pipeline failures, noop, missing-tool, missing-data
  5. Uses WIQL dedup: searches for existing open WI with same title → appends comment; else creates new
  6. Never fails the pipeline — filing errors are logged as warnings

Test plan

  • cargo build
  • cargo test conclusion — 4 Rust tests pass (1 types + 3 compiler integration)
  • cargo test flatten_work_item — 6 codemod tests pass
  • npx vitest run src/conclusion — 8 TypeScript tests pass (conclusion.js logic)
  • npx vitest run src/shared/__tests__/wit.test.ts — WI write helper tests pass
  • npm run typecheck — TypeScript type-checks clean

@jamesadevine jamesadevine force-pushed the copilot/always-run-step-report-failures branch from 1720ec7 to 67e3050 Compare June 17, 2026 21:28
@github-actions

Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Generally solid — the Conclusion job wires up correctly and the TypeScript logic handles edge cases well. Two issues worth fixing before merge.

Findings

🐛 Bugs / Logic Issues

  • [src/compile/agentic_pipeline.rs, build_conclusion_job] serde_json::to_string().unwrap_or_default() silently swallows serialization errors, violating the project's anyhow propagation convention. While Vec<String> serialization won't practically fail, if it did you'd silently emit AW_WORK_ITEM_TAGS="" and tags would be dropped with no error in the build log. Should be:

    let tags_json = serde_json::to_string(&conclusion_config.tags)
        .context("failed to serialize conclusion tags")?;
  • [src/compile/agentic_pipeline.rs, build_conclusion_job] The DownloadPipelineArtifact@2 step condition is SucceededOrFailed, but when the Agent job fails early (before SafeOutputs publishes the artifact), the download step will turn red in the pipeline UI — even though conclusion.js handles the missing manifest gracefully via its own Always condition. This creates confusing noise. Consider condition: always() + continueOnError: true on the download step so operators see the intended "artifact unavailable, continuing" signal rather than a red failure.

⚠️ Suggestions

  • Silent breaking change for safe-outputs: noop: work-item: / safe-outputs: missing-tool: work-item: configs. The WorkItemReportConfig, NoopConfig, and MissingToolConfig structs were removed. Since FrontMatter.safe_outputs is an opaque HashMap<String, serde_json::Value>, existing pipelines with these sub-keys will still parse but silently lose their WI-filing behavior — no compile error, no warning. This is a real behavior regression for any consumer who configured these. The test fixtures do show the migration (work-item: enabled: falsenoop: {}), but there's no codemod registered in src/compile/codemods/ to automate it. Given the project's codemod framework (docs/codemods.md), adding one here would be the right call.

✅ What Looks Good

  • WIQL injection in wit.ts findWorkItemByTitle is correctly handled: title.replaceAll("'", "''") is the proper WIQL string-literal escape, and the title originates from sanitized operator-controlled front matter (not agent-supplied content).
  • Artifact name consistency: safe_outputs artifact is published by the SafeOutputs job and downloaded by the same name in the Conclusion job — consistent throughout.
  • EXECUTED_NDJSON_FILENAME = "safe-outputs-executed.ndjson" matches the TypeScript EXECUTED_MANIFEST_FILENAME constant and the Rust ndjson.rs export — no mismatch.
  • fileSignal error handling: errors are caught as warnings rather than re-thrown, ensuring the Conclusion step never itself fails the pipeline due to a WI API error.
  • wire_explicit_dependencies correctly gates the Teardown dependency on has_teardown, so pipelines without a Teardown job get dependsOn: [Agent, Detection, SafeOutputs] only.
  • always() on both job and conclusion.js step correctly ensures the reporter runs even when all upstream jobs failed or were cancelled.
  • ConclusionConfig uses deny_unknown_fields — typos in conclusion: front matter will fail fast at parse time.
  • Good test coverage: 3 front-matter unit tests in types.rs + 3 compiler integration tests in compiler_tests.rs + TypeScript vitest suite.

Generated by Rust PR Reviewer for issue #1076 · 1K AIC · ⌖ 12.5 AIC · ⊞ 35.4K ·

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Needs changes — two correctness bugs that will silently break the core feature (failure detection and per-tool opt-out), plus a data-loss codemod bug.


Findings

🐛 Bugs / Logic Issues

1. $[dependencies.*.result] will not expand in step env: blocks (src/compile/agentic_pipeline.rs, lines ~1162–1173)

let agent_result = format!("$[dependencies.{}.result]", agent_id.as_str());
// ...
conclusion_step = conclusion_step
    .with_env("AW_AGENT_RESULT", EnvValue::Literal(agent_result))
    .with_env("AW_DETECTION_RESULT", EnvValue::Literal(detection_result))
    .with_env("AW_SAFEOUTPUTS_RESULT", EnvValue::Literal(safeoutputs_result));

ADO only evaluates $[...] runtime expressions inside job-level variables: definitions and condition: fields — not in step env: blocks. These three env vars will receive the literal strings $[dependencies.Agent.result] etc. at runtime, so conclusion.js will see "$[dependencies.Agent.result]" instead of "Succeeded" or "Failed", and its failure-detection logic will never trigger.

The fix is to hoist these to job-level variables: on the Conclusion job and reference them as $(AW_AGENT_RESULT) in the step env. Notably, the existing test comment in compiler_tests.rs at the bottom of the file documents exactly this pitfall ("ADO ONLY evaluates $[ ... ] inside variables: mappings and condition: fields").


2. Per-tool configs are emitted by Rust but never consumed by conclusion.js (src/compile/agentic_pipeline.rs vs scripts/ado-script/src/conclusion/index.ts)

The Rust compiler passes AW_NOOP_CONFIG, AW_MISSING_TOOL_CONFIG, AW_MISSING_DATA_CONFIG as JSON env vars:

for tool_key in &["noop", "missing-tool", "missing-data"] {
    if let Some(tool_config) = front_matter.safe_outputs.get(*tool_key) {
        let env_key = format!("AW_{}_CONFIG", tool_key.to_uppercase().replace('-', "_"));
        // ...
    }
}

But loadConfig() in conclusion.js reads only global flat vars (AW_WORK_ITEM_TYPE, AW_WORK_ITEM_AREA_PATH, etc.) and never reads AW_NOOP_CONFIG et al. Consequences:

  • noop.title-prefix, noop.work-item-type, noop.area-path, noop.tags — all silently ignored.
  • Per-tool opt-out (noop.report-as-work-item: false) does not work despite being documented in the PR and in docs/conclusion.md.
  • The only opt-out that actually functions is the global report-failure-as-work-item: false.

The conclusion test (test_conclusion_job_emits_expected_env_vars_for_conclusion_script) verifies that AW_NOOP_CONFIG is emitted with the right JSON, but there is no test verifying that conclusion.js actually reads and applies it.


3. Codemod silently drops non-mapping work-item: values (src/compile/codemods/0003_flatten_work_item_config.rs, ~line 100)

let Value::Mapping(wi_map) = wi_val else {
    // work-item was present but not a mapping — put it back as-is
    continue;   // <── wi_val was already removed; this drops it
};

tool_map.remove(&wi_key) is called unconditionally before this let-else. If work-item: is present but isn't a Mapping (e.g., work-item: true), the value is removed and never re-inserted — silent data loss. The comment acknowledges the intent to restore it, but the code doesn't. Fix:

let Value::Mapping(wi_map) = wi_val else {
    tool_map.insert(wi_key, wi_val); // restore
    continue;
};

4. Default work-item type is "Bug" in TypeScript but "Task" everywhere else (scripts/ado-script/src/conclusion/index.ts, loadConfig)

workItemType: readOptionalEnv(env, "AW_WORK_ITEM_TYPE") ?? "Bug",

AW_WORK_ITEM_TYPE is never set by the Rust compiler (no global work-item-type env var is emitted for the conclusion step). The PR description and old WorkItemReportConfig default both say "Task". All conclusion work items will be filed as Bugs. This should be "Task" to match the documented default, or the Rust compiler should emit AW_WORK_ITEM_TYPE from the front matter.


✅ What Looks Good

  • The architectural separation of concerns is clean: noop/missing-tool are now pure pass-through recorders, and the Conclusion job owns all WI filing. This is a good design.
  • The wire_explicit_dependencies update (adding Teardown-conditional dep) is correct — condition: always() on the Conclusion job ensures it runs even when dependencies are skipped.
  • WorkItemReportConfig duplication between Rust and TypeScript is reduced; the TypeScript wit.ts helpers look well-structured.
  • Codemod 0003 correctly detects conflicts between flat fields and nested work-item: and bails with an actionable error.
  • The WIQL single-quote escaping (replaceAll("'", "''")) is the correct convention for WIQL string literals.

Generated by Rust PR Reviewer for issue #1076 · 197.3 AIC · ⌖ 13.2 AIC · ⊞ 35.4K ·

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Solid architecture with one wiring gap and a few logic nuances worth addressing before merge.


Findings

🐛 Bugs / Logic Issues

conclusion.js reads global WI config env vars that build_conclusion_job never emits (src/compile/agentic_pipeline.rs, scripts/ado-script/src/conclusion/index.ts)

loadConfig() consumes AW_WORK_ITEM_TYPE, AW_WORK_ITEM_AREA_PATH, AW_WORK_ITEM_ITERATION_PATH, AW_WORK_ITEM_TAGS, AW_WORK_ITEM_TITLE, and AW_INCLUDE_STATS from the environment. build_conclusion_job never emits any of them. These serve as global fallbacks in buildWorkItemConfig — most visibly for the pipeline_failure signal, which has no per-tool config entry and therefore always gets workItemType: "Task", empty area/iteration paths, and no tags.

Users can configure per-tool settings via noop.area-path: etc., but there is currently no front-matter path to control the global pipeline-failure WI type, area, iteration path, tags, or title template. Either:

  • Expose the missing fields in the safe-outputs: front-matter block and emit the corresponding env vars in build_conclusion_job, or
  • Document the current limitation explicitly in docs/conclusion.md so users aren't surprised.

report-failure-as-work-item: false disables all WI filing, not just pipeline failures (scripts/ado-script/src/conclusion/index.ts:main)

if (!config.reportFailureAsWorkItem) {
  logInfo("Conclusion work-item filing disabled via AW_REPORT_FAILURE_AS_WORK_ITEM=false");
  return 0;  // exits before filing noop/missing-tool/missing-data WIs too
}

The name and the PR description ("Global opt-out — no failure work items at all") suggest this flag is intentionally global. But buildWorkItemConfig also redundantly uses it as the enabled field passed to fileOrAppendWorkItem:

return {
  enabled: config.reportFailureAsWorkItem,  // always true here because of the early return above
  ...
};

This is dead code — enabled can never be false at that callsite. Low severity, but worth cleaning up to avoid confusion in future maintenance.


No per-tool opt-out path for pipeline_failure signals (scripts/ado-script/src/conclusion/index.ts)

getToolConfigKey("pipeline_failure") returns "pipeline_failure", but config.toolConfigs only contains noop, missing_tool, and missing_data. So toolConfig is always undefined for pipeline-failure events, meaning:

  • The per-tool opt-out check in fileSignal (if (toolConfig && !toolConfig.reportAsWorkItem)) is silently skipped.
  • The only way to disable pipeline-failure WI filing is the global flag — which disables all WI filing.

This should be documented; it's a footgun for users who want "no pipeline-failure WIs but still want noop WIs."


⚠️ Suggestions

grep "ado-script.zip" checksums.txt | sha256sum -c - is too broad (src/compile/agentic_pipeline.rs download_script)

A partial match on "ado-script.zip" will also hit lines like ado-script.zip.sig. Use an anchored pattern to avoid accidental mismatch:

grep -E '\bado-script\.zip$' checksums.txt | sha256sum -c -

Conclusion job unconditionally runs checkout_self_step()

The Conclusion job only runs conclusion.js (downloaded at runtime). A self-checkout adds latency for no purpose unless there's an implicit pool requirement driving it.


No integration test for the Teardown + Conclusion coexistence path

wire_explicit_dependencies conditionally adds Teardown to Conclusion's dependsOn when has_teardown, but there's no fixture that exercises both. Worth adding a conclusion_with_teardown.md fixture to prevent future regressions on that branch.


✅ What Looks Good

  • WIQL escaping in findWorkItemByTitlereplaceAll("'", "''") is the correct SQL-style WIQL escaping; @project binding correctly avoids project-name injection.
  • ADO runtime expression hoisting — Job-level $[dependencies.{job}.result] variables referenced as $(AW_*_RESULT) macros in the step env is the correct ADO pattern and is well-commented.
  • Manifest parsing resiliencereadManifestEntries handles missing file, read errors, and malformed lines independently, and DownloadPipelineArtifact@2 is marked continue_on_error: true, so a failed SafeOutputs job won't block the conclusion script.
  • Codemod 0003 conflict detectionbail! on overlapping flat keys is the right call; no silent data loss on ambiguous migrations.
  • Module rename (safeoutputssafe_outputs) is a clean consistency win throughout.

Generated by Rust PR Reviewer for issue #1076 · 350.4 AIC · ⌖ 12.8 AIC · ⊞ 33.8K ·

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: needs changes — two correctness bugs and a documentation/front-matter mismatch that would produce hard parse errors for users following the docs.

Findings

🐛 Bugs / Logic Issues

EnvValue::AdoMacro("$(SC_WRITE_TOKEN)") — double macro expansion

src/compile/agentic_pipeline.rsbuild_conclusion_job:

if has_write_sc {
    conclusion_step = conclusion_step.with_env(
        "SYSTEM_ACCESSTOKEN",
        EnvValue::AdoMacro("$(SC_WRITE_TOKEN)"),
    );
}

lower_env_value lowers AdoMacro(name) as format!("$({name})"), so this emits $($(SC_WRITE_TOKEN)) in the generated YAML — a double-wrapped macro that ADO evaluates as a literal string. The Conclusion job will have the wrong (empty) token whenever a write service-connection is configured, silently breaking work-item filing.

The correct form is EnvValue::secret("SC_WRITE_TOKEN") (or PipelineVar), which lowers to $(SC_WRITE_TOKEN). Note that EnvValue::AdoMacro is also being constructed directly (bypassing the ado_macro() allowlist factory), and "SC_WRITE_TOKEN" is not in ALLOWED_ADO_MACROS anyway — the fallthrough else branch correctly uses EnvValue::secret("System.AccessToken"), so the same pattern should be applied here.


docs/conclusion.md describes a conclusion: front-matter block that doesn't exist

docs/conclusion.md (new file) and docs/front-matter.md (+2 lines) both document config under a top-level conclusion: key:

conclusion:
  work-item-type: Bug
  area-path: "MyProject\\MyTeam"

But FrontMatter has #[serde(deny_unknown_fields)] and no conclusion field was added to the struct (the types.rs diff is only a module-path rename + tests). Any user who writes conclusion: in front matter will get a hard parse error from the compiler. The actual implementation reads configuration from safe-outputs: (confirmed by build_conclusion_job's front_matter.safe_outputs.get(...) calls and the test fixture at tests/fixtures/conclusion_basic.md).

The PR description itself says "Lives entirely under safe-outputs: — no separate conclusion: block", so these doc additions appear to be leftover from an earlier design.


docs/conclusion.md field names and defaults don't match the implementation

The table in docs/conclusion.md documents:

  • work-item-title → but the code reads title-prefix
  • Default for work-item-type"Bug" → but buildWorkItemConfig (TypeScript) defaults to "Task"

The test fixture uses title-prefix and work-item-type: Bug (user-set), so users reading the docs will use wrong field names.

🔒 Security Concerns

ALLOWED_ADO_MACROS allowlist bypass

src/compile/agentic_pipeline.rs — the new code constructs EnvValue::AdoMacro(...) directly rather than calling EnvValue::ado_macro(...). The ado_macro() factory validates the name against ALLOWED_ADO_MACROS; the direct variant constructor skips that check. The rest of the codebase consistently uses the factory to ensure compile-time macro hygiene. (This is a secondary consequence of the double-macro bug above.)

⚠️ Suggestions

  • Variable shadowing (src/compile/agentic_pipeline.rs, ~line 1190): inside the per-tool loop, let prefix = format!("AW_{}", ...) shadows the prefix: &JobPrefix<'_> parameter. Rename to env_prefix to avoid confusion and the Rust unused_variable / shadowing warnings.

  • Unnecessary checkout step: build_conclusion_job starts with checkout_self_step() but the job only runs a downloaded bundle (conclusion.js) and a pipeline artifact. No repo code is needed; removing the checkout saves ~30s per run.

✅ What Looks Good

  • The three-layer opt-out design (global report-failure-as-work-item, per-tool report-as-work-item, noop: false) is clean and correctly wired end-to-end.
  • wire_explicit_dependencies correctly adds has_teardown as an optional dep so Conclusion runs after Teardown when present.
  • The WIQL single-quote escaping (title.replaceAll("'", "''")) in wit.ts is correct for ADO WIQL injection defence.
  • Moving work-item filing out of noop.rs/missing_tool.rs Stage 3 executors and into the dedicated Conclusion job is a sound architectural improvement.
  • Codemod 0003_flatten_work_item_config is well-tested (6 cases including conflict detection and idempotency).

Generated by Rust PR Reviewer for issue #1076 · 477.8 AIC · ⌖ 12.7 AIC · ⊞ 33.8K ·

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Looks good overall — clean architecture shift, solid test coverage, and the migration path (codemod + backward-compat) is well-handled. A few issues worth addressing before merge.

Findings

🐛 Bugs / Logic Issues

  • conclusion/index.ts:~390renderTitle dead code / titlePrefix template substitution silently dropped

    In buildWorkItemConfig, renderTitle is always called with undefined as the template:

    title: toolConfig?.titlePrefix ??
      renderTitle(undefined, config.pipelineName, signal.kind, signal.defaultTitle),

    renderTitle returns undefined when template is undefined, so this always reduces to toolConfig?.titlePrefix ?? undefined. The {pipeline_name} / {signal} substitution logic inside renderTitle is never exercised. Intent unclear: if titlePrefix is meant to support {pipeline_name} placeholders (as renderTitle implies), the call should be renderTitle(toolConfig?.titlePrefix, ...). If not, remove the dead call.

  • agentic_pipeline.rs:~1190v.to_string() for boolean env var emits JSON-encoded string

    if let Some(v) = obj.get("report-as-work-item") {
        conclusion_step = conclusion_step.with_env(
            format!("{env_prefix}_REPORT_AS_WORK_ITEM"),
            EnvValue::Literal(v.to_string()),
        );
    }

    serde_json::Value::to_string() on Bool(false) gives "false" (correct), but on String("false") gives "\"false\"" (a JSON-encoded string, including the quotes). readBooleanEnv in TypeScript will then log a warning and default to true — silently inverting the intended opt-out. For all other string fields in the same block (title-prefix, area-path, etc.) the code correctly uses .as_str(). This field should too: v.as_bool().map(|b| b.to_string()), and skip the env var if the value isn't a boolean.

⚠️ Suggestions

  • agentic_pipeline.rs / conclusion/index.tspipeline_failure signal is not configurable via front matter

    The compiler emits AW_NOOP_*, AW_MISSING_TOOL_*, AW_MISSING_DATA_* env vars for per-tool customisation, but no AW_PIPELINE_FAILURE_* vars. loadConfig() in conclusion/index.ts similarly has no pipeline_failure entry in toolConfigs. This means work-item type, area path, and tags for pipeline failure work items are hardcoded (Task, no area/iteration path). Given the PR description explicitly positions this as a per-tool opt-out feature, the omission may be intentional — if so, worth a comment in the code noting it as a deliberate limitation.

  • conclusion/index.ts:~470enabled field in WorkItemReportConfig is unreachable from conclusion.js

    buildWorkItemConfig sets enabled: config.reportFailureAsWorkItem, but main() already returns early when that flag is false before any call to fileSignal/fileOrAppendWorkItem. The if (!config.enabled) guard inside fileOrAppendWorkItem (in wit.ts) is dead code from conclusion.js's call site. Not a bug, but a misleading contract — the enabled field exists to be driven externally, yet conclusion.js always passes true. Consider omitting enabled from the config passed here or documenting the invariant.

✅ What Looks Good

  • WIQL injection protection correctly preserved and ported to TypeScript (title.replaceAll("'", "''"))
  • continue_on_error: true on the artifact download step — right call for fault tolerance when SafeOutputs was skipped/failed
  • Dependency wiring handles optional Teardown gracefully with the has_teardown guard
  • Codemod 0003 is thorough: conflict detection, idempotency test, enabled→report-as-work-item rename all covered
  • withRetry wrapper on all ADO SDK calls in wit.ts — good resilience
  • Stage 3 pass-through (noop/missing-tool become plain success in execute) is the right model; execute_impl no longer needs credentials it previously degraded without

Generated by Rust PR Reviewer for issue #1076 · 536.6 AIC · ⌖ 12.7 AIC · ⊞ 35.7K ·

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Solid implementation with one real resilience bug worth fixing before merge; everything else is high quality.


Findings

🐛 Bugs / Logic Issues

  • src/compile/agentic_pipeline.rs — Conclusion job can fail the pipeline on infra errors

    UseNode@1 and the Download ado-script bundle bash step have no condition: always() or continue_on_error: true. Since the job runs with condition: always(), individual steps default to condition: succeeded(). If UseNode@1 fails (unusual but possible), the download step is skipped — but the conclusion.js bash step still executes (it has condition: always()) and runs node .... If the agent pool doesn't already have Node.js on $PATH, that bash step exits 127, the Conclusion job is marked Failed, and the overall pipeline shows as failed even when the agent ran successfully. This directly contradicts the PR description's "Never fails the pipeline" guarantee.

    Minimal fix: add continue_on_error = true to the UseNode@1 TaskStep (same as the artifact-download step), or add a Node fallback guard to the conclusion.js bash step.

  • src/compile/agentic_pipeline.rs — Duplicated comment block

    Around the for tool_key in &["noop", "missing-tool", "missing-data"] loop there are two consecutive copies of the same comment:

    // Pass per-tool configs as individual flat env vars (gh-aw pattern).
    // Each field gets its own env var — avoids JSON-in-env-var corruption in ADO.
    // Pass per-tool configs as individual flat env vars (gh-aw pattern).  ← duplicate
    // Each field gets its own env var — avoids JSON-in-env-var corruption in ADO.  ← duplicate
    

    Copy-paste artifact; easy to clean up.

⚠️ Suggestions

  • scripts/ado-script/src/conclusion/index.tstitlePrefix naming is misleading

    PerToolConfig.titlePrefix and the front-matter field title-prefix act as a full title template (with {pipeline_name} / {signal} substitutions that replace the default title entirely), not as a string prepended to something. A user reading title-prefix: "[ado-aw] Noop" would reasonably expect the pipeline name to be appended automatically. Consider renaming to title-template (with a codemod if needed) or at minimum expanding the front-matter docs to clarify this is the complete title.

  • scripts/ado-script/src/conclusion/index.tspipeline_failure silently falls through toolConfigs

    In loadConfig(), toolConfigs contains keys noop, missing_tool, missing_data — but not pipeline_failure. In fileSignal(), config.toolConfigs[getToolConfigKey("pipeline_failure")] returns undefined, so the per-tool opt-out branch is skipped and filing always proceeds. This is intentional, but it's a gotcha for future contributors. A short comment on the pipeline_failure case in fileSignal() (or an explicit guard) would prevent a future author from being surprised when trying to add per-tool config for failures.

✅ What Looks Good

  • Job-variable hoisting for $[dependencies.X.result] — the conclusion_variables approach is exactly right; ADO only evaluates runtime expressions in variables: and condition:, so hoisting to job variables and referencing via $(name) macros in the step env is the correct idiom.
  • ir/lower.rsEnvValue::AdoMacro double-expansion guard — new defensive check that rejects $(Already.Wrapped) names in AdoMacro is a smart addition that prevents silent empty-string expansions in generated YAML.
  • WIQL dedup logic in wit.tsreplaceAll("'", "''") is the correct WIQL escaping, and the NOT IN ('Closed', 'Resolved', 'Done') state filter covers all three built-in ADO process templates. Mirrors the deleted Rust implementation faithfully.
  • process.exit(0) crash handler — ensures the Conclusion job never hard-fails the pipeline from a TypeScript runtime crash, which is the right default for a best-effort reporting job.
  • Codemod 0003_flatten_work_item_config — conflict detection (bail! when flat field and nested work-item: coexist) and the idempotency test are both solid; the transformation is safe to run repeatedly.
  • SanitizeContent still applied — noop/missing_tool results are sanitized in Stage 3 before being written to safe-outputs-executed.ndjson, so the Conclusion script reads operator/sanitized content, not raw agent strings.
  • Module rename safeoutputssafe_outputs — clean and the path updates are exhaustive across common.rs, mcp.rs, execute.rs, main.rs, types.rs, etc.

Generated by Rust PR Reviewer for issue #1076 · 318.9 AIC · ⌖ 12.6 AIC · ⊞ 34.1K ·

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Good architectural direction — moving WI filing from Stage 3 into an always-running Conclusion job is the right separation of concerns. One real bug will silently neutralise the manifest-based signal reporting (noop/missing_tool/missing_data) on every pipeline run.


Findings

🐛 Bugs / Logic Issues

src/compile/agentic_pipeline.rssafe-outputs-executed.ndjson never reaches the safe_outputs artifact

The Conclusion job downloads the safe_outputs artifact and reads safe-outputs-executed.ndjson from it:

// build_conclusion_job – download step
.with_input("artifact", "safe_outputs")
.with_input("path", "$(Pipeline.Workspace)/conclusion_inputs");
// ...and the env var:
EnvValue::Literal("$(Pipeline.Workspace)/conclusion_inputs".to_string()),

But the execute_safe_outputs step in build_safeoutputs_job writes the executed NDJSON file to $(Pipeline.Workspace)/analyzed_outputs_$(Build.BuildId)/:

"ado-aw execute ... --safe-output-dir \"$(Pipeline.Workspace)/analyzed_outputs_$(Build.BuildId)\" ..."

The safe_outputs artifact is published from $(Agent.TempDirectory)/staging. The existing copy_logs_safeoutputs_step only copies logs — it never copies safe-outputs-executed.ndjson to staging. The Conclusion job will therefore always log:

Conclusion manifest not found: $(Pipeline.Workspace)/conclusion_inputs/safe-outputs-executed.ndjson

...and skip noop/missing_tool/missing_data signal reporting entirely. Pipeline-failure reporting (driven by upstream job results, not the manifest) is unaffected and will still work.

Fix — add a copy into copy_logs_safeoutputs_step or a dedicated step before the Publish:

cp "$(Pipeline.Workspace)/analyzed_outputs_$(Build.BuildId)/safe-outputs-executed.ndjson" \
   "$(Agent.TempDirectory)/staging/safe-outputs-executed.ndjson" 2>/dev/null || true

⚠️ Suggestions

src/compile/agentic_pipeline.rs:~1254use inside function body

use crate::compile::ir::job::JobVariable;
let conclusion_variables = vec![...];

use inside a function body is valid Rust but unusual here: all other IR imports are at the top of the file. Moving it alongside use super::ir::job::{Job, Pool} keeps things consistent.


✅ What Looks Good

  • ADO job-variable hoisting pattern for upstream results is exactly right — $[dependencies.X.result] in a job variable, referenced as $(VAR) in the step env block. Lowering path is correct.
  • EnvValue::AdoMacro validation guard in lower.rs is a solid defence-in-depth addition; the double-wrapped $($(...)) case was a real footgun.
  • report-as-work-item / serde_json::Value::to_string note (the comment about JSON-encoded strings producing "\"false\"" vs "false") — thorough and prevents a silent opt-out inversion.
  • WIQL single-quote escaping in wit.ts::findWorkItemByTitle is sufficient for operator-controlled title inputs; only ' needs doubling in WIQL string literals.
  • Checksum verification (sha256sum -c -) in the bundle download step is good supply-chain hygiene.
  • Codemod handles conflict detection, idempotency, and all three diagnostic tool keys correctly.
  • continue_on_error = true on the DownloadPipelineArtifact step is the right choice — the artifact may not exist when SafeOutputs was skipped.

Generated by Rust PR Reviewer for issue #1076 · 922.5 AIC · ⌖ 12.5 AIC · ⊞ 34.1K ·

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Solid feature with good test coverage — one correctness bug needs fixing before merge.

Findings

🐛 Bugs / Logic Issues

  • src/compile/agentic_pipeline.rs ~L1100, conclusion_script — The conclusion bash step has condition: always() but does not check whether conclusion.js was successfully downloaded before calling node. If the "Download ado-script bundle" bash step fails (e.g., network error, wrong version in a release), conclusion.js won't exist at /tmp/ado-aw-scripts/ado-script/conclusion.js. The step then runs node /nonexistent/path, exits non-zero, and the Conclusion job fails — directly contradicting the PR description's "Never fails the pipeline" guarantee.

    The fix is a simple file-existence guard:

    if command -v node >/dev/null 2>&1 && [ -f /tmp/ado-aw-scripts/ado-script/conclusion.js ]; then
      node /tmp/ado-aw-scripts/ado-script/conclusion.js
    else
      echo "##vso[task.logissue type=warning]conclusion.js unavailable; skipping conclusion reporting"
    fi

    Alternatively, continue_on_error = true on the conclusion bash step achieves the same result with less precision.

⚠️ Suggestions

  • scripts/ado-script/src/shared/wit.ts findWorkItemByTitle — The WIQL title-escaping comment from the deleted Rust code (// This title value comes from operator-controlled front-matter configuration...) was important for security audit purposes — it explains why single-quote doubling is safe here and that this is not agent-supplied content. The TypeScript equivalent has no such comment. Worth porting for reviewers and future maintainers.

  • title-prefix field name vs. renderTitle behaviour — The field is named title-prefix but renderTitle() uses it as a full title template supporting {pipeline_name}, {pipeline}, and {signal} interpolations. A user supplying "[ado-aw] Failure" gets that as the entire title (not a prefix prepended to the pipeline name). The name title-template or at least a doc clarification would prevent the "prefix" misinterpretation. Minor UX confusion, but worth noting before the front-matter API is widely used.

✅ What Looks Good

  • Clean separation of concerns: Stage-3 noop/missing-tool executors are now pure pass-throughs (no ADO HTTP calls), and work-item filing is fully delegated to the Conclusion job. The behavioral change is well-tested in execute.rs (old warning → new plain success).
  • DownloadPipelineArtifact@2 correctly has continue_on_error = true — even if the SafeOutputs artifact is missing, the Conclusion job proceeds and logs a warning rather than failing.
  • process.exit(0) on crash in conclusion/index.ts ensures the TypeScript bundle itself never fails the pipeline. The only gap is the pre-TypeScript Node.js invocation described above.
  • WIQL single-quote escaping (title.replaceAll("'", "''")) is correct.
  • The $[dependencies.X.result]-in-EnvValue::Literal-hoisted-to-job-variables pattern is the established ADO approach, and the in-code comments explain the ADO constraint clearly.
  • Codemod 0003_flatten_work_item_config with conflict detection and idempotency test is well-structured.
  • EnvValue::AdoMacro double-expansion guard (lower.rs) is a nice defensive addition bundled with this PR.

Generated by Rust PR Reviewer for issue #1076 · 317.5 AIC · ⌖ 20.3 AIC · ⊞ 34.1K ·

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Solid architecture, but there's a blocking bug: the documented report-failure-as-work-item key is rejected at compile time by the existing validation gate.

Findings

🐛 Bugs / Logic Issues

  • [src/compile/common.rsvalidate_safe_outputs_keys] report-failure-as-work-item: false triggers a compile-time "unrecognised tool name" error.
    validate_safe_outputs_keys iterates every key in safe_outputs and rejects anything not in ALL_KNOWN_SAFE_OUTPUTS, NON_MCP_SAFE_OUTPUT_KEYS, DEBUG_ONLY_TOOLS, or the "memory" migration escape-hatch. report-failure-as-work-item is none of these, so any pipeline using:

    safe-outputs:
      report-failure-as-work-item: false

    will fail compilation with "safe-outputs contains unrecognised tool name(s)". The types.rs unit test only exercises parsing, not the validation pass, so this is untested. Fix: add a special case (or add it to NON_MCP_SAFE_OUTPUT_KEYS) in validate_safe_outputs_keys.

  • [src/compile/agentic_pipeline.rsbuild_conclusion_job] Conclusion job bypasses supply-chain: mirrors.
    The bash download step always fetches from https://github.com/githubnext/ado-aw/releases/... regardless of whether supply-chain: is configured. Pipelines that mirror ado-script.zip via an internal feed for air-gapped environments will have the Conclusion job fail at the download step. The existing ado_script extension's download logic (which is supply-chain:-aware) isn't reused here.

🔒 Security Concerns

  • [scripts/ado-script/src/shared/wit.tsfindWorkItemByTitle] WIQL escaping is correct, but the comment in the old Rust code that explained why it's safe (operator-controlled, not agent-controlled) was not ported.
    The ''' doubling is the right WIQL defence. Worth adding an inline comment similar to the deleted Rust one clarifying the trust boundary — especially since buildTitle now appends config.pipelineName, which is author-controlled front matter, while the dedup title becomes agent-observable. Not a current vulnerability but a future footgun.

⚠️ Suggestions

  • [scripts/ado-script/src/conclusion/index.tsbuildTitle] Title length not validated before ADO API call.
    buildTitle concatenates titlePrefix + " " + pipelineName without bounds checking. ADO work item titles have a 255-character limit; exceeding it causes createWorkItem to throw, which fileSignal catches and logs as a warning. The work item is silently not filed. Consider truncating or rejecting at compile time (e.g. validate_safe_outputs_keys could check string length of title-prefix).

  • [src/compile/agentic_pipeline.rsbuild_conclusion_job] UseNode@1 and the download bash step lack explicit condition: always().
    Steps within a job with condition: always() still follow the default "run if previous step succeeded" step-level condition. If UseNode@1 fails, the download step is skipped, but conclusion_step has condition: always() and will still run — correctly falling back to the command -v node guard. This is arguably fine, but it is worth noting the asymmetry; adding condition: always() to the download bash step would make the Conclusion job fully robust to node install failures.

✅ What Looks Good

  • Correct EXECUTED_NDJSON_FILENAME alignment: ndjson.rs defines "safe-outputs-executed.ndjson" and conclusion/index.ts uses the same constant — no naming divergence.
  • Clean noop/missing-tool refactor: removing ADO work-item filing from Stage 3 and delegating it to the always-running Conclusion job is architecturally correct and the behavior change is well-tested in execute.rs.
  • WIQL single-quote escaping in wit.ts::findWorkItemByTitle matches the deleted Rust implementation exactly.
  • Codemod conflict detection in 0003_flatten_work_item_config.rs correctly bail!s rather than silently overwriting when flat fields and a work-item: block coexist.
  • Module rename (safeoutputssafe_outputs) is systematic; all internal references updated.
  • Job result variable hoisting: the $[dependencies.X.result]$(AW_X_RESULT) pattern is the correct ADO approach for surfacing runtime expressions into step env blocks — well-executed here.

Generated by Rust PR Reviewer for issue #1076 · 410.7 AIC · ⌖ 12.9 AIC · ⊞ 34.1K ·

jamesadevine and others added 15 commits June 18, 2026 14:17
Add an always-running Conclusion job to the canonical pipeline shape that
reports pipeline failures and diagnostic signals (noop, missing-tool,
missing-data) to Azure DevOps work items.

Key changes:
- New ConclusionConfig front-matter type (conclusion: block)
- New conclusion.js ado-script bundle (TypeScript, ncc-bundled)
- build_conclusion_job() in agentic_pipeline.rs with condition: always()
- Work-item write helpers in shared/wit.ts (create, comment, WIQL dedup)
- Removed work-item filing from noop/missing-tool safe-output executors
  (moved to conclusion job - separation of concerns)
- Renamed src/safeoutputs/ to src/safe_outputs/ (snake_case convention)

Pipeline shape: Setup -> Agent -> Detection -> SafeOutputs -> Teardown -> Conclusion

Co-authored-by: Copilot <[email protected]>
…oss-platform path in conclusion test

- noop/missing-tool now return plain success (not warning) since work-item
  filing moved to the Conclusion job
- conclusion test uses stringContaining for cross-platform path separators

Co-authored-by: Copilot <[email protected]>
- Use anyhow::Context instead of unwrap_or_default for tags serialization
  (propagates errors instead of silently dropping tags)
- Change DownloadPipelineArtifact step to condition: always() with
  continueOnError: true (avoids red failure noise when artifact is
  unavailable due to early Agent failure)

Co-authored-by: Copilot <[email protected]>
…tten codemod

Non-breaking change: conclusion job config now lives under safe-outputs:
(aligned with gh-aw's pattern) instead of a separate conclusion: block.

Three opt-out levels:
- Global: safe-outputs: report-failure-as-work-item: false
- Per-tool: safe-outputs: noop: report-as-work-item: false
- Disable tool: safe-outputs: noop: false

The Conclusion job is always emitted when safe-outputs: exists (gh-aw
pattern where noop is always-on). Per-tool config (title-prefix,
work-item-type, area-path, tags) is passed as JSON env vars
(AW_NOOP_CONFIG, AW_MISSING_TOOL_CONFIG, AW_MISSING_DATA_CONFIG).

Adds codemod 0003_flatten_work_item_config that migrates the old nested
work-item: sub-block to the flat form (title -> title-prefix, enabled
-> report-as-work-item).

Co-authored-by: Copilot <[email protected]>
…onfigured

When permissions.write is set, the Conclusion job must use the
service-connection-minted token (SC_WRITE_TOKEN) instead of
System.AccessToken — matching the SafeOutputs job's pattern.

Co-authored-by: Copilot <[email protected]>
1. Hoist $[dependencies.*.result] to job-level variables (ADO only
   evaluates runtime expressions in variables:/condition:, NOT step env).
   Step env now uses $(AW_AGENT_RESULT) macro references.

2. conclusion.js now reads per-tool configs from AW_NOOP_CONFIG,
   AW_MISSING_TOOL_CONFIG, AW_MISSING_DATA_CONFIG env vars. Per-tool
   opt-out (report-as-work-item: false) and per-tool overrides
   (title-prefix, work-item-type, area-path, tags) now actually work.

3. Codemod 0003 no longer drops non-mapping work-item: values — restores
   the value to the mapping if it is not a Mapping (e.g. work-item: true).

4. Default work-item type changed from Bug to Task (matching the
   documented default and the old WorkItemReportConfig convention).

Filed #1081 for the type-level fix (EnvValue::RuntimeExpression variant).

Co-authored-by: Copilot <[email protected]>
Matches gh-aw pattern: each per-tool config field gets its own env var
(AW_NOOP_TITLE_PREFIX, AW_NOOP_WORK_ITEM_TYPE, etc.) instead of a
single AW_NOOP_CONFIG JSON blob. Avoids ADO variable expansion
corrupting JSON payloads.

Also removes dead global env var reads from conclusion.js (AW_WORK_ITEM_TYPE,
AW_WORK_ITEM_AREA_PATH, etc.) that the compiler never emitted.

Co-authored-by: Copilot <[email protected]>
…kout

1. SC_WRITE_TOKEN: use EnvValue::secret (not AdoMacro) to avoid
   $($(SC_WRITE_TOKEN)) double expansion and allowlist bypass
2. Remove checkout_self_step from Conclusion job — only needs
   downloaded bundle and pipeline artifact, saves ~30s
3. Rename shadowed prefix → env_prefix in per-tool loop
4. Fix docs/conclusion.md: config lives under safe-outputs: not a
   stale conclusion: block; fix field name (title-prefix not
   work-item-title) and default (Task not Bug)
5. Fix docs/front-matter.md: remove conclusion: field reference

Co-authored-by: Copilot <[email protected]>
AdoMacro values must be bare variable names (e.g. "Build.Reason");
the $() wrapper is added by lowering. Direct construction bypassing
the ado_macro() factory could pass pre-wrapped names like
"$(SC_WRITE_TOKEN)" which would emit $($(SC_WRITE_TOKEN)) — a
double expansion that ADO silently drops to an empty string.

The guard rejects names containing $, (, or ) in both
lower_env_value and lower_env_value_as_expr_atom, with an
actionable error message pointing to PipelineVar/secret.

Co-authored-by: Copilot <[email protected]>
…r fix

1. titlePrefix now supports {pipeline_name}/{signal} placeholders via
   renderTitle (was dead code — always called with undefined template)
2. report-as-work-item env var uses as_bool()/as_str() instead of
   v.to_string() which JSON-encodes strings as "\"false\"" — silently
   inverting the opt-out
3. Document pipeline_failure has no per-tool config (deliberate)
4. Document enabled field is always true from conclusion.js (guarded
   by main() early return and per-tool opt-out in fileSignal)

Co-authored-by: Copilot <[email protected]>
…fy pipeline_failure

1. conclusion.js step now guards against missing node binary —
   logs a warning and exits 0 instead of failing with exit 127.
   This ensures the Conclusion job never fails the pipeline on
   infra issues (UseNode failure, missing runner tooling).
2. Remove duplicated comment block in build_conclusion_job.
3. Add comment in fileSignal explaining why pipeline_failure has
   no per-tool config entry (intentional, matches gh-aw).

Co-authored-by: Copilot <[email protected]>
…bVariable import

1. Copy safe-outputs-executed.ndjson from analyzed_outputs to the
   safe_outputs artifact staging directory. Without this, the
   Conclusion job could never find the manifest — noop/missing-tool/
   missing-data signal reporting was silently broken.
2. Move JobVariable import to file-level (consistent with other IR
   imports).

Co-authored-by: Copilot <[email protected]>
…align title-prefix

1. Copy safe-outputs-executed.ndjson into the safe_outputs artifact
   so the Conclusion job can find noop/missing-tool/missing-data
   entries. Without this, diagnostic signal reporting was silently
   skipped (pipeline-failure reporting was unaffected).

2. Extend bash guard to also check conclusion.js exists on disk
   before running node — handles download failures gracefully.

3. Align title-prefix with gh-aw: it is a prefix prepended to the
   pipeline name (${titlePrefix} ${pipelineName}), not a template
   with placeholder substitution. Matches gh-aw missing_issue_helpers.

4. Move JobVariable import to file-level (consistency with other IR
   imports).

Co-authored-by: Copilot <[email protected]>
validate_safe_outputs_keys rejected report-failure-as-work-item as an
unrecognised tool name, breaking the documented global opt-out. Add
SAFE_OUTPUT_CONFIG_KEYS for global config-level keys (not tool names)
and allow them in the validation pass.

Co-authored-by: Copilot <[email protected]>
…ion job

Addresses review feedback plus two latent bugs found while re-evaluating
against main (which now has the supply-chain mirror feature, #1080):

- Conclusion job now reuses ado_script::install_and_download_steps_typed,
  which (a) respects the supply-chain feed mirror and (b) unzips to
  /tmp/ado-aw-scripts/ (the hand-rolled copy double-nested to
  /tmp/ado-aw-scripts/ado-script/ado-script/, so conclusion.js was never
  at the referenced path).
- release.yml now auto-globs ado-script/*.js instead of a manual file
  list, so every built bundle ships. conclusion.js was missing from the
  manual list and would never have reached a release.
- New bundle-coverage vitest guards that every src/<name>/ bundle dir is
  wired into the npm build chain (the safety net for auto-globbing).
- buildTitle truncates to ADO's 255-char System.Title limit.
- Ported the WIQL trust-boundary comment to wit.ts.

Co-authored-by: Copilot <[email protected]>
@jamesadevine jamesadevine force-pushed the copilot/always-run-step-report-failures branch from c3188ff to 623d410 Compare June 18, 2026 13:40
@github-actions

Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Solid implementation of the Conclusion job feature — architecture is well-thought-out and follows existing conventions. Three items worth addressing before merge.


Findings

🐛 Bugs / Logic Issues

scripts/ado-script/src/shared/wit.ts:155WorkItemReportConfig.includeStats field is dead code

includeStats: boolean is declared in the TypeScript WorkItemReportConfig interface and set to true in buildWorkItemConfig, but fileOrAppendWorkItem never reads it. The stats block is always appended unconditionally in the report builder functions (appendStats). The field mirrors the now-deleted Rust WorkItemReportConfig but has no effect in the TS implementation.

src/compile/agentic_pipeline.rs:1228 — Implicit all-tools opt-in may surprise pipeline authors

When safe-outputs: noop: {} is the only configured tool (triggering the Conclusion job), the TypeScript defaults reportAsWorkItem: true for all diagnostic signal kinds (missing-tool, missing-data, pipeline_failure) — not just noop. No AW_MISSING_TOOL_REPORT_AS_WORK_ITEM env var is emitted for unconfigured tools, so the TS default silently opts them in. Pipelines that configure only noop: will also get work items for missing-tool and missing-data by default. This may or may not be intentional behaviour — it mirrors the gh-aw pattern — but it should be clearly documented in docs/conclusion.md.

⚠️ Suggestions

tests/compiler_tests.rs — No test for Conclusion dependsOn when Teardown exists

test_conclusion_job_is_emitted_with_expected_condition_and_dependencies verifies the no-Teardown path (deps == [Agent, Detection, SafeOutputs]). There's no test for the case where a teardown: block exists — i.e. that Teardown appears in the Conclusion dependsOn list. The has_teardown branch in wire_explicit_dependencies is untested.

scripts/ado-script/src/conclusion/index.ts:193extractNamedValue regex fallback is loose

The fallback pattern /tool[_ -]?name\s*:\s*([^\r\n,;]+)/i applied to sanitized-but-agent-observable context strings could match unintended substrings (e.g. "no tool name: available in registry" → extracts "available in registry"). The field tool_name is already in the manifest schema and carries the right value when populated; the regex path adds noise. Consider tightening to an exact match or removing it.

✅ What Looks Good

  • WIQL injection: single-quote doubling in findWorkItemByTitle is the correct WIQL escape; the comment accurately describes why the input is trusted (compile-time front matter) while keeping the escape as defence-in-depth.
  • buildTitle title truncation to 255 chars: correctly prevents ADO API rejections for long titlePrefix + pipelineName combinations while preserving dedup stability (same inputs → same title).
  • ADO runtime-expression hoisting pattern: $[dependencies.X.result] goes into variables: via EnvValue::Literal; step env references via EnvValue::PipelineVar ($(AW_AGENT_RESULT)). Matches the project's established pattern and avoids the $[...]-in-step-env gotcha caught by the existing test.
  • Graceful degradation in conclusion script: the command -v node guard and continue_on_error: true on the artifact download ensure the Conclusion job never fails the pipeline even if Node or the artifact is unavailable.
  • Codemod test coverage: all six tests for 0003_flatten_work_item_config are thorough, including conflict detection and idempotency.
  • fileSignal missing-project guard: early return with a warning when SYSTEM_TEAMPROJECT is unset is the correct behaviour rather than letting the SDK throw.

Generated by Rust PR Reviewer for issue #1076 · 347.7 AIC · ⌖ 12.4 AIC · ⊞ 35.7K ·

…ep-report-failures

# Conflicts:
#	.github/workflows/release.yml
#	AGENTS.md
#	docs/ado-script.md
#	scripts/ado-script/package.json
#	src/compile/agentic_pipeline.rs
#	src/compile/ir/tasks.rs
#	src/runtimes/node/extension.rs
#	src/safe_outputs/add_pr_comment.rs
@github-actions

Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Solid implementation with one correctness gap in the split-review case and a sanitization regression to address.


Findings

🐛 Bugs / Logic Issues

SafeOutputs_Reviewed failures silently missed by Conclusion

src/compile/agentic_pipeline.rswire_explicit_dependencies + build_conclusion_job

When a pipeline has require-approval tools (split case: both SafeOutputs and SafeOutputs_Reviewed jobs exist), the Conclusion job:

  1. Only depends on [Agent, Detection, SafeOutputs, Teardown?]not SafeOutputs_Reviewed. Conclusion can fire while SafeOutputs_Reviewed is still waiting for human approval.
  2. Only reads $[dependencies.SafeOutputs.result] via AW_SAFEOUTPUTS_RESULT. There is no AW_SAFEOUTPUTS_REVIEWED_RESULT variable. A reviewer rejection (causing SafeOutputs_Reviewed to fail) produces zero signal in the Conclusion report.

Concrete scenario: all tools are require-approval → human rejects → SafeOutputs_Reviewed fails → Conclusion already reported success / has no visibility into the reviewed job's failure.

The analogous Teardown gap is documented and intentional (you don't want Teardown blocked behind human approval). But the reporting gap is different — Conclusion is specifically meant to detect failures. Either add SafeOutputs_Reviewed to the dep list (accepted design trade-off: Conclusion waits for human gate) or add AW_SAFEOUTPUTS_REVIEWED_RESULT via a second JobVariable + env mapping so conclusion.js can inspect it.


Codemod doesn't handle missing-data.work-item: (silent pass-through)

src/compile/codemods/0003_flatten_work_item_config.rsDIAGNOSTIC_KEYS

The codemod targets ["noop", "missing-tool", "missing-data"], which is good. However, looking at the old code being removed from mod.rs, only noop and missing-tool had WorkItemReportConfig / nesting — missing-data did not. The codemod will silently no-op on missing-data.work-item: (which cannot exist in real front matter), but this is harmless. No action required — just confirm missing-data never had a work-item: sub-block in the old schema.


🔒 Security Concerns

title-prefix, area-path, iteration-path, work-item-type bypass SanitizeConfig

src/compile/agentic_pipeline.rsbuild_conclusion_job, per-tool config loop (approx line 1570–1610)

The old WorkItemReportConfig was decorated with #[derive(SanitizeConfig)] and sanitize_config_fields() was called before any value was used. The new code reads directly from the opaque serde_json::Value map:

if let Some(v) = obj.get("title-prefix").and_then(|v| v.as_str()) {
    conclusion_step = conclusion_step
        .with_env(format!("{env_prefix}_TITLE_PREFIX"), EnvValue::Literal(v.to_string()));
}

These values come from operator-controlled front matter (not agent-generated content), so the risk is lower than agent content — but the project's explicit convention (src/sanitize.rs, SanitizeConfig derive) is to sanitize config fields before embedding them in pipeline artifacts. An operator could craft an area-path with characters that are benign in env vars but problematic once conclusion.js passes them to the ADO REST API (e.g., embedded quotes in WIQL-influenced contexts).

The minimal fix is to call crate::sanitize::sanitize(v) on each string value before wrapping in EnvValue::Literal, matching what the old code did.


⚠️ Suggestions

AW_AGENT_RESULT uses EnvValue::Literal for $[...] — add a comment

src/compile/agentic_pipeline.rs ~line 1595

JobVariable {
    name: "AW_AGENT_RESULT".to_string(),
    value: EnvValue::Literal(format!("$[dependencies.{}.result]", agent_id.as_str())),
},

This intentionally uses EnvValue::Literal to pass a raw ADO runtime expression — correct because $[...] expressions are only evaluated in variables: and condition: blocks, not in step env: blocks (hence the hoist). The new AdoMacro guard added in lower.rs doesn't block this path. A brief inline comment here would prevent a future reader from wondering if this should be EnvValue::AdoMacro instead.


DownloadPipelineArtifact@2 has continue_on_error = true without explanation

src/compile/agentic_pipeline.rsbuild_conclusion_job

The step silently succeeds when the safe_outputs artifact doesn't exist (e.g., the SafeOutputs job was skipped). This is the right behaviour, but a one-line comment ("artifact may not exist if SafeOutputs was skipped; conclusion.js handles missing dir gracefully") would make the intent clear.


✅ What Looks Good

  • The EnvValue::AdoMacro double-wrap guard added to lower.rs is a good defensive catch — the test with a pre-wrapped $(SC_WRITE_TOKEN) name confirms the fix is exercised.
  • The decision to read report-as-work-item via as_bool() / as_str() (not to_string()) to avoid "\"false\"" JSON encoding is exactly right and well-commented.
  • The SAFE_OUTPUT_CONFIG_KEYS constant + validate_safe_outputs_keys update correctly allows report-failure-as-work-item without treating it as an unknown tool name.
  • install_and_download_steps_typed made pub(crate) with proper doc comment — clean factoring that keeps the supply-chain mirror path consistent across all consumers.
  • Codemod conflict detection (bail on work-item: + flat fields coexisting) is correct and matches the project's convention.
  • Module rename safeoutputssafe_outputs is consistent and the similarity-index renames confirm no logic was lost.

Generated by Rust PR Reviewer for issue #1076 · 542.8 AIC · ⌖ 12.7 AIC · ⊞ 34.8K ·

jamesadevine and others added 2 commits June 29, 2026 11:23
…usion WI config

Address PR review feedback:
- Conclusion job now depends on and surfaces AW_SAFEOUTPUTS_REVIEWED_RESULT in
  the mixed manual-review split, so a reviewer rejection is reported.
- Sanitize title-prefix/work-item-type/area-path/iteration-path config fields
  before embedding in pipeline env vars.
- Clarify AW_AGENT_RESULT literal and DownloadPipelineArtifact continue_on_error.

Co-authored-by: Copilot <[email protected]>
@github-actions

Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Solid implementation with a few issues worth addressing before merge.

Findings

🐛 Bugs / Logic Issues

  • [agentic_pipeline.rs, build_conclusion_job] front_matter.name not sanitized before AW_PIPELINE_NAME

    All other user-configurable strings (e.g. title-prefix, work-item-type, area-path) go through crate::sanitize::sanitize(v) before being emitted as env vars, but the pipeline name does not:

    .with_env("AW_PIPELINE_NAME", EnvValue::Literal(front_matter.name.clone()))

    This ends up verbatim in the ADO work item title (buildTitle(titlePrefix, pipelineName)) and body (appendStats). Pipeline names are operator-controlled so the risk is low, but it's inconsistent with the rest of the env var construction. Apply crate::sanitize::sanitize(&front_matter.name) for consistency.

  • [conclusion/index.ts] AW_REPORT_FAILURE_AS_WORK_ITEM=false is a global kill-switch, not a pipeline-failure-only toggle

    The early return in main() skips all signals (noop, missing-tool, missing-data, and pipeline failures):

    if (!config.reportFailureAsWorkItem) {
      return 0;  // ← skips noop + missing-tool + missing-data too
    }

    A user who sets report-failure-as-work-item: false expecting only pipeline-failure work items to be suppressed (while keeping noop/missing-tool reporting) will silently get nothing. Separately, there's no path to suppress only pipeline failure — the per-tool opt-out works for noop/missing-tool/missing-data but pipeline_failure has no toolConfigs entry, so safe-outputs.pipeline_failure.report-as-work-item: false would be silently ignored. The docs describe this as a "global opt-out," but the key name report-failure-as-work-item implies narrower scope than the actual behaviour. Consider either renaming to report-as-work-item (global) or adding pipeline_failure to toolConfigs so per-tool opt-out works symmetrically.

⚠️ Suggestions

  • [agentic_pipeline.rs, build_conclusion_job] Conclusion job can fail if node exits non-zero

    The PR description says "Never fails the pipeline," but the conclusion bash step has no guard:

    node /tmp/ado-aw-scripts/ado-script/conclusion.js
    # ↑ no || true; exit code propagates to the step

    The TypeScript handles internal errors gracefully (process.exit(0) on crash), so this is only triggered by OOM, SIGKILL, or an unhandled promise rejection escaping the top-level .then. Still, for a job whose defining contract is "always runs, never fails," adding || true (or continue_on_error = true on the step) would fully honour that guarantee.

  • [compile/codemods/0003_flatten_work_item_config.rs] Silent unwrap_or_default on YAML key

    In the hoisting loop:

    let key_str = k.as_str().unwrap_or_default().to_string();

    A non-string YAML key (e.g. integer) silently becomes "", and the resulting empty-string key is inserted into the parent mapping. Unreachable in practice since serde_yaml front-matter is always string-keyed, but a bail or continue would be more defensive.

✅ What Looks Good

  • $[...] job-variable hoisting is exactly right — runtime expression inside variables:, referenced as $(name) in step env. The inline comment explaining why EnvValue::AdoMacro is not used here is valuable.
  • JobPrefix::id() wildcard arm correctly falls "Conclusion" through to the unprefixed branch without any code change needed; the doc comment update accurately describes the new behaviour.
  • download_artifact.continue_on_error = true correctly handles the case where SafeOutputs was skipped and the artifact doesn't exist.
  • SAFE_OUTPUT_CONFIG_KEYS allowlist cleanly separates config keys from tool names in validate_safe_outputs_keys, avoiding false "unknown key" errors.
  • Codemod tests cover the important cases: flatten, rename (title → title-prefix, enabled → report-as-work-item), no-op when clean, conflict detection, and idempotency. That last one is particularly important for a codemod that rewrites front matter.
  • New noop.rs / missing_tool.rs are correctly simplified — execute_impl just returns success now that filing is deferred to the Conclusion job. The old files had ~280 lines of ADO API calls that are now rightfully in TypeScript.

Generated by Rust PR Reviewer for issue #1076 · 357.7 AIC · ⌖ 12.6 AIC · ⊞ 36.4K ·

jamesadevine and others added 2 commits June 29, 2026 14:54
- Sanitize front_matter.name before AW_PIPELINE_NAME for consistency with
  the other operator-controlled config fields (review: sanitization gap).
- Guard the conclusion node invocation with '|| true' so the always-run
  Conclusion job never fails on a non-zero node exit (OOM/SIGKILL), honoring
  the 'always runs, never fails' contract.
- Make the codemod hoist loop skip non-string YAML keys defensively instead
  of coercing them to an empty-string key via unwrap_or_default.
- Clarify docs that report-failure-as-work-item is a global master kill-switch
  that suppresses all signals (the global-vs-per-tool scope is intentional and
  covered by an existing conclusion unit test).

Co-authored-by: Copilot <[email protected]>
… step

A blanket '|| true' in the bash body swallows every non-zero node exit with
zero visibility. Use ADO's continueOnError on the conclusion.js step instead:
the pipeline still stays green (honoring the 'always runs, never fails'
contract) but a genuine failure (node OOM/SIGKILL, unhandled rejection) is
surfaced in the timeline as a warning rather than silently hidden. Add a test
asserting the step emits continueOnError: true.

Co-authored-by: Copilot <[email protected]>
@github-actions

Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Well-structured feature with solid test coverage. Two findings worth addressing before merge — one is a silent semantic break in the codemod, one is a minor sanitization inconsistency.


Findings

🐛 Bugs / Logic Issues

  • src/compile/codemods/0003_flatten_work_item_config.rs — The title → title-prefix migration silently changes behaviour. Before the migration, title was the exact ADO work item title used for WIQL dedup. After, title-prefix is used as "${prefix} ${pipelineName}" by conclusion.js. Any open work items created with the old exact title will no longer match the dedup query, so a comment-append becomes a new work item creation — potentially creating duplicates for active pipelines. The codemod summary should call this out, or the migrated key should carry a note distinguishing the semantic shift.

⚠️ Suggestions

  • src/compile/agentic_pipeline.rs:build_conclusion_job (~line 1545 of the new function) — tags is serialized raw JSON (serde_json::to_string(tags)) without calling sanitize() on individual elements, while every other string field in the same block (title-prefix, area-path, iteration-path, work-item-type) goes through crate::sanitize::sanitize(v). These are operator-controlled, so exploitability is low, but the inconsistency could trip up a future refactor. One option: iterate the tags and sanitize each as_str() before serializing.

  • scripts/ado-script/src/conclusion/index.ts:buildWorkItemConfigenabled: true is hardcoded here while the actual opt-out is handled upstream in fileSignal via toolConfig.reportAsWorkItem. The comment explains this, but the WorkItemReportConfig.enabled field is now a dead code path in conclusion.js (it's always true). Worth a clarifying comment in fileOrAppendWorkItem that callers in the conclusion context always opt out at a higher level, so this guard is for direct wit.ts consumers only.

  • wire_explicit_dependencies — When Teardown is present, Conclusion is given [Agent, Detection, SafeOutputs, Teardown] as explicit dependencies. Since Teardown already depends on SafeOutputs, the direct SafeOutputsConclusion edge is transitive-redundant. Harmless in ADO, but slightly noisy in the emitted YAML.

✅ What Looks Good

  • condition: always() + continueOnError: true on the conclusion step is exactly right — failure is surfaced as a warning rather than masking the real signal.
  • SAFE_OUTPUT_CONFIG_KEYS cleanly separates report-failure-as-work-item from tool names so it doesn't spuriously fail validate_safe_outputs_keys.
  • WIQL single-quote escaping is correct in both the Rust and TypeScript paths.
  • partition_safe_outputs_by_approval() is correctly reused to detect the mixed-split case, and the consistency between build_conclusion_job's has_reviewed_job (front-matter-derived) and wire_explicit_dependencies's (jobs-list-derived) holds in all the relevant cases.
  • The readBooleanEnv / readOptionalEnv TypeScript helpers are well-guarded against the serde_json::Value::to_string quoting pitfall (correctly flagged in the Rust comment).
  • Test coverage is solid: the three compiler integration tests cover the basic case, the absence-without-safe-outputs case, and the mixed manual-review split case.

Generated by Rust PR Reviewer for issue #1076 · 347.7 AIC · ⌖ 12.5 AIC · ⊞ 36.4K ·

@jamesadevine jamesadevine merged commit 9a1ba2f into main Jun 29, 2026
9 checks passed
jamesadevine added a commit that referenced this pull request Jul 1, 2026
- Reject ADO runtime expressions ('$[ ... ]') on provider keys: ADO does not
  evaluate runtime expressions inside a step env: block (the literal is passed
  verbatim, ref #1076), so only macros '$(...)' are now permitted. Previously
  '$[...]' was accepted at validation but would fail IR lowering / ship broken.
- Make provider-key matching case-sensitive (exact) across is_provider_expr_env_key,
  copilot_byom_active, copilot_byom_credential_keys, the base-URL host lookup, and
  the copilot_provider_env prefix filter. The Copilot CLI only reads uppercase
  COPILOT_PROVIDER_*; a lowercase lookalike is never usable, so exact-case matching
  fails closed instead of half-activating BYOM and breaking silently at runtime.
- Replace the magic number 17 with a COPILOT_PROVIDER_PREFIX const + starts_with.
- Document that WIRE_API is deliberately excluded from the credential set, and that
  AWF injects placeholder COPILOT_PROVIDER_* values into the agent container
  (api-proxy-sidecar.md) so --exclude-env + sidecar keep the real credential out.
- Update tests/docs accordingly; add case-sensitivity and $[...]-rejection tests.

Co-authored-by: Copilot <[email protected]>
jamesadevine added a commit that referenced this pull request Jul 1, 2026
* feat(engine): allow ADO expressions on Copilot BYOM provider env keys

Closes #1261. Aligns engine.env validation with gh-aw's BYOM/BYOK model:
an allowlist of COPILOT_PROVIDER_* keys may carry ADO macro/runtime
expressions so Foundry provider credentials can be sourced from Setup-job
outputs; all other keys remain literal-only. A literal
COPILOT_PROVIDER_BASE_URL host is auto-added to the AWF network allowlist.

Co-authored-by: Copilot <[email protected]>

* feat(engine): isolate Copilot BYOM provider credentials via AWF api-proxy sidecar

When engine.env activates Copilot BYOM/BYOK mode (a COPILOT_PROVIDER_* credential
key is present), the compiler now enables the AWF api-proxy sidecar
(--enable-api-proxy) on the agent step and pre-pulls its container image. The
sidecar holds the real provider credential and injects it only at the proxy
layer, so the secret never reaches the agent container or the Copilot CLI
process. The credential keys are also passed as --exclude-env flags
(defense-in-depth). Verified AWF v0.27.9 (the pinned version) exposes
--enable-api-proxy and ships the copilot-byok provider.

Co-authored-by: Copilot <[email protected]>

* feat(engine): extend Copilot BYOM credential isolation to the Detection stage

The Detection (threat-analysis) stage now inherits the same COPILOT_PROVIDER_*
routing and AWF api-proxy credential isolation as the main agent, matching
gh-aw (whose detection engine config inherits the main engine's env). Without
this, Foundry-only workflows — which have no GitHub Copilot token — could not
run threat detection at all.

- engine.rs: add copilot_provider_env() (renders the COPILOT_PROVIDER_* subset)
  and extract render_engine_env_entry() shared by copilot_env/copilot_provider_env.
- agentic_pipeline.rs: thread detection_provider_env; run_threat_analysis_step
  applies the provider env + --enable-api-proxy/--exclude-env; detection job
  pre-pulls the api-proxy image when BYOM is active. Extract awf_api_proxy_flags()
  shared by the agent and detection AWF invocations.

Co-authored-by: Copilot <[email protected]>

* fix(engine): address Rust PR review feedback on BYOM isolation

- Validate the BYOM COPILOT_PROVIDER_BASE_URL host with is_valid_hostname before
  adding it to the AWF allow-domains list, consistent with the api-target path
  (rejects IPv6 literals / non-DNS-safe hosts). [security]
- Derive AWF --exclude-env from the actual provider credential keys present in
  engine.env (case-preserving) instead of a hardcoded uppercase set, so a
  user-written lowercase key cannot activate the sidecar yet still leak into the
  agent container. [isolation bug]
- Gate BYOM detection on the Copilot engine type to prevent a future non-Copilot
  engine from erroneously activating the sidecar. [footgun]
- Reword the engine.env expression validation errors to render clearly
  ('$(...)', '${{ }}', '$[...]') and drop the stray unbalanced paren.
- Tighten the integration assertion to the exact emitted (unquoted) provider env
  line; add unit tests for host rejection and case-preserving key derivation.

Co-authored-by: Copilot <[email protected]>

* fix(engine): tighten BYOM provider env validation per PR review

- Reject ADO runtime expressions ('$[ ... ]') on provider keys: ADO does not
  evaluate runtime expressions inside a step env: block (the literal is passed
  verbatim, ref #1076), so only macros '$(...)' are now permitted. Previously
  '$[...]' was accepted at validation but would fail IR lowering / ship broken.
- Make provider-key matching case-sensitive (exact) across is_provider_expr_env_key,
  copilot_byom_active, copilot_byom_credential_keys, the base-URL host lookup, and
  the copilot_provider_env prefix filter. The Copilot CLI only reads uppercase
  COPILOT_PROVIDER_*; a lowercase lookalike is never usable, so exact-case matching
  fails closed instead of half-activating BYOM and breaking silently at runtime.
- Replace the magic number 17 with a COPILOT_PROVIDER_PREFIX const + starts_with.
- Document that WIRE_API is deliberately excluded from the credential set, and that
  AWF injects placeholder COPILOT_PROVIDER_* values into the agent container
  (api-proxy-sidecar.md) so --exclude-env + sidecar keep the real credential out.
- Update tests/docs accordingly; add case-sensitivity and $[...]-rejection tests.

Co-authored-by: Copilot <[email protected]>

* refactor(engine): drop detection provider-env string round-trip

Address the remaining (non-blocking) design note from PR review: the detection
step's provider env no longer flows as a rendered YAML string that is re-wrapped
and re-parsed via parse_env_block.

- copilot_provider_env now returns validated raw Vec<(String, String)> pairs
  instead of a KEY: "VALUE" YAML string.
- Split validation out of render_engine_env_entry into validate_engine_env_entry,
  shared by the agent (string) path and the detection (typed) path so both
  validate identically.
- Extract env_value_from_str as the single macro-vs-literal classifier, shared
  by parse_env_block and the detection path; run_threat_analysis_step now applies
  the typed EnvValues directly (no env:\n wrap + reparse).

Behavior-preserving: emitted YAML is byte-identical (macro $(Setup.FOUNDRY_TOKEN)
still lowers to an unquoted PipelineVar). The agent path keeps its string form
because it aggregates multiple env sources (engine + awf-path + agent-extension).

Co-authored-by: Copilot <[email protected]>

* docs(engine): clarify BYOM env classification + provider-key scopes per review

Address the two minor (non-blocking) suggestions from the latest PR review:

- Document env_value_from_str's classification boundary: only an exact single
  $(NAME) wrapper becomes PipelineVar; compound/partial macros (e.g. $(A)$(B),
  prefix-$(X)) fall through to Literal but still expand correctly at runtime
  (ADO expands $( ) references inside quoted step-env values). Add unit tests
  pinning both the single-macro and compound/partial-literal paths.
- Add a scope note on copilot_provider_env distinguishing the two provider-key
  sets: ALL COPILOT_PROVIDER_* keys are forwarded to the detection env (routing),
  whereas only the narrower COPILOT_BYOM_CREDENTIAL_ENV_KEYS drive api-proxy
  --exclude-env isolation.
- Fix a doc-comment placement slip so parse_env_block's docs sit on
  parse_env_block again (were left above env_value_from_str by the prior refactor).

No behavior change.

Co-authored-by: Copilot <[email protected]>

* fix(engine): warn on unresolvable literal COPILOT_PROVIDER_BASE_URL

Previously, a literal but malformed COPILOT_PROVIDER_BASE_URL (e.g. a scheme-less
`my-foundry/openai/v1`, or an IPv6 literal) caused provider_base_url_host to
return None and the &&-chain in required_hosts to silently short-circuit — the
host was never added to the AWF allowlist and the agent failed at runtime with a
firewall-blocked network error, contradicting the documented promise that a
literal base URL is added automatically.

Introduce resolve_provider_base_url_host -> ProviderBaseUrlHost (None / Host /
Unresolved), shared by required_hosts (consumes Host) and a new
Engine::network_host_warnings (surfaces Unresolved). generate_allowed_domains
now prints the non-fatal warning telling the operator to add the host via
network.allowed. Compile stays non-fatal.

Add unit tests (malformed + IPv6 warn; valid/expression/absent do not) and
document the warning in docs/engine.md.

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Copilot <[email protected]>
jamesadevine added a commit that referenced this pull request Jul 2, 2026
Every ado-script bundle is a build artifact shipped via the release
ado-script.zip and downloaded to /tmp at pipeline runtime; the in-repo
copies are gitignored. conclusion.js was the sole exception — PR #1076
added the bundle but omitted it from scripts/ado-script/.gitignore, so
the built file was committed and tracked by accident. Nothing consumes
the repo copy (only the /tmp runtime path is referenced), so untrack it
and ignore it alongside the other bundles.

Co-authored-by: Copilot <[email protected]>
jamesadevine added a commit that referenced this pull request Jul 2, 2026
* fix(ado-script): read auto-injected collection URI in getWebApi

conclusion.js could not file the pipeline_failure work item because the
Conclusion step never injected ADO_COLLECTION_URI, which the shared
getWebApi() auth helper required. ADO does not auto-inject that renamed
var, and the Conclusion step builder only set SYSTEM_ACCESSTOKEN.

Align ado-script with the Rust `ado-aw execute` path by reading ADO's
auto-injected collection URI directly: SYSTEM_COLLECTIONURI, falling back
to SYSTEM_TEAMFOUNDATIONCOLLECTIONURI. This removes the collection URI
from the per-step injection contract entirely, so no compiler-emitted
step can forget it again (fixes already-compiled locks too).

Also drop the now-redundant ADO_COLLECTION_URI exports from the gate and
synthetic-PR step builders, update gate_e2e to set SYSTEM_COLLECTIONURI,
refresh docs, and add auth.test.ts precedence coverage.

Fixes #1307

Co-authored-by: Copilot <[email protected]>

* chore(ado-script): stop tracking built conclusion.js bundle

Every ado-script bundle is a build artifact shipped via the release
ado-script.zip and downloaded to /tmp at pipeline runtime; the in-repo
copies are gitignored. conclusion.js was the sole exception — PR #1076
added the bundle but omitted it from scripts/ado-script/.gitignore, so
the built file was committed and tracked by accident. Nothing consumes
the repo copy (only the /tmp runtime path is referenced), so untrack it
and ignore it alongside the other bundles.

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Copilot <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant