feat(compile): migrate legacy {{ workspace }} markers and validate checkout-aware paths#1263
Conversation
…eckout-aware paths
Before the native-IR migration, the compiler folded {{ workspace }},
{{ working_directory }} and {{ trigger_repo_directory }} markers across
the whole generated YAML (including custom steps). After the migration
these markers flow through verbatim and are no longer substituted.
Rather than restore runtime substitution of a fixed-path anchor (an
antipattern under multi-checkout, where $(Build.SourcesDirectory) is the
shared root of every checked-out repo), this:
- Adds codemod 0004_legacy_path_markers, which rewrites the markers in
front matter to the explicit ADO path they resolved to, derived from
the source's own workspace:/repos:.
- Adds a warning-only path_layout_check pass that flags checkout-aware
path mistakes (references to not-checked-out repos, the multi-checkout
self-subfolder form under single checkout, runtime-import targets to
not-checked-out repos) and deprecated markers left in the agent body.
- Extracts shared pure resolvers (resolve_working_directory_expr,
contains_template_marker) from compute_effective_workspace.
Co-authored-by: Copilot <[email protected]>
🔍 Rust PR ReviewSummary: Looks good overall — clean architecture, solid tests, correct codemod ordering. Two issues worth addressing before merge. Findings🐛 Bugs / Logic Issues
|
Address Rust PR review feedback on #1263: - contains_template_marker now advances one code-point past a non-matching {{ (matching replace_marker) so nested {{ ... }} forms are detected consistently and the codemod early-exit gate cannot skip a substitutable marker. - sources_dir_segments advances i by start + seg.len() instead of start, avoiding re-scanning an extracted segment on back-to-back double prefixes. - replace_marker uses a let-else break instead of expect() on the string-walk hot path. Adds regression tests for the nested-marker consistency and double-prefix segment cases. Co-authored-by: Copilot <[email protected]>
🔍 Rust PR ReviewSummary: Looks good overall — clean refactor, solid test coverage, and a well-motivated approach. One real (low-severity) bug to fix and a couple of minor observations. Findings🐛 Bugs / Logic Issues
Both helpers check for
The PR's description says The existing Suggested fix for both helpers: skip any // in contains_template_marker
let preceded_by_dollar = marker_start > 0 && input.as_bytes()[marker_start - 1] == b'$';
if !preceded_by_dollar {
if let Some(close) = input[start..].find("}}") ... { return true; }
}
// in replace_marker
if input[i..].starts_with("{{") {
let preceded_by_dollar = i > 0 && input.as_bytes()[i - 1] == b'$';
if !preceded_by_dollar { ... }
}
|
Address second Rust PR review on #1263: contains_template_marker and replace_marker now skip any {{ immediately preceded by a dollar sign, so an ADO template expression like ${{ workspace }} is neither detected as a legacy marker nor substituted (which would have produced a stray $ before the replacement). Adds a regression test covering the exclusion for both helpers, and documents the deliberate narrowing of the step-path check to executable step blocks plus the empty-segment no-progress-risk invariant in sources_dir_segments. Co-authored-by: Copilot <[email protected]>
🔍 Rust PR ReviewSummary: Looks good — sound logic, well-tested, no bugs or security issues found. Two minor visibility nits. Findings
|
Address third Rust PR review on #1263: resolve_working_directory_expr and collect_path_layout_warnings are only used within the crate, so narrow them from pub to pub(crate) to keep the public API surface honest. Co-authored-by: Copilot <[email protected]>
🔍 Rust PR ReviewSummary: Solid implementation — logic is correct, well-tested, and the design choices (codemod vs. runtime substitution, warning-only validation) are sound. A few minor points worth noting for follow-up. Findings✅ What Looks Good
|
Summary
Before the native-IR migration, the compiler folded the directory markers
{{ workspace }},{{ working_directory }}and{{ trigger_repo_directory }}across the entire generated YAML — including user-authoredsteps:/post-steps:/setup:/teardown:. After the migration these markers flow through verbatim and are no longer substituted (the reported regression).Rather than restore runtime substitution of a fixed-path anchor — an antipattern under multi-checkout, where
$(Build.SourcesDirectory)is the shared root of every checked-out repo — this PR migrates existing sources and adds checkout-aware validation:Codemod
0004_legacy_path_markers— rewrites the markers (interior whitespace ignored, so both{{ workspace }}and{{workspace}}are handled) in every front-matter string scalar to the explicit ADO path they resolved to, derived from the source's ownworkspace:/repos::{{ workspace }}/{{ working_directory }}→$(Build.SourcesDirectory),$(Build.SourcesDirectory)/$(Build.Repository.Name), or$(Build.SourcesDirectory)/<alias>{{ trigger_repo_directory }}→ the trigger ("self") repo dirIdempotent, pure, registered after
m0001;${{ parameters }}and{{#runtime-import}}spans are preserved.Warning-only
path_layout_checkpass (hooked intocompile_pipeline_inner) — flags$(Build.SourcesDirectory)/<seg>references to declared-but-not-checked-out repos, the multi-checkout self-subfolder form used under a single checkout,{{#runtime-import …}}targets to not-checked-out repos, and deprecated markers left in the agent body (which the codemod cannot migrate). Never fails the compile.Shared pure resolvers — extracted
resolve_effective_workspace(pure core) fromcompute_effective_workspace, plusresolve_working_directory_exprandcontains_template_marker, reused by both the codemod and the validation pass to avoid drift.Docs —
docs/codemods.md(new section),docs/front-matter.md(deprecation note), and theAGENTS.mdarchitecture tree.Test plan
cargo build✓cargo test✓ — 2310+ pass, including:path_layout_checkunit tests (each warning case + dedup + extraction helpers)compile_migrates_legacy_workspace_marker_in_steps) asserting source rewrite + lock content + codemod warningcargo clippy --all-targets --all-features✓ clean