feat(compile): runtime prompt loading via {{#runtime-import}} markers

[jamesadevine]

Summary

Adopts gh-aw's {{#runtime-import path}} marker model and the inlined-imports front-matter toggle. Agent prompt bodies are now loaded at pipeline runtime by default, so edits to the markdown body no longer require ado-aw compile.

Behaviour

• inlined-imports: false (new default) — the agent body is loaded at pipeline runtime via a {{#runtime-import …}} marker resolved by a new import.js ado-script bundle. The compiled YAML contains the marker (and the resolver step), not the body text.
• inlined-imports: true — legacy behaviour preserved bytes-for-bytes for the simple case; author-written {{#runtime-import shared/snippet.md}} markers are resolved at compile time so the body is fully self-contained.
• Author UX bonus — {{#runtime-import path}} (required) and {{#runtime-import? path}} (optional, skip-if-missing) work inside any agent's markdown body, with the same semantics as gh-aw.

The Stage-2 threat-analysis prompt is not runtime-imported. It's a tooling-shipped template that's include_str!'d into the ado-aw binary and inlined into the emitted YAML at compile time, matching gh-aw's pattern (their threat_detection.md ships with the setup action and is read directly from disk — no marker, no resolver).

Consolidated single extension (addresses reviewer feedback)

One always-on AdoScriptExtension owns all ado-script wiring. It exposes two features through the existing trait hooks — no new template markers, no ScriptAssets registry:

Feature	Hook	Job that consumes the bundle
Gate evaluator (gate.js)	setup_steps()	Setup
Runtime-import resolver	prepare_steps()	Agent

ADO jobs use isolated VMs — /tmp is not shared. The bundle is therefore downloaded once per consuming job. When both features are active, install + download steps appear in both Setup and Agent. That's correct architecture given ADO's topology, not waste.

filters:	inlined-imports	Setup-job steps	Agent-job extra steps
inactive	true	(none)	(none)
inactive	false	(no Setup job)	install + download + resolver
active	true	install + download + gate	(none)
active	false	install + download + gate	install + download + resolver

Implementation

• New consolidated extension src/compile/extensions/ado_script.rs (~470 lines). One internal helper install_and_download_steps() produces the install+download YAML; both setup_steps() and prepare_steps() call it. The Rust source has one place for the install/download YAML; the emitted YAML carries it once per consuming job.
• resolve_imports_inline() (compile-time resolver for inlined-imports: true) lives in the same module.
• Deleted: src/compile/extensions/trigger_filters.rs, src/compile/extensions/runtime_prompt.rs, src/compile/script_assets.rs, the script_assets: ScriptAssets field on CompileContext and its four constructor sites, the prepend block at common.rs:2123-2131, and the {{ agent_prompt_resolver_steps }} template marker (all four base templates).
• 1ES double-call eliminated: compile_shared now detects when extra_replacements already binds {{ setup_job }} (the 1ES path) and skips its own redundant generate_setup_job invocation. Each extension's setup_steps() is now invoked exactly once per pipeline.
• NodeTool@0 displayName updated from "Install Node.js 20.x for gate evaluator" to "Install Node.js 20.x" (the bundle now serves both gate.js and import.js).
• Docs refreshed: docs/ado-script.md (per-job download model), docs/runtime-imports.md, docs/filter-ir.md, docs/template-markers.md, AGENTS.md.

Test plan

Per-job placement tests (pin the reviewer-found bug)

4 new tests in tests/compiler_tests.rs that split the emitted YAML by job block and assert exactly which job(s) contain Download ado-aw scripts:

Test	Expectation
test_gate_only_pipeline_downloads_bundle_in_setup_job_not_agent	Download in Setup only
test_imports_only_pipeline_downloads_bundle_in_agent_job_not_setup	Download in Agent only (no Setup job)
test_both_features_active_downloads_bundle_in_both_jobs	Download in BOTH (exactly 2 occurrences)
test_neither_feature_active_emits_no_node_or_download_anywhere	No NodeTool@0 and no Download anywhere

Without the consolidation, tests (2) and (3) fail with "Agent job is missing the script bundle download" — exactly the cross-job VM isolation bug the reviewer caught.

Automated

• cargo build clean.
• cargo test --bin ado-aw: 1595 unit tests pass.
• cargo test --tests: 108 compiler_tests + every integration suite green.
• cargo clippy --all-targets --all-features clean (only pre-existing warnings).
• cd scripts/ado-script && npm test: 199 tests across 26 files.
• cd scripts/ado-script && npm run test:smoke: 3/3 pass.
• cd scripts/ado-script && npm run typecheck clean.

Manual

Hand-inspected emitted YAML for the four (gate × imports) combinations:

Combo	NodeTool@0	Download	Resolver	Setup job present	Agent job present
gate_only	1	1	0	Yes	Yes
imports_only	1	1	1	No	Yes
both	2	2	1	Yes	Yes
neither	0	0	0	No	Yes

All four match the design.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[github-actions]

🔍 Rust PR Review

Summary: Has a critical cross-job VM isolation bug that will cause inlined-imports: false (the new default) pipelines to fail at runtime, plus a minor issue in the TypeScript error path.

---

Findings

🐛 Bugs / Logic Issues

[CRITICAL] src/compile/script_assets.rs + src/compile/common.rs:3197-3215 — ScriptAssets download runs in the wrong ADO job

RuntimePromptExtension::setup_steps() calls ctx.script_assets.request(), which causes ScriptAssets::emit_steps() to include the NodeTool@0 + Download ado-aw scripts steps in generate_setup_job. That function produces a separate top-level - job: Setup block (look at common.rs:2168). The node import.js resolver steps, however, are placed inside the - job: Agent block via {{ agent_prompt_resolver_steps }} (base.yml line 111).

Azure DevOps jobs run on separate, isolated VMs — /tmp is not shared between jobs. So when the Agent job tries to execute:

- bash: |
    set -eo pipefail
    node '/tmp/ado-aw-scripts/ado-script/dist/import/index.js' /tmp/awf-tools/agent-prompt.md

.../tmp/ado-aw-scripts/ won't exist (it was unzipped in the Setup job on a different VM). Every pipeline with inlined-imports: false (the new default) will fail at this step at runtime.

The existing TriggerFiltersExtension doesn't have this problem because its gate.js steps are also returned from setup_steps() and placed in the Setup job — they run on the same VM as the download. The import resolver needs to follow the same pattern: either emit the download steps as part of agent_resolver_steps inline in the Agent job, or add a per-job download mechanism.

The automated tests don't catch this because they only verify that the compiled YAML contains the string "Download ado-aw scripts", not which job it appears in.

---

[Minor] scripts/ado-script/src/import/index.ts:42-46 — Writes partially-expanded file before exiting on error

const expanded = original.replace(MARKER, (_whole, optional, rawPath) => {
    if (!existsSync(absPath)) {
        hadError ??= `file not found: ${rawPath}`;
        return "";   // ← replaces required marker with empty string
    }
    // ...
});
// ...
if (hadError) { fail(hadError); }
writeFileSync(target, expanded, "utf8");  // ← already wrote above ← wait, no:

Actually the writeFileSync only runs if there's no hadError check issue — looking again: fail() calls process.exit(1) so writeFileSync is NOT reached on error. ✓ (I withdraw this finding — the control flow is correct since fail() exits.)

---

⚠️ Suggestions

• src/compile/script_assets.rs:40 — The NodeTool@0 display name says "Install Node.js 20.x for gate evaluator" but this step now also serves the import resolver. Minor: consider updating to "Install Node.js 20.x".

• src/compile/onees.rs:131-143 — onees.rs::generate_setup_job calls ext.setup_steps(ctx) for all extensions, and then compile_shared calls common.rs::generate_setup_job which does the same. For 1ES, the result of the second call is never used (the {{ setup_job }} marker is already replaced by extra_replacements). This is harmless due to ScriptAssets::request() being idempotent, but the double-call is wasteful and could surprise future extension authors whose setup_steps() have side effects.

---

✅ What Looks Good

• resolve_imports_inline in runtime_prompt.rs is clean and well-tested: path traversal is impossible (the check rejects whitespace, absolute paths are used as-is, relative paths join against the compile-time base dir), error messages are actionable, the five unit tests cover the important cases.
• ScriptAssets dedup pattern is elegant — single AtomicBool + idempotent request() correctly collapses N extension calls to exactly one NodeTool + download pair. The existing gate.js refactor (removing duplicate steps from TriggerFiltersExtension) is a clean win.
• inlined-imports: bool field has proper #[serde(rename = "inlined-imports", default)] — defaults to false without requiring the field to be Option<bool>.
• The import.js single-pass design (no recursive expansion of imported snippets) is correct and intentional; the inline comment explains why.
• Integrity-check tests in codemod_tests.rs (body-only edits pass under default, fail under inlined-imports: true) directly validate the stated UX goal.

Generated by Rust PR Reviewer for issue #625 · ● 2.1M · ◷

[github-actions]

🔍 Rust PR Review

Summary: Looks good — well-designed feature with strong test coverage. Three minor findings, none blocking.

---

Findings

⚠️ Suggestions

• src/compile/script_assets.rs:37 — Stale display name in shared NodeTool step
  The NodeTool@0 step still displays "Install Node.js 20.x for gate evaluator" but is now also used for the import resolver. Since ScriptAssets is shared by both TriggerFiltersExtension and RuntimePromptExtension, the display name should be something like "Install Node.js 20.x for ado-aw scripts" to avoid confusion in pipeline run logs.

• src/compile/common.rs:3195 — Resolve runtime imports step missing timeoutInMinutes
  The NodeTool and Download steps both have timeoutInMinutes: 5, but the resolver bash step (node '{import_eval_path}' /tmp/awf-tools/agent-prompt.md) omits it. The script is fast in practice, but consistency with the surrounding steps would prevent a stalled Node process from tying up the pipeline until the job-level timeout fires.

• src/compile/onees.rs:65 / src/compile/common.rs:3057 — ext.setup_steps() called twice for 1ES
  For the 1ES compile path, each extension's setup_steps() (and its ctx.script_assets.request() side-effect) is called once inside onees::generate_setup_job and then again inside compile_shared → common::generate_setup_job. The second call's result is silently discarded because the extra_replacements entry for {{ setup_job }} wins before the shared replacement runs. This is currently harmless — all side-effects are idempotent and AtomicBool is fine — but a future extension with a non-idempotent setup_steps would quietly drop steps on the 1ES target. Worth a comment, or a short-circuit in compile_shared when extra_replacements already covers {{ setup_job }}.

---

✅ What Looks Good

• ScriptAssets deduplication is clean: request() + emit_steps() pattern correctly emits NodeTool + curl/sha256/unzip exactly once regardless of how many extensions request it, and the unit tests pin both the no-request and multi-request cases.
• resolve_imports_inline error handling is solid: unterminated markers, empty paths, whitespace-in-paths, missing required files, and missing optional files all handled with clear anyhow errors or silent drops respectively.
• Integrity check semantics are correct for the new default: body-only edits don't trip the check (inlined-imports: false), front-matter edits still do, and the three new codemod_tests.rs tests cover all three cases end-to-end.
• ##vso[...] injection in import.js is not possible — the [^\s}]+ capture group prevents any whitespace (including newlines) in the imported path, so the fail() output can't be split across lines to inject additional VSO commands.
• Test coverage is comprehensive: 12 compile-output tests across 3 modes × 4 targets, 5 unit tests for inline resolution, 7 vitest specs for the import bundle, 3 smoke tests, and the 1ES setup-job regression test.

Generated by Rust PR Reviewer for issue #625 · ● 3.6M · ◷

[jamesadevine]

Thanks for the careful review — the critical cross-job VM isolation finding was spot on. Force-pushed the fix.

What changed in this revision

• Critical bug fixed (per-job download). The ado-script.zip install + download now lands in the same job as each consumer: Setup job for the gate evaluator, Agent job for the runtime-import resolver. When both features are active the bundle is downloaded in both jobs — correct architecture given ADO's VM isolation, not duplication waste.

• Consolidated to a single always-on AdoScriptExtension at src/compile/extensions/ado_script.rs. It exposes the gate via setup_steps() and the runtime-import resolver via prepare_steps() — both hooks already exist on the CompilerExtension trait, so no new template markers were needed. The previous {{ agent_prompt_resolver_steps }} marker is gone (resolver flows through the existing {{ prepare_steps }} block instead).

• ScriptAssets registry removed along with the script_assets: ScriptAssets field on CompileContext. The registry's "one shared download serves all consumers" premise was the bug; per-job emission replaces it.

• {{ setup_job }} double-binding fixed. compile_shared now detects when extra_replacements has already bound {{ setup_job }} (the 1ES override path) and skips its own redundant generate_setup_job invocation. Each extension's setup_steps() is invoked exactly once per pipeline.

• NodeTool@0 displayName updated from "Install Node.js 20.x for gate evaluator" to "Install Node.js 20.x".

Test pinning

4 new placement tests in tests/compiler_tests.rs parse the emitted YAML by job block (extract_job_block helper) and assert exactly which job(s) contain Download ado-aw scripts:

• gate-only → Setup only
• imports-only → Agent only (no Setup job)
• both → both jobs (exactly 2 downloads)
• neither → no NodeTool@0, no download anywhere

On main before this revision, the imports-only and both-active tests fail with "Agent job is missing the script bundle download" — your critical finding, codified.

Net file shape

• New: src/compile/extensions/ado_script.rs.
• Deleted: src/compile/extensions/trigger_filters.rs, src/compile/extensions/runtime_prompt.rs, src/compile/script_assets.rs, the {{ agent_prompt_resolver_steps }} template marker.
• 24 files changed, ~+5 net lines.

[jamesadevine]

@copilot resolve the merge conflicts in this pull request

[Copilot]

@copilot resolve the merge conflicts in this pull request

Resolved by merging origin/main into this branch and fixing the conflict in docs/filter-ir.md. Commit: 44f4c85.