feat(toolkit-lib): validate command by kaizencc · Pull Request #1508 · aws/aws-cdk-cli

kaizencc · 2026-05-13T23:41:28Z

Summary

Adds a validate() method to the Toolkit class in @aws-cdk/toolkit-lib. This command:

Synthesizes the CDK app
Reads the policy-validation-report.json from the cloud assembly output directory (written by aws-cdk-lib during synthesis)
Returns a structured ValidateResult with overall pass/fail status and per-plugin reports

This is the toolkit-lib layer only — CLI wiring (cdk validate) will follow separately.

Changes

New action types (lib/actions/validate/index.ts): ValidateOptions, ValidateResult — uses schema types from @aws-cdk/cloud-assembly-schema (PR feat(cloud-assembly-schema): add policy validation report schema types #1515) rather than redefining them
New validate() method on the Toolkit class — reads the report JSON and casts to typed schema, no manual field mapping
IO messages: CDK_TOOLKIT_I9600 (passed), CDK_TOOLKIT_E9600 (failed), CDK_TOOLKIT_I9601 (no report found)
ToolkitAction: added validate to the union type

Dependencies

feat(cloud-assembly-schema): add policy validation report schema types #1515 — schema types in @aws-cdk/cloud-assembly-schema (must merge first)
Corresponding change in aws-cdk-lib to always write policy-validation-report.json to the assembly directory (currently gated behind a context flag)

Test plan

Unit tests covering: success, failure, missing report, IO message emission, violation parsing, plugin version handling
Integration test with a CDK app that has validation plugins configured (blocked on aws-cdk-lib change)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

github-actions · 2026-05-13T23:41:42Z

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

codecov-commenter · 2026-05-13T23:58:42Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.20%. Comparing base (791408c) to head (85e1659).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1508   +/-   ##
=======================================
  Coverage   88.20%   88.20%           
=======================================
  Files          76       76           
  Lines       10834    10834           
  Branches     1492     1492           
=======================================
  Hits         9556     9556           
  Misses       1250     1250           
  Partials       28       28

Flag	Coverage Δ
suite.unit	`88.20% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

rix0rrr · 2026-05-15T10:42:42Z

+    if (status === 'failure') {
+      await ioHelper.notify(IO.CDK_TOOLKIT_E9600.msg('Policy validation failed', result));
+    } else {
+      await ioHelper.notify(IO.CDK_TOOLKIT_I9600.msg('Policy validation passed', result));
+    }


Can we pick slightly less mechanical language here?

"policy policy policy"

Let's speak like a human.

"cdk validate found problems" (or something to that effect), "no problems found", let's not pretend to be IBM and we all wear ties in triplicate.

rix0rrr · 2026-05-19T10:00:46Z

+    const selectStacks = stacksOpt(options);
+    await using assembly = await synthAndMeasure(ioHelper, cx, selectStacks);
+
+    const reportPath = path.join(assembly.directory, POLICY_VALIDATION_REPORT_FILE);


Should be an artifact in the manifest, no?

Although I can live with this for a bit.

https://github.com/orgs/aws/projects/178/views/44?pane=issue&itemId=189647151

i tracked it and will do it separately

rix0rrr · 2026-05-19T10:01:20Z

+        status: 'success',
+        pluginReports: [],
+      };
+      await ioHelper.notify(IO.CDK_TOOLKIT_I9601.msg('No policy validation report found'));


This is very mechanical. What does this mean to me as a user?

Right now it means no plugins / no construct annotations / default rule set is turned off.

I could be convinced for this to not be possible and we always write a report no matter what

there's no default rule set right now so the message wont mention that for now

rix0rrr · 2026-05-19T10:01:43Z

+
+    const reportJson = await fs.readJson(reportPath);
+
+    if (!Array.isArray(reportJson.pluginReports)) {


Can we do a JSON schema validation so we don't have to bother with this?

rix0rrr · 2026-05-19T10:09:36Z

+    if (status === 'failure') {
+      await ioHelper.notify(IO.CDK_TOOLKIT_E9600.msg('Policy validation failed', result));
+    } else {
+      await ioHelper.notify(IO.CDK_TOOLKIT_I9600.msg('Policy validation passed', result));
+    }


"policy policy policy"

Let's speak like a human.

"cdk validate found problems" (or something to that effect), "no problems found", let's not pretend to be IBM and we all wear ties in triplicate.

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

…lidate Instead of redefining the policy validation report types locally, import them from @aws-cdk/cloud-assembly-schema (added in PR #1515). This removes ~130 lines of duplicate type definitions and simplifies the validate method to a typed cast instead of manual field mapping. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Handle malformed report: throw ToolkitError if pluginReports is missing or not an array - Test constructStack (ConstructTraceJson): add recursive trace to fixture and assert nested id/construct/location fields - Assert IO message data payload contains full ValidateResult - Test missing title field gracefully results in undefined Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Replace 'as const' with explicit PolicyValidationReportStatus type annotation - Extract 'policy-validation-report.json' to a file-level constant Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- PolicyValidationReportStatus -> PolicyValidationReportConclusion - PluginReportJson.summary.pluginName -> PluginReportJson.pluginName - PluginReportJson.summary.status -> PluginReportJson.conclusion - PluginReportJson.version -> PluginReportJson.pluginVersion - PolicyViolationJson.fix -> PolicyViolationJson.suggestedFix - ViolatingConstructJson restructured: resourceLogicalId/templatePath moved to cloudFormationResource, constructStack -> stackTraces - ValidateResult.status -> ValidateResult.conclusion - Update test fixtures and assertions to match

- Use friendlier message when no validation report exists: explain that no plugins are configured rather than stating a file is missing - Remove manual pluginReports array check; trust the report format since it's written by aws-cdk-lib (not user-authored)

…ated loading

- "No validation plugins configured" instead of "No policy validation report found" - "cdk validate found problems" instead of "Policy validation failed" - "No problems found" instead of "All policy checks passed"

The policy-validation-report.json fixtures were missing the required `version` field, causing Manifest.loadValidationReport to throw "Invalid semver string: undefined" during schema validation.

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

kaizencc had a problem deploying to automation May 13, 2026 23:41 — with GitHub Actions Failure

kaizencc temporarily deployed to automation May 13, 2026 23:41 — with GitHub Actions Inactive

github-actions Bot added the p2 label May 13, 2026

kaizencc temporarily deployed to no-approval May 13, 2026 23:41 — with GitHub Actions Inactive

aws-cdk-automation requested a review from a team May 13, 2026 23:41

kaizencc temporarily deployed to run-tests May 13, 2026 23:46 — with GitHub Actions Inactive

aws-cdk-automation temporarily deployed to run-tests May 13, 2026 23:52 — with GitHub Actions Inactive

kaizencc had a problem deploying to no-approval May 14, 2026 18:37 — with GitHub Actions Failure

kaizencc had a problem deploying to no-approval May 14, 2026 18:45 — with GitHub Actions Failure

kaizencc had a problem deploying to no-approval May 14, 2026 22:25 — with GitHub Actions Failure

rix0rrr reviewed May 15, 2026

View reviewed changes

kaizencc marked this pull request as ready for review May 18, 2026 15:39

kaizencc had a problem deploying to no-approval May 18, 2026 15:40 — with GitHub Actions Failure

kaizencc mentioned this pull request May 18, 2026

feat(cli): cdk validate command (behind --unstable flag) #1527

Open

3 tasks

rix0rrr reviewed May 19, 2026

View reviewed changes

kaizencc had a problem deploying to no-approval May 19, 2026 17:53 — with GitHub Actions Failure

rix0rrr approved these changes May 20, 2026

View reviewed changes

kaizencc and others added 13 commits May 20, 2026 13:35

feat(toolkit): validate command

80d4a5e

chore: tests

c5ec73e

chore: self mutation

fedcdd7

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

chore(toolkit): cleanup validate method style

b3352c5

- Replace 'as const' with explicit PolicyValidationReportStatus type annotation - Extract 'policy-validation-report.json' to a file-level constant Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

chore(toolkit): use friendlier validation messages with emojis

a13e1c8

refactor(toolkit): use Manifest.loadValidationReport for schema-valid…

4520aab

…ated loading

chore(toolkit): use human-friendly messages for validate

a77be5f

- "No validation plugins configured" instead of "No policy validation report found" - "cdk validate found problems" instead of "Policy validation failed" - "No problems found" instead of "All policy checks passed"

fix(toolkit): add version field to validation report test fixtures

152e026

The policy-validation-report.json fixtures were missing the required `version` field, causing Manifest.loadValidationReport to throw "Invalid semver string: undefined" during schema validation.

chore: self mutation

85e1659

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(toolkit-lib): validate command#1508

feat(toolkit-lib): validate command#1508
kaizencc merged 13 commits into
mainfrom
conroyka/validate

kaizencc commented May 13, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 13, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented May 13, 2026 •

edited

Loading

Uh oh!

rix0rrr May 15, 2026

Uh oh!

rix0rrr May 19, 2026

Uh oh!

rix0rrr May 19, 2026

Uh oh!

kaizencc May 19, 2026

Uh oh!

rix0rrr May 19, 2026

Uh oh!

kaizencc May 19, 2026

Uh oh!

kaizencc May 19, 2026

Uh oh!

rix0rrr May 19, 2026

Uh oh!

kaizencc May 20, 2026

Uh oh!

rix0rrr May 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants


		const reportJson = await fs.readJson(reportPath);

		if (!Array.isArray(reportJson.pluginReports)) {

Conversation

kaizencc commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Dependencies

Test plan

Uh oh!

github-actions Bot commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dependency Review

Scanned Files

Uh oh!

codecov-commenter commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kaizencc commented May 13, 2026 •

edited

Loading

github-actions Bot commented May 13, 2026 •

edited

Loading

codecov-commenter commented May 13, 2026 •

edited

Loading