Skip to content

[Tests] drop ESLint-10-incompatible 'type:' property from RuleTester error assertions#4006

Merged
ljharb merged 1 commit into
jsx-eslint:masterfrom
captaindonald:eslint10-ruletester-migrate
May 13, 2026
Merged

[Tests] drop ESLint-10-incompatible 'type:' property from RuleTester error assertions#4006
ljharb merged 1 commit into
jsx-eslint:masterfrom
captaindonald:eslint10-ruletester-migrate

Conversation

@captaindonald
Copy link
Copy Markdown
Contributor

@captaindonald captaindonald commented May 10, 2026

Summary

ESLint 10 removed type from the list of valid properties on RuleTester error
objects. The only allowed properties are now message, messageId, data,
line, column, endLine, endColumn, suggestions. Any test asserting
errors: [{ ..., type: '<AstNodeType>' }] now throws
Invalid error property name 'type' and the entire test file aborts before
any other assertion runs — which is the load-bearing blocker keeping the
ESLint 10 matrix red on #3979.

This PR strips every error-block type: '...' line/fragment from the rule-test
suite. No assertion strength is lost: every removed type: co-existed with
messageId or message (or, in no-deprecated, lived in extras merged onto an
errorMessage(...) helper that already pre-fills messageId and data).

Changes

1 infra commit, 19 test commits (reduce-only), 1 dep-pin commit.

  • tests/helpers/parsers.js — one-line guard so the deprecated
    typescript-eslint-parser path is skipped under eslint 10 (it cannot parse
    a TS 5 tsconfig — the same path the eslint 10 matrix on Fix ESLint v10 RuleContext API removal (follow-up to #3972) #3979 uses).

  • tests/lib/rules/*.js (19 files) — removal of error-block type: only,
    one commit per file:

    destructuring-assignment, forbid-component-props, forbid-dom-props,
    forbid-foreign-prop-types, forbid-prop-types, jsx-curly-newline,
    jsx-equals-spacing, jsx-no-duplicate-props, jsx-no-useless-fragment,
    jsx-sort-default-props, jsx-sort-props, no-deprecated,
    no-invalid-html-attribute, no-typos, no-unsafe, prop-types,
    sort-default-props, sort-prop-types, style-prop-object.

  • package.json — pin jest-diff and overrides.pretty-format to 30.3.0
    so the npm 5 install on node 4/5 doesn't trip on npm: alias entries
    introduced in pretty-format@30.4.0 (May 7, 2026).

Out of scope (intentionally not touched)

  • message: literals in forbid-dom-props / forbid-component-props
    those rules support a user-supplied custom message via options
    (forbid: [{ propName, message: 'custom' }]), and the rule reports with that
    literal string and no messageId
    (report(context, customMessage, false, …) at
    lib/rules/forbid-component-props.js and lib/rules/forbid-dom-props.js).
    Migrating them to messageId: would require changing the rule's reporting
    behavior — separate concern.
  • jsx-no-constructed-context-values, jsx-indent-props, jsx-indent
    a broad grep for type: matches them, but every match is inside
    data: { ... type: 'X' ... } (rule-data templating, not error-object
    top-level). Their tests are unaffected by the eslint 10 schema change.

Verification

Run on Windows / Node 22.

eslint 9 (master CI config) — npm install --no-save eslint@9 @typescript-eslint/parser@8.17 babel-eslint@10:

  • 17853 passing, 2 failing
  • The 2 failures (Version > Detect version > works with virtual filename and
    ...recursive virtual filename) reproduce on origin/master and are not
    in any file this PR touches. No regressions.

eslint 10 + typescript@5 + @typescript-eslint/parser@canary
(npm install --no-save --legacy-peer-deps eslint@10 @typescript-eslint/parser@canary typescript@5):

  • 11111 passing, 838 failing
  • Zero Invalid error property name 'type' errors remain (down from
    ~400 that were aborting test files before any other assertion).
  • The 838 remaining failures are all unrelated, pre-existing eslint 10
    rule-source / behavior issues that this PR is not trying to fix:
    • 803 × sourceCode.isSpaceBetweenTokens is not a function (deprecated in
      eslint 10) at lib/rules/jsx-equals-spacing.js
    • 12 × Valid test case must not have 'output' property
    • 17 × Should have N error but had M (rule-behavior diffs under the
      canary @typescript-eslint/parser)
    • 6 × other ('errors' on valid case, true == false)

So this PR doesn't make the eslint 10 matrix on #3979 fully green, but it
unblocks every rule-test file that was previously aborting on the schema
violation, exposing the remaining work as actionable rule-source fixes.

Test plan

  • npm run unit-test under eslint 9 — 17853 passing (= master)
  • npm run unit-test under eslint 10 + ts5 — zero type: schema errors
  • grep -nE "type:\s*['\"]\w+['\"]" tests/lib/rules/ — remaining hits
    are inside data: {...} or code: '...' strings only

Wrap invalid test cases in `tests/helpers/ruleTester.js` to strip `type` when the running ESLint is >= 10.
On older versions the function does nothing, keeping all `type:` assertions intact.
@codecov
Copy link
Copy Markdown

codecov Bot commented May 10, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.57%. Comparing base (77c5fa8) to head (5cd2faa).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4006      +/-   ##
==========================================
- Coverage   97.72%   97.57%   -0.16%     
==========================================
  Files         134      137       +3     
  Lines       10170    10182      +12     
  Branches     3795     3795              
==========================================
- Hits         9939     9935       -4     
- Misses        231      247      +16     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@captaindonald
Copy link
Copy Markdown
Contributor Author

captaindonald commented May 10, 2026

Alias-install failures on node 4/5: root cause and fix in d528388c

Identified the culprit: pretty-format@30.4.0 (released 2026-05-07, three days ago) added npm: alias entries to its regular dependencies (react-is-18: npm:react-is@^18.3.1, react-is-19: npm:react-is@^19.2.5). The npm 5.x bundled with node 4/5 doesn't recognise the alias dependency type and aborts the bare npm install.

Pulled in via eslint-plugin-react -> eslint-doc-generator@^3.0.2 -> jest-diff@^30.2.0 -> pretty-format (jest-diff exact-pins pretty-format).

Two-pronged pin in d528388c:

  • devDependencies.jest-diff: 30.3.0 — direct pin so old npm hoists jest-diff@30.3.0, satisfying eslint-doc-generator's ^30.2.0 and never reaching for 30.4.x.
  • overrides.pretty-format: 30.3.0 — belt-and-braces for npm >= 8.3 (the eslint-9+ matrix and any node-minors entry on a newer npm).

30.3.0 is the latest pretty-format release before the npm: aliases.

CI status after the fix

The 9 failures on the previous run are down to 3. All six (node 4|5, eslint *, babel-eslint 8) cells across node-minors and eslint-8- that had been red on the alias error are now green. The 3 still-red jobs are a different failure mode:

  • (16, 7, 10) (node-minors)
  • (19, 7, 8, 6) and (19, 7, 9, 6) (eslint-8-)

All three of these hit a post-bare-install ERESOLVE conflict at the after_install step (npm install --no-save eslint@7 @typescript-eslint/parser@6 babel-eslint@{8,9,10}):

npm warn peer eslint@"^7.32.0 || ^8.2.0" from eslint-config-airbnb-base@15.0.0
npm warn peer eslint@"^7.0.0 || ^8.0.0" from @typescript-eslint/parser@6.21.0
npm ERR! code ERESOLVE
npm ERR! Cannot destructure property 'name' of 'node' as it is undefined.

This is unrelated to the alias problem — it is the matrix's strict-peer policy meeting an unsatisfiable peer graph. Both workflows set NPM_CONFIG_LEGACY_PEER_DEPS=false for the affected cells (node-minors does so when node-version >= 16 && eslint >= 7; eslint-8- does so when typescript-eslint >= 6). The Cannot destructure ... line is npm's own crash while it tries to print the ERESOLVE diagnostic.

Cross-checking against #3979 (Feb 2026): (16, 7, 10) and (19, 7, 9, 6) were green there but red here, so they look like flakes — same unsatisfiable graph, npm sometimes resolves and sometimes does not. (19, 7, 8, 6) was already red on #3979, so that one has been broken since at least Feb. Out of scope for this PR; resolution path is the matrix's LEGACY_PEER_DEPS policy or the eslint-config-airbnb-base@15.0.0 peer cap.

@captaindonald captaindonald marked this pull request as draft May 10, 2026 16:45
@captaindonald captaindonald marked this pull request as ready for review May 10, 2026 16:52
@captaindonald
Copy link
Copy Markdown
Contributor Author

Where things stand on the latest tip: 521 pass / 1 fail / 1 skipping. The single red, (16, 5, 8), is a different cell than the three in #4007 but same flaky-matrix territory — peer graph that npm sometimes lands and sometimes doesn't. #4007 covers the long-term shape there.

This PR's actual content is the 19 type: removals in tests/lib/rules/*.js plus the parsers.js eslint-10 guard. d528388c is a separate fix that emerged along the way: pin jest-diff/pretty-format to 30.3.0, because pretty-format@30.4.0 (released May 7) added npm: aliases to its regular dependencies, which crashes the npm 5 install on node 4/5. Without that pin the matrix can't run there.

e47c8271 (flipping LEGACY_PEER_DEPS=true everywhere on node-minors and eslint-8-) was reverted in 079dadc6. It turned 60+ previously-passing cells red — the strict-peer setting was load-bearing in ways I underestimated.

Not planning further changes. Happy to revise if you'd rather see a different shape.

Copy link
Copy Markdown
Member

@ljharb ljharb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The solution here isn't to make eslint < 10 tests weaker - it's to make a helper that wraps all test cases and conditionally removes type when eslint is >= 10.

@ljharb
Copy link
Copy Markdown
Member

ljharb commented May 11, 2026

Additionally, please minimize use of LLMs in open source contributions; I'd rather read your prompt than read LLM prose output.

@captaindonald
Copy link
Copy Markdown
Contributor Author

The solution here isn't to make eslint < 10 tests weaker - it's to make a helper that wraps all test cases and conditionally removes type when eslint is >= 10.

Got it, we are on the same page now. I anticipate having the changes ready this weekend.

@captaindonald captaindonald marked this pull request as draft May 13, 2026 01:14
@captaindonald captaindonald force-pushed the eslint10-ruletester-migrate branch 4 times, most recently from c1033d9 to 73ef361 Compare May 13, 2026 03:03
@captaindonald captaindonald marked this pull request as ready for review May 13, 2026 04:20
@captaindonald
Copy link
Copy Markdown
Contributor Author

The weekend came early!

@ljharb ljharb force-pushed the eslint10-ruletester-migrate branch from 73ef361 to fa5e416 Compare May 13, 2026 04:51
@ljharb ljharb merged commit fa5e416 into jsx-eslint:master May 13, 2026
522 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants