Sync with python/mypy master#9
Closed
georgesittas wants to merge 220 commits into
Closed
Conversation
…/test_diff_cache.py` (python#21096) In Debian, we build the `mypy` Debian package from the Python package source dists published to https://pypi.org/project/mypy/ ; so to support running the `mypy/test/test_diff_cache.py` tests, include `misc/{diff-cache,apply-cache-diff}.py` in the `MANIFEST.in`
When a user mistypes a module name in an import statement, along with the module not found error, mypy now suggests similar module names that might be what they intended as a note.
This PR collects all visible names in the current scope and uses fuzzy matching to provide suggestions in the error messages
) This PR improves error messages when positional argument is missing from a function call. Previously when a user forgets a positional argument, mypy would previously emit multiple type errors because the subsequent arguments would be "shifted" and mismatched with their expected types. Instead of showing multiple type errors, it emits a single consolidated message that suggests which argument might be missing.
…tax (python#21108) Closes python#20366. The "invariant by default" rule applies to the legacy `TypeVar` syntax. Under PEP 695 (`class MyClass[T]: ...`), mypy infers variance from usage instead. Updated the wording in `common_issues.rst` to make this distinction clear. --------- Co-authored-by: Leo Ji <nuglifeleoji@gmail.com>
@ilevkivskyi made a bunch of improvements to the feature.
This also contains a few small unrelated changes to mypyc docs.
…python#21115) This makes it consistent with the `--native-parser` flag. Now `pip install mypy[native-parser]` will install native parser support.
The release branch has been cut: https://github.com/python/mypy/tree/release-1.20
Add sections for major features and list items for small user-visible changes. Document expected major changes in the mypy 2.0 release. Related issue: python#20726
I was reviewing the commits and I saw the dos/MS Windows line endings; fixed with `dos2unix`
…python#21112) Closes python#17828. The docs used "value restriction" to describe type variables like `T: (str, bytes)`, but the runtime attribute is `TypeVar.__constraints__`. This PR updates the terminology across the affected doc pages to use "value-constrained type variable" consistently, matching the runtime naming and the term discussed in python#17828. Changes: - `generics.rst`: rename section title, update prose, add new anchor `value-constrained-type-variables` (old anchor `type-variable-value-restriction` kept for backward compat) - `more_types.rst`: update cross-reference text - `command_line.rst`: update cross-reference text Made with [Cursor](https://cursor.com) Co-authored-by: Leo Ji <nuglifeleoji@gmail.com>
Add specialized `librt.vecs.vec` iterator classes. There are no primitives for `iter(v)` yet, though we could add them later. Generally using `iter(v)` should be avoided when using vecs, since the generic iterator protocol is inefficient for unboxed items. I used coding agent assist but reviewed generated code carefully (this is mostly boilerplate).
…d/README.md (python#21131) Not required for anything, but it seems nice. CREDITS was excluded back in 2020, but it seems polite to include it in the sdists python#9592 / python@f5f5485 With this PR, CREDITS won't be part of the wheel, but `typeshed/stdlib/_typeshed/README.md` will; let me know if I should exclude it from the wheel
This is used in interpreted/non-native code only, where it will speed up 'in' operations significantly. This is also added for consistency with built-in sequence types. I used coding agents assist but did changes in small increments and reviewed each change manually.
Fixes python#21137 This flag has nothing to with type checking strictness, it is a convenience option for config maintenance.
Helps with confusing symptoms in python#21132 It is unfortunate that it makes the logic a little more ad hoc / might lead to minimal repros being a little more confusing. But I think this is still a better and more helpful state to be in than before python#20660 (relevant PR from the narrowing rewrite). Hopefully we can add the ability to check unreachable code, which will fix this and other issues.
This matches runtime better, e.g. we also show exception notes
Fixes python#21143 (on the same lines as python#20704, i.e. there still exists a deeper fix that could be made) I also update a comment that is a little out of date since python#20863 and code in checkexpr.py that is out of date for modern typeshed
Fixes python#21149 (the regression part of it, still working on the bad behaviour present in previous mypy versions)
…ore" comment` note (python#19904) This should allow the user to more easily compare what is already there; especially if pretty is off, such as in mypy_primer diffs.
Source commit: python/typeshed@c5e47fa
Fixes python#21176 This seems to resolve all the HDD performance issues. Although this makes cache less durable, this is fine, as FS is not perfect in this sense either (and it doesn't really need to be that durable).
This is to unbreak master. We can apply an actual fix later. Refs for context: * python#21120 * python/cpython@e31c551 cc @JukkaL
Fixes python#21141 I am not sure how typeshed patches work. If possible, I would add some tests here, otherwise I will add tests in a follow-up PR.
The plan is to get this small feature release out today (Mon May 11). Related issue: python#21450
Fixes python#21467. `analyze_unbound_tvar` accessed `t.args[0]` unconditionally when a class base was `Unpack` with no arguments (e.g. `class C(Protocol[Unpack]): ...`), raising `IndexError: tuple index out of range` during semantic analysis. This adds a guard returning `None` in that case, so the existing "Free type variable expected in Protocol[...]" error is reported instead of crashing. Added a regression test in `check-typevar-tuple.test`.
Fixes [Mypyc python#1192](mypyc/mypyc#1192) Fix a reference leak in mypyc-generated code for `bytes + bytes`. The `bytes + bytes` primitive for `CPyBytes_Concat` was marked as stealing the left operand with `steals=[True, False]`. However, `CPyBytes_Concat` does not steal or decref either argument; it allocates and returns a new `bytes` object. Because of this mismatch, the refcount pass skipped emitting a `dec_ref` for owned left operands. For code such as: ```python return (1).to_bytes(4, "big") + (2).to_bytes(4, "big") ```` the generated IR decrefed only the right operand after `CPyBytes_Concat`, leaving the owned left operand alive. Chained concatenations amplified the leak because intermediate concat results could also become leaked left operands. This PR removes the incorrect `steals=[True, False]` annotation from the `bytes + bytes` primitive, so both operands are treated as non-stolen and owned operands are decrefed normally after the call. The tests verify that the generated IR decrefs both owned operands after `CPyBytes_Concat`.
The negative narrowing here could be more aggressive, but we will need better literal handling to unblock it Builds on python#21456 Fixes python#13684 in combination with previous PRs. There is one remaining diagnostic, but that one is desirable
Fixes a duplicated word in the overload implementation compatibility docs. No code changes. Co-authored-by: Noa Levi <275430404+lphuc2250gma@users.noreply.github.com>
This PR serves as the foundation for the smaller alternative of the [`char` proposal request](python#21418). It builds on top of the existing `librt.strings` and `ord(char)` specialization, so code like the following can lower to a direct codepoint check i.e without materializing a 1-character str: ```py from librt.strings import isspace from mypy_extensions import i32 c: i32 = ... if (isspace(c)): ... ``` <br /> Semantics: - Input type is `i32` - Negative inputs return `False` - For all valid Unicode codepoints, behavior matches `str.isspace()` on the corresponding 1-character string (ensured by exhaustive test too) This PR adds only `isspace`; It does not add any new type-system surface, and it does not introduce the rest of the codepoint helperfamily. I'll be contributing these next if this direction looks good. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
) Fixes python#21475 Avoid the crash while we don't support this. The actual feature is tracked in python#21346
Free threading support and limitations weren't documented at all. Some features that we currently support were still documented as unsupported. This includes some dunders and numeric use cases. We now support simple numeric use cases via `vec[float]`.
This is mostly needed for python#21260
2nd PR building python#21418 Wraps `Py_UNICODE_ISDIGIT` for the codepoint fast path, mirroring the already-merged `librt.strings.isspace`. I've microbenchmarked both codepoint primitives and they're ~1.6x faster than their counterpart `str` specialized primitives.
Fixes python#20391 The problem became more prominent with type variables with defaults, so better be consistent.
3rd PR for python#21418, mirroring `librt.strings.isdigit`. Measured on a microbenchmark this is roughly 30-40% faster for a char
…hon#21481) With `separate=True` and cross-module inheritance, when only the subclass module is recompiled incrementally and the parent is loaded from mypy's incremental cache, `find_attr_initializers` gathers no defaults from the parent. The subclass therefore has no `__mypyc_defaults_setup` of its own, and `ClassIR.get_method` walks the MRO and returns the parent's. Without this fix, `emit_attr_defaults_func_call` emits a raw `CPyDef_<parent>___...(...)` call. The parent's header only declares that function as a pointer inside `struct export_table_<group>`, so the symbol isn't reachable as a free function from the subclass's compilation unit and clang/gcc fail with: ``` error: call to undeclared function 'CPyDef_<parent_module>___<Parent>_____mypyc_defaults_setup'; ISO C99 and later do not support implicit function declarations ``` A cold build doesn't hit this because the parent's `defs.body` is populated (everything is freshly parsed), so the subclass gets its own `__mypyc_defaults_setup` and the call is intra-group. Likewise, an incremental change that propagates through interface-hash deps to the parent makes it get reparsed too, avoiding the trigger. The bug requires an invalidation pattern that touches the subclass but **not** the parent. ## Fix `emit_attr_defaults_func_call` in `mypyc/codegen/emitclass.py` now applies `emitter.get_group_prefix(defaults_fn.decl)` when emitting the call, matching the pattern already used by the other cross-group call sites in this file (`emit_setup_or_dunder_new_call`, `generate_constructor_for_class`, etc.). `get_group_prefix` returns `""` for same-group calls (so intra-group behaviour is unchanged) and `"exports_<group>."` when the target lives in a different group. It also registers the target group in `context.group_deps` so the right header gets `#include`d. ## Tests Added `testIncrementalCrossModuleInheritedAttrDefaults` in `mypyc/test-data/run-multimodule.test`, a two-step test that reproduces the bug under `TestRunSeparate`: parent in `other_b.py` with attribute defaults, empty subclass in `other_a.py`, step 2 modifies `other_a.py` to trigger a recompile without touching the parent. Verified to fail under `TestRunSeparate` (with the implicit-declaration error) on the unpatched tree and to pass under all three modes (`TestRun`, `TestRunMultiFile`, `TestRunSeparate`) with the fix. Local checks: - `pre-commit run --all-files` — pass - `python runtests.py self` — pass - mypyc unit tests (1077 + 1 skipped) — pass - `TestRunSeparate` + `TestRunMultiFile` run tests (110) — pass ## Real-world impact Surfaced first against `sqlglot-mypy==1.20.0.post6` (a downstream of mypy used to compile sqlglot) in CI when `build/` and `.mypy_cache/` were preserved across GitHub Actions runs and a PR happened to edit only subclass modules. The incremental compile produced malformed C and failed with the implicit-declaration error. Stripped of sqlglot specifics, the bug requires only `separate=True` + cross-module inheritance + parent with attribute defaults + subclass without its own defaults + an invalidation pattern where the subclass is rechecked but the parent is not. These conditions are common in any non-trivial codebase using mypyc with `separate=True`, so this should be considered a latent issue affecting incremental builds of such codebases. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…Union[TypedDict, dict[K, V]] (python#21511) When a variable is typed as `Union[TypedDict, dict[int, float]]`, mypy incorrectly raised `Expected TypedDict key to be string literal` when assigning a dict literal with non-string keys like `{1: 5.2}`. The plain `dict[int, float]` alternative makes the assignment valid, so this was a false positive. The root cause was that `match_typeddict_call_with_dict`, which is used as a probe to check whether a dict literal could match a TypedDict, was emitting errors from `validate_typeddict_kwargs` during the matching phase rather than silently returning `False`. Wrapping the call in `filter_errors()` suppresses those spurious diagnostics. Fixes python#21510 Co-authored-by: Claude Agent <agent@example.com>
Fixes python#19106 Tuple constructor is quite imprecise, and I don't think we can get anything decent without some special-casing, so I am doing just that, adding one more `special_sig`.
Fixes python#21471. ## Summary - Avoid special-casing `Annotated[...]` as its inner type when the subscript has no arguments. - Let the normal type analyzer report the existing `Annotated[...]` argument error instead of crashing. - Add a regression test for `list[Annotated[()]]` in type application context. Prepared with AI assistance; I reviewed and verified the changes. --------- Co-authored-by: cyphercodes <cyphercodes@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Fixes python#19110 Two (mostly unrelated) things here: * Handle empty tuple index in tuple subclasses * Move tuple special-casing in map type to inner function to handle situations with indirect tuple subclasses.
4th PR of python#21418 This is 5x faster than `str.isalpha()` for ASCII.
…thon#21490) ## Problem In `separate=True` mode, each group's `__native_internal_<mod>.h` reaches into a sibling group's export-table header via an angled include: ```c // __native_internal_caller.h #include <other_group/__native_other.h> ``` The consumer's `.o` file bakes in byte offsets from that struct at C compile time. If the sibling group's struct layout shifts (e.g. a class is inserted earlier in the source, shifting all subsequent offsets), the consumer's `.o` must be recompiled, otherwise it silently resolves offsets to the wrong object at runtime. Two bugs in the old `get_header_deps` combined to hide this dep from setuptools' `Extension.depends`: 1. **No transitive walk.** The cross-group include lives inside `__native_internal_<mod>.h`, not in the `.c` file itself. The old code only scanned the `.c` file's direct includes and never opened the resolved headers. 2. **Single-pass resolution.** Deps were resolved per-group before sibling groups had written their headers to disk, so the cross-group header didn't yet exist even if we had looked. ## Fix - **`_extract_includes` / `_INCLUDE_RE`**: distinguishes `#include "foo"` (quoted) from `#include <foo>` (angled), matching the preprocessor's own search rules — quoted form searches the includer's directory first; angled form searches `-I` paths only. - **`resolve_cfile_deps`**: transitive walker that opens each resolved header and follows its `#include` directives, bounded by a visited set. Headers that don't exist under `target_dir` (lib-rt headers like `<Python.h>`) are dropped since they never change between builds. - **Two-pass loop in `mypyc_build`**: first pass writes all groups' files to disk; second pass resolves deps so every group's headers are present when cross-group includes are looked up. ## Impact on non-incremental builds None. `Extension.depends` is only consulted by setuptools when a `.o` already exists. Cold builds compile everything unconditionally. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Collaborator
Author
|
Closing since this branch is now the fresh one and contains everything in merge + some of our changes on top. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merges
upstream/master(python/mypy) intorelease-1.20to bring the fork fully up to date.Conflict resolutions
mypy/version.py: kept fork version (1.20.0.post9)pyproject.toml: kept fork'sast-serializedependency innative-parserextramypyc/build.py: took upstream — its refactored loop subsumes the fork's cached-group and dep-resolution fixesmypyc/codegen/emit.py: took upstream — thenative_function_call()refactor was already upstreamed via [mypyc] Fix cross-group call to inherited __mypyc_defaults_setup python/mypy#21481mypyc/codegen/emitmodule.py: took upstream — same logic, upstream usesemit_linesfor claritymypyc/lib-rt/misc_ops.c,module_shim*.tmpl: took upstream — comment wording onlymypyc/irbuild/expression.py: merged both sides — fork addschar_rprimitive/is_char_rprimitive/is_str_rprimitive/is_tagged/short_int_rprimitive, upstream addsint64_rprimitive/is_bytearray_rprimitive/is_bytes_rprimitive/is_tuple_rprimitive/vec_api_by_item_typemypyc/doc/librt.rst: took upstream (adds random/strings/time/vecs table entries) with upstream's__link syntaxmypyc/test-data/run-multimodule.test: took upstream (fork had nothing to add here)mypyc/test-data/run-classes.test: kept fork'stestCompiledSubclassAttrAccessWithAllowInterpretedSubclassestest; took upstream'sfoo = Foo()inside loop for the leak testmypyc/test/test_misc.py: took upstream style, re-inserted fork'stest_cached_group_deps_populated_from_disk_cfiletest