fix-shared-libpython-link
1019
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
05f6e6f681 |
fix(subinterpreter): don't touch the thread state before create() attaches one (#6127)
`subinterpreter::create()` documents that "the main interpreter and its GIL
are not required to be held prior to calling this function", but its first
statement is `error_scope err_scope;`, i.e. `PyErr_Fetch()`, before
`main_guard` attaches a thread state. With no current `PyThreadState`,
`PyErr_Fetch()` -> `_PyErr_GetRaisedException(NULL)` dereferences null and
the process dies (SIGSEGV; 0xC0000005 on Windows). `~error_scope` is the
mirror image: it calls `PyErr_Restore()` after `main_guard` has already
swapped the thread state back away.
Two ordinary situations reach `create()` with no thread state:
- an embedder that ends its initialization with `PyEval_SaveThread()`,
which is the documented way to hand the GIL back after
`Py_InitializeFromConfig()`;
- any worker thread that has never touched Python.
Existing tests never hit this because they all run under the
`py::scoped_interpreter guard{}` in catch.cpp, which keeps the GIL held on
the main thread for the whole run.
Move `error_scope` inside the `main_guard` scope. The case it exists for is
unaffected: a caller that already holds the main GIL takes
`subinterpreter_scoped_activate`'s `simple_gil_` fast path, which keeps the
same thread state, so its pending error is still saved across
`Py_NewInterpreterFromConfig()` and restored afterwards. A caller sitting on
some other interpreter never had its error indicator touched in the first
place, since everything inside the block runs on the main interpreter's
thread state and `PyThreadState_Swap()` does not move error indicators. It
also makes the `pybind11_fail()` path unwind in a safer order: `~error_scope`
now runs while `main_guard` is still alive.
Add "Create Subinterpreter without a thread state", covering both a thread
that dropped its thread state via `gil_scoped_release` and a thread that
never had one. It segfaults without the fix and passes with it.
Verified on Windows / MSVC 14.51 / CPython 3.13.14: test_with_catch goes
from 33 to 34 test cases, all passing.
Assisted-by: ClaudeCode:claude-opus-5
|
||
|
|
6dd1756f98 |
chore(deps): update pre-commit hooks (#6126)
* chore(deps): update pre-commit hooks updates: - [github.com/pre-commit/mirrors-clang-format: v22.1.5 → v22.1.8](https://github.com/pre-commit/mirrors-clang-format/compare/v22.1.5...v22.1.8) - [github.com/astral-sh/ruff-pre-commit: v0.15.20 → v0.16.1](https://github.com/astral-sh/ruff-pre-commit/compare/v0.15.20...v0.16.1) - [github.com/pre-commit/mirrors-mypy: v2.1.0 → v2.3.0](https://github.com/pre-commit/mirrors-mypy/compare/v2.1.0...v2.3.0) - [github.com/codespell-project/codespell: v2.4.2 → v2.4.3](https://github.com/codespell-project/codespell/compare/v2.4.2...v2.4.3) * style: pre-commit fixes * fix: adapt to the ruff 0.16 default rule set The hook update pulled in ruff 0.16, which expanded its default rules and autofixed `self: S -> S` to `-> Self`. That fix added a runtime `typing_extensions` import to setup_helpers.py, which must stay dependency-free; it broke every test job whose environment does not supply that package. The import is now guarded by TYPE_CHECKING. Also replace the `exec` in docs/conf.py with an importlib module load, parenthesize implicit string concatenations in list literals, mark unused unpacked variables with a leading underscore, and make noxfile.py executable to match its shebang. The remaining new rules are silenced per-file, with reasons: the chrono tests check naive local time on purpose, and the other rules conflict with how the tests are written. Assisted-by: ClaudeCode:claude-opus-5 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Henry Schreiner <henryfs@princeton.edu> |
||
|
|
59565095fa |
Delegate py::print to Python's native print (#6121)
* refactor: delegate py::print to native print * style: use auto for print result * test: focus py::print coverage on delegation * docs: describe native py::print delegation * fix: keep py::print silent when builtin is missing * test: cover py::print with unavailable stdout * docs: clarify py::print builtins lookup |
||
|
|
63d627c574 |
feat!: drop support for Python 3.8, MSVC 2017 (#6110)
* feat!: drop support for Python 3.8 The minimum supported version is now Python 3.9. pybind11 v3.0 was the last release that supports Python 3.8. The deprecation note said that support goes away in 3.1. Remove the code paths that this makes dead: - the `_PyObject_Vectorcall` fallback in `cast.h` - the `frame->f_code` and `frame->f_back` fallbacks in `pytypes.h` - the `PyFrame_FastToLocals` path in `get_type_override` - the conditional `Py_VISIT(Py_TYPE(self))` in `tp_traverse` - `PYBIND11_PYCONFIG_SUPPORT_PY_VERSION_HEX`, the pre-PyConfig interpreter init, and the `widen_chars` helpers in `embed.h` Assisted-by: ClaudeCode:claude-opus-5 * ci(appveyor): use Python 3.9 The AppVeyor job set `PYTHON: 38`, which makes the path `C:\Python38`. The image gives Python 3.9.13 as `C:\Python39`. Assisted-by: ClaudeCode:claude-opus-5 * feat!: require MSVC 2019 or newer Python 3.9 is the new minimum, so MSVC 2017 is no longer needed. Raise the compile-time floor to _MSC_VER 1920 and remove the workarounds that only applied below it: std::launder, fold expressions, weak_from_this, aligned new/delete, the C4100 warning helper, and the func_handle syntax error. AppVeyor now builds with Visual Studio 2019. Assisted-by: ClaudeCode:claude-opus-5 * fix(appveyor): build against the Python that has the test packages CMake 4 has no FindPythonLibs, so pybind11 uses FindPython. FindPython reads the registry before PATH and selected `C:\Python314-x64`, but the test packages go into the Python on PATH. Pass `Python_ROOT_DIR` to name the correct one. Also set `CMAKE_ARCH`. It was never set, so the architecture came from the generator. Visual Studio 2017 defaults to Win32, but Visual Studio 2019 defaults to x64, which made this x86 job build 64-bit code. Assisted-by: ClaudeCode:claude-opus-5 * fix(appveyor): give the linker Python's libs directory The build compiled but failed to link with LNK1104 on a bare `python39.lib`. That name comes from the `#pragma comment(lib, ...)` in pyconfig.h, so the linker needs the directory. CMake 4 has no FindPythonLibs, and FindPython does not add it for this Debug x86 build. Put it on LIB instead. The directory listing is temporary, to confirm the library is present. Assisted-by: ClaudeCode:claude-opus-5 * fix(appveyor): link the release Python library in Debug The image ships python39_d.lib next to python39.lib, so FindPython picks the debug import library for a Debug build. pybind11 undefines _DEBUG around Python.h, so pyconfig.h asks for python39.lib instead and the link failed with LNK1104. Name the release library for the debug slot. Setting LIB does not work, because MSBuild replaces it from the toolset, and it would link both import libraries. Assisted-by: ClaudeCode:claude-opus-5 * chore(appveyor): print link settings to debug LNK1104 Revert the two attempted fixes. Neither changed the failure: setting LIB does not survive MSBuild, and naming the release library for Python_LIBRARY_DEBUG had no effect. Print the Python cache entries and the link settings of a generated project file instead, to see what the linker really gets. Temporary. Assisted-by: ClaudeCode:claude-opus-5 * fix(appveyor): take the release Python library in Debug The generated project file linked C:\Python39\libs\python39_d.lib in the Debug configuration, because the image ships debug binaries next to the release ones. pybind11 undefines _DEBUG around Python.h, so pyconfig.h asks for python39.lib in a #pragma comment(lib), which nothing on the link line satisfies and no library directory holds. Map Debug to the release artifacts. Assisted-by: ClaudeCode:claude-opus-5 * chore(appveyor): drop the temporary link diagnostic Assisted-by: ClaudeCode:claude-opus-5 * fix(tests): guard the unraisable warning filter for pytest < 6 The distro pytest in the Clang and GCC Docker jobs has no PytestUnraisableExceptionWarning, so an unconditional filterwarnings mark makes pytest fail with an INTERNALERROR after the tests pass. Assisted-by: ClaudeCode:claude-opus-5 Claude-Session: https://claude.ai/code/session_01Aimf6HuSz1vLRwBnbxmCTc * docs: address review items on version hints, embed docs, and a PyPy xfail Extend Python_ADDITIONAL_VERSIONS through 3.15, describe the PyConfig behavior of initialize_interpreter, and drop the stale Python 3.8 wording from the PyPy xfail reason. Assisted-by: ClaudeCode:claude-opus-5 * Update README.rst Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com> --------- Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com> |
||
|
|
ea9e6e6a23 |
feat: small command line helpers for quick tests (#4272)
* feat: small command line helpers for quick tests
Add --cflags, --ldflags, --embed, and --file to the pybind11 command
line tool, based on python-config. This makes quick one-off compiles
easy:
c++ $(python3 -m pybind11 --file=example.cpp)
Assisted-by: ClaudeCode:claude-fable-5
* refactor: quote paths, clarify CLI output logic, strengthen tests
- Quote include and library dirs in get_cflags/get_ldflags so paths with
spaces work, sharing one quote helper in commands.py
- Restructure the --file/--cflags/--ldflags printing into a single print
- Note the Unix-compiler orientation in help text and docs
- Test -L presence for --embed and quoting of paths with spaces
Assisted-by: ClaudeCode:claude-fable-5
* refactor: simplify flag helpers and keep import pybind11 light
- Replace _get_config_var(name, fmt, quote) with a plain _config(name)
string helper; formatting and quoting happen at the call sites
- Defer sysconfig/shlex imports so import pybind11 does not pay for the
CLI (~1ms -> ~0.08ms for pybind11.commands)
- Fetch EXT_SUFFIX once in main()
- Import commands normally in the quoting test instead of importlib
file loading
Assisted-by: ClaudeCode:claude-fable-5
* fix: shared-library flags on all Unix platforms
The --file/--ldflags output only added -shared and -fPIC on Linux, so
other Unix platforms (FreeBSD, Solaris, AIX) linked an executable and
failed on the missing main. Gate on os.name instead.
Assisted-by: ClaudeCode:claude-opus-5
|
||
|
|
652c69437b |
fix: add py::mod_gil_used() spelling, support pedantic tests (#5797)
* tests: add linking warnings/error Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * tests: check pedantic C++14 * fix: some pedantic warnings Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * feat: add py::mod_gil_used() Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * fix: use not_supported() Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * style: pre-commit fixes * Update CMakeLists.txt * fix: remove the true/false parameter from mod_gil_not_used Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * style: pre-commit fixes * Update tests/CMakeLists.txt * fix: deprecate mod_gil_not_used(bool) instead of removing it Keep the bool constructor public and mark it deprecated, so existing code that spells mod_gil_not_used(false) still compiles. mod_gil_used() now sets the flag through its friendship, which keeps the deprecated constructor the only bool overload and stops pybind11 warning against itself. Also use the py:: alias in exo_planet_pybind11.cpp to match its sibling home_planet_very_lonely_traveler.cpp. Assisted-by: ClaudeCode:claude-opus-5 * fix: add trailing comma to macros picked up in rebase test_smart_ptr.cpp and standalone_enum_module.cpp came from master after this branch was written, so they still invoked variadic macros with no variadic argument. That is a pedantic error below C++20. Assisted-by: ClaudeCode:claude-opus-5 --------- Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
|
5540f96802 |
fix: only add string_view life support for transient sources (#6096)
* revert: "revert: add life support to handles cast to string_view (#6092)"
This re-applies #6092 (reverting #6097) so the follow-up fixes in this PR can build on it.
Assisted-by: ClaudeCode:claude-opus-4.8
* fix: don't throw from string_view life support outside a bound function
PR #6092 added loader_life_support::add_patient(src) to keep the source
object alive when loading a string view, fixing a real use-after-free when
a container of views is built from a non-sequence iterable (e.g. a
generator): list_caster materializes a temporary tuple that owns the
strings and destroys it when load() returns, before the bound function
body runs.
add_patient throws when there is no life support frame, so casting to a
view outside a bound function (e.g. a manual py::cast<std::string_view>)
now raises instead of relying on the caller-owned source, a regression
from #6092.
For these view-into-src cases registration is best effort: inside a bound
function it keeps src alive (fixing the UAF), and outside one the caller
owns src's lifetime as before. Add try_add_patient(), which returns false
instead of throwing when there is no frame, and use it at the three view
load sites. add_patient() keeps its strict contract for value-creating
conversions.
Assisted-by: ClaudeCode:claude-opus-4.8
* fix: only add string_view life support for transient sources
Refine the previous commit. Best-effort registration (try_add_patient)
silently produces a dangling view when a container of views is built from
a generator outside a bound function: there the materialized temporary is
released before the view is used, and with no frame nothing keeps it
alive. Such a cast cannot be made safe, so it should fail loudly, while a
view into a durable, caller-owned object needs no life support at all.
The view caster cannot tell a durable source from a pybind11-managed
transient one; that provenance lives in the container caster. Introduce an
ambient transient_source_guard that the list, set, map, and array casters
set around their generator/materialized paths, and have the string caster
keep the source alive only when loading from a transient source (via the
throwing add_patient, so try_add_patient is no longer needed). This means:
- views into durable sources (direct arguments, sequences, manual casts)
add no life support and no longer throw outside a bound function, and
- a generator used outside a frame throws, rather than silently dangling.
The guard restores (rather than clears) the previous value, so a durable
container nested in a transient one is correctly treated as transient.
Verified with AddressSanitizer: the in-frame generator case is clean, the
out-of-frame durable cases succeed, and the out-of-frame generator case
throws.
Assisted-by: ClaudeCode:claude-opus-4.8
* Revert "fix: only add string_view life support for transient sources"
This reverts commit
|
||
|
|
f95b7a5860 |
fix: include complex in std::complex input annotation (#6113)
* fix: include `complex` in std::complex input annotation The input annotation for `std::complex<T>` omitted the builtin `complex` type, which typeshed only makes a `typing.SupportsComplex` on Python 3.11+. Assisted-by: ClaudeCode:claude-opus-5 Claude-Session: https://claude.ai/code/session_013JABmnjt9oAh29p1pytAB3 * Add comment to explain why the explicit `complex` is needed. --------- Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com> |
||
|
|
17b3b605a5 |
test: remove dead PR3068 workaround branch (#6109)
`WORKAROUND_ENABLING_ROLLBACK_OF_PR3068` compared the 5-element `sys.version_info` tuple to `(3, 9)`, so it was always False. The guarded branch never ran. Remove the constant and keep only the branch that runs. Assisted-by: ClaudeCode:claude-opus-5 |
||
|
|
57e7a8de34 |
fix(stl_bind): correct __delitem__ for negative-step slices and re-enable contiguous erase fast path (#6088)
* fix(stl_bind): correct __delitem__ for negative-step slices and re-enable contiguous erase fast path The slice __delitem__ binding advanced the erase index by step - 1 for all steps. That correction is only valid for positive steps, where erasing shifts later elements down by one. For negative steps the visited indices are strictly decreasing and erasing never shifts them, so the extra -1 deleted the wrong elements (e.g. del v[::-2] on [0,1,2,3] yielded [1,2] instead of [0,2]) and del v[::-1] walked off the front of the vector (v.begin() - 1, observed SIGBUS). Switch to the signed slice::compute overload so negative steps stay signed, advance by step for negative steps and step - 1 for positive ones, and drop the && false that had disabled the O(n) contiguous fast path since 2016. Assisted-by: ClaudeCode:claude-fable-5 * refactor: address review — static_cast and parametrized test Use static_cast instead of a C-style cast for the slice.compute() size argument, and convert the __delitem__ slice test to pytest.mark.parametrize over the slice cases. Assisted-by: ClaudeCode:claude-fable-5 * test(stl_bind): cover slice deletion edge cases * fix(stl_bind): erase strided slices in descending order * Eliminate a variable and avoid redundant index increment (i + 1, ++i). The control flow handles all relevant boundaries: - slicelength == 0: excluded by the outer guard. - slicelength == 1: erases once, decrements to zero, and breaks without touching start. - Larger slices: updates start exactly when another erase remains. - slicelength cannot underflow because the loop exits when it reaches zero. - Mutating slicelength is harmless because it is not used afterward. - The potentially dangerous final start += step remains eliminated. It also removes the separate loop counter. The compiler would probably optimize the former i + 1, ++i mechanics away, but the new source expresses the real state more directly: "number of erasures remaining." The unconditional while (true) is safe because entry is strictly guarded by slicelength > 0, and the decrement guarantees eventual termination. --------- Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com> |
||
|
|
36341ba9ba | ci: test Python 3.15 (#6107) | ||
|
|
28b98f2d39 |
ci: update cibuildwheel to v4.1 and pyodide to 314 (#6095)
* ci: update cibuildwheel to v4.1 and pyodide to 314 Bump pypa/cibuildwheel from v3.4 to v4.1 across the Pyodide, iOS, and Android jobs. Move the Pyodide build to cp314-pyodide_wasm32 (pyodide 314.0.0). Drop the iOS CIBW_SKIP: cp314-* workaround now that pypa/cibuildwheel#2494 is resolved. Assisted-by: ClaudeCode:claude-opus-4.8 * test: xfail array resize on numpy<2.4 + Python 3.14 Removing the cp314 iOS skip surfaced a pre-existing numpy bug, not a cibuildwheel regression: numpy<2.4 has a resize(refcheck=True) regression on Python 3.14 where the reference held by the bound function isn't detected, so a resize that should raise instead succeeds and the later reshape rejects the non-square size. Fixed in numpy>=2.4, but the iOS test environment has no numpy wheel newer than 2.3.5.post1, so xfail the test for that specific combination. Assisted-by: ClaudeCode:claude-opus-4.8 * test: link numpy resize aliasing issue and trim comment iOS numpy wheels updated to 2.5.0, so the xfail is a general numpy<2.4 + Python 3.14 guard rather than iOS-specific. Reference numpy/numpy#30265. Assisted-by: ClaudeCode:claude-opus-4.8 |
||
|
|
db92ad940f |
revert: "add life support to handles cast to string_view (#6092)" (#6097)
This reverts commit
|
||
|
|
59d7cb28c1 |
fix: add life support to handles cast to string_view (#6092)
* Fix handling of string_view to prevent GC'ing strings before they are used. * style: pre-commit fixes * Update cast.h --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
|
ff1a35cb84 |
ci: update GraalPy from 24.2 to 25.0 (Python 3.12 based) (#6083)
* ci: update GraalPy from 24.2 to 25.0 (Python 3.12 based) - Replace graalpy-24.2 with graalpy-25.0 in CI matrix - Replace graalpy-24.1 with graalpy-24.2 (shift older version up) - Remove dead GRAALPY_VERSION < (24, 2) xfail guards from tests - Remove unused GRAALPY_VERSION from tests/env.py - Update internals.h comment to reference v25.0 Assisted-by: OpenCode:glm-5 * fix: use numpy 2.2.x for GraalPy 3.12 (graalpy312 wheels) numpy 1.26.x only has graalpy311 wheels; GraalPy 25.0 (Python 3.12) requires numpy 2.2.x which has graalpy312 wheels on the GraalVM index. Assisted-by: OpenCode:glm-5 * fix: simplify numpy requirement for GraalPy (drop 3.11 branch) We no longer test GraalPy 3.11, so the version split is unnecessary. Assisted-by: OpenCode:glm-5 |
||
|
|
6079989cf7 |
Pypy testing no longer xfails (#6077)
* test passes on PyPy macOS * adjust tests for pypy HEAD * fixes from review * pypy 7.3.23 was released, drop some PyPy testing * pin to pypy version 7.3.23 * Restore xfail for cross-module translator platforms Keep the expected failure for Android and FreeBSD while limiting the PyPy-specific part to versions before 7.3.23. Android CIBW still raises RuntimeError for this test, matching the existing tracked platform issue. --------- Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com> |
||
|
|
46ebf5031b |
feat(subinterpreter): reusable PyThreadState via subinterpreter_thread_state (#6073)
* feat(subinterpreter): add opt-in TLS-cached thread state mode subinterpreter_scoped_activate previously created and destroyed a fresh PyThreadState on every activation when the calling OS thread was not already running the target interpreter. Workloads that repeatedly re-enter the same sub-interpreter from the same thread therefore churn thread states and lose per-thread interpreter state between activations (see pybind/pybind11#6040). Add an opt-in subinterpreter_thread_state::cached policy: on first use a PyThreadState is created and stored in OS-thread-local storage keyed by the target interpreter; subsequent activations on that thread only swap it in/out and never destroy it. The default stays transient, so existing behavior is unchanged. Since pybind11 does not control thread lifetime, cleanup is explicit: subinterpreter::release_cached_thread_state() releases the calling thread's cached state for one interpreter, and the static release_all_cached_thread_states() releases all of the calling thread's cached states as an end-of-thread hook. The TLS map's destructor only frees its own nodes and never touches the Python C API, so an unreleased state leaks rather than crashing at thread exit. Includes test coverage and embedding docs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * style: pre-commit fixes * refactor(subinterpreter): replace cached enum/TLS with subinterpreter_thread_state RAII Address review feedback on the original "cached" mode by switching to an explicit two-RAII design suggested by @b-pass: "Create a class ... to RAII-manage the PyThreadState but start its lifetime in an already released state. You could create another class (or modify scoped_activate) to scoped/RAII activate the inactive threadstate." Removed - enum subinterpreter_thread_state { transient, cached } and the defaulted ctor parameter on subinterpreter_scoped_activate. - detail::subinterpreter_thread_state_cache thread_local map. - subinterpreter::release_cached_thread_state() and subinterpreter::release_all_cached_thread_states(). This eliminates: the hidden per-thread map, the "release_all" footgun across pybind11 modules (the cache was module-local), and the implicit "must not be active when called" contract on the release functions. Added - Public class subinterpreter_thread_state that owns one PyThreadState for a given subinterpreter on its constructing OS thread, created in a released state (not current, no GIL). Non-copyable, non-movable (PyThreadState is bound to its creating OS thread). - subinterpreter_scoped_activate(subinterpreter_thread_state &) overload: swaps the owned PyThreadState in on entry, swaps it out on exit, does not touch its lifetime. Behavior - The existing subinterpreter_scoped_activate(subinterpreter const &) overload is unchanged (still transient: New on entry, Delete on exit). All previously-working code keeps working. - With subinterpreter_thread_state, one OS thread can alternate between multiple subinterpreters and each PyThreadState is preserved across activations -- the use case that gil_scoped_release/acquire + a long-lived scoped_activate cannot solve alone (the per-thread internals.tstate slot holds only one inactive tstate). - The dtor of subinterpreter_thread_state guards against the "destroyed-while-active" contract violation: if Swap reveals the cached tstate was current, do not Swap back to a now-deleted pointer (the safe-when-active fix b-pass requested for the old release_* functions, applied at the natural location instead). Lifetime contract is enforced by ordinary C++ scope: typical placement is `thread_local`. No new release/cleanup APIs are required. Tests cover (a) tstate identity preserved across activations on a thread, (b) transient and reusing modes do not share state, (c) different OS threads get distinct PyThreadStates, and (d) the multi-subinterpreter alternation case. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(subinterpreter): address review on #6073 (same-thread checks, test scoping) Per @b-pass's review: - ~subinterpreter_thread_state(): add a PYBIND11_DETAILED_ERROR_MESSAGES- guarded check that destruction happens on the OS thread that created the PyThreadState (same PyThread_get_thread_native_id pattern as ~subinterpreter), failing with pybind11_fail otherwise. - subinterpreter_scoped_activate(subinterpreter_thread_state &): add the matching DETAILED_ERROR_MESSAGES check that activation happens on the creating OS thread, enforcing the newly documented rule. - docs: document that activating a subinterpreter_thread_state on another OS thread is illegal. - tests: keep each subinterpreter (and its subinterpreter_thread_state) in an enclosing scope so destruction order is thread-state -> subinterpreter -> unsafe_reset_internals_for_single_interpreter(). The previous top-level declarations ran the reset while the subinterpreters were still alive, which is the likely cause of the CI crashes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs: fix codespell (re-used -> reused) in embedding.rst Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
|
f891299e6a |
Fix custom __str__ for enum_ (#6078)
* fix: prioritize custom enum __str__
* test: strengthen custom enum __str__ regression
* refactor: move enum __str__ handling into enum_
Keep class_::def() generic and let enum_ own the enum-specific
behavior for custom __str__ overloads. This avoids using the private
__entries attribute as a runtime sentinel for py::enum_ while preserving
the prepend behavior that lets user-defined enum __str__ methods take
precedence over the generated default.
One caveat is that this applies to normal py::enum_ API usage. Code that
intentionally upcasts an enum_ binding to class_& and then calls
class_::def("__str__", ...) will bypass this enum_ override and keep the
generic class_ behavior.
* fix: avoid enum def ambiguity on MinGW
Remove the inherited class_::def overload set from enum_ and add an explicit forwarding overload for non-string def() calls. GCC 15 on MinGW otherwise sees the duplicate dependent-base def(const char *, ...) template as ambiguous with enum_::def(const char *, ...), breaking the C++11 build.
---------
Co-authored-by: ctmd1234567 <ctmd1234567@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
|
||
|
|
28bbda9ddf |
[skip ci] test: reject shared_ptr casts for custom holders (#6069)
Lock down the current behavior discussed in issue #6064: py::cast() from std::shared_ptr<T> remains rejected for custom holder bindings that are not std::shared_ptr or py::smart_holder. This extracts the incompatible-holder coverage from PR #6068 and adds a test shaped like issue #6064, while PR #6065 and PR #6066 explore alternative support paths. |
||
|
|
5a5f21deb6 |
fix: Invert input/output context if a Python function is called from C++ (#6055)
When the Callable itself is an input (parameter) to a C++ function, its arguments are outputs (C++ passes them to the Python callback), and vice versa. Therefore, we must invert them if C++ calls a Python function, but keep them the same in the other direction. |
||
|
|
678b6735e4 |
build: support Eigen 5 (#6036)
* build: support Eigen 5 fix #6034 * build: probe Eigen 3 and 5 separately in CMake config mode Avoid relying on package-specific handling of a bounded version range when discovering Eigen through Eigen3Config.cmake. Made-with: Cursor * [skip ci] build: clarify Eigen 5 module fallback comment Explain that the MODULE-mode fallback only exists for older Eigen 3 setups so the remaining fallback path does not look like an unresolved Eigen 5 issue. Made-with: Cursor * [skip ci] docs: add Eigen 5 entry to v3.0.4 changelog Document the Eigen 5 CMake package detection fix in the 3.0.4 release notes before merging the PR. Made-with: Cursor --------- Co-authored-by: Eisuke Kawashima <e-kwsm@users.noreply.github.com> Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com> |
||
|
|
7029f99cc6 |
fix: segfault when moving scoped_ostream_redirect (#6033)
* fix: segfault when moving `scoped_ostream_redirect` The default move constructor left the stream (`std::cout`) pointing at the moved-from `pythonbuf`, whose internal buffer and streambuf pointers were nulled by the move. Any subsequent write through the stream dereferenced null, causing a segfault. Replace `= default` with an explicit move constructor that re-points the stream to the new buffer and disarms the moved-from destructor. * fix: mark move constructor noexcept to satisfy clang-tidy * fix: use bool flag instead of nullptr sentinel for moved-from state Using `old == nullptr` as the moved-from sentinel was incorrect because nullptr is a valid original rdbuf() value (e.g. `std::ostream os(nullptr)`). Replace with an explicit `active` flag so the destructor correctly restores nullptr buffers. Add tests for the nullptr-rdbuf edge case. * fix: remove noexcept and propagate active flag from source - Remove noexcept: pythonbuf inherits from std::streambuf whose move is not guaranteed nothrow on all implementations. Suppress clang-tidy with NOLINTNEXTLINE instead. - Initialize active from other.active so that moving an already moved-from object does not incorrectly re-activate the redirect. - Only rebind the stream and disarm the source when active. * test: add unflushed ostream redirect regression Cover the buffered-before-move case for `scoped_ostream_redirect`, which still crashes despite the current move fix. This gives the PR a direct reproducer for the remaining bug path. Made-with: Cursor * fix: disarm moved-from pythonbuf after redirect move The redirect guard now survives moves, but buffered output could still remain in the moved-from `pythonbuf` and be flushed during destruction through moved-out Python handles. Rebuild the destination put area from the transferred storage and clear the source put area so unflushed bytes follow the active redirect instead of crashing in the moved-from destructor. Made-with: Cursor --------- Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com> |
||
|
|
0db7f72dc2 |
Handle result from PyObject_VisitManagedDict (#6032)
* Handle result from PyObject_VisitManagedDict * add unit test * style: pre-commit fixes * use different variable name This avoids a warning on msvc about Py_Visit shadowing the vret variable. * skip test_get_referrers on unsupported runtimes The managed-dict referrer check is only known to work on CPython 3.13.13+ and 3.14.4+, while earlier releases and non-CPython interpreters can report different traversal behavior. Made-with: Cursor --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com> |
||
|
|
98e50b702d |
fix: avoid copy constructor instantiation in shared_ptr fallback cast (#6028)
* tests: add regressions for shared_ptr reference_internal fallback * fix: avoid copy constructor instantiation in shared_ptr fallback cast * Remove stray empty line * tests: rename PyTorch shared_ptr regression test files * refactor: add cast_non_owning helper for reference-like casts Name the non-owning generic cast path so callers do not have to rediscover that reference-like policies must pass null copy/move constructor callbacks. This keeps the shared_ptr reference_internal fallback self-documenting and points future maintainers toward the safe API. Made-with: Cursor * tests: guard deprecated-copy warning probes with __has_warning Use __has_warning for the Clang-only regression test so older compiler jobs skip unsupported warning groups instead of failing with -Wunknown-warning-option. A simple __clang_major__ >= 13 guard would be shorter, but it bakes in a version cutoff; __has_warning is slightly more verbose while being more robust to vendor builds, backports, and future packaging differences. Made-with: Cursor --------- Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com> |
||
|
|
83f71d8b82 |
fix: detect virtual inheritance in add_base to prevent pointer offset crash (#6017)
Virtual inheritance places the base subobject at a dynamic offset, but
load_impl Case 2a uses reinterpret_cast which assumes a fixed offset.
This caused segfaults when dispatching inherited methods through virtual
bases (e.g. SftVirtDerived2::name()).
Add an is_static_downcastable SFINAE trait that detects whether
static_cast<Derived*>(Base*) is valid. When it is not (virtual
inheritance), set multiple_inheritance = true in add_base to force the
implicit_casts path, which correctly adjusts pointers at runtime.
Remove the workaround .def("name", &SftVirtDerived2::name) from
test_smart_ptr.cpp that was papering over the issue.
Made-with: Cursor
|
||
|
|
524d72b36d |
fix: strdup "self" arg in def_property_static, partially revert #6010 (gh-5976) (#6015)
* fix: strdup args added after initialize_generic in def_property_static (gh-5976) `def_property_static` calls `process_attributes::init` on already-initialized function records (after `initialize_generic`'s strdup loop has run). Args added at this stage (e.g. "self" via `append_self_arg_if_needed`) remain as string literals, so `destruct()` would call `free()` on them. Fix by strdup'ing name/descr of any args appended by the late `process_attributes::init` call. Root cause introduced by gh-5486. Made-with: Cursor * Partially revert gh-6010: remove py_is_finalizing() workarounds Now that the root cause (free of string literals in def_property_static, gh-5976) is fixed in the previous commit, the py_is_finalizing() guards introduced in gh-6010 are no longer needed: - tp_dealloc_impl: remove early return during finalization (was leaking all function records instead of properly destroying them) - destruct(): remove guard around arg.value.dec_ref() - common.h: remove py_is_finalizing() helper (no remaining callers) The genuine fix from gh-6010 (PyObject_Free + Py_DECREF ordering in tp_dealloc_impl) is retained. Made-with: Cursor * test: add embedding test for py::enum_ across interpreter restart (gh-5976) py::enum_ is the primary trigger for gh-5976 because its constructor creates properties via def_property_static / def_property_readonly_static, which call process_attributes::init on already-initialized function records. Yet none of the existing embedding tests used py::enum_ at all. Add an PYBIND11_EMBEDDED_MODULE with py::enum_ and a test case that imports it, finalize/reinitializes the interpreter, and re-imports it. This exercises the def_property_static code path that was fixed in the preceding commit. Note: on Python 3.14.2 (and likely 3.12+), tp_dealloc_impl is not called during Py_FinalizeEx for function record PyObjects — they simply leak because types are effectively immortalized. As a result, this test cannot trigger the original free()-on-string-literal crash on this Python version. However, it remains valuable as a regression guard: on Python builds where finalization does clean up function records (or if CPython changes this behavior), the test would catch the crash. It also verifies that py::enum_ survives interpreter restart correctly, which was previously untested. Made-with: Cursor * test: skip enum restart test on Python 3.12 (pre-existing crash) Made-with: Cursor * Add test_standalone_enum_module.py, standalone_enum_module.cpp * Make standalone_enum_module.cpp more similar to #5976 reproducer. Also fix clang-tidy error. * This crashes when testing locally: ( cd /wrk/forked/pybind11/tests && PYTHONPATH=/wrk/bld/pybind11_gcc_v3.14.2_df793163d58_default/lib /wrk/bld/pybind11_gcc_v3.14.2_df793163d58_default/TestVenv/bin/python3 -m pytest test_standalone_enum_module.py ) ============================= test session starts ============================== platform linux -- Python 3.14.2, pytest-9.0.2, pluggy-1.6.0 installed packages of interest: build==1.4.2 numpy==2.4.3 scipy==1.17.1 C++ Info: 13.3.0 C++20 __pybind11_internals_v12_system_libstdcpp_gxx_abi_1xxx_use_cxx11_abi_1__ PYBIND11_SIMPLE_GIL_MANAGEMENT=False rootdir: /wrk/forked/pybind11/tests configfile: pytest.ini plugins: timeout-2.4.0, xdist-3.8.0 collected 1 item test_standalone_enum_module.py F [100%] =================================== FAILURES =================================== ________________________ test_enum_import_exit_no_crash ________________________ def test_enum_import_exit_no_crash(): # Modeled after reproducer under issue #5976 > env.check_script_success_in_subprocess( f""" import sys sys.path.insert(0, {os.path.dirname(env.__file__)!r}) import standalone_enum_module as m assert m.SomeEnum.__class__.__name__ == "pybind11_type" """, rerun=1, ) test_standalone_enum_module.py:10: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ code = 'import sys\nsys.path.insert(0, \'/wrk/forked/pybind11/tests\')\nimport standalone_enum_module as m\nassert m.SomeEnum.__class__.__name__ == "pybind11_type"' def check_script_success_in_subprocess(code: str, *, rerun: int = 8) -> None: """Runs the given code in a subprocess.""" import os import subprocess import sys import textwrap if ANDROID or IOS or sys.platform.startswith("emscripten"): pytest.skip("Requires subprocess support") code = textwrap.dedent(code).strip() try: for _ in range(rerun): # run flakily failing test multiple times subprocess.check_output( [sys.executable, "-c", code], cwd=os.getcwd(), stderr=subprocess.STDOUT, text=True, ) except subprocess.CalledProcessError as ex: > raise RuntimeError( f"Subprocess failed with exit code {ex.returncode}.\n\n" f"Code:\n" f"```python\n" f"{code}\n" f"```\n\n" f"Output:\n" f"{ex.output}" ) from None E RuntimeError: Subprocess failed with exit code -6. E E Code: E ```python E import sys E sys.path.insert(0, '/wrk/forked/pybind11/tests') E import standalone_enum_module as m E assert m.SomeEnum.__class__.__name__ == "pybind11_type" E ``` E E Output: E munmap_chunk(): invalid pointer _ = 0 code = 'import sys\nsys.path.insert(0, \'/wrk/forked/pybind11/tests\')\nimport standalone_enum_module as m\nassert m.SomeEnum.__class__.__name__ == "pybind11_type"' os = <module 'os' (frozen)> rerun = 1 subprocess = <module 'subprocess' from '/wrk/cpython_installs/v3.14.2_df793163d58_default/lib/python3.14/subprocess.py'> sys = <module 'sys' (built-in)> textwrap = <module 'textwrap' from '/wrk/cpython_installs/v3.14.2_df793163d58_default/lib/python3.14/textwrap.py'> env.py:68: RuntimeError =========================== short test summary info ============================ FAILED test_standalone_enum_module.py::test_enum_import_exit_no_crash - Runti... ============================== 1 failed in 0.23s =============================== ERROR: completed_process.returncode=1 * Add "Added in PR #6015" comments, for easy reference back to this PR * test: use PYBIND11_CATCH2_SKIP_IF for Python 3.12 enum restart skip Replace #if/#else/#endif preprocessor guard with runtime PYBIND11_CATCH2_SKIP_IF so the test is always compiled and shows [ SKIPPED ] in output on Python 3.12. Made-with: Cursor * fix: suppress MSVC C4127 in PYBIND11_CATCH2_SKIP_IF macro The constant condition in PYBIND11_CATCH2_SKIP_IF triggers MSVC warning C4127 (conditional expression is constant), which becomes a build error under /WX. Made-with: Cursor |
||
|
|
3b62426106 |
Fix static_pointer_cast build failure with virtual inheritance in holder_caster_foreign_helpers.h (#6014)
* Add regression test for #5989: static_pointer_cast fails with virtual inheritance When a class uses virtual inheritance and its holder type is shared_ptr, passing a shared_ptr of the derived type as a method argument triggers a compilation error because static_pointer_cast cannot downcast through a virtual base (dynamic_pointer_cast is needed instead). Made-with: Cursor * Fix #5989: use dynamic_pointer_cast for virtual inheritance in esft downcast Replace the unconditional static_pointer_cast in set_via_shared_from_this with a SFINAE-dispatched esft_downcast helper that falls back to dynamic_pointer_cast when static_cast through a virtual base is ill-formed. Also add a workaround in the test binding (.def("name") on SftVirtDerived2) for a separate pre-existing issue with inherited method dispatch through virtual bases. Made-with: Cursor |
||
|
|
c0bbd8b709 |
fix: bind noexcept and ref-qualified methods from unregistered base classes (#5992)
* Strip noexcept from cpp17 function type bindings * Fix a bug and increase test coverage * Does this fix it? * Silence clang-tidy issue * Simplify method adapter with macro and add missing rvalue adaptors + tests * Supress clang-tidy errors * Improve test coverage * Add additional static assert * Try to resolve MSVC C4003 warning * Simplify method adaptor into 2 template instatiations with enable_if_t * Fix ambiguous STL template * Close remaining qualifier consistency gaps for member pointer bindings. A production-code review after #2234 showed that ref-qualified member pointers were still inconsistently handled across def_buffer, vectorize, and overload_cast, so this adds the missing overloads with focused tests for each newly-supported signature. Co-authored-by: Cursor <cursoragent@cursor.com> * Clarify why def_buffer/vectorize omit rvalue-qualified overloads. These comments were added while reviewing the qualifier coverage follow-up, to document that buffer/vectorized calls operate on existing Python-owned instances and should not move-from self. Co-authored-by: Cursor <cursoragent@cursor.com> * Add compile-only overload_cast guard for ref-qualified methods. This was added as a maintenance follow-up to the qualifier-consistency work, so future changes that introduce overload_cast ambiguity or wrong ref/noexcept resolution fail at compile time. Co-authored-by: Cursor <cursoragent@cursor.com> * Refactor overload_cast_impl qualifier overloads with a macro. As part of the qualifier-consistency maintenance follow-up, this reduces duplication in overload_cast_impl while preserving the same ref/noexcept coverage and keeping pedantic-clean macro expansion. Co-authored-by: Cursor <cursoragent@cursor.com> * Expose __cpp_noexcept_function_type to Python tests and use explicit skip guards. This replaces hasattr-based optional assertions with skipif-gated noexcept-only tests so skipped coverage is visible in pytest output while keeping non-noexcept checks always active. Co-authored-by: Cursor <cursoragent@cursor.com> * Add static_assert in method_adaptor to guard that T is a member function pointer. Suggested by @Skylion007 in PR #5992 review comment [T007]. Made-with: Cursor * automatic clang-format change (because of #6002) --------- Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
ce9f83c700 |
Fix crash in def_readwrite for non-smart-holder properties of smart-holder classes (v2) (#6008)
* Add tests that cause crash in def_readwrite
- Occurs with non-smart-holder property of smart-holder class
* Fix crash in def_readwrite for non-smart-holder properties of smart-holder classes
* Use default policy
* Address PR comments
* Add test for cast error path
* style: pre-commit fixes
* Revert "Use default policy"
This reverts commit
|
||
|
|
dd95d53f0a |
Fix memory leak: clear managed dict in pybind11_object_dealloc on Python 3.13+ (#5999)
* fix: clear managed dict in pybind11_object_dealloc on Python 3.13+ On Python 3.14, PyObject_GC_Del (tp_free) no longer implicitly clears the managed dict of objects with Py_TPFLAGS_MANAGED_DICT. Without an explicit PyObject_ClearManagedDict() call before tp_free(), objects stored in the __dict__ of py::dynamic_attr() instances have their refcounts permanently abandoned, causing memory leaks — capsule destructors for numpy arrays (and other objects) never run. Adds a regression test: stores a py::capsule in the __dict__ of a DynamicClass instance and asserts the capsule destructor is called when the instance is deleted. * [tests]: mark test_dynamic_attr_dealloc_frees_dict_contents to be strict=False xfail on PYPY * [docs]: clarify Python version comments in pybind11_object_dealloc Distinguish between when the API is available (3.13+, where PyObject_ClearManagedDict was introduced) and when the leak actually manifests (3.14+, where tp_free stopped implicitly clearing the managed dict). --------- Co-authored-by: Yury Matveev <yury.matveev@desy.de> |
||
|
|
4d51aefc2c |
chore(deps): update pre-commit hooks (#6002)
* chore(deps): update pre-commit hooks updates: - [github.com/pre-commit/mirrors-clang-format: v21.1.8 → v22.1.0](https://github.com/pre-commit/mirrors-clang-format/compare/v21.1.8...v22.1.0) - [github.com/astral-sh/ruff-pre-commit: v0.14.14 → v0.15.4](https://github.com/astral-sh/ruff-pre-commit/compare/v0.14.14...v0.15.4) - [github.com/adhtruong/mirrors-typos: v1.42.3 → v1.44.0](https://github.com/adhtruong/mirrors-typos/compare/v1.42.3...v1.44.0) - [github.com/PyCQA/pylint: v4.0.4 → v4.0.5](https://github.com/PyCQA/pylint/compare/v4.0.4...v4.0.5) - [github.com/python-jsonschema/check-jsonschema: 0.36.1 → 0.37.0](https://github.com/python-jsonschema/check-jsonschema/compare/0.36.1...0.37.0) * style: pre-commit fixes --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
|
cd538ed118 |
Re-enable Android tests in CIBW workflow (#6001)
* Re-enable Android tests in CIBW workflow * Skip subprocess tests on Android * Remove Android workarounds no longer necessary with current cibuildwheel version * Skip more subprocess tests on Android |
||
|
|
1a863647c5 |
[ci skip] fix(tests): make async callback test deterministic (#5986)
Replace the fixed sleep in test_async_callbacks with a bounded wait for all expected callback results, so detached worker scheduling no longer causes sporadic CI failures. Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
ccb7129f54 |
Improve performance of enum_ operators by going back to specific implementation (#5887)
* Improve performance of enum_ operators by going back to specific implementation test_enum needs a patch because ops are now overloaded and this affects their docstrings. * outline call_impl to save on code size This does cause more move constructions, as shown by the needed update to test_copy_move. Up to reviewers whether they want more code size or more moves. * add function_ref.h to PYBIND11_HEADERS. * Update test_copy_move tests with C++17 passing values just so we can see mostly-not-red tests * Remove stray TODO * fix clang-tidy * fix clang-tidy again. add function_ref.h to test_files.py * Add static assertion for function_ref lifetime safety in call_impl Add a static_assert to document and enforce that function_ref is trivially copyable, ensuring safe pass-by-value usage. This also documents the lifetime safety guarantees: function_ref is created from cap->f which lives in the capture object, and is only used synchronously within call_impl without being stored beyond its scope. * Add #undef cleanup for enum operator macros Undefine all enum operator macros after their last use to prevent macro pollution and follow the existing code pattern. This matches the cleanup pattern used for the previous enum operator macros. * Rename PYBIND11_THROW to PYBIND11_ENUM_OP_THROW_TYPE_ERROR Rename the macro to be more specific and avoid potential clashes with public macros. The new name clearly indicates it's scoped to enum operations and describes its purpose (throwing a type error). * Clarify comments in function_ref.h Replace vague comments about 'extensions to <functional>' and 'functions' with a clearer description that this is a header-only class template similar to std::function but with non-owning semantics. This makes it clear that it's template-only and requires no additional library linking. --------- Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com> |
||
|
|
e8e8d6ab22 |
Expand float and complex strict mode to allow ints and ints/float (for PEP 484 compatibility). (#5879)
* init Signed-off-by: Michael Carlstrom <rmc@carlstrom.com> * Add constexpr to is_floating_point check This is known at compile time so it can be constexpr * Allow noconvert float to accept int * Update noconvert documentation * Allow noconvert complex to accept int and float * Add complex strict test * style: pre-commit fixes * Update unit tests so int, becomes double. * style: pre-commit fixes * remove if (constexpr) Signed-off-by: Michael Carlstrom <rmc@carlstrom.com> * fix spelling error * bump order in #else * Switch order in c++11 only section Signed-off-by: Michael Carlstrom <rmc@carlstrom.com> * ci: trigger build * ci: trigger build * Allow casting from float to int The int type caster allows anything that implements __int__ with explicit exception of the python float. I can't see any reason for this. This modifies the int casting behaviour to accept a float. If the argument is marked as noconvert() it will only accept int. * tests for py::float into int * Update complex_cast tests * Add SupportsIndex to int and float * style: pre-commit fixes * fix assert * Update docs to mention other conversions * fix pypy __index__ problems * style: pre-commit fixes * extract out PyLong_AsLong __index__ deprecation Signed-off-by: Michael Carlstrom <rmc@carlstrom.com> * style: pre-commit fixes * Add back env.deprecated_call Signed-off-by: Michael Carlstrom <rmc@carlstrom.com> * remove note Signed-off-by: Michael Carlstrom <rmc@carlstrom.com> * remove untrue comment Signed-off-by: Michael Carlstrom <rmc@carlstrom.com> * fix noconvert_args Signed-off-by: Michael Carlstrom <rmc@carlstrom.com> * resolve error Signed-off-by: Michael Carlstrom <rmc@carlstrom.com> * Add comment Signed-off-by: Michael Carlstrom <rmc@carlstrom.com> * [skip ci] tests: Add overload resolution test for float/int breaking change Add test_overload_resolution_float_int() to explicitly test the breaking change where int arguments now match float overloads when registered first. The existing tests verify conversion behavior (int -> float, int/float -> complex) but do not test overload resolution when both float and int overloads exist. This test fills that gap by: - Testing that float overload registered before int overload matches int(42) - Testing strict mode (noconvert) overload resolution breaking change - Testing complex overload resolution with int/float/complex overloads - Documenting the breaking change explicitly This complements existing tests which verify 'can it convert?' by testing 'which overload wins when multiple can convert?' * Add test to verify that custom __index__ objects (not PyLong) work correctly with complex conversion. These should be consistent across CPython, PyPy, and GraalPy. * Improve comment clarity for PyPy __index__ handling Replace cryptic 'So: PYBIND11_INDEX_CHECK(src.ptr())' comment with clearer explanation of the logic: - Explains that we need to call PyNumber_Index explicitly on PyPy for non-PyLong objects - Clarifies the relationship to the outer condition: when convert is false, we only reach this point if PYBIND11_INDEX_CHECK passed above This makes the code more maintainable and easier to understand during review. * Undo inconsequential change to regex in test_enum.py During merge, HEAD's regex pattern was kept, but master's version is preferred. The order of ` ` and `\|` in the character class is arbitrary. Keep master's order (already fixed in PR #5891; sorry I missed looking back here when working on 5891). * test_methods_and_attributes.py: Restore existing `m.overload_order(1.1)` call and clearly explain the behavior change. * Reject float → int conversion even in convert mode Enabling implicit float → int conversion in convert mode causes silent truncation (e.g., 1.9 → 1). This is dangerous because: 1. It's implicit - users don't expect truncation when calling functions 2. It's silent - no warning or error 3. It can hide bugs - precision loss is hard to detect This change restores the explicit rejection of PyFloat_Check for integer casters, even in convert mode. This is more in line with Python's behavior where int(1.9) must be explicit. Note that the int → float conversion in noconvert mode is preserved, as that's a safe widening conversion. * Revert test changes that sidestepped implicit float→int conversion This reverts all test modifications that were made to accommodate implicit float→int conversion in convert mode. With the production code change that explicitly rejects float→int conversion even in convert mode, these test workarounds are no longer needed. Changes reverted: - test_builtin_casters.py: Restored cant_convert(3.14159) and np.float32 conversion with deprecated_call wrapper - test_custom_type_casters.py: Restored TypeError expectation for m.ints_preferred(4.0) - test_methods_and_attributes.py: Restored TypeError expectation for m.overload_order(1.1) - test_stl.py: Restored float literals (2.0) that were replaced with strings to avoid conversion - test_factory_constructors.py: Restored original constructor calls that were modified to avoid float→int conversion Also removes the unused avoid_PyLong_AsLong_deprecation fixture and related TypeVar imports, as all uses were removed. * Replace env.deprecated_call() with pytest.deprecated_call() The env.deprecated_call() function was removed, but two test cases still reference it. Replace with pytest.deprecated_call(), which is the standard pytest context manager for handling deprecation warnings. Since we already require pytest>=6 (see tests/requirements.txt), the compatibility function is obsolete and pytest.deprecated_call() is available. * Update test expectations for swapped NoisyAlloc overloads PR 5879 swapped the order of NoisyAlloc constructor overloads: - (int i, double) is now placement new (comes first) - (double d, double) is now factory pointer (comes second) This swap is necessary because pybind11 tries overloads in order until one matches. With int → float conversion now allowed: - create_and_destroy(4, 0.5): Without the swap, (double d, double) would match first (since int → double conversion is allowed), bypassing the more specific (int i, double) overload. With the swap, (int i, double) matches first (exact match), which is correct. - create_and_destroy(3.5, 4.5): (int i, double) fails (float → int is rejected), then (double d, double) matches, which is correct. The swap ensures exact int matches are preferred over double matches when an int is provided, which is the expected overload resolution behavior. Update the test expectations to match the new overload resolution order. * Resolve clang-tidy error: /__w/pybind11/pybind11/include/pybind11/cast.h:253:46: error: repeated branch body in conditional chain [bugprone-branch-clone,-warnings-as-errors] 253 | } else if (PyFloat_Check(src.ptr())) { | ^ /__w/pybind11/pybind11/include/pybind11/cast.h:258:10: note: end of the original 258 | } else if (convert || PYBIND11_LONG_CHECK(src.ptr()) || PYBIND11_INDEX_CHECK(src.ptr())) { | ^ /__w/pybind11/pybind11/include/pybind11/cast.h:283:16: note: clone 1 starts here 283 | } else { | ^ * Add test coverage for __index__ and __int__ edge cases: incorrectly returning float These tests ensure that: - Invalid return types (floats) are properly rejected - The fallback from __index__ to __int__ works correctly in convert mode - noconvert mode correctly prevents fallback when __index__ fails * Minor comment-only changes: add PR number, for easy future reference * Ensure we are not leaking a Python error is something is wrong elsewhere (e.g. UB, or bug in Python beta testing). See also: https://github.com/pybind/pybind11/pull/5879#issuecomment-3521099331 * [skip ci] Bump PYBIND11_INTERNALS_VERSION to 12 (for PRs 5879, 5887, 5960) --------- Signed-off-by: Michael Carlstrom <rmc@carlstrom.com> Co-authored-by: gentlegiantJGC <gentlegiantJGC@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com> |
||
|
|
3ae5a173c5 |
Add fallback implementation of PyCriticalSection_BeginMutex for Python 3.13t (#5981)
* Add failback implementation of `PyCriticalSection_BeginMutex` for Python 3.13t * Add comment for Python version * Use `_PyCriticalSection_BeginSlow` * Add forward declaration * Fix forward declaration * Remove always true condition `defined(PY_VERSION_HEX)` * Detect musllinux * Add manylinux test * Use direct mutex locking for Python 3.13t `_PyCriticalSection_BeginSlow` is a private CPython function not exported on Linux. For Python < 3.14.0rc1, use direct `mutex.lock()`/`mutex.unlock()` instead of critical section APIs. * Empty commit to trigger CI * Empty commit to trigger CI * Empty commit to trigger CI * Run apt update before apt install * Remove unnecessary prefix * Add manylinux test with Python 3.13t * Simplify pycritical_section with std::unique_lock fallback for Python < 3.14 * Fix potential deadlock in make_iterator_impl for Python 3.13t Refactor pycritical_section into a unified class with internal version checks instead of using a type alias fallback. Skip locking in make_iterator_impl for Python < 3.14.0rc1 to avoid deadlock during type registration, as pycritical_section cannot release the mutex during Python callbacks without PyCriticalSection_BeginMutex. * Add reference for xfail message |
||
|
|
5f2c678916 |
Add helpers to array that return the size and strides as a std::span (#5974)
* Add helper functions to pybind11::array to return the shape and strides as a std::span. These functions are hidden with macros unless PYBIND11_CPP20 is defined and the <span> include has been found. * style: pre-commit fixes * tests: Add unit tests for shape_span() and strides_span() Add comprehensive unit tests for the new std::span helper functions: - Test 0D, 1D, 2D, and 3D arrays - Verify spans match regular shape()/strides() methods - Test that spans can be used to construct new arrays - Tests are conditionally compiled only when PYBIND11_HAS_SPAN is defined * Use __cpp_lib_span feature test macro instead of __has_include Replace __has_include(<span>) check with __cpp_lib_span feature test macro to resolve ambiguity where some pre-C++20 systems might have a global header called <span> that isn't the C++20 std::span. The check is moved after <version> is included, consistent with how __cpp_lib_char8_t is handled. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix: Use py::ssize_t instead of ssize_t in span tests On Windows/MSVC, ssize_t is not available in the standard namespace without proper includes. Use py::ssize_t (the pybind11 typedef) instead to ensure cross-platform compatibility. Fixes compilation errors on: - Windows/MSVC 2022 (C++20) - GCC 10 (C++20) Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
e7754de037 |
Revert internals destruction and add test for internals recreation (#5972)
* Bump internals version * Prevent internals destruction before all pybind11 types are destroyed * Use Py_XINCREF and Py_XDECREF * Hold GIL before decref * Use weakrefs * Remove unused code * Move code location * Move code location * Move code location * Try add tests * Fix PYTHONPATH * Fix PYTHONPATH * Skip tests for subprocess * Revert to leak internals * Revert to leak internals * Revert "Revert to leak internals" This reverts commit |
||
|
|
95d4af7ff9 |
Fix CI issue: numpy requirement for Python 3.14 on ARM64 Windows (#5977)
* Fix pip install conflicts in tests/requirements.txt
- Fix numpy version conflict for Python 3.14 on ARM64 by excluding
Python 3.14+ from the ARM64-specific numpy>=2.3.0 requirement
- Add scipy requirement for Python 3.13 on Windows
Co-authored-by: Cursor <cursoragent@cursor.com>
* Revert "Fix pip install conflicts in tests/requirements.txt"
This reverts commit
|
||
|
|
4f81a12507 |
Fix deadlock in test with free threading (#5973)
Importing "widget_module" re-enables the GIL. In current versions of CPython, this requires pausing all threads attached to all interpreters. The spinning on sync/num without a py::gil_scoped_release causes occasional deadlocks. |
||
|
|
53ccc32f84 |
fix: cross-compilation updates (#5829)
* Android updates * Revert removal of Interpreter |
||
|
|
e44aae2268 |
chore: bump CMake max policy to 4.2 (#5944)
* chore: bupm CMake to 4.1 Signed-off-by: Henry Schreiner <henryfs@princeton.edu> * ci: use newest version of CMake in a few places Signed-off-by: Henry Schreiner <henryfs@princeton.edu> * chore: bupm CMake to 4.2 Signed-off-by: Henry Schreiner <henryfs@princeton.edu> --------- Signed-off-by: Henry Schreiner <henryfs@princeton.edu> |
||
|
|
10f8708979 |
Change function calls to use vectorcall (#5948)
* Make argument_vector re-usable for other types. * Attempt to collect args into array for vectorcall * Revert "Attempt to collect args into array for vectorcall" This reverts commit |
||
|
|
c761608a22 |
chore(deps): update pre-commit hooks (#5951)
* chore(deps): update pre-commit hooks updates: - [github.com/pre-commit/mirrors-clang-format: v21.1.6 → v21.1.8](https://github.com/pre-commit/mirrors-clang-format/compare/v21.1.6...v21.1.8) - [github.com/astral-sh/ruff-pre-commit: v0.14.7 → v0.14.10](https://github.com/astral-sh/ruff-pre-commit/compare/v0.14.7...v0.14.10) - [github.com/pre-commit/mirrors-mypy: v1.19.0 → v1.19.1](https://github.com/pre-commit/mirrors-mypy/compare/v1.19.0...v1.19.1) - [github.com/crate-ci/typos: v1.40.0 → v1](https://github.com/crate-ci/typos/compare/v1.40.0...v1) - [github.com/python-jsonschema/check-jsonschema: 0.35.0 → 0.36.0](https://github.com/python-jsonschema/check-jsonschema/compare/0.35.0...0.36.0) * fix(tests): allow numpy>=2.3.0 for ARM64 to fix Windows ARM CI MSYS2 upgraded from numpy 2.3.x to 2.4.0, which no longer satisfies the ~=2.3.0 constraint. Relaxing to >=2.3.0 allows the MSYS2-provided numpy 2.4.0 to be used. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com> |
||
|
|
b93c0f7ed8 |
Fix ambiguous str(handle) constructor for object-derived types (#5949)
* Fix ambiguous `str(handle)` constructor for object-derived types Templatize `str(handle h)` with SFINAE to exclude types derived from `object`, resolving ambiguity with `str(const object&)` when calling `py::str()` on types like `kwargs`, `dict`, etc. The template now only accepts `handle` or `PyObject*`, while all `object`-derived types use the `str(const object&)` overload. * fix(tests): CIBW test fixes from b-pass→vectorcall branch - Install multiple-interpreter test modules into wheel (CMakeLists.txt) The mod_per_interpreter_gil, mod_shared_interpreter_gil, and mod_per_interpreter_gil_with_singleton modules were being built but not installed into the wheel when using scikit-build-core. - Pin numpy 2.4.0 for Python 3.14 CI tests (requirements.txt) NumPy 2.4.0 is the first version with official Python 3.14 wheels. - Add IOS platform constant to tests/env.py - Skip subinterpreter tests on iOS (test_multiple_interpreters.py) Subinterpreters are not supported in the iOS simulator environment. - Enable pytest timeout of 120s for CIBW tests (pyproject.toml) Provides a safety net to catch hanging tests before CI job timeout. - Disable pytest-timeout for Pyodide (no signal.setitimer) Pyodide runs in WebAssembly without POSIX signals. - Add -v flag for verbose pytest output in CIBW tests |
||
|
|
fee2527dfa |
Fix concurrency consistency for internals_pp_manager under multiple-interpreters (#5947)
* Add per-interpreter storage for `gil_safe_call_once_and_store`
* Disable thread local cache for `internals_pp_manager`
* Disable thread local cache for `internals_pp_manager` for multi-interpreter only
* Use anonymous namespace to separate these type_ids from other tests with the same class names.
* style: pre-commit fixes
* Revert internals_pp_manager changes
* This is the crux of fix for the subinterpreter_before_main failure.
The pre_init needs to check if it is in a subinterpreter or not. But in 3.13+ this static initializer runs in the main interpreter. So we need to check this later, during the exec phase.
* Continue to do the ensure in both places, there might be a reason it was where it was...
Should not hurt anything to do it extra times here.
* Change get_num_interpreters_seen to a boolean flag instead.
The count was not used, it was just checked for > 1, we now accomplish this by setting the flag.
* Spelling typo
* Work around older python versions, only need this check for newish versions
* Add more comments for test case
* Add more comments for test case
* Stop traceback propagation
* Re-enable subinterpreter support on ubuntu 3.14 builds
Was disabled in
|
||
|
|
0057e4945d |
Add per-interpreter storage for gil_safe_call_once_and_store (#5933)
* Add new argument to `gil_safe_call_once_and_store::call_once_and_store_result` * Add per-interpreter storage for `gil_safe_call_once_and_store` * Make `~gil_safe_call_once_and_store` a no-op * Fix C++11 compatibility * Improve thread-safety and add default finalizer * Try fix thread-safety * Try fix thread-safety * Add a warning comment * Simplify `PYBIND11_INTERNALS_VERSION >= 12` * Try fix thread-safety * Try fix thread-safety * Revert get_pp() * Update comments * Move call-once storage out of internals * Revert internal version bump * Cleanup outdated comments * Move atomic_bool alias into pybind11::detail namespace The `using atomic_bool = ...` declaration was at global scope, polluting the global namespace. Move it into pybind11::detail to avoid potential conflicts with user code. * Add explicit #include <unordered_map> for subinterpreter support The subinterpreter branch uses std::unordered_map but relied on transitive includes. Add an explicit include for robustness. * Remove extraneous semicolon after destructor definition Style fix: remove trailing semicolon after ~call_once_storage() destructor body. * Add comment explaining unused finalize parameter Clarify why the finalize callback parameter is intentionally ignored when subinterpreter support is disabled: the storage is process-global and leaked to avoid destructor calls after interpreter finalization. * Add comment explaining error_scope usage Clarify why error_scope is used: to preserve any existing Python error state that might be cleared or modified by dict_getitemstringref. * Improve exception safety in get_or_create_call_once_storage_map() Use std::unique_ptr to hold the newly allocated storage map until the capsule is successfully created. This prevents a memory leak if capsule creation throws an exception. * Add timeout-minutes: 3 to cpptest workflow steps Add a 3-minute timeout to all C++ test (cpptest) steps across all platforms to detect hangs early. This uses GitHub Actions' built-in timeout-minutes property which works on Linux, macOS, and Windows. * Add progress reporter for test_with_catch Catch2 runner Add a custom Catch2 streaming reporter that prints one line per test case as it starts and ends, with immediate flushing to keep CI logs current. This makes it easy to see where the embedded/interpreter tests are spending time and to pinpoint which test case is stuck when builds hang (e.g., free-threading issues). The reporter: - Prints "[ RUN ]" when each test starts - Prints "[ OK ]" or "[ FAILED ]" when each test ends - Prints the Python version once at the start via Py_GetVersion() - Uses StreamingReporterBase for immediate output (not buffered) - Is set as the default reporter via CATCH_CONFIG_DEFAULT_REPORTER This approach gives visibility into all tests without changing their behavior, turning otherwise opaque 90-minute CI timeouts into locatable issues in the Catch output. * clang-format auto-fix (overlooked before) * Disable "Move Subinterpreter" test on free-threaded Python 3.14+ This test hangs in Py_EndInterpreter() when the subinterpreter is destroyed from a different thread than it was created on. The hang was observed: - Intermittently on macOS with Python 3.14.0t - Predictably on macOS, Ubuntu, and Windows with Python 3.14.1t and 3.14.2t Root cause analysis points to an interaction between pybind11's subinterpreter creation code and CPython's free-threaded runtime, specifically around PyThreadState_Swap() after PyThreadState_DeleteCurrent(). See detailed analysis: https://github.com/pybind/pybind11/pull/5933 * style: pre-commit fixes * Add test for gil_safe_call_once_and_store per-interpreter isolation This test verifies that gil_safe_call_once_and_store provides separate storage for each interpreter when subinterpreter support is enabled. The test caches the interpreter ID in the main interpreter, then creates a subinterpreter and verifies it gets its own cached value (not the main interpreter's). Without per-interpreter storage, the subinterpreter would incorrectly see the main interpreter's cached object. * Add STARTING/DONE timestamps to test_with_catch output Print UTC timestamps at the beginning and end of the test run to make it immediately clear when tests started and whether they ran to completion. The DONE message includes the Catch session result value. Example output: [ STARTING ] 2025-12-21 03:23:20.497Z [ PYTHON ] 3.14.2 ... [ RUN ] Threads [ OK ] Threads [ DONE ] 2025-12-21 03:23:20.512Z (result 0) * Disable stdout buffering in test_with_catch Ensure test output appears immediately in CI logs by disabling stdout buffering. Without this, output may be lost if the process is killed by a timeout, making it difficult to diagnose which test was hanging. * EXPERIMENT: Re-enable hanging test to verify CI log buffering fix This is a temporary commit to verify that the unbuffered stdout fix makes the hanging test visible in CI logs. REVERT THIS COMMIT after confirming the output appears. * Revert "Disable stdout buffering in test_with_catch" This reverts commit |
||
|
|
799f591ec3 |
Re-enable Move Subinterpreter test for free-threaded Python 3.14 (#5940)
* Remove skip for Move Subinterpreter test on free-threaded Python 3.14+ * Fix deadlock by detaching from the main interpreter before joining the thread. * style: pre-commit fixes --------- Co-authored-by: b-pass <b-pass@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
|
78381e5e28 |
Improve C++ test infrastructure: progress reporter, timeouts, and skip hanging Move Subinterpreter test (#5942)
* Improve C++ test infrastructure and disable hanging test This commit improves the C++ test infrastructure to ensure test output is visible in CI logs, and disables a test that hangs on free-threaded Python 3.14+. Changes: ## CI/test infrastructure improvements - .github/workflows: Added `timeout-minutes: 3` to all C++ test steps to prevent indefinite hangs. - tests/**/CMakeLists.txt: Added `USES_TERMINAL` to C++ test targets (cpptest, test_cross_module_rtti, test_pure_cpp) to ensure output is shown immediately rather than buffered and possibly lost on crash/timeout. - tests/test_with_catch/catch.cpp: Added a custom Catch2 progress reporter with timestamps, Python version info, and a SIGTERM handler to make test execution and failures clearly visible in CI logs. ## Disabled hanging test - The "Move Subinterpreter" test is disabled on free-threaded Python 3.14+ due to a hang in Py_EndInterpreter() when the subinterpreter is destroyed from a different thread than it was created on. Work on fixing the underlying issue will continue under PR #5940. Context: We were in the dark for months (since we started testing with Python 3.14t) because CI logs gave no clue about the root cause of hangs. This led to ignoring intermittent hangs (mostly on macOS). Our hand was forced only with the Python 3.14.1 release, when hangs became predictable on all platforms. For the full development history of these changes, see PR #5933. * Add test summary to progress reporter Print the total number of test cases and assertions at the end of the test run, making it easy to spot if tests are disabled or added. Example output: [ PASSED ] 20 test cases, 1589 assertions. * Add PYBIND11_CATCH2_SKIP_IF macro to skip tests at runtime Catch2 v2 doesn't have native skip support (v3 does with SKIP()). This macro allows tests to be skipped with a visible message while still appearing in the test list. Use this for the Move Subinterpreter test on free-threaded Python 3.14+ so it shows as skipped rather than being conditionally compiled out. Example output: [ RUN ] Move Subinterpreter [ SKIPPED ] Skipped on free-threaded Python 3.14+ (see PR #5940) [ OK ] Move Subinterpreter * Fix clang-tidy bugprone-macro-parentheses warning in PYBIND11_CATCH2_SKIP_IF |
||
|
|
1006933415 |
chore: also use typos (#5931)
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com> |