fix-shared-libpython-link
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Henry SchreinerandRalf W. Grosse-Kunstleve
|
63d627c574 |
feat!: drop support for Python 3.8, MSVC 2017 (#6110)
* feat!: drop support for Python 3.8 The minimum supported version is now Python 3.9. pybind11 v3.0 was the last release that supports Python 3.8. The deprecation note said that support goes away in 3.1. Remove the code paths that this makes dead: - the `_PyObject_Vectorcall` fallback in `cast.h` - the `frame->f_code` and `frame->f_back` fallbacks in `pytypes.h` - the `PyFrame_FastToLocals` path in `get_type_override` - the conditional `Py_VISIT(Py_TYPE(self))` in `tp_traverse` - `PYBIND11_PYCONFIG_SUPPORT_PY_VERSION_HEX`, the pre-PyConfig interpreter init, and the `widen_chars` helpers in `embed.h` Assisted-by: ClaudeCode:claude-opus-5 * ci(appveyor): use Python 3.9 The AppVeyor job set `PYTHON: 38`, which makes the path `C:\Python38`. The image gives Python 3.9.13 as `C:\Python39`. Assisted-by: ClaudeCode:claude-opus-5 * feat!: require MSVC 2019 or newer Python 3.9 is the new minimum, so MSVC 2017 is no longer needed. Raise the compile-time floor to _MSC_VER 1920 and remove the workarounds that only applied below it: std::launder, fold expressions, weak_from_this, aligned new/delete, the C4100 warning helper, and the func_handle syntax error. AppVeyor now builds with Visual Studio 2019. Assisted-by: ClaudeCode:claude-opus-5 * fix(appveyor): build against the Python that has the test packages CMake 4 has no FindPythonLibs, so pybind11 uses FindPython. FindPython reads the registry before PATH and selected `C:\Python314-x64`, but the test packages go into the Python on PATH. Pass `Python_ROOT_DIR` to name the correct one. Also set `CMAKE_ARCH`. It was never set, so the architecture came from the generator. Visual Studio 2017 defaults to Win32, but Visual Studio 2019 defaults to x64, which made this x86 job build 64-bit code. Assisted-by: ClaudeCode:claude-opus-5 * fix(appveyor): give the linker Python's libs directory The build compiled but failed to link with LNK1104 on a bare `python39.lib`. That name comes from the `#pragma comment(lib, ...)` in pyconfig.h, so the linker needs the directory. CMake 4 has no FindPythonLibs, and FindPython does not add it for this Debug x86 build. Put it on LIB instead. The directory listing is temporary, to confirm the library is present. Assisted-by: ClaudeCode:claude-opus-5 * fix(appveyor): link the release Python library in Debug The image ships python39_d.lib next to python39.lib, so FindPython picks the debug import library for a Debug build. pybind11 undefines _DEBUG around Python.h, so pyconfig.h asks for python39.lib instead and the link failed with LNK1104. Name the release library for the debug slot. Setting LIB does not work, because MSBuild replaces it from the toolset, and it would link both import libraries. Assisted-by: ClaudeCode:claude-opus-5 * chore(appveyor): print link settings to debug LNK1104 Revert the two attempted fixes. Neither changed the failure: setting LIB does not survive MSBuild, and naming the release library for Python_LIBRARY_DEBUG had no effect. Print the Python cache entries and the link settings of a generated project file instead, to see what the linker really gets. Temporary. Assisted-by: ClaudeCode:claude-opus-5 * fix(appveyor): take the release Python library in Debug The generated project file linked C:\Python39\libs\python39_d.lib in the Debug configuration, because the image ships debug binaries next to the release ones. pybind11 undefines _DEBUG around Python.h, so pyconfig.h asks for python39.lib in a #pragma comment(lib), which nothing on the link line satisfies and no library directory holds. Map Debug to the release artifacts. Assisted-by: ClaudeCode:claude-opus-5 * chore(appveyor): drop the temporary link diagnostic Assisted-by: ClaudeCode:claude-opus-5 * fix(tests): guard the unraisable warning filter for pytest < 6 The distro pytest in the Clang and GCC Docker jobs has no PytestUnraisableExceptionWarning, so an unconditional filterwarnings mark makes pytest fail with an INTERNALERROR after the tests pass. Assisted-by: ClaudeCode:claude-opus-5 Claude-Session: https://claude.ai/code/session_01Aimf6HuSz1vLRwBnbxmCTc * docs: address review items on version hints, embed docs, and a PyPy xfail Extend Python_ADDITIONAL_VERSIONS through 3.15, describe the PyConfig behavior of initialize_interpreter, and drop the stale Python 3.8 wording from the PyPy xfail reason. Assisted-by: ClaudeCode:claude-opus-5 * Update README.rst Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com> --------- Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com> |
||
|
Henry Schreinerandpre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
652c69437b |
fix: add py::mod_gil_used() spelling, support pedantic tests (#5797)
* tests: add linking warnings/error Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * tests: check pedantic C++14 * fix: some pedantic warnings Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * feat: add py::mod_gil_used() Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * fix: use not_supported() Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * style: pre-commit fixes * Update CMakeLists.txt * fix: remove the true/false parameter from mod_gil_not_used Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * style: pre-commit fixes * Update tests/CMakeLists.txt * fix: deprecate mod_gil_not_used(bool) instead of removing it Keep the bool constructor public and mark it deprecated, so existing code that spells mod_gil_not_used(false) still compiles. mod_gil_used() now sets the flag through its friendship, which keeps the deprecated constructor the only bool overload and stops pybind11 warning against itself. Also use the py:: alias in exo_planet_pybind11.cpp to match its sibling home_planet_very_lonely_traveler.cpp. Assisted-by: ClaudeCode:claude-opus-5 * fix: add trailing comma to macros picked up in rebase test_smart_ptr.cpp and standalone_enum_module.cpp came from master after this branch was written, so they still invoked variadic macros with no variadic argument. That is a pedantic error below C++20. Assisted-by: ClaudeCode:claude-opus-5 --------- Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Henry SchreinerandRalf W. Grosse-Kunstleve
|
5540f96802 |
fix: only add string_view life support for transient sources (#6096)
* revert: "revert: add life support to handles cast to string_view (#6092)"
This re-applies #6092 (reverting #6097) so the follow-up fixes in this PR can build on it.
Assisted-by: ClaudeCode:claude-opus-4.8
* fix: don't throw from string_view life support outside a bound function
PR #6092 added loader_life_support::add_patient(src) to keep the source
object alive when loading a string view, fixing a real use-after-free when
a container of views is built from a non-sequence iterable (e.g. a
generator): list_caster materializes a temporary tuple that owns the
strings and destroys it when load() returns, before the bound function
body runs.
add_patient throws when there is no life support frame, so casting to a
view outside a bound function (e.g. a manual py::cast<std::string_view>)
now raises instead of relying on the caller-owned source, a regression
from #6092.
For these view-into-src cases registration is best effort: inside a bound
function it keeps src alive (fixing the UAF), and outside one the caller
owns src's lifetime as before. Add try_add_patient(), which returns false
instead of throwing when there is no frame, and use it at the three view
load sites. add_patient() keeps its strict contract for value-creating
conversions.
Assisted-by: ClaudeCode:claude-opus-4.8
* fix: only add string_view life support for transient sources
Refine the previous commit. Best-effort registration (try_add_patient)
silently produces a dangling view when a container of views is built from
a generator outside a bound function: there the materialized temporary is
released before the view is used, and with no frame nothing keeps it
alive. Such a cast cannot be made safe, so it should fail loudly, while a
view into a durable, caller-owned object needs no life support at all.
The view caster cannot tell a durable source from a pybind11-managed
transient one; that provenance lives in the container caster. Introduce an
ambient transient_source_guard that the list, set, map, and array casters
set around their generator/materialized paths, and have the string caster
keep the source alive only when loading from a transient source (via the
throwing add_patient, so try_add_patient is no longer needed). This means:
- views into durable sources (direct arguments, sequences, manual casts)
add no life support and no longer throw outside a bound function, and
- a generator used outside a frame throws, rather than silently dangling.
The guard restores (rather than clears) the previous value, so a durable
container nested in a transient one is correctly treated as transient.
Verified with AddressSanitizer: the in-frame generator case is clean, the
out-of-frame durable cases succeed, and the out-of-frame generator case
throws.
Assisted-by: ClaudeCode:claude-opus-4.8
* Revert "fix: only add string_view life support for transient sources"
This reverts commit
|
||
|
Ralf W. Grosse-Kunstleve
|
524d72b36d |
fix: strdup "self" arg in def_property_static, partially revert #6010 (gh-5976) (#6015)
* fix: strdup args added after initialize_generic in def_property_static (gh-5976) `def_property_static` calls `process_attributes::init` on already-initialized function records (after `initialize_generic`'s strdup loop has run). Args added at this stage (e.g. "self" via `append_self_arg_if_needed`) remain as string literals, so `destruct()` would call `free()` on them. Fix by strdup'ing name/descr of any args appended by the late `process_attributes::init` call. Root cause introduced by gh-5486. Made-with: Cursor * Partially revert gh-6010: remove py_is_finalizing() workarounds Now that the root cause (free of string literals in def_property_static, gh-5976) is fixed in the previous commit, the py_is_finalizing() guards introduced in gh-6010 are no longer needed: - tp_dealloc_impl: remove early return during finalization (was leaking all function records instead of properly destroying them) - destruct(): remove guard around arg.value.dec_ref() - common.h: remove py_is_finalizing() helper (no remaining callers) The genuine fix from gh-6010 (PyObject_Free + Py_DECREF ordering in tp_dealloc_impl) is retained. Made-with: Cursor * test: add embedding test for py::enum_ across interpreter restart (gh-5976) py::enum_ is the primary trigger for gh-5976 because its constructor creates properties via def_property_static / def_property_readonly_static, which call process_attributes::init on already-initialized function records. Yet none of the existing embedding tests used py::enum_ at all. Add an PYBIND11_EMBEDDED_MODULE with py::enum_ and a test case that imports it, finalize/reinitializes the interpreter, and re-imports it. This exercises the def_property_static code path that was fixed in the preceding commit. Note: on Python 3.14.2 (and likely 3.12+), tp_dealloc_impl is not called during Py_FinalizeEx for function record PyObjects — they simply leak because types are effectively immortalized. As a result, this test cannot trigger the original free()-on-string-literal crash on this Python version. However, it remains valuable as a regression guard: on Python builds where finalization does clean up function records (or if CPython changes this behavior), the test would catch the crash. It also verifies that py::enum_ survives interpreter restart correctly, which was previously untested. Made-with: Cursor * test: skip enum restart test on Python 3.12 (pre-existing crash) Made-with: Cursor * Add test_standalone_enum_module.py, standalone_enum_module.cpp * Make standalone_enum_module.cpp more similar to #5976 reproducer. Also fix clang-tidy error. * This crashes when testing locally: ( cd /wrk/forked/pybind11/tests && PYTHONPATH=/wrk/bld/pybind11_gcc_v3.14.2_df793163d58_default/lib /wrk/bld/pybind11_gcc_v3.14.2_df793163d58_default/TestVenv/bin/python3 -m pytest test_standalone_enum_module.py ) ============================= test session starts ============================== platform linux -- Python 3.14.2, pytest-9.0.2, pluggy-1.6.0 installed packages of interest: build==1.4.2 numpy==2.4.3 scipy==1.17.1 C++ Info: 13.3.0 C++20 __pybind11_internals_v12_system_libstdcpp_gxx_abi_1xxx_use_cxx11_abi_1__ PYBIND11_SIMPLE_GIL_MANAGEMENT=False rootdir: /wrk/forked/pybind11/tests configfile: pytest.ini plugins: timeout-2.4.0, xdist-3.8.0 collected 1 item test_standalone_enum_module.py F [100%] =================================== FAILURES =================================== ________________________ test_enum_import_exit_no_crash ________________________ def test_enum_import_exit_no_crash(): # Modeled after reproducer under issue #5976 > env.check_script_success_in_subprocess( f""" import sys sys.path.insert(0, {os.path.dirname(env.__file__)!r}) import standalone_enum_module as m assert m.SomeEnum.__class__.__name__ == "pybind11_type" """, rerun=1, ) test_standalone_enum_module.py:10: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ code = 'import sys\nsys.path.insert(0, \'/wrk/forked/pybind11/tests\')\nimport standalone_enum_module as m\nassert m.SomeEnum.__class__.__name__ == "pybind11_type"' def check_script_success_in_subprocess(code: str, *, rerun: int = 8) -> None: """Runs the given code in a subprocess.""" import os import subprocess import sys import textwrap if ANDROID or IOS or sys.platform.startswith("emscripten"): pytest.skip("Requires subprocess support") code = textwrap.dedent(code).strip() try: for _ in range(rerun): # run flakily failing test multiple times subprocess.check_output( [sys.executable, "-c", code], cwd=os.getcwd(), stderr=subprocess.STDOUT, text=True, ) except subprocess.CalledProcessError as ex: > raise RuntimeError( f"Subprocess failed with exit code {ex.returncode}.\n\n" f"Code:\n" f"```python\n" f"{code}\n" f"```\n\n" f"Output:\n" f"{ex.output}" ) from None E RuntimeError: Subprocess failed with exit code -6. E E Code: E ```python E import sys E sys.path.insert(0, '/wrk/forked/pybind11/tests') E import standalone_enum_module as m E assert m.SomeEnum.__class__.__name__ == "pybind11_type" E ``` E E Output: E munmap_chunk(): invalid pointer _ = 0 code = 'import sys\nsys.path.insert(0, \'/wrk/forked/pybind11/tests\')\nimport standalone_enum_module as m\nassert m.SomeEnum.__class__.__name__ == "pybind11_type"' os = <module 'os' (frozen)> rerun = 1 subprocess = <module 'subprocess' from '/wrk/cpython_installs/v3.14.2_df793163d58_default/lib/python3.14/subprocess.py'> sys = <module 'sys' (built-in)> textwrap = <module 'textwrap' from '/wrk/cpython_installs/v3.14.2_df793163d58_default/lib/python3.14/textwrap.py'> env.py:68: RuntimeError =========================== short test summary info ============================ FAILED test_standalone_enum_module.py::test_enum_import_exit_no_crash - Runti... ============================== 1 failed in 0.23s =============================== ERROR: completed_process.returncode=1 * Add "Added in PR #6015" comments, for easy reference back to this PR * test: use PYBIND11_CATCH2_SKIP_IF for Python 3.12 enum restart skip Replace #if/#else/#endif preprocessor guard with runtime PYBIND11_CATCH2_SKIP_IF so the test is always compiled and shows [ SKIPPED ] in output on Python 3.12. Made-with: Cursor * fix: suppress MSVC C4127 in PYBIND11_CATCH2_SKIP_IF macro The constant condition in PYBIND11_CATCH2_SKIP_IF triggers MSVC warning C4127 (conditional expression is constant), which becomes a build error under /WX. Made-with: Cursor |
||
|
Scott Wolchok
|
30748f863f |
Avoid heap allocation for function calls with a small number of args (#5824)
* Avoid heap allocation for function calls with a small number of arguments
We don't have access to llvm::SmallVector or similar, but given the
limited subset of the `std::vector` API that
`function_call::args{,_convert}` need and the "reserve-then-fill"
usage pattern, it is relatively straightforward to implement custom
containers that get the job done.
Seems to improves time to call the collatz function in
pybind/pybind11_benchmark significantly; numbers are a little noisy
but there's a clear improvement from "about 60 ns per call" to "about
45 ns per call" on my machine (M4 Max Mac), as measured with
`timeit.repeat('collatz(4)', 'from pybind11_benchmark import
collatz')`.
* clang-tidy
* more clang-tidy
* clang-tidy NOLINTBEGIN/END instead of NOLINTNEXTLINE
* forgot to increase inline size after removing std::variant
* constexpr arg_vector_small_size, use move instead of swap to hopefully clarify second_pass_convert
* rename test_embed to test_low_level
* rename test_low_level to test_with_catch
* Be careful to NOINLINE slow paths
* rename array/vector members to iarray/hvector. Move comment per request. Add static_asserts for our untagged union implementation per request.
* drop is_standard_layout assertions; see https://github.com/pybind/pybind11/pull/5824#issuecomment-3308616072
|