This adds support for a `py::args_kw_only()` annotation that can be
specified between `py::arg` annotations to indicate that any following
arguments are keyword-only. This allows you to write:
m.def("f", [](int a, int b) { /* ... */ },
py::arg("a"), py::args_kw_only(), py::arg("b"));
and have it work like Python 3's:
def f(a, *, b):
# ...
with respect to how `a` and `b` arguments are accepted (that is, `a` can
be positional or by keyword; `b` can only be specified by keyword).
Currently user specializations of the form
template <typename itype> struct polymorphic_type_hook<itype, std::enable_if_t<...>> { ... };
will fail if itype is also polymorphic, because the existing specialization will also
be enabled, which leads to 2 equally viable candidates. With this change, user provided
specializations have higher priority than the built in specialization for polymorphic types.
This commit introduces the use of C++17-style fold expressions when
casting tuples & the argument lists of functions.
This change can improve performance of the resulting bindings: because
fold expressions have short-circuiting semantics, pybind11 e.g. won't
try to cast the second argument of a function if the first one failed.
This is particularly effective when working with functions that have
many overloads with long argument lists.
* test pair-copyability on C++17 upwards
The stdlib falsely detects containers like M=std::map<T, U>
as copyable, even when one of T and U is not copyable.
Therefore we cannot rely on the stdlib dismissing std::pair<T, M>
by itself, even on C++17.
* fix is_copy_assignable
bind_map used std::is_copy_assignable which suffers from the same problems
as std::is_copy_constructible, therefore the same fix has been applied.
* created tests for copyability
Don't assume that just because the language version is C++17 that the
standard library offers all C++17 features, too. When using clang-6.0
and --std=c++17 on Ubuntu 18.04 with libstdc++, __cpp_sized_deallocation
is false.
Pybind11 provides a cast operator between opaque void* pointers on the
C++ side and capsules on the Python side. The py::cast<void *>
expression was not aware of this possibility and incorrectly triggered a
compile-time assertion ("Unable to cast type to reference: value is
local to type caster") that is now fixed.
* Support C++17 aligned new statement
This patch makes pybind11 aware of nonstandard alignment requirements in
bound types and passes on this information to C++17 aligned 'new'
operator. Pre-C++17, the behavior is unchanged.
* Fix potential crash when calling an overloaded function
The crash would occur if:
- dispatcher() uses two-pass logic (because the target is overloaded and some arguments support conversions)
- the first pass (with conversions disabled) doesn't find any matching overload
- the second pass does find a matching overload, but its return value can't be converted to Python
The code for formatting the error message assumed `it` still pointed to the selected overload,
but during the second-pass loop `it` was nullptr. Fix by setting `it` correctly if a second-pass
call returns a nullptr `handle`. Add a new test that segfaults without this fix.
* Make overload iteration const-correct so we don't have to iterate again on second-pass error
* Change test_error_after_conversions dependencies to local classes/variables
* stl.h: propagate return value policies to type-specific casters
Return value policies for containers like those handled in in 'stl.h'
are currently broken.
The problem is that detail::return_value_policy_override<C>::policy()
always returns 'move' when given a non-pointer/reference type, e.g.
'std::vector<...>'.
This is sensible behavior for custom types that are exposed via
'py::class_<>', but it does not make sense for types that are handled by
other type casters (STL containers, Eigen matrices, etc.).
This commit changes the behavior so that
detail::return_value_policy_override only becomes active when the type
caster derives from type_caster_generic.
Furthermore, the override logic is called recursively in STL type
casters to enable key/value-specific behavior.
The current code requires implicitly that integral types are cast-able to floating point. In case of strongly-typed integrals (e.g. as explained at http://www.ilikebigbits.com/blog/2014/5/6/type-safe-identifiers-in-c) this is not always the case.
This commit uses SFINAE to move the numeric conversions into separate `cast()` implementations to avoid the issue.
* Add basic support for tag-based static polymorphism
Sometimes it is possible to look at a C++ object and know what its dynamic type is,
even if it doesn't use C++ polymorphism, because instances of the object and its
subclasses conform to some other mechanism for being self-describing; for example,
perhaps there's an enumerated "tag" or "kind" member in the base class that's always
set to an indication of the correct type. This might be done for performance reasons,
or to permit most-derived types to be trivially copyable. One of the most widely-known
examples is in LLVM: https://llvm.org/docs/HowToSetUpLLVMStyleRTTI.html
This PR permits pybind11 to be informed of such conventions via a new specializable
detail::polymorphic_type_hook<> template, which generalizes the previous logic for
determining the runtime type of an object based on C++ RTTI. Implementors provide
a way to map from a base class object to a const std::type_info* for the dynamic
type; pybind11 then uses this to ensure that casting a Base* to Python creates a
Python object that knows it's wrapping the appropriate sort of Derived.
There are a number of restrictions with this tag-based static polymorphism support
compared to pybind11's existing support for built-in C++ polymorphism:
- there is no support for this-pointer adjustment, so only single inheritance is permitted
- there is no way to make C++ code call new Python-provided subclasses
- when binding C++ classes that redefine a method in a subclass, the .def() must be
repeated in the binding for Python to know about the update
But these are not much of an issue in practice in many cases, the impact on the
complexity of pybind11's innards is minimal and localized, and the support for
automatic downcasting improves usability a great deal.
- PYBIND11_MAKE_OPAQUE now takes ... rather than a single argument and
expands it with __VA_ARGS__; this lets templated, comma-containing
types get through correctly.
- Adds a new macro PYBIND11_TYPE() that lets you pass the type into a
macro as a single argument, such as:
PYBIND11_OVERLOAD(PYBIND11_TYPE(R<1,2>), PYBIND11_TYPE(C<3,4>), func)
Unfortunately this only works for one macro call: to forward the
argument on to the next macro call (without the processor breaking it
up again) requires also adding the PYBIND11_TYPE(...) to type macro
arguments in the PYBIND11_OVERLOAD_... macro chain.
- updated the documentation with these two changes, and use them at a couple
places in the test suite to test that they work.
The `py::args` or `py::kwargs` arguments aren't properly referenced
when added to the function_call arguments list: their reference counts
drop to zero if the first (non-converting) function call fails, which
means they might be cleaned up before the second pass call runs.
This commit adds a couple of extra `object`s to the `function_call`
where we can stash a reference to them when needed to tie their
lifetime to the function_call object's lifetime.
(Credit to YannickJadoul for catching and proposing a fix in #1223).
Pybind11's default conversion to int always produces a long on Python 2 (`int`s and `long`s were unified in Python 3). This patch fixes `int` handling to match Python 2 on Python 2; for short types (`size_t` or smaller), the number will be returned as an `int` if possible, otherwise `long`. Requires Python 2.5+.
This is needed for things like `sys.exit`, which refuse to accept a `long`.
Building with the (VS2017) /permissive- flag puts the compiler into
stricter standards-compliant mode. It shouldn't cause the compiler to
work differently--it just disallows some non-conforming code--so should
be perfectly fine for the test suite under all VS2017 builds.
This commit also fixes one failure under non-permissive mode.
This changes the caster to return a reference to a (new) local `CharT`
type caster member so that binding lvalue-reference char arguments
works (currently it results in a compilation failure).
Fixes#1116
`type_descr` is now applied only to the final signature so that it only
marks the argument types, but not nested types (e.g. for tuples) or
return types.
The current C++14 constexpr signatures don't require relaxed constexpr,
but only `auto` return type deduction. To get around this in C++11,
the type caster's `name()` static member functions are turned into
`static constexpr auto` variables.
The current PYBIND11_INTERNALS_ID depends on the version of the library
in order to isolate binary incompatible internals capsules. However,
this does not preclude conflicts between modules built from different
(binary incompatible) commits with the same version number.
For example, if one module was built with an early v2.2.dev and
submitted to PyPI, it could not be loaded alongside a v2.2.x release
module -- it would segfault because of incompatible internals with
the same ID.
This PR changes the ID to depend on PYBIND11_INTERNALS_VERSION which is
independent of the main library version. It's an integer which should be
incremented whenever a binary incompatible change is made to internals.
PYBIND11_INTERNALS_KIND is also introduced for a similar reason.
The same versioning scheme is also applied to `type_info` and the
`module_local` type attribute.
The main point of `py::module_local` is to make the C++ -> Python cast
unique so that returning/casting a C++ instance is well-defined.
Unfortunately it also makes loading unique, but this isn't particularly
desirable: when an instance contains `Type` instance there's no reason
it shouldn't be possible to pass that instance to a bound function
taking a `Type` parameter, even if that function is in another module.
This commit solves the issue by allowing foreign module (and global)
type loaders have a chance to load the value if the local module loader
fails. The implementation here does this by storing a module-local
loading function in a capsule in the python type, which we can then call
if the local (and possibly global, if the local type is masking a global
type) version doesn't work.
This allows you to use:
cls.def(py::init(&factory_function));
where `factory_function` returns a pointer, holder, or value of the
class type (or a derived type). Various compile-time checks
(static_asserts) are performed to ensure the function is valid, and
various run-time type checks where necessary.
Some other details of this feature:
- The `py::init` name doesn't conflict with the templated no-argument
`py::init<...>()`, but keeps the naming consistent: the existing
templated, no-argument one wraps constructors, the no-template,
function-argument one wraps factory functions.
- If returning a CppClass (whether by value or pointer) when an CppAlias
is required (i.e. python-side inheritance and a declared alias), a
dynamic_cast to the alias is attempted (for the pointer version); if
it fails, or if returned by value, an Alias(Class &&) constructor
is invoked. If this constructor doesn't exist, a runtime error occurs.
- for holder returns when an alias is required, we try a dynamic_cast of
the wrapped pointer to the alias to see if it is already an alias
instance; if it isn't, we raise an error.
- `py::init(class_factory, alias_factory)` is also available that takes
two factories: the first is called when an alias is not needed, the
second when it is.
- Reimplement factory instance clearing. The previous implementation
failed under python-side multiple inheritance: *each* inherited
type's factory init would clear the instance instead of only setting
its own type value. The new implementation here clears just the
relevant value pointer.
- dealloc is updated to explicitly set the leftover value pointer to
nullptr and the `holder_constructed` flag to false so that it can be
used to clear preallocated value without needing to rebuild the
instance internals data.
- Added various tests to test out new allocation/deallocation code.
- With preallocation now done lazily, init factory holders can
completely avoid the extra overhead of needing an extra
allocation/deallocation.
- Updated documentation to make factory constructors the default
advanced constructor style.
- If an `__init__` is called a second time, we have two choices: we can
throw away the first instance, replacing it with the second; or we can
ignore the second call. The latter is slightly easier, so do that.
We currently allocate instance values when creating the instance itself
(except when constructing the instance for a `cast()`), but there is no
particular reason to do so: the instance itself and the internals (for
a non-simple layout) are allocated via Python, with no reason to
expect better locality from the invoked `operator new`. Moreover, it
makes implementation of factory function constructors trickier and
slightly less efficient: they don't use the pre-eallocate the memory,
which means there is a pointless allocation and free.
This commit makes the allocation lazy: instead of preallocating when
creating the instance, the allocation happens when the instance is
first loaded (if null at that time).
In addition to making it more efficient to deal with cases that don't
need preallocation, this also allows for a very slight performance
increase by not needing to look up the instances types during
allocation. (There is a lookup during the eventual load, of course, but
that is happening already).
This adds a PYBIND11_NAMESPACE macro that expands to the `pybind11`
namespace with hidden visibility under gcc-type compilers, and otherwise
to the plain `pybind11`. This then forces hidden visibility on
everything in pybind, solving the visibility issues discussed at end
end of #949.
Attempting to mix py::module_local and non-module_local classes results
in some unexpected/undesirable behaviour:
- if a class is registered non-local by some other module, a later
attempt to register it locally fails. It doesn't need to: it is
perfectly acceptable for the local registration to simply override
the external global registration.
- going the other way (i.e. module `A` registers a type `T` locally,
then `B` registers the same type `T` globally) causes a more serious
issue: `A.T`'s constructors no longer work because the `self` argument
gets converted to a `B.T`, which then fails to resolve.
Changing the cast precedence to prefer local over global fixes this and
makes it work more consistently, regardless of module load order.
This commit adds a `py::module_local` attribute that lets you confine a
registered type to the module (more technically, the shared object) in
which it is defined, by registering it with:
py::class_<C>(m, "C", py::module_local())
This will allow the same C++ class `C` to be registered in different
modules with independent sets of class definitions. On the Python side,
two such types will be completely distinct; on the C++ side, the C++
type resolves to a different Python type in each module.
This applies `py::module_local` automatically to `stl_bind.h` bindings
when the container value type looks like something global: i.e. when it
is a converting type (for example, when binding a `std::vector<int>`),
or when it is a registered type itself bound with `py::module_local`.
This should help resolve potential future conflicts (e.g. if two
completely unrelated modules both try to bind a `std::vector<int>`.
Users can override the automatic selection by adding a
`py::module_local()` or `py::module_local(false)`.
Note that this does mildly break backwards compatibility: bound stl
containers of basic types like `std::vector<int>` cannot be bound in one
module and returned in a different module. (This can be re-enabled with
`py::module_local(false)` as described above, but with the potential for
eventual load conflicts).
The builtin exception handler currently doesn't work across modules
under clang/libc++ for builtin pybind exceptions like
`pybind11::error_already_set` or `pybind11::stop_iteration`: under
RTLD_LOCAL module loading clang considers each module's exception
classes distinct types. This then means that the base exception
translator fails to catch the exceptions and the fall through to the
generic `std::exception` handler, which completely breaks things like
`stop_iteration`: only the `stop_iteration` of the first module loaded
actually works properly; later modules raise a RuntimeError with no
message when trying to invoke their iterators.
For example, two modules defined like this exhibit the behaviour under
clang++/libc++:
z1.cpp:
#include <pybind11/pybind11.h>
#include <pybind11/stl_bind.h>
namespace py = pybind11;
PYBIND11_MODULE(z1, m) {
py::bind_vector<std::vector<long>>(m, "IntVector");
}
z2.cpp:
#include <pybind11/pybind11.h>
#include <pybind11/stl_bind.h>
namespace py = pybind11;
PYBIND11_MODULE(z2, m) {
py::bind_vector<std::vector<double>>(m, "FloatVector");
}
Python:
import z1, z2
for i in z2.FloatVector():
pass
results in:
Traceback (most recent call last):
File "zs.py", line 2, in <module>
for i in z2.FloatVector():
RuntimeError
This commit fixes the issue by adding a new exception translator each
time the internals pointer is initialized from python builtins: this
generally means the internals data was initialized by some other
module. (The extra translator(s) are skipped under libstdc++).
This commit adds a PYBIND11_UNSHARED_STATIC_LOCALS macro that forces a
function to have hidden visibility under gcc and gcc-compatible
compilers. gcc, in particular, needs this to to avoid sharing static
local variables across modules (which happens even under a RTLD_LOCAL
dlopen()!). clang doesn't appear to have this issue, but the forced
visibility on internal pybind functions certainly won't hurt it and icc.
This updates the workaround from #862 to use this rather than the
version-specific template.
