Mailman 3 April 2016 - Cplusplus-sig

enhancement: Py_DECREF / Py_XDECREF assertion
by Wilfried Karel April 28, 2016

April 28, 2016

Dear Stefan, I've created a pull request: https://github.com/boostorg/python/pull/64 Cheers, Willi. > Hi Willi, > > On 26.04.2016 16:19, Wilfried Karel wrote: >> Dear list, >> >> I suggest to insert an assertion before calling Py_DECREF/Py_XDECREF >> which ensures that the reference count before the call is larger than >> zero. Otherwise, this means that a PyObject that has already been >> destroyed is planned to be destroyed a second time - which will not >> happen, as the reference count is of unsigned integer type, and so it >> will become a very large number - but it keeps a PyObject alive that >> references destroyed memory. >> >> I suggest to insert such assertions, because this easily detectable >> programming error is hard to track based on a possibly resulting >> segmentation fault at some later point during program execution (as has >> happened to me). Maybe this would be detected by a debug-build of >> Python, but in my environment, it would be troublesome to link it, >> together with providing possibly needed debug-builds of other Python >> extension packages (e.g. NumPy). >> >> Thus, I suggest to insert >> assert( Py_REFCNT(m_ptr) > 0 ); >> >> at the beginning of >> api::object_base::~object_base() >> and >> void decref(T* p) >> >> and to insert >> assert( p == nullptr || Py_REFCNT(python::upcast<PyObject>(p)) > 0 ); >> at the beginning of >> void xdecref(T* p) > > I'm curious: how do you manage to get non-garbage-collected objects with > zero ref-counts ? > > If you can reproduce such cases I think your suggestion is indeed a good > idea. In that case, would you mind submitting a patch / PR (against the > 'devel' branch) ? (And seeing you using 'nullptr' above reminds me: > please make sure the code is backward compatible with pre-C++-11). > Please include a test case to demonstrate the failing and fixed behavior. > >> Cheers, >> Willi. > > Many thanks ! > > Stefan > > -- > > ...ich hab' noch einen Koffer in Berlin... -- _____ _____ _____ Technische Universität Wien DI Wilfried Karel /____// ___// / Dept. of Geodesy & Geoinform. wilfried.karel@ // __ / /__ / // / Research Groups Photogrammetry geo.tuwien.ac.at //__/// /__ / // / & Remote Sensing +43 1 58801 12244 /____//____//____/ Gusshausstr. 27-29, A-1040 Wien

1 0

Trying to implement a Boost Python stl_forward_iterator
by Matthew Conte April 27, 2016

April 27, 2016

Hello list, I'm having trouble creating a boost python forward iterator to get a numpy array as input of a C++ function that takes an stl begin and end iterator as a template and modifies it, here's the code for the C++ function: *template<typename _InputIt>* *void fill_increasing(_InputIt begin, _InputIt end, typename std::iterator_traits<_InputIt>::value_type start, typename std::iterator_traits<_InputIt>::value_type increment)* *{* * typedef typename std::iterator_traits<_InputIt>::value_type _Type;* * if (std::is_floating_point<_Type>::value)* * {* * std::size_t count = 0;* * while (begin != end)* * {* * *begin = start + ((_Type)count*increment);* * ++begin;* * ++count;* * }* * }* * else {* * while (begin != end)* * {* * *begin = start;* * start += increment;* * ++begin;* * }* * }* *} * There is already an implementation of an stl input iterator in Boost Python (i.e. stl_iterator.hpp) but it's not mutable, so that won't help me here, so I tried to create my own stl_forward_iterator based on the input iterator. *namespace boost {* * namespace python* * {* * // An STL forward iterator over a python sequence* * template<typename ValueT>* * struct stl_forward_iterator* * : boost::iterator_facade<* * stl_forward_iterator<ValueT>* * , ValueT* * , std::forward_iterator_tag* * >* * {* * stl_forward_iterator()* * : impl_()* * {* * }* * // ob is the python sequence* * stl_forward_iterator(boost::python::object const &ob)* * : impl_(ob)* * {* * }* * private:* * friend class boost::iterator_core_access;* * void increment()* * {* * this->impl_.increment();* * }* * ValueT& dereference() const* * {* * return extract<ValueT&>(this->impl_.current().get())();* * }* * bool equal(stl_forward_iterator<ValueT> const &that) const* * {* * return this->impl_.equal(that.impl_);* * }* * objects::stl_input_iterator_impl impl_;* * };* *}* *}* The only change I really made was changing the return of the *deference* function to a reference instead of just a value. And I use this in my main application as follows: *template<typename T>* *void wrap_fill_increasing(boost::python::numeric::array& array, T start, T increment) {* * boost::python::stl_forward_iterator<T> begin(array);* * boost::python::stl_forward_iterator<T> end;* * vec_fill_increasing(begin, end, start, increment);* *}* *BOOST_PYTHON_MODULE(example) {* * import_array();* * p::numeric::array::set_module_and_type("numpy", "ndarray");* * p::def("fill_increasing", &wrap_fill_increasing<int>);* *}* *int main(int argc, char **argv){ PyImport_AppendInittab("example", &PyInit_example); Py_Initialize(); PyRun_SimpleString( "import example\n" "import numpy\n" "z3 = numpy.zeros((1024,), dtype=numpy.int <http://numpy.int>)\n" "example.fill_increasing(z3, 0, 1)\n" "print(z3)\n" ); Py_Finalize();}* But I get the error: *TypeError: No registered converter was able to extract a C++ reference to type int from this Python object of type numpy.int32* And it occurs on the *dereference* function in my stl_forward_iterator and I don't know how to resolve this. If something that can help he in this scenario doesn't exist, I really think there should be one as a suggestion. Any help would be greatly appreciated, thank you.

1 0

enhancement: Py_DECREF / Py_XDECREF assertion
by Wilfried Karel April 27, 2016

April 27, 2016

Dear list, I suggest to insert an assertion before calling Py_DECREF/Py_XDECREF which ensures that the reference count before the call is larger than zero. Otherwise, this means that a PyObject that has already been destroyed is planned to be destroyed a second time - which will not happen, as the reference count is of unsigned integer type, and so it will become a very large number - but it keeps a PyObject alive that references destroyed memory. I suggest to insert such assertions, because this easily detectable programming error is hard to track based on a possibly resulting segmentation fault at some later point during program execution (as has happened to me). Maybe this would be detected by a debug-build of Python, but in my environment, it would be troublesome to link it, together with providing possibly needed debug-builds of other Python extension packages (e.g. NumPy). Thus, I suggest to insert assert( Py_REFCNT(m_ptr) > 0 ); at the beginning of api::object_base::~object_base() and void decref(T* p) and to insert assert( p == nullptr || Py_REFCNT(python::upcast<PyObject>(p)) > 0 ); at the beginning of void xdecref(T* p) Cheers, Willi. -- _____ _____ _____ Technische Universität Wien DI Wilfried Karel /____// ___// / Dept. of Geodesy & Geoinform. wilfried.karel@ // __ / /__ / // / Research Groups Photogrammetry geo.tuwien.ac.at //__/// /__ / // / & Remote Sensing +43 1 58801 12244 /____//____//____/ Gusshausstr. 27-29, A-1040 Wien

2 1

Multiple registration safety and unit testing
by David Sankel April 21, 2016

April 21, 2016

Hello all, I'd like to allow a class to be registered more than once and wanted to get your opinion if this change would be a good idea. 'boost::python::class_' always adds a class to the current scope and registers it in the global registry. The global registry code (particularly the 'insert' function in 'src/converter/registry.cpp') asserts that the class hasn't been registered already. This, of course, prevents multiple registrations of the same type. There is a use case, though, where multiple registrations of the same type is both necessary and safe. Generally, a c++ component which has a 'class_' call may not know the name of the module it will eventually end up in. It would still be desirable to write a unit test for this component. If we write a unit test and put the 'class_' into some dummy module we run into a problem where other unit tests might call 'class_' in another scope. The safety of multiple 'class_' calls stems from always using the same conversion function. Semantically everything is a-okay. I'm proposing to change the precondition of the 'boost::python::converter::registry::insert' function from: The behavior is undefined unless the specified 'type_info' object has not already been registered. to: The behavior is undefined unless the specified 'type_info' object has not already been registered with a semantically different converter Any objections? -- David Sankel

2 3

Python subclassing from C++
by eric.frederich＠siemens.com April 4, 2016

April 4, 2016

Hello, I'd like to expose an std::map to Python. I found a great example here: https://sourceforge.net/p/cctbx/code/HEAD/tree/trunk/scitbx/stl/map_wrapper… It implements a bunch of methods to make it act like a dict (hooray duck typing) But I think it misses some like __eq__, __ne__, pop. I think this could be done better by extending collections.MutableMapping. This way you'd only have to implement the 5 abstract methods rather than the 20 or so it currently does. How can I do this? Is it possible to define the parent class when creating a Python class from C++ using boost::python::class_? I haven't been able to find a way. Thanks, ~Eric This message and any attachments are solely for the use of intended recipients. The information contained herein may include trade secrets, protected health or personal information, privileged or otherwise confidential information. Unauthorized review, forwarding, printing, copying, distributing, or using such information is strictly prohibited and may be unlawful. If you are not an intended recipient, you are hereby notified that you received this email in error, and that any review, dissemination, distribution or copying of this email and any attachment is strictly prohibited. If you have received this email in error, please contact the sender and delete the message and any attachment from your system. Thank you for your cooperation

1 0