Re: [C++-sig] [Numpy-discussion] Overloading sqrt(5.5)*myvector
I should have added: This structure worked with the older version of VPython which used Numeric, but it doesn't work in the beta version which uses numpy. Since I don't know enough about either numpy or Boost, I'm left guessing which subsystem is the source of my difficulties, and clueless about how to remedy them. Bruce Sherwood Bruce Sherwood wrote:
Thanks for the comment, which limits the range of possible solutions. The VPython vector class is implemented in C++, not in Python. I made up the simple test in my previous note to try out the solution that had been offered and which you have usefully ruled out. Here is the relevant part of the vector class, which indeed doesn't look like an ndarray:
inline vector operator*( const double s) const throw() { return vector( s*x, s*y, s*z); }
and here is the free function for right multiplication:
inline vector operator*( const double& s, const vector& v) { return vector( s*v.x, s*v.y, s*v.z); }
Maybe the unsolvable problem is in the Boost definitions:
py::class_<vector>("vector", py::init< py::optional<double, double, double> >()) .def( self * double()) .def( double() * self)
Left multiplication is fine, but right multiplication isn't.
Bruce Sherwood
Robert Kern wrote:
Bruce Sherwood wrote:
Thanks for the suggestion. It hadn't occurred to me to try to override numpy as you suggest. However, when I try the code shown below as the start of a test of this scheme, I get the following error:
Traceback (most recent call last): File "C:\Documents and Settings\Bruce\My Documents\0VPythonWork\vectors.py", line 24, in <module> numpy.float64.__mul__ = new_mul TypeError: can't set attributes of built-in/extension type 'numpy.float64'
I'm copying this to the numpy discussion list, as maybe someone there will see where to go starting from your suggestion.
Like most (or all) builtin-types, the numpy float scalars do not permit replacing their methods from Python.
I'm not familiar with vpython's vector. If you can make it "not look like an ndarray", then you should be able to just implement __rmul__ on vector.
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Hi Bruce, I have to add that I don't know the answer to your question either, but I do know that it is solvable and that once the list experts return, enlightenment will soon follow. My confidence comes from knowing the Python internals for how left and right multiplication are performed. As long as the "left" __mul__ operator returns NotImplemented, then the __rmul__ method will be attempted (see http://docs.python.org/ref/numeric-types.html). Of course, I don't know how to declare such a beast in Boost, having never used it, but I'm sure it is possible. My intuition is that the first problem you need to solve is getting Boot to generate the appropriate __rmul__ method. The second problem, if it even exists, is ensuring that __mul__ returns NotImplemented. Best of luck, -Kevin On Dec 27, 2007 10:15 AM, Bruce Sherwood <Bruce_Sherwood@ncsu.edu> wrote:
I should have added: This structure worked with the older version of VPython which used Numeric, but it doesn't work in the beta version which uses numpy. Since I don't know enough about either numpy or Boost, I'm left guessing which subsystem is the source of my difficulties, and clueless about how to remedy them.
Bruce Sherwood
Thanks for the comment, which limits the range of possible solutions. The VPython vector class is implemented in C++, not in Python. I made up the simple test in my previous note to try out the solution that had been offered and which you have usefully ruled out. Here is the relevant part of the vector class, which indeed doesn't look like an ndarray:
inline vector operator*( const double s) const throw() { return vector( s*x, s*y, s*z); }
and here is the free function for right multiplication:
inline vector operator*( const double& s, const vector& v) { return vector( s*v.x, s*v.y, s*v.z); }
Maybe the unsolvable problem is in the Boost definitions:
py::class_<vector>("vector", py::init< py::optional<double, double, double> >()) .def( self * double()) .def( double() * self)
Left multiplication is fine, but right multiplication isn't.
Bruce Sherwood
Robert Kern wrote:
Bruce Sherwood wrote:
Thanks for the suggestion. It hadn't occurred to me to try to override numpy as you suggest. However, when I try the code shown below as the start of a test of this scheme, I get the following error:
Traceback (most recent call last): File "C:\Documents and Settings\Bruce\My Documents\0VPythonWork\vectors.py", line 24, in <module> numpy.float64.__mul__ = new_mul TypeError: can't set attributes of built-in/extension type ' numpy.float64'
I'm copying this to the numpy discussion list, as maybe someone there will see where to go starting from your suggestion.
Like most (or all) builtin-types, the numpy float scalars do not permit replacing their methods from Python.
I'm not familiar with vpython's vector. If you can make it "not look
Bruce Sherwood wrote: like an
ndarray", then you should be able to just implement __rmul__ on vector.
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
I realized belatedly that I should upgrade from Boost 1.33 to 1.34. Alas, that didn't cure my problem. Bruce Sherwood Bruce Sherwood wrote:
I should have added: This structure worked with the older version of VPython which used Numeric, but it doesn't work in the beta version which uses numpy. Since I don't know enough about either numpy or Boost, I'm left guessing which subsystem is the source of my difficulties, and clueless about how to remedy them.
Bruce Sherwood
Bruce Sherwood wrote:
Thanks for the comment, which limits the range of possible solutions. The VPython vector class is implemented in C++, not in Python. I made up the simple test in my previous note to try out the solution that had been offered and which you have usefully ruled out. Here is the relevant part of the vector class, which indeed doesn't look like an ndarray:
inline vector operator*( const double s) const throw() { return vector( s*x, s*y, s*z); }
and here is the free function for right multiplication:
inline vector operator*( const double& s, const vector& v) { return vector( s*v.x, s*v.y, s*v.z); }
Maybe the unsolvable problem is in the Boost definitions:
py::class_<vector>("vector", py::init< py::optional<double, double, double> >()) .def( self * double()) .def( double() * self)
Left multiplication is fine, but right multiplication isn't.
Bruce Sherwood
Robert Kern wrote:
Bruce Sherwood wrote:
Thanks for the suggestion. It hadn't occurred to me to try to override numpy as you suggest. However, when I try the code shown below as the start of a test of this scheme, I get the following error:
Traceback (most recent call last): File "C:\Documents and Settings\Bruce\My Documents\0VPythonWork\vectors.py", line 24, in <module> numpy.float64.__mul__ = new_mul TypeError: can't set attributes of built-in/extension type 'numpy.float64'
I'm copying this to the numpy discussion list, as maybe someone there will see where to go starting from your suggestion.
Like most (or all) builtin-types, the numpy float scalars do not permit replacing their methods from Python.
I'm not familiar with vpython's vector. If you can make it "not look like an ndarray", then you should be able to just implement __rmul__ on vector.
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
On Dec 29, 2007 7:47 AM, Bruce Sherwood <Bruce_Sherwood@ncsu.edu> wrote:
I realized belatedly that I should upgrade from Boost 1.33 to 1.34. Alas, that didn't cure my problem.
Can you post small and complete example of what you are trying to achieve? -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/
Roman Yakovenko wrote:
On Dec 29, 2007 7:47 AM, Bruce Sherwood <Bruce_Sherwood@ncsu.edu> wrote:
I realized belatedly that I should upgrade from Boost 1.33 to 1.34. Alas, that didn't cure my problem.
Can you post small and complete example of what you are trying to achieve?
I don't have a "small and complete" example available, but I'll summarize from earlier posts. VPython (vpython.org) has its own vector class to mimic the properties of 3D vectors used in physics, in the service of easy creation of 3D animations. There is a beta version which imports numpy and uses it internally; the older production version uses Numeric. Boost python and thread libraries are used to connect the C++ VPython code to Python. There is operator overloading that includes scalar*vector and vector*scalar, both producing vector. With Numeric, sqrt produced a float, which was a scalar for the operator overloading. With numpy, sqrt produces a numpy.float64 which is caught by vector*scalar but not by scalar*vector, which means that scalar*vector produces an ndarray rather than a vector, which leads to a big performance hit in existing VPython programs. The overloading and Boost code is the same in the VPython/Numeric and VPython/numpy versions. I don't know whether the problem is with numpy or with Boost or with the combination of the two. Here is the relevant part of the vector class: inline vector operator*( const double s) const throw() { return vector( s*x, s*y, s*z); } and here is the free function for right multiplication: inline vector operator*( const double& s, const vector& v) { return vector( s*v.x, s*v.y, s*v.z); } Maybe the problem is in the Boost definitions: py::class_<vector>("vector", py::init< py::optional<double, double, double> >()) .def( self * double()) .def( double() * self) Left multiplication is fine, but right multiplication isn't. A colleague suggested the following Boost declarations but cautioned that he wasn't sure of the syntax for referring to operator, and indeed this doesn't compile: .def( "__mul__", &vector::operator*(double), "Multiply vector times scalar") .def( "__rmul__", &operator*(const double&, const vector&), "Multiply scalar times vector") I would really appreciate a Boost or numpy expert being able to tell me what's wrong (if anything) with these forms. However, I may have a useful workaround as I described in a post to the numpy discussion list. A colleague suggested that I do something like this for sqrt and other such mathematical functions: def sqrt(x): try: return mathsqrt(x) except TypeError: return numpysqrt(x) That is, first try the simple case of a scalar argument, handled by the math module sqrt, and only use the numpy sqrt routine in the case of an array argument. Even with the overhead of the try/except machinery, one gets must faster square roots for scalar arguments this way than with the numpy sqrt. Bruce Sherwood
I found by timing measurements that a faster scheme with less penalty for the case of sqrt(array) looks like this: def sqrt(x): if type(x) is float: return mathsqrt(x) return numpysqrt(x) Bruce Sherwood wrote:
Roman Yakovenko wrote:
On Dec 29, 2007 7:47 AM, Bruce Sherwood <Bruce_Sherwood@ncsu.edu> wrote:
I realized belatedly that I should upgrade from Boost 1.33 to 1.34. Alas, that didn't cure my problem.
Can you post small and complete example of what you are trying to achieve?
I don't have a "small and complete" example available, but I'll summarize from earlier posts. VPython (vpython.org) has its own vector class to mimic the properties of 3D vectors used in physics, in the service of easy creation of 3D animations. There is a beta version which imports numpy and uses it internally; the older production version uses Numeric. Boost python and thread libraries are used to connect the C++ VPython code to Python.
There is operator overloading that includes scalar*vector and vector*scalar, both producing vector. With Numeric, sqrt produced a float, which was a scalar for the operator overloading. With numpy, sqrt produces a numpy.float64 which is caught by vector*scalar but not by scalar*vector, which means that scalar*vector produces an ndarray rather than a vector, which leads to a big performance hit in existing VPython programs. The overloading and Boost code is the same in the VPython/Numeric and VPython/numpy versions. I don't know whether the problem is with numpy or with Boost or with the combination of the two.
Here is the relevant part of the vector class:
inline vector operator*( const double s) const throw() { return vector( s*x, s*y, s*z); }
and here is the free function for right multiplication:
inline vector operator*( const double& s, const vector& v) { return vector( s*v.x, s*v.y, s*v.z); }
Maybe the problem is in the Boost definitions:
py::class_<vector>("vector", py::init< py::optional<double, double, double> >()) .def( self * double()) .def( double() * self)
Left multiplication is fine, but right multiplication isn't.
A colleague suggested the following Boost declarations but cautioned that he wasn't sure of the syntax for referring to operator, and indeed this doesn't compile:
.def( "__mul__", &vector::operator*(double), "Multiply vector times scalar") .def( "__rmul__", &operator*(const double&, const vector&), "Multiply scalar times vector")
I would really appreciate a Boost or numpy expert being able to tell me what's wrong (if anything) with these forms. However, I may have a useful workaround as I described in a post to the numpy discussion list. A colleague suggested that I do something like this for sqrt and other such mathematical functions:
def sqrt(x): try: return mathsqrt(x) except TypeError: return numpysqrt(x)
That is, first try the simple case of a scalar argument, handled by the math module sqrt, and only use the numpy sqrt routine in the case of an array argument. Even with the overhead of the try/except machinery, one gets must faster square roots for scalar arguments this way than with the numpy sqrt.
Bruce Sherwood
Okay, I've implemented the scheme below that was proposed by Scott Daniels on the VPython mailing list, and it solves my problem. It's also much faster than using numpy directly: even with the "def "and "if" overhead: sqrt(scalar) is over 3 times faster than the numpy sqrt, and sqrt(array) is very nearly as fast as the numpy sqrt. Thanks to those who made suggestions. There remains the question of why operator overloading of the kind I've described worked with Numeric and Boost but not with numpy and Boost. There is also the question of whether it would pay for numpy to make what is probably an exceedingly fast check and do much faster calculations of sqrt(scalar) and other such mathematical functions. Bruce Sherwood Bruce Sherwood wrote:
I found by timing measurements that a faster scheme with less penalty for the case of sqrt(array) looks like this:
def sqrt(x): if type(x) is float: return mathsqrt(x) return numpysqrt(x)
Bruce Sherwood wrote:
Roman Yakovenko wrote:
On Dec 29, 2007 7:47 AM, Bruce Sherwood <Bruce_Sherwood@ncsu.edu> wrote:
I realized belatedly that I should upgrade from Boost 1.33 to 1.34. Alas, that didn't cure my problem.
Can you post small and complete example of what you are trying to achieve?
I don't have a "small and complete" example available, but I'll summarize from earlier posts. VPython (vpython.org) has its own vector class to mimic the properties of 3D vectors used in physics, in the service of easy creation of 3D animations. There is a beta version which imports numpy and uses it internally; the older production version uses Numeric. Boost python and thread libraries are used to connect the C++ VPython code to Python.
There is operator overloading that includes scalar*vector and vector*scalar, both producing vector. With Numeric, sqrt produced a float, which was a scalar for the operator overloading. With numpy, sqrt produces a numpy.float64 which is caught by vector*scalar but not by scalar*vector, which means that scalar*vector produces an ndarray rather than a vector, which leads to a big performance hit in existing VPython programs. The overloading and Boost code is the same in the VPython/Numeric and VPython/numpy versions. I don't know whether the problem is with numpy or with Boost or with the combination of the two.
Here is the relevant part of the vector class:
inline vector operator*( const double s) const throw() { return vector( s*x, s*y, s*z); }
and here is the free function for right multiplication:
inline vector operator*( const double& s, const vector& v) { return vector( s*v.x, s*v.y, s*v.z); }
Maybe the problem is in the Boost definitions:
py::class_<vector>("vector", py::init< py::optional<double, double, double> >()) .def( self * double()) .def( double() * self)
Left multiplication is fine, but right multiplication isn't.
A colleague suggested the following Boost declarations but cautioned that he wasn't sure of the syntax for referring to operator, and indeed this doesn't compile:
.def( "__mul__", &vector::operator*(double), "Multiply vector times scalar") .def( "__rmul__", &operator*(const double&, const vector&), "Multiply scalar times vector")
I would really appreciate a Boost or numpy expert being able to tell me what's wrong (if anything) with these forms. However, I may have a useful workaround as I described in a post to the numpy discussion list. A colleague suggested that I do something like this for sqrt and other such mathematical functions:
def sqrt(x): try: return mathsqrt(x) except TypeError: return numpysqrt(x)
That is, first try the simple case of a scalar argument, handled by the math module sqrt, and only use the numpy sqrt routine in the case of an array argument. Even with the overhead of the try/except machinery, one gets must faster square roots for scalar arguments this way than with the numpy sqrt.
Bruce Sherwood
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
On Dec 29, 2007 3:00 PM, Bruce Sherwood <Bruce_Sherwood@ncsu.edu> wrote:
Thanks to those who made suggestions. There remains the question of why operator overloading of the kind I've described worked with Numeric and Boost but not with numpy and Boost. There is also the question of whether it would pay for numpy to make what is probably an exceedingly fast check and do much faster calculations of sqrt(scalar) and other such mathematical functions.
I think the problem you ran into with Numeric has to do with Python's optimizations for built-ins. In order to override members of e.g. class "dict," you have to provide a trivial subclass (or do something funny with a metaclass) because, I think, otherwise Python hard-wires method calls to the extension method instead of looking them up in __dict__. That's about all I know on the subject, unfortunately, but here's Guido on this issue: http://mail.python.org/pipermail/python-dev/2003-November/040070.html -Max -- "The presentation or 'gift' of the Holy Ghost simply confers upon a man the right to receive at any time, when he is worthy of it and desires it, the power and light of truth of the Holy Ghost, although he may often be left to his own spirit and judgment." --Joseph F. Smith (manual, p. 69) Be pretty if you are, Be witty if you can, But be cheerful if it kills you.
That doesn't sound like my problem. In VPython it's not that its vector class overrides a Python class. Rather there is overloading of operators involving the vector class. The rules for doing this in C++ and Boost worked with Numeric but not with numpy. It's not clear whether the problem is with Boost or numpy. Bruce Sherwood Maximilian Wilson wrote:
On Dec 29, 2007 3:00 PM, Bruce Sherwood <Bruce_Sherwood@ncsu.edu> wrote:
Thanks to those who made suggestions. There remains the question of why operator overloading of the kind I've described worked with Numeric and Boost but not with numpy and Boost. There is also the question of whether it would pay for numpy to make what is probably an exceedingly fast check and do much faster calculations of sqrt(scalar) and other such mathematical functions.
I think the problem you ran into with Numeric has to do with Python's optimizations for built-ins. In order to override members of e.g. class "dict," you have to provide a trivial subclass (or do something funny with a metaclass) because, I think, otherwise Python hard-wires method calls to the extension method instead of looking them up in __dict__. That's about all I know on the subject, unfortunately, but here's Guido on this issue:
http://mail.python.org/pipermail/python-dev/2003-November/040070.html
-Max
Hi I'm having a strange problem when using custom converters, they seem to work fine when I have a function/method that returns an object type but fails if I try use the object as a boost::python::object. The error I get is: TypeError: No to_python (by-value) converter found for C++ type: class Interface For example: Interface* TestConverter(Interface* interface) { return interface; } Works correctly, the python class instance that inherits from Interface passes correctly through the TestConverter method when called from python. However, if I try this: Interface* TestConverter(Interface* interface) { boost::python::object obj(interface); return interface; } Then the TypeError is thrown. I would have thought the converter would be called for this case as well? Here's the converter code: template < typename T > struct ConverterScriptObject { ConverterScriptObject() { // Register custom the to python converter boost::python::to_python_converter< T*, ConverterScriptObject< T > >(); } static PyObject* convert(T* object) { PyObject* pyobject = Py_None; if (object) { PyObject* script_object = object->GetScriptObject(); if (script_object) { pyobject = script_object; } } Py_INCREF(pyobject); return pyobject; } }; ConverterScriptObject< Interface >(); Thanks, Lloyd
participants (5)
-
Bruce Sherwood -
Kevin Jacobs <jacobs＠bioinformed.com> -
Lloyd Weehuizen -
Maximilian Wilson -
Roman Yakovenko