Mailman 3 March 2015 - cython-devel

Re: [Cython] operator() bug in cython
by Ulrich Dobramysl June 23, 2015

June 23, 2015

On Fri Feb 13 2015 at 11:07:39 PM Greg Ewing <greg.ewing(a)canterbury.ac.nz> wrote: > Ulrich Dobramysl wrote: > > > Thanks! I haven't figured out how to call heap allocated objects, as the > > code > > --- > > cdef OperatorTest *t = new OperatorTest() > > t() > > --- > > is not translatable by Cython. > > Have you tried: > > t[0]() > > ? > Thanks, I didn't think about the t[0] trick! > I haven't been following the development of Cython's internals, > so I may be speaking naively here, but it looks wrong to me that > the cname of that entry should be 'operator()' rather than the > c-level name of the variable that the NameNode refers to. > > The cname of the entry is correct IMO, because it cannot have any knowledge about the cname of the object it is being called as later. The logic dealing with the special case of an operator() member function of a c++ class in SimpleCallNode.analyse_c_function_call() needs to be changed slightly to not treat operator() as an overloaded entry. Rather, the function type of the SimpleCallNode needs to be set to the type of the looked-up operator() entry. Then, at the code generation stage, the correct cname of the SimpleCallNode entry is used and not the entry of operator(). With the patch below my test case compiles and runs without problems: ---- diff --git a/Cython/Compiler/ExprNodes.py b/Cython/Compiler/ExprNodes.py index f99ec6e..88522c9 100644 --- a/Cython/Compiler/ExprNodes.py +++ b/Cython/Compiler/ExprNodes.py @@ -4569,11 +4569,13 @@ class SimpleCallNode(CallNode): args = self.args if func_type.is_cpp_class: - overloaded_entry = self.function.type.scope.lookup("operator()") - if overloaded_entry is None: + call_operator = self.function.type.scope.lookup("operator()") + if call_operator is None: self.type = PyrexTypes.error_type self.result_code = "<error>" return + self.function.type = call_operator.type + overloaded_entry = None elif hasattr(self.function, 'entry'): overloaded_entry = self.function.entry elif (isinstance(self.function, IndexNode) and @@ -4603,7 +4605,7 @@ class SimpleCallNode(CallNode): else: entry = None func_type = self.function_type() - if not func_type.is_cfunction: + if not (func_type.is_cfunction or func_type.is_cpp_class): error(self.pos, "Calling non-function type '%s'" % func_type) self.type = PyrexTypes.error_type self.result_code = "<error>" ---- What do you think? I don't know a lot about cython's internals, so there might be use cases which break this code. Ulrich

2 2

Cython inserting unqualified module name into sys.module on python 3?
by Nathaniel Smith April 3, 2015

April 3, 2015

Hi all, Can anyone shed any light on this? https://github.com/numpy/numpy/issues/5680 -n -- Nathaniel J. Smith -- http://vorpus.org

3 3

Re: [Cython] Bringing Cython and PyPy closer together
by Stefan Behnel March 29, 2015

March 29, 2015

[copied here from PyPy mailing list] Stefan Behnel, 15.02.2012 12:32: > The current state of the discussion seems to be that PyPy provides ways to > talk to C code, but nothing as complete as CPython's C-API in the sense > that it allows efficient two-way communication between C code and Python > objects. Thus, we need to either improve this or look for alternatives. > > In order to get us more focussed on what can be done and what the > implications are, so that we may eventually be able to decide what should > be done, I started a Wiki page for a PyPy backend CEP (Cython Enhancement > Proposal). > > http://wiki.cython.org/enhancements/pypy The discussion so far makes me rather certain that the most promising short-term solution is to make Cython generate C code that PyPy's cpyext can handle. This should get us a rather broad set of running code somewhat quickly, while requiring the least design-from-scratch type of work in a direction that does not yet allow us to see if it will really make existing code work or not. On top of the basic cpyext interface, it should then be easy to implement obvious optimisations like native C level calls to Cython wrapped functions from PyPy (and potentially also the other direction) and otherwise avoid boxing/unboxing where unnecessary, e.g. for builtins. After all, it all boils down to native code at some point and I'm sure there are various ways to exploit that. Also, going this route will help both projects to get to know each other better. I think that's a required basis if we really aim for designing a more high-level interface at some point. The first steps I see are: - get Cython's test suite to run on PyPy - analyse the failing tests and decide how to fix them - adapt the Cython generated C code accordingly, special casing for PyPy where required Here is a "getting started" guide that tells you how testing works in Cython: http://wiki.cython.org/HackerGuide Once we have the test suite runnable, we can set up a PyPy instance on our CI server to get feed-back on any advances. https://sage.math.washington.edu:8091/hudson/ So, any volunteers or otherwise interested parties to help in getting this to work? Anyone in for financial support? Stefan

2 9

"long long" versus "PY_LONG_LONG"
by Lisandro Dalcin March 29, 2015

March 29, 2015

At some point Cython lost the ability of using PY_LONG_LONG instead of "long long" in generated C code. The big offenders are the following two files: Cython/Utility/Overflow.c Cython/Utility/TypeConversion.c Do we want to continue using PY_LONG_LONG? Otherwise, what about Microsoft compilers? Going back to PY_LONG_LONG everywhere is quite easy right now, let me know and I'll contribute the patch for the upcoming 0.22.1 release (if that ever happens). -- Lisandro Dalcin ============ Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459

2 3

PyPy3 fixes
by Lisandro Dalcin March 19, 2015

March 19, 2015

1) Stefan, I had to push this fix, just to be sure, please double check. https://github.com/cython/cython/commit/6bd3f7b9e494d1259082aecfc0366da15fc… 2) PyPy3 defines some legacy (Py2) Py_TPFLAGS_XXX values. Do you like the following fix? IOW, I check if the value is #define'd, if not I #define it to 0 (zero): diff --git a/Cython/Utility/ModuleSetupCode.c b/Cython/Utility/ModuleSetupCode.c index 6477fb2..3ae8a0c 100644 --- a/Cython/Utility/ModuleSetupCode.c +++ b/Cython/Utility/ModuleSetupCode.c @@ -59,13 +59,16 @@ #define __Pyx_DefaultClassType PyType_Type #endif -#if PY_MAJOR_VERSION >= 3 +#if !defined(Py_TPFLAGS_CHECKTYPES) #define Py_TPFLAGS_CHECKTYPES 0 +#endif +#if !defined(Py_TPFLAGS_HAVE_INDEX) #define Py_TPFLAGS_HAVE_INDEX 0 +#endif +#if !defined(Py_TPFLAGS_HAVE_NEWBUFFER) #define Py_TPFLAGS_HAVE_NEWBUFFER 0 #endif - -#if PY_VERSION_HEX < 0x030400a1 && !defined(Py_TPFLAGS_HAVE_FINALIZE) +#if !defined(Py_TPFLAGS_HAVE_FINALIZE) #define Py_TPFLAGS_HAVE_FINALIZE 0 #endif -- Lisandro Dalcin ============ Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459

3 3

Sorry: 0.22.x and master broken
by Lisandro Dalcin March 17, 2015

March 17, 2015

Folks, I'm very sorry. In an attempt to fix issues with PyPy3, I pushed a fix to 0.22.x (https://github.com/cython/cython/commit/8573e7898a27ef607a3264909cc3bf24d4e…) and merged to master, but now one test is failing with CPython 3 (https://travis-ci.org/cython/cython/jobs/54762494). I did not run the full test suite with python3, my fault. I'll look at this issue tomorrow and push a fix ASAP. -- Lisandro Dalcin ============ Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459

1 0

Generated function has invalid name when converting from python object to C struct defined in C++ namespace
by Daniel Grunwald March 17, 2015

March 17, 2015

Hello, I have Cython code like this: cdef extern from "cpp_library.h" namespace "CppLibrary": struct SomeStruct: int i void do_with_struct(SomeStruct s) def run(): do_with_struct(object()) With cython 0.21.1, invalid C++ code is generated: struct CppLibrary::SomeStruct; static struct CppLibrary::SomeStruct __pyx_convert__from_py_CppLibrary::SomeStruct(PyObject *); With cython master, a compiler error occurs: @cname("__pyx_convert__from_py_CppLibrary::SomeStruct") cdef SomeStruct __pyx_convert__from_py_CppLibrary::SomeStruct(obj) except *: ^ FromPyStructUtility:11:50: Expected an identifier or literal Cython should mangle the struct name to ensure the helper function has a valid name. Regards, -- Daniel Grunwald Axivion GmbH Nobelstr. 15 70569 Stuttgart Deutschland Tel: +49 711 6204378-33 Fax: +49 711 6204378-99 Geschaeftsfuehrung: Stefan Bellon, Thomas Eisenbarth, Sebastian Rummler Sitz der Gesellschaft: Stuttgart Registergericht: Amtsgericht Stuttgart, HRB 720590

1 0

Cython magic annotate option is broken under IPython 3.0
by Nathan Goldbaum March 11, 2015

March 11, 2015

Hi all, To reproduce this issue, be on cython 0.22 and IPython 3.0 and run the following notebook: http://nbviewer.ipython.org/gist/ngoldbaum/855a629d997aa7959254 Googling the error returns some several year old discussions in IPython related to the autoreload magic and I'm not sure if that is a red herring here: http://mail.scipy.org/pipermail/ipython-dev/2012-May/009288.html For how I'm working around this by calling cython at the command line in my notebook and then loading the annotated HTML with an IFrame. Thanks for your help!

1 0

Py3, TypeConversion.c, PyLong internals
by Lisandro Dalcin March 10, 2015

March 10, 2015

Robert, please look at this line: https://github.com/cython/cython/blob/0.22.x/Cython/Utility/TypeConversion.… You are still using a PyLong internal detail to test for signedness. I'm getting a strange error with PyPy3 that might be related to this, see this traceback: https://travis-ci.org/mpi4py/mpi4py/jobs/53793609#L5650 The code generating this traceback is linked below, the__setitem__method of the variable "m" is coded to only accept "usigned char" values. https://bitbucket.org/mpi4py/mpi4py/src/86e4235ee2ec31832031ce020421810c51d… Any suggestions about how to check signedness in Python 3 (CPython and PyPy) without using Py_SIZE() ? -- Lisandro Dalcin ============ Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459

1 0

Proposal: extern implementations of functions
by Jeroen Demeyer March 7, 2015

March 7, 2015

Hello Cython developers, I would like to propose a new mechanism to implement functions in an external .c file. The goal is to solve the problem I was having in https://groups.google.com/forum/#!topic/cython-users/TsbBNyvwZn4 With the attached patch, you can do the following: * in foo.pxd, declare the function as usual: cdef void myfunc() * in foo.c, actually implement the function: static void myfunc() { /* ... */ } * in foo.pyx, declare the function extern: cdef extern from "foo.c": void myfunc() With my patch, the extern declaration in foo.pyx will be interpreted as implementing the function. Therefore, myfunc() is now available in the module foo, and it can also be cimported by other modules. What do you think of this proposed feature? Jeroen.

1 0