Mailman 3 June 2015 - cython-devel

[RFE] Add dummy compiler directive decorators for pure python mode
by Carlos Pita July 28, 2015

July 28, 2015

Hi all, pure python mode will benefit from the addition of dummy decorators for compiler directives to Shadow.py, like in: @cython.boundscheck(False) def f(): pass AFAICS this is not currently valid inside the interpreter. Cheers -- Carlos

3 5

cython-0.23 monkey-patches stdlib?
by Neal Becker June 29, 2015

June 29, 2015

According to https://github.com/cython/cython/blob/master/CHANGES.rst the default build would attempt to modify python stdlib files? How would this work on distributions? I am maintainer for Cython on Fedora. So installing Cython would try to overwrite stdlib files? That sounds difficult. For one thing, it means more than 1 package would own the same files. And I don't know if the general user community would like this approach.

2 2

generating documentation from Cython source code (was: [cython-users] Compiling a Cython module *without* having the required libraries)
by Stefan Behnel June 28, 2015

June 28, 2015

Nikolaj van Omme schrieb am 28.06.2015 um 21:02: > On 15-06-28 02:52 PM, Stefan Behnel wrote: >> Nathann Cohen schrieb am 28.06.2015 um 18:44: >>>> Could you generate Python stub modules for them by grepping out def/cpdef >>>> functions and extension types together with their docstrings from the >>>> Cython sources? >>> Would there be a not-so-awful way to do this parsing? >>> >>> We may have classes and methods inside. Possibly several layers of it >>> (sorry, I know >_<). >> Well, the *right* way would let Cython do the parsing and integrate it with >> Sphinx, maybe via a Sphinx plugin. Either Cython could generate Python >> stubs (should be easy to do, see e.g. Cython/CodeWriter.py), or it could >> write out some XML/JSON/whatever representation of the source code metadata >> that other tools could use for further processing. >> >> The Python stub approach has the advantage of supporting normal Python >> tools, a dedicated file format has the advantage of allowing a more >> complete representation of the source code metadata. >> >> Both would be nice to have, I guess. But for the purpose of generating API >> documentation, Python stubs should get you most of the way already without >> requiring plugins for all documentation tools. Signature types could still >> be represented by mapping them to Python annotations (with a "# type: >> ignore" marker to disable PEP 484 type checks...). > > We are facing the same issues. Maybe we could team up and work on a > solution together? Please do. :) Can we move further discussion to the cython-devel mailing list? Stefan

1 0

Re: [Cython] operator() bug in cython
by Ulrich Dobramysl June 23, 2015

June 23, 2015

On Fri Feb 13 2015 at 11:07:39 PM Greg Ewing <greg.ewing(a)canterbury.ac.nz> wrote: > Ulrich Dobramysl wrote: > > > Thanks! I haven't figured out how to call heap allocated objects, as the > > code > > --- > > cdef OperatorTest *t = new OperatorTest() > > t() > > --- > > is not translatable by Cython. > > Have you tried: > > t[0]() > > ? > Thanks, I didn't think about the t[0] trick! > I haven't been following the development of Cython's internals, > so I may be speaking naively here, but it looks wrong to me that > the cname of that entry should be 'operator()' rather than the > c-level name of the variable that the NameNode refers to. > > The cname of the entry is correct IMO, because it cannot have any knowledge about the cname of the object it is being called as later. The logic dealing with the special case of an operator() member function of a c++ class in SimpleCallNode.analyse_c_function_call() needs to be changed slightly to not treat operator() as an overloaded entry. Rather, the function type of the SimpleCallNode needs to be set to the type of the looked-up operator() entry. Then, at the code generation stage, the correct cname of the SimpleCallNode entry is used and not the entry of operator(). With the patch below my test case compiles and runs without problems: ---- diff --git a/Cython/Compiler/ExprNodes.py b/Cython/Compiler/ExprNodes.py index f99ec6e..88522c9 100644 --- a/Cython/Compiler/ExprNodes.py +++ b/Cython/Compiler/ExprNodes.py @@ -4569,11 +4569,13 @@ class SimpleCallNode(CallNode): args = self.args if func_type.is_cpp_class: - overloaded_entry = self.function.type.scope.lookup("operator()") - if overloaded_entry is None: + call_operator = self.function.type.scope.lookup("operator()") + if call_operator is None: self.type = PyrexTypes.error_type self.result_code = "<error>" return + self.function.type = call_operator.type + overloaded_entry = None elif hasattr(self.function, 'entry'): overloaded_entry = self.function.entry elif (isinstance(self.function, IndexNode) and @@ -4603,7 +4605,7 @@ class SimpleCallNode(CallNode): else: entry = None func_type = self.function_type() - if not func_type.is_cfunction: + if not (func_type.is_cfunction or func_type.is_cpp_class): error(self.pos, "Calling non-function type '%s'" % func_type) self.type = PyrexTypes.error_type self.result_code = "<error>" ---- What do you think? I don't know a lot about cython's internals, so there might be use cases which break this code. Ulrich

2 2

Feature Request: Variadic Templates
by Michael Enßlin June 22, 2015

June 22, 2015

Hi, I'd like to see support for variadic template arguments, such as this: test.h: template<typename RetT, typename ... VARARGS> RetT foo(VARARGS ...); test.pyx: cdef extern from "test.h": cdef RetT foo[RetT, ... VARARGS](... VARARGS) def test(): cdef int i = foo[int, float, int](1, 2.5, 3) This would allow Cython's libcpp to easily support many of C++11's new types, including std::tuple and std::function. Support for the latter in particular would prove quite useful in passing around function pointers between Cython and C++. I have tried to implement this feature myself, but I'm entirely unfamiliar with Cython's codebase, and all my attempts ended in unsatisfactorily hacky, unstable code. I believe that any experienced Cython developer on this list would be able to properly implement this in a matter of a few hours, as it seems like a rather minor feature. The current workaround for the non-existance of this feature involves typedefs for every possible number of arguments, like this: template<typename ... Ts> struct S {}; using S0 = S<>; template<typename T0> using S1 = S<T0>; template<typename T0, typename T1> using S2 = S<T0, T1>; template<typename T0, typename T1, typename T2> using S3 = S<T0, T1, T2>; then exporting all of S0, S1, ... individually in the .pxd file. This is inconvenient, but acceptable. Now assume that S has a member function f, template<typename ... Us> void f(Ts..., Us ...); due to the nature of C++, the same trick as above does not work, and the user will be forced to clutter the C++ code with a function body for every possible len(Us). Even worse, the .pxd file will now contain a quadratic number of wrappers (one for every possible combination of len(Ts), len(Us)), all of them extremely prone to mistakes. Thanks for reading, ~ Michael

1 0

Cython 0.22.1 released
by Stefan Behnel June 20, 2015

June 20, 2015

Hi all, this is to inform you that the major bug fix release 0.22.1 for the 0.22 release series has finally been published. There are no code changes from the release candidate. The full change log follows. Downloads: https://pypi.python.org/pypi/Cython/0.22.1 http://cython.org/release/Cython-0.22.1.tar.gz http://cython.org/release/Cython-0.22.1.zip Changelog: https://github.com/cython/cython/blob/146fd20d5b3603995dbee6ed77f1919f410cd… Documentation: http://docs.cython.org/ Have fun, Stefan 0.22.1 (2015-06-20) =================== Bugs fixed ---------- * Crash when returning values on generator termination. * In some cases, exceptions raised during internal isinstance() checks were not propagated. * Runtime reported file paths of source files (e.g for profiling and tracing) are now relative to the build root directory instead of the main source file. * Tracing exception handling code could enter the trace function with an active exception set. * The internal generator function type was not shared across modules. * Comparisons of (inferred) ctuples failed to compile. * Closures inside of cdef functions returning ``void`` failed to compile. * Using ``const`` C++ references in intermediate parts of longer expressions could fail to compile. * C++ exception declarations with mapping functions could fail to compile when pre-declared in .pxd files. * C++ compilation could fail with an ambiguity error in recent MacOS-X Xcode versions. * C compilation could fail in pypy3. * Fixed a memory leak in the compiler when compiling multiple modules. * When compiling multiple modules, external library dependencies could leak into later compiler runs. Fix by Jeroen Demeyer. This fixes ticket 845.

1 0

Bug: Extension Type inheriting from int cause a MemoryError
by Stephen LARROQUE June 19, 2015

June 19, 2015

Hello, I am trying to make an extension type inheriting from int or cython.int (to do arithmetic operations in Galois Fields). However, this causes a MemoryError because it seems such extension type is not freed correctly. Other than that, it works perfectly well. Here is a test case to reproduce the bug on Python 2.7.9 Win32 (I'm running Windows 7, but it may happen on other platform, I did not test) with Cython v0.22 on Anaconda: ######################## # mode: run import sys cdef class ExtendedInt(int): pass _ERRORS = u""" MemoryError """ # Simple test case to raise a MemoryError by generating lots of ExtendedInt total_it = 1000 for i in xrange(total_it): for j in xrange(10000000): ExtendedInt(j) sys.stdout.write("\rGenerating lists of ExtendedInt : %i/%i" % (i, total_it)) ######################## My current workaround as advised in the cython-users list: https://groups.google.com/forum/#!topic/cython-users/WJ5BJ5moAh4 is to inherit from standard (object) and then reimplement all int magic methods, which is cumbersome and less optimized, but at least there's no MemoryError.

2 2

Bug: comprehensions clear current exception
by Andrew Svetlov June 16, 2015

June 16, 2015

I have an issue in aiohttp library: https://github.com/KeepSafe/aiohttp/issues/410 The source of problem is: when I execute the following code body = ', '.join("'{}': {!r}".format(k, v) for k, v in self.items()) in except block Cython clears exception. Changing from comprehension to regular iteration works well: lst = [] for k, v in self._items: lst.append("'{}': {!r}".format(k, v)) body = ', '.join(lst) In pyre Python both versions don't clear current exception. I think it's a Cython bug. -- Thanks, Andrew Svetlov

2 2

Feature request: Flag to enable warnings when 'extern' functions are not declared 'except +'
by Michael Enßlin June 12, 2015

June 12, 2015

Hi everybody, as you surely have guessed from the amount of Bug reports I have recently submitted to this mailing list, I'm currently working on a rather large Cython project. More precisely: I'm using Cython as the glue layer between the Python and C++ components of openage; an overview and source code may be found here: https://github.com/mic-e/openage/blob/pyinterface/doc/implementation/pyinte… https://github.com/mic-e/openage/tree/pyinterface/cpp/pyinterface https://github.com/mic-e/openage/tree/pyinterface/openage/cppinterface In openage, almost all C++ functions that are exposed to Cython SHOULD be declared as 'except +'. Forgetting to do so can cause hard-to-debug issues, including useless Exception messages, corruption of CPython and right-out program termination, if those methods should throw a C++ exception. As a solution, I'd like to see Cython warnings if an extern cimport has no 'except' declaration. To intentionally omit the "except +", I suggest a new except declaration, "except -", or, even better, the C++11 keyword "noexcept" (since precisely those methods that are declared 'noexcept' should be cimported without 'except+'). Of course, those warnings (or errors?) should be disabled by default, and enabled by a special command-line flag. Any thoughts on this? ~ mic_e

1 0

Can't call functions with multi-token template arguments such as 'char *'
by Michael Enßlin June 12, 2015

June 12, 2015

Hi, it seems to be impossible to use anything but a single word as a template type for functions. Classes don't suffer from this issue, as seen below. As a workaround, complex types can be ctypedef-d to a single word (also seen below). $ cat t10.pyx cdef extern from "nope.h": cdef cppclass bar[T]: void func(T arg) void foo[T](T arg) ctypedef char * charptr def test(): # works cdef bar[char *] barobj barobj.func(NULL) # works foo[int](5) # works foo[charptr](NULL) # fails foo[char *](NULL) $ cython --cplus t10.pyx Error compiling Cython file: ------------------------------------------------------------ ... # works foo[charptr](NULL) # fails foo[char *](NULL) ^ ------------------------------------------------------------ t10.pyx:27:14: Expected an identifier or literal Happy debugging, ~ mic_e

1 0