Mailman 3 April 2020 - cython-devel

Cython 3.0 alpha 1 released
by Stefan Behnel May 19, 2020

May 19, 2020

Dear Cython users and devs, today, I'm happy to announce the first alpha release of Cython 3.0. https://pypi.org/project/Cython/3.0a1/ It took us a while to get to this point, well more than a year's time, but we received a lot of help along the way, most notably from David Woods, Jeroen Demeyer and Matti Picus. Thanks a lot, and also to the many other contributors! For this release, we already have 182 closed issues and merged PRs, including 109 PRs contributed by non-core devs! ** What is Cython 3.0? Cython 3.0 is our effort to bring Cython up to date with modern Python 3, after an 18 year long development history. According to the Python mailing list archive [1] and Greg's download directory [2], Cython's predecessor Pyrex 0.1 was announced and released to the public on April 4th, 2002. Cython is finally coming of age. :) ** So, what's new? Too much. Way too much for this announcement. Cython 3.0 comes with a very long list of new features, bug fixes and modernisations, a few of which are backwards incompatible, but in a good way. See the latest changelog: https://github.com/cython/cython/blob/master/CHANGES.rst Here's a short teaser list anyway: - Python 3 semantics by default, legacy Python 2 semantics via directive. - No more deprecated NumPy C-API usage. - Unicode module names, imports, and identifiers (PEP-3131, PEP-489). - Support for the fast vectorcall protocol (PEP-590). - Inlined properties on external cdef classes. - Faster dispatch for fused functions. - First steps towards supporting CPython's stable ABI (PEP-384). ** How alpha is it? Well. It's not complete yet [3]. It still has some known issues. There are still some unmerged PRs waiting. Support for the limited API is … limited. But, it's in a good shape and probably ready enough for you to make use of all those cool new features. Please give it a try and report back if you find issues that we can still improve on in the upcoming pre-releases. PRs very welcome! As always, if it works for you, use it. Generate your C code locally with it, test it, then ship it to your users. Once the code is compiled, they don't even have to know that you've used an alpha version. :) Have fun, Stefan [1] https://mail.python.org/pipermail/python-list/2002-April/126661.html [2] https://www.csse.canterbury.ac.nz/greg.ewing/python/Pyrex/oldtar/ [3] https://github.com/cython/cython/milestone/58

2 6

Re: [Cython] [cython-users] Problems with "private" functions/methods in Cython 3.0a2
by Stefan Behnel April 23, 2020

April 23, 2020

Hi, thanks for the very detailed report. I'm CC-ing cython-devel, but let's keep the discussion for this on the cython-users list. The background here is Github issue 1382 and PR 3123 https://github.com/cython/cython/issues/1382 https://github.com/cython/cython/pull/3123 This was newly merged for alpha 2, and I consider it somewhat provisional. It makes "private" Python names follow Python semantics, but if that poses excessive breakage for existing code, then we'll have to find a pragmatic way to deal with it. 'Toni Barth' via cython-users schrieb am 23.04.20 um 19:07: > I just updated Cython from 3.0a1 to 3.0a2 in one of my projects and > noticed that it seems to introduce issues with private > functions/methods, means methods with leading __. > > I noticed two cases which now fail in my project. I'm however unsure if > thats supposed to happen or if its a problem with the new Cython build. > > > Case 1: private function not declared > > > crash.pxd: > > > cpdef __foo() > > cdef class Bar: > cpdef run(Bar self) > > > crash.pyx: > > > cpdef __foo(): > print("foo") > > cdef class Bar: > cpdef run(Bar self): > __foo() > > print("bar") > > > Running this with the following command: > > py -3 -c "import pyximport;pyximport.install();import > crash;crash.Bar().run()" > > > Result: > > crash.c > C:\Program Files (x86)\Microsoft Visual > Studio\Shared\Python36_64\lib\site-packages\Cython\Compiler\Main.py:344: > FutureWarning: Cython directive 'language_level' not set, using '3str' > for now (Py3). This has changed from earlier releases! File: > D:\crash\crash.pxd > tree = Parsing.p_module(s, pxd, full_module_name) > > Error compiling Cython file: > ------------------------------------------------------------ > ... > cpdef __foo(): > print("foo") > > cdef class Bar: > cpdef run(Bar self): > __foo() > ^ > ------------------------------------------------------------ > > crash.pyx:6:4: undeclared name not builtin: __foo At least this looks like a bug to me. Global module names should not be impacted by this change. > Error compiling Cython file: > ------------------------------------------------------------ > ... > cpdef __foo(): > print("foo") > > cdef class Bar: > cpdef run(Bar self): > __foo() > ^ > ------------------------------------------------------------ > > crash.pyx:6:4: '__foo' redeclared > > Error compiling Cython file: > ------------------------------------------------------------ > ... > cpdef __foo() > ^ > ------------------------------------------------------------ > > crash.pxd:1:12: Previous declaration is here Lovely :) > Another problem, probably related, occurs when trying to override a > private method in a derived class: > > > Case 2: overriding private class methods > > > base.pxd: > > > cdef class Base: > cdef void __foo(Base self) > cpdef run(Base self) > > > base.pyx: > > > cdef class Base: > > cdef void __foo(Base self): > print("foo!") > > cpdef run(Base self): > self.__foo() > > > extend.pxd: > > > from base cimport Base > > cdef class Extend(Base): > pass > > > extend.pyx: > > > cdef class Extend(Base): > > cdef void __foo(Extend self): > print("bar") > > Running with the following command: > > py -3 -c "import pyximport;pyximport.install();import > extend;extend.Extend().run()" > > > Results in the following output: > > > extend.c > C:\Program Files (x86)\Microsoft Visual > Studio\Shared\Python36_64\lib\site-packages\Cython\Compiler\Main.py:344: > FutureWarning: Cython directive 'language_level' not set, using '3str' > for now (Py3). This has changed from earlier releases! File: > D:\crash\extend.pxd > tree = Parsing.p_module(s, pxd, full_module_name) > > Error compiling Cython file: > ------------------------------------------------------------ > ... > cdef class Extend(Base): > > cdef void __foo(Extend self): > ^ > ------------------------------------------------------------ > > extend.pyx:3:7: C method '_Extend__foo' not previously declared in > definition part of extension type 'Extend' This, OTOH, is intended. Private methods are not meant to be inherited and overridden. That's what makes them "private" :) That's consistent with Python semantics. I think it's somewhat up for debate if this should apply in the same way to cdef classes as for Python classes. But from a language design point of view, having this kind of difference between the two seems a bad idea. Do you feel like argueing against this? :) Stefan

2 4

Help with cygdb bug
by Volker Weißmann April 23, 2020

April 23, 2020

Hello, cy exec print(localvar) currently does not work. And I need your help to fix it. If I'm not mistaken, the function "_fill_locals_dict" in libcython.py is supposed to find and set all local variables. The "for name, cyvar in iterator:" correctly iterates over all local variables, but cyvar.type is not PythonObject but CObject. The cython line "localvar= 49" gets translated to the C-line "__pyx_v_localvar = 49;". Notice how the type of__pyx_v_localvar is long and not a static PyObject* . If an equivalent "static PyObject*" would exist in the C-code, I would know how to make 'cy exec print(localvar)' work (the reason why global variables work, is because they are of type static PyObject*). I see two ways to make `cy exec print(localvar)` work: 1. Let cygdb generate C-code to convert a long (or any other type) to an equivalent static PyObject* (For long's it would be __Pyx_PyInt_From_long). Then let cygdb execute this code by running gdb.parse_and_eval(code). Then read the resulting variable with something like code = ''' (PyObject *) PyDict_SetItem( (PyObject *) %d, (PyObject *) %d, (PyObject *) %s) ''' % (local_dict_pointer, pystringp, cname) gdb.parse_and_eval(code) 2. Compile the argument of cy exec (in this case "print(localvar)") to C-code (while getting the local variables right) and run it using gdb.parse_and_eval(code). Ideally, cy exec var=123 should work like cy set var=123. Example to reproduce it: codefile.pyx: def func(): localvar = 49 print(localvar) func() setup.py: #!/usr/bin/env python from setuptools import setup, Extension from Cython.Build import cythonize extensions = Extension('codefile', sources=['codefile.pyx']) setup( author = 'Volker Weissmann', author_email = 'volker.weissmann(a)gmx.de', ext_modules = cythonize( extensions, gdb_debug=True, ), ) .cygdbinit: cy break codefile:3 cy run cy print localvar To reproduce it, run: python setup.py build_ext --inplace cygdb . -- -q --args python -c "import codefile" With a debug version of Python.

2 1