Mailman 3 August 2014 - cython-devel

Re: [Cython] Bringing Cython and PyPy closer together
by Stefan Behnel March 29, 2015

March 29, 2015

[copied here from PyPy mailing list] Stefan Behnel, 15.02.2012 12:32: > The current state of the discussion seems to be that PyPy provides ways to > talk to C code, but nothing as complete as CPython's C-API in the sense > that it allows efficient two-way communication between C code and Python > objects. Thus, we need to either improve this or look for alternatives. > > In order to get us more focussed on what can be done and what the > implications are, so that we may eventually be able to decide what should > be done, I started a Wiki page for a PyPy backend CEP (Cython Enhancement > Proposal). > > http://wiki.cython.org/enhancements/pypy The discussion so far makes me rather certain that the most promising short-term solution is to make Cython generate C code that PyPy's cpyext can handle. This should get us a rather broad set of running code somewhat quickly, while requiring the least design-from-scratch type of work in a direction that does not yet allow us to see if it will really make existing code work or not. On top of the basic cpyext interface, it should then be easy to implement obvious optimisations like native C level calls to Cython wrapped functions from PyPy (and potentially also the other direction) and otherwise avoid boxing/unboxing where unnecessary, e.g. for builtins. After all, it all boils down to native code at some point and I'm sure there are various ways to exploit that. Also, going this route will help both projects to get to know each other better. I think that's a required basis if we really aim for designing a more high-level interface at some point. The first steps I see are: - get Cython's test suite to run on PyPy - analyse the failing tests and decide how to fix them - adapt the Cython generated C code accordingly, special casing for PyPy where required Here is a "getting started" guide that tells you how testing works in Cython: http://wiki.cython.org/HackerGuide Once we have the test suite runnable, we can set up a PyPy instance on our CI server to get feed-back on any advances. https://sage.math.washington.edu:8091/hudson/ So, any volunteers or otherwise interested parties to help in getting this to work? Anyone in for financial support? Stefan

2 9

Automatic conversion with fixed-size C arrays
by Kurt Smith Oct. 25, 2014

Oct. 25, 2014

Hi devs, Being able to convert between simple C structs and Python dictionaries is great, but it doesn't work if there is a fixed-size array field in the struct. It seems like the following struct should be convertible safely: cdef struct state_t: int i, j, k float x[3] float v[3] Along with that, I'd expect that converting between fixed-sized C arrays and Python iterables should work as well: def auto_convert(list ll): cdef int arr[10] = ll # ... return arr If this is deemed something worth doing and if someone is willing to give pointers when I get stuck, I'm offering to put in the development time to implement this. Thoughts? Kurt

4 17

aritmetic with arrays in Cython
by Ian Henriksen Oct. 24, 2014

Oct. 24, 2014

Hi all, I noticed that some time ago there was a pull request ( https://github.com/cython/cython/pull/144) open that was trying to implement basic arithmetic operations with arrays. This seems to have also been proposed in CEP 518 ( https://github.com/cython/cython/wiki/enhancements-simd). What is the status on this? This is a feature I'd really like to see in Cython. I'd be happy to help with it, though I don't currently know enough about the internals of Cython to do it without some guidance. What would be the best way to do this? I've had three ideas thus far that might allow something like this to work, but I'd really like to get some second opinions before I get too far with any of them. 1. Make these array operations available as a part of Cython's memory view objects. 2. Write a Cython wrapper around a NumPy-like C++ library. https://github.com/ContinuumIO/libdynd seems to be a good candidate, though I'm not as familiar with it. (This doesn't strike me as something that would be a part of the main Cython project, but it could be a good way to make this sort of functionality available. It seems like a good way to avoid a lot of duplicate work though.) 3. Write a Fortran backend for Cython that uses Fortran's arrays wherever memory views are used. (This could be a bit crazy, but with the iso_c_binding module, it might be possible. It could also allow better optimizations for arrays to be applied by the compiler.) What would be the best way to go about this? Thanks! -Ian Henriksen

5 20

Travis-ci builds of OSX wheels
by Matthew Brett Oct. 16, 2014

Oct. 16, 2014

Hi, We just got scikit-image OSX wheel builds more or less fully automated on travis-ci : https://travis-ci.org/scikit-image/scikit-image-wheels https://github.com/scikit-image/scikit-image-wheels Any interest in getting the same thing working for Cython? Cheers, Matthew

2 8

Cython 0.21 alpha 1 released
by Stefan Behnel Sept. 8, 2014

Sept. 8, 2014

Hi, I uploaded a first alpha release of Cython 0.21. http://cython.org/release/Cython-0.21a1.tar.gz http://cython.org/release/Cython-0.21a1.tar.gz.asc There were a couple of major changes in this version, including internal improvements to some basic syntax structures, so please give it a try with your code to help us fix some more bugs before the final release. Complete changelog: https://github.com/cython/cython/blob/deb348795f79342843e692358140ea4581985… Have fun, Stefan

2 8

Compiler crash in RemoveUnreachableCode
by Andriy Kornatskyy Sept. 7, 2014

Sept. 7, 2014

The cython compiler crash report below. Steps to reproduce: 1. virtualenv env 2. env/bin/easy_install cython 3. env/bin/easy_install wheezy.http The wheezy.http has dependency on wheezy.core. If I install those two packages separately there is no error, only if through dependencies. Thanks. Andriy Kornatskyy File "Visitor.py", line 178, in Cython.Compiler.Visitor.TreeVisitor._visit (/var/folders/g8/2kym1h8n7qbgrwg4qkfqw1gw0000gn/T/easy_install-JVXbGE/Cython-0.20/Cython/Compiler/Visitor.c:4437) File "Visitor.py", line 137, in Cython.Compiler.Visitor.TreeVisitor._raise_compiler_error (/var/folders/g8/2kym1h8n7qbgrwg4qkfqw1gw0000gn/T/easy_install-JVXbGE/Cython-0.20/Cython/Compiler/Visitor.c:3655) Cython.Compiler.Errors.CompilerCrash: Error compiling Cython file: ------------------------------------------------------------ ... """ """ __version__ = '0.1.129' ^ ------------------------------------------------------------ src/wheezy/core/__init__.py:5:14: Compiler crash in RemoveUnreachableCode ModuleNode.body = StatListNode(__init__.py:5:14) Compiler crash traceback from this point on: File "Visitor.py", line 170, in Cython.Compiler.Visitor.TreeVisitor._visit (/var/folders/g8/2kym1h8n7qbgrwg4qkfqw1gw0000gn/T/easy_install-JVXbGE/Cython-0.20/Cython/Compiler/Visitor.c:4275) File “/.../env/lib/python2.7/site-packages/Cython-0.20-py2.7-macosx-10.9-x86_64.egg/Cython/Compiler/ParseTreeTransforms.py", line 2135, in visit_StatListNode if not self.current_directives['remove_unreachable']: TypeError: 'NoneType' object has no attribute '__getitem__'

2 7

Bug: operator[] does not respect except +
by J Robert Ray Sept. 5, 2014

Sept. 5, 2014

I apologize for not having a working demonstration, but I have found this bug and verified it is still an issue in Cython 0.20.2. No try/catch block is generated for operator[]. In contrast, a try/catch block does get generated for operator(). If it is any use, here is a paraphrasing of the pyx code and the generated cpp code. cdef extern from "blah.h" namespace "blah": cdef cppclass Blah: int operator[](int) except + ... cdef class PyBlah: ... def demo(self, int index): return deref(self.thisptr)[index] ... /* "Blah.pyx":107 * def demo(self, int index): * return deref(self.thisptr)[index] # <<<<<<<<<<<<<< */ __Pyx_XDECREF(__pyx_r); __pyx_t_1 = __Pyx_PyInt_From_int(((*__pyx_v_self->thisptr)[__pyx_v_index])); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 107; __pyx_clineno = __LINE__; goto __pyx_L1_error;} __Pyx_GOTREF(__pyx_t_1); __pyx_r = __pyx_t_1; __pyx_t_1 = 0; goto __pyx_L0; Please let me know if I'm doing anything wrong. Thanks!

2 2

[Desired feature] Process .pxd before .py
by Alexandru Ionut Grama Aug. 28, 2014

Aug. 28, 2014

Hello to all I've start to use cython in order to use call some API modules written in python from C/C++. Those modules written in python must remain untouched, and all .py files should be "decorated" with their counterpart .pxd files. I would like to mark as public some API functions from .py files, following the indications from http://docs.cython.org/src/tutorial/pxd_files.html. I've started to write a little example: suma_alex.py: def suma_Alex(a,b): return a+b suma_alex.pxd: cdef public int suma_Alex(int a,int b) The combination of those two files should turn into something like this for the compiler: cdef public int suma_Alex(int a,int b): return a+b On generated .h file I should obtain a function prototype like this: __PYX_EXTERN_C DL_IMPORT(int) suma_Alex(int, int); Instead of that, I obtain a prototype like this: __PYX_EXTERN_C DL_IMPORT(int) __pyx_f_10suma_alex_suma_Alex(int, int); Digging into code and executing cythonize with pdb, I followed that is executed a pipeline for .py file before .pxd file's pipeline. Correct me if I'm wrong, but this makes the name of .c function be generated according to py file instead of pxd file. That makes an add of string "__pyx_f_10suma_alex_" before function name, because the compiler doesn't know about "public" key (on python doesn't exists). Including public key on a pxd file for this function makes the compiler remove "static" key and generate a .h file, but doesn't change the name prototype into a simple "suma_Alex". If I use cython on a pyx file containing the code below, it works perfectly as expected. cdef public int suma_Alex(int a,int b): return a+b The questions are: - I'm doing what I want to do on a right way? - Is this feature included on cython development version? I've tried on 0.20.2 - If is not implemented, I want to colaborate implementing it. Could you please provide me some guidelines with the methods/modules that I should modify? King regards, Alexandru. -- *---------------------------------------------------------------Alexandru Ionut Grama**email: gramaalexandruionut(a)gmail.com <gramaalexandruionut(a)gmail.com>*

2 2

timeframe for subtype support in freelists? (was: Cython 0.21 beta 1 released)
by Stefan Behnel Aug. 28, 2014

Aug. 28, 2014

Peter Schay schrieb am 19.08.2014 um 00:46: > One question: Is there any expected time-frame for freelists to support > cdef subclasses? Soon? Never? Somewhere in between? Hard to say. This isn't easy as supporting subtypes means that we need full control over what the parent types are doing in their allocation/deallocation chain, and that may not be entirely trivial to determine. The current call chain isn't made for that. In any case, no-one's currently working on this AFAIK. Stefan

1 0

error LNK2001: unresolved external symbol PyInit_init
by Andriy Kornatskyy Aug. 28, 2014

Aug. 28, 2014

Here is an issue when trying to install wheezy.template (it uses cythonize) into environment with cython. Host: windows 7 64bit Python version: 3.4.1 32 bit wheezy.template version: 0.1.151 When installing either from pip3 or downloading the source with python3 setup.py install c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\BIN\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:C:\Python34\libs /LIBPATH:C:\Python34\PCbuild /EXPORT: PyInit_init build\temp.win32-3.4\Release\src\wheezy\template__init.obj /O UT:build\lib.win32-3.4\wheezy\template__init.pyd /IMPLIB:build\temp.win32-3.4 \Release\src\wheezy\template__init.lib /MANIFESTFILE:build\temp.win32-3.4\Rel ease\src\wheezy\template__init.pyd.manifest LINK : error LNK2001: unresolved external symbol PyInit_init build\temp.win32-3.4\Release\src\wheezy\template__init__.lib : fatal error LNK1120: 1 unresolved externals error: command 'c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\BIN\link.exe' failed with exit status 1120 Some details are here: https://bitbucket.org/akorn/wheezy.template/issue/10/installation-on-window… Thanks. Andriy Kornatskyy

4 12