Mailman 3 March 2012 - cython-devel

Re: [Cython] Bringing Cython and PyPy closer together
by Stefan Behnel March 29, 2015

March 29, 2015

[copied here from PyPy mailing list] Stefan Behnel, 15.02.2012 12:32: > The current state of the discussion seems to be that PyPy provides ways to > talk to C code, but nothing as complete as CPython's C-API in the sense > that it allows efficient two-way communication between C code and Python > objects. Thus, we need to either improve this or look for alternatives. > > In order to get us more focussed on what can be done and what the > implications are, so that we may eventually be able to decide what should > be done, I started a Wiki page for a PyPy backend CEP (Cython Enhancement > Proposal). > > http://wiki.cython.org/enhancements/pypy The discussion so far makes me rather certain that the most promising short-term solution is to make Cython generate C code that PyPy's cpyext can handle. This should get us a rather broad set of running code somewhat quickly, while requiring the least design-from-scratch type of work in a direction that does not yet allow us to see if it will really make existing code work or not. On top of the basic cpyext interface, it should then be easy to implement obvious optimisations like native C level calls to Cython wrapped functions from PyPy (and potentially also the other direction) and otherwise avoid boxing/unboxing where unnecessary, e.g. for builtins. After all, it all boils down to native code at some point and I'm sure there are various ways to exploit that. Also, going this route will help both projects to get to know each other better. I think that's a required basis if we really aim for designing a more high-level interface at some point. The first steps I see are: - get Cython's test suite to run on PyPy - analyse the failing tests and decide how to fix them - adapt the Cython generated C code accordingly, special casing for PyPy where required Here is a "getting started" guide that tells you how testing works in Cython: http://wiki.cython.org/HackerGuide Once we have the test suite runnable, we can set up a PyPy instance on our CI server to get feed-back on any advances. https://sage.math.washington.edu:8091/hudson/ So, any volunteers or otherwise interested parties to help in getting this to work? Anyone in for financial support? Stefan

2 9

dynamically compile and import pure python modules
by mark florisson April 29, 2012

April 29, 2012

Hey, For the fused type runtime dispatch I found it very convenient to use the with statement, but that is not supported in Python 2.4. However, the compiler could dynamically compile compiler code with the compiler itself and import it (pyximport), if it is not needed to compile Cython itself. I gave it a try and it seems to work like a charm (but probably needs more testing :): https://github.com/markflorisson88/cython/commit/0c2983056919f7f4d30a809724… It seems like a bit of a hack, so I thought I'd bring it up here and ask for opinions, i.e. ask if anyone thinks it is a terrible idea. Mark

3 12

NumPy dependency in Jenkins builds
by Stefan Behnel April 12, 2012

April 12, 2012

mark florisson, 23.03.2012 14:26: > This may be OT for this thread ... in which case it's quite common to start a new one ... > but sas numpy removed at some point from Jenkins? I'm seeing this for > all python versions since Februari 25: > > Following tests excluded because of missing dependencies on your > system: > run.memoryviewattrs > run.numpy_ValueError_T172 > run.numpy_bufacc_T155 > run.numpy_cimport > run.numpy_memoryview > run.numpy_parallel > run.numpy_test > ALL DONE May be my fault. I think when I unified the build jobs, I might have disabled it because we didn't have a NumPy version for Py3 at the time. I'll look into it. Stefan

2 5

GSoC 2012
by Stefan Behnel April 2, 2012

April 2, 2012

Hi, I noticed that people start rushing for the next season on Python's GSoC mailing lists. Do we have any interested developers here, or general ideas about suitable topics? I would expect that we'll do as in the last years and participate under Python's umbrella. Please reply on the cython-devel mailing list. Stefan

6 13

Cython 0.16 Release Candidate
by mark florisson April 1, 2012

April 1, 2012

We are happy to announce the first release candidate for Cython 0.16, you can grab it from here: http://wiki.cython.org/ReleaseNotes-0.16 Since the beta last month several issues have been fixed, including the NumPy array attribute access deprecation, which are now rewritten to use the NumPy macros. Another serious bug in the reversed(range()) optimization was disabled, a C compiler bug on OS X Lion in OpenMP code was circumvented, the documentation for fused types has been improved, some bugs related to fused C++ and buffers types have been fixed, and nogil was added to the included STL declarations. I encourage everyone to give it a try, see if your project still compiles and works, and if it doesn't to take up those torches and pitchforks from your desks, you know where to find us. Enjoy!

2 1

0.16 release
by mark florisson March 31, 2012

March 31, 2012

Hey, It's been almost three months since we talked about a 0.16 release, I think it's quite ready. It would already be a big release, it would be good to see how people like it, and to catch any issues etc before we pile on more features. Mark

7 46

weird code in argument unpacking (memoryview related?)
by Stefan Behnel March 30, 2012

March 30, 2012

Hi, this code in Nodes.py around line 3950, at the end of the DefNodeWrapper's method generate_keyword_unpacking_code(), was added as part of the memory view changes, back in July last year: """ # convert arg values to their final type and assign them for i, arg in enumerate(all_args): if arg.default and not arg.type.is_pyobject: code.putln("if (values[%d]) {" % i) if arg.default and not arg.type.is_pyobject: code.putln('} else {') code.putln( "%s = %s;" % ( arg.entry.cname, arg.calculate_default_value_code(code))) if arg.type.is_memoryviewslice: code.put_incref_memoryviewslice(arg.entry.cname, have_gil=True) code.putln('}') """ By being overly complicated, it hides rather well what it's actually meant to achieve. It doesn't do at all what its comment says and also refers to the default value of the argument, which is already assigned way before this place in the code. Therefore, I'd be surprised if it generated anything but dead C code, because the case that "values[i]" is NULL should never happen. Mark, could you comment on this? Is this just a left-over from a broken merge or something? Stefan

2 2

Gsoc project
by Philip Herron March 29, 2012

March 29, 2012

Hey I got linked to your idea http://groups.google.com/group/cython-users/browse_thread/thread/cb8aa58083… by David Malcolm on his plugin mailing list. I am looking to apply to Gsoc once again this year i have done gsoc 2010 and 2011 on GCC implementing my own GCC front-end for python which is still in very early stages since its a huge task. But i am tempted to apply to this project to implement a more self contained project to give back to the community more promptly while that hacking on my own front-end on my own timer. And i think it would benefit me to get to understand in more detail different aspects of python which is what i need and would gain very much experience from. I was wondering if you could give me some more details on how this could all work i am not 100% familiar with cython but i think i understand it to a good extend from playing with it for most of my evening. I just want to make sure i understand the basic use case of this fully, When a user could have something like: -header foo.h extern int add (int, int); -source foo.c #include "foo.h" int add (int x, int y) { return x+y; } We use the plugin to go over the decls created and create a pxd file like: cdef int add (int a, int b): return a + b Although this is a really basic example i just want to make sure i understand whats going on. Maybe some more of you have input? I guess this would be best suited as a proposal for Python rather than GCC? --Phil

8 13

Question on "numpy_common.pxi" in NumPy tests
by Stefan Behnel March 29, 2012

March 29, 2012

Hi, the NumPy related tests use a file "numpy_common.pxi" that contains this useless code: """ cdef extern from *: bint FALSE "0" void import_array() void import_umath() if FALSE: import_array() import_umath() """ Does this serve any purpose? It currently leads to build failures of the tests in C++ mode because the "import_umath()" call seems to inline code with a bare "return" statement, which is not allowed in the module init function (which returns a reference to the module). Is there any reason why this can't just die? Stefan

3 7

test
by Oleksandr Kreshchenko March 28, 2012

March 28, 2012

1 0