Mailman 3 February 2012 - cython-devel

jenkins is down
by Vitja Makarov Feb. 25, 2012

Feb. 25, 2012

Jenkins app is down now. Before it got down it was raising exceptions. Can some one restart jenkins? -- vitja.

2 3

Broken C-API generation for ext modules
by Lisandro Dalcin Feb. 24, 2012

Feb. 24, 2012

The commit below from Stefan broke C-API generation for extension modules. The problem is that the code of __Pyx_ImportModule() and __Pyx_ImportType() depeds on the #definition of __Pyx_PyIdentifier_FromString , and such #define is not emitted in C-API headers. Stefan, given that __Pyx_PyIdentifier_FromString() is only used in __Pyx_ImportModule() and __Pyx_ImportType(), what's your opinion about just rolling back your commit? commit 500f9a40f5ad441c2c204d076cfc4f82a41d531b Author: Stefan Behnel <scoder(a)users.berlios.de> Date: Sat Sep 10 00:14:05 2011 +0200 minor code simplification in utility functions by using macro for Py2/3 dependent PyString/PyUnicode_FromString() calls -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169

2 2

p3k pyregr tests problem
by Vitja Makarov Feb. 22, 2012

Feb. 22, 2012

Hi! I've noticed problems with py3k pyregr tests now it shows ~8K tests instead of 13K Is that related to changes in cython or python? https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests-pyregr/… -- vitja.

2 7

star-imports in Cython/Includes/cpython considered harmful
by Stefan Behnel Feb. 22, 2012

Feb. 22, 2012

Hi, I just noticed that the star-imports in the cpython package have serious side effects. The cpython/__init__.pxd file has this in it: """ from cpython.version cimport * from cpython.ref cimport * from cpython.exc cimport * from cpython.module cimport * from cpython.mem cimport * ... """ This means that a direct cimport of any of those recursively cimported names from the cpython package namespace will always trigger a complete cimport of all cpython.* modules, thus 1) wasting compile time 2) polluting the cpython package namespace 3) polluting the internal state of Cython's list of cimported modules 4) leading to things being defined in the generated C code that don't need to be there, specifically the object structs of PyBoolObject and PyComplexObject, which we provide definitions for in cpython.bool and cpython.complex. The point where I noticed this is when I got test errors in PyPy because it doesn't expose the bool/complex structs. That wasn't because the test actually used them, it was purely because it had this cimport in it: from cpython cimport PyObject If it had used this instead: from cpython.object cimport PyObject Cython would still have parsed all of those .pxd files, but at least it wouldn't have used those declarations to do any harm. Now, I'm 100% certain that there is user code out there that does this as well. Fixing this bug by removing all the star-imports will break that code, but keeping the cimports in the package will trap the average lazy user into cimporting stuff from there directly. I'll add a big warning to the package for now, but I wonder if we shouldn't accept breaking this code at some point (and sooner is always better than later). Stefan

2 2

Fwd: Re: [Numpy-discussion] Proposed Roadmap Overview
by Dag Sverre Seljebotn Feb. 21, 2012

Feb. 21, 2012

This has got to be the most incredible and interesting turn of events I've seen in a while! :-) Dag -------- Original Message -------- Subject: Re: [Numpy-discussion] Proposed Roadmap Overview Date: Mon, 20 Feb 2012 22:04:00 -0600 From: Travis Oliphant <travis(a)continuum.io> Reply-To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> Interesting you bring this up. I actually have a working prototype of using Python to emit LLVM. I will be showing it at the HPC tutorial that I am giving at PyCon. I will be making this available after PyCon to a wider audience as open source. It uses llvm-py (modified to work with LLVM 3.0) and code I wrote to do the translation from Python byte-code to LLVM. This LLVM can then be "JIT"ed. I have several applications that I would like to use this for. It would be possible to write "more of NumPy" using this approach. Initially, it makes it *very* easy to create a machine-code ufunc from Python code. There are other use-cases of having loops written in Python and plugged in to a calculation, filtering, or indexing framework that this system will be useful for. There is still a need for a core data-type object, a core array object, and a core calculation object. Maybe some-day these cores can be shrunk to a smaller subset and more of something along the lines of LLVM generation from Python can be used. But, there is a lot of work to do before that is possible. But, a lot of the currently pre-compiled loops can be done on the fly instead using this approach. There are several things I'm working on in that direction. This is not PyPy. It certainly uses the same ideas that they are using, but instead it fits into the CPython run-time and doesn't require changing the whole ecosystem. If you are interested in this work let me know. I think I'm going to call the project numpy-llvm, or fast-py, or something like that. It is available on github and will be open source (but it's still under active development). Here is an example of the code to create a ufunc using the system (this is like vectorize, but it creates machine code and by-passes the interpreter and so is 100x faster). from math import sin, pi def sinc(x): if x==0: return 1.0 else: return sin(x*pi)/(pi*x) from translate import Translate t = Translate(sinc) t.translate() print t.mod res = t.make_ufunc('sinc') -Travis On Feb 20, 2012, at 10:55 AM, Sturla Molden wrote: > Den 20.02.2012 17:42, skrev Sturla Molden: >> There are still other options than C or C++ that are worth considering. >> One would be to write NumPy in Python. E.g. we could use LLVM as a >> JIT-compiler and produce the performance critical code we need on the >> fly. >> >> > > LLVM and its C/C++ frontend Clang are BSD licenced. It compiles faster > than GCC and often produces better machine code. They can therefore be > used inside an array library. It would give a faster NumPy, and we could > keep most of it in Python. > > Sturla > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org <mailto:NumPy-Discussion@scipy.org> > http://mail.scipy.org/mailman/listinfo/numpy-discussion

4 5

Cython Library
by mark florisson Feb. 16, 2012

Feb. 16, 2012

Hey, I created a CEP for the optional ability to have Cython create a version-dependent library to share types and common utility functions internal to the compiler, you can find it here: http://wiki.cython.org/enhancements/libcython . It also addresses entry caching, both for pxd files and Cython utility codes. Mark

1 0

Bringing Cython and PyPy closer together
by Stefan Behnel Feb. 15, 2012

Feb. 15, 2012

Hi, following up on the thread on the PyPy mailing list where this topic was started, I've started a CEP in our Wiki in order to focus the different ideas and opinions. http://wiki.cython.org/enhancements/pypy The current state of the discussion seems to be that PyPy provides ways to talk to C code, but nothing as complete as CPython's C-API in the sense that it allows efficient two-way communication between C code and Python objects. Thus, we need to either improve this or look for alternatives. Please add to the CEP as you see fit. Stefan

1 0

overwriting pull requests and the related discussion
by Stefan Behnel Feb. 15, 2012

Feb. 15, 2012

Hi, I'd like to suggest that instead of overwriting pull requests and all of their comments on github by pushing replaced commits over them, it would be better to keep any existing discussions accessible by rejecting the current pull request and creating a new one. Does that make sense for everyone? Stefan

3 2

Cython compatibility thread on PyPy mailing list
by Stefan Behnel Feb. 14, 2012

Feb. 14, 2012

Hi, just wanted to point you to a thread that currently runs on the PyPy-dev mailing list about how Cython and PyPy could improve their interoperability. http://thread.gmane.org/gmane.comp.python.pypy/9437/focus=9452 Stefan

1 0

Discussion with Guido van Rossum and (hopefully) core python-dev on scientific Python and Python3
by Fernando Perez Feb. 13, 2012

Feb. 13, 2012

Hi folks, [ I'm broadcasting this widely for maximum reach, but I'd appreciate it if replies can be kept to the *numpy* list, which is sort of the 'base' list for scientific/numerical work. It will make it much easier to organize a coherent set of notes later on. Apology if you're subscribed to all and get it 10 times. ] As part of the PyData workshop (http://pydataworkshop.eventbrite.com) to be held March 2 and 3 at the Mountain View Google offices, we have scheduled a session for an open discussion with Guido van Rossum and hopefully as many core python-dev members who can make it. We wanted to seize the combined opportunity of the PyData workshop bringing a number of 'scipy people' to Google with the timeline for Python 3.3, the first release after the Python language moratorium, being within sight: http://www.python.org/dev/peps/pep-0398. While a number of scientific Python packages are already available for Python 3 (either in released form or in their master git branches), it's fair to say that there hasn't been a major transition of the scientific community to Python3. Since there is no more development being done on the Python2 series, eventually we will all want to find ways to make this transition, and we think that this is an excellent time to engage the core python development team and consider ideas that would make Python3 generally a more appealing language for scientific work. Guido has made it clear that he doesn't speak for the day-to-day development of Python anymore, so we all should be aware that any ideas that come out of this panel will still need to be discussed with python-dev itself via standard mechanisms before anything is implemented. Nonetheless, the opportunity for a solid face-to-face dialog for brainstorming was too good to pass up. The purpose of this email is then to solicit, from all of our community, ideas for this discussion. In a week or so we'll need to summarize the main points brought up here and make a more concrete agenda out of it; I will also post a summary of the meeting afterwards here. Anything is a valid topic, some points just to get the conversation started: - Extra operators/PEP 225. Here's a summary from the last time we went over this, years ago at Scipy 2008: http://mail.scipy.org/pipermail/numpy-discussion/2008-October/038234.html, and the current status of the document we wrote about it is here: file:///home/fperez/www/site/_build/html/py4science/numpy-pep225/numpy-pep225.html. - Improved syntax/support for rationals or decimal literals? While Python now has both decimals (http://docs.python.org/library/decimal.html) and rationals (http://docs.python.org/library/fractions.html), they're quite clunky to use because they require full constructor calls. Guido has mentioned in previous discussions toying with ideas about support for different kinds of numeric literals... - Using the numpy docstring standard python-wide, and thus having python improve the pathetic state of the stdlib's docstrings? This is an area where our community is light years ahead of the standard library, but we'd all benefit from Python itself improving on this front. I'm toying with the idea of giving a lighting talk at PyConn about this, comparing the great, robust culture and tools of good docstrings across the Scipy ecosystem with the sad, sad state of docstrings in the stdlib. It might spur some movement on that front from the stdlib authors, esp. if the core python-dev team realizes the value and benefit it can bring (at relatively low cost, given how most of the information does exist, it's just in the wrong places). But more importantly for us, if there was truly a universal standard for high-quality docstrings across Python projects, building good documentation/help machinery would be a lot easier, as we'd know what to expect and search for (such as rendering them nicely in the ipython notebook, providing high-quality cross-project help search, etc). - Literal syntax for arrays? Sage has been floating a discussion about a literal matrix syntax (https://groups.google.com/forum/#!topic/sage-devel/mzwepqZBHnA). For something like this to go into python in any meaningful way there would have to be core multidimensional arrays in the language, but perhaps it's time to think about a piece of the numpy array itself into Python? This is one of the more 'out there' ideas, but after all, that's the point of a discussion like this, especially considering we'll have both Travis and Guido in one room. - Other syntactic sugar? Sage has "a..b" <=> range(a, b+1), which I actually think is both nice and useful... There's also the question of allowing "a:b:c" notation outside of [], which has come up a few times in conversation over the last few years. Others? - The packaging quagmire? This continues to be a problem, though python3 does have new improvements to distutils. I'm not really up to speed on the situation, to be frank. If we want to bring this up, someone will have to provide a solid reference or volunteer to do it in person. - etc... I'm putting the above just to *start* the discussion, but the real point is for the rest of the community to contribute ideas, so don't be shy. Final note: while I am here commiting to organizing and presenting this at the discussion with Guido (as well as contacting python-dev), I would greatly appreciate help with the task of summarizing this prior to the meeting as I'm pretty badly swamped in the run-in to pydata/pycon. So if anyone is willing to help draft the summary as the date draws closer (we can put it up on a github wiki, gist, whatever), I will be very grateful. I'm sure it will be better than what I'll otherwise do the last night at 2am :) Cheers, f ps - to the obvious question about webcasting the discussion live for remote participation: yes, we looked into it already; no, unfortunately it appears it won't be possible. We'll try to at least have the audio recorded (and possibly video) for posting later on. pps- if you are close to Mountain View and are interested in attending this panel in person, drop me a line at fernando.perez(a)berkeley.edu. We have a few spots available *for this discussion only* on top of the pydata regular attendance (which is long closed, I'm afraid). But we'll need to provide Google with a list of those attendees in advance. Please indicate if you are a core python committer in your email, as we'll give priority for this overflow pool to core python developers (but will otherwise accommodate as many people as Google lets us).

1 0