Hey, On Thu, Feb 28, 2008 at 9:17 AM, Robert Kern wrote:

On Thu, Feb 28, 2008 at 7:35 AM, Neal Becker wrote:

...

I tried numpyx.pyx with cython-0.9.6.12.

These were written for and still work with Pyrex. If it doesn't work with Cython then that is either a bug in Cython or an intentional incompatibility of Cython.

I just updated all this code to run with cython here: https://code.launchpad.net/~fdo.perez/+junk/numpy-cython Do we want to just switch over to cython and use this instead? If people think that the cython switch is OK for 1.0.5 I can do it, but I'm not about to touch this before others approve... Cheers, f

On Wed, Apr 09, 2008 at 01:15:23AM +0200, Stéfan van der Walt wrote:

I'm in favour of switching -- pyrex is as good as unmaintained, and in contrast cython is improving by the day. Do the changes cause problems with pyrex, or can we even have both working at the same time? Either way, we should add a note to the docstring.

How about waiting for after 1.0.5 release. It should be anytime soon now, and let us not risk breaking the workflow of potential contributors for little gain. I have little to say on the subject, as I am not helping to get the release out, but here are my 2 cents. Gaël

On Tue, Apr 08, 2008 at 04:20:28PM -0700, Robert Kern wrote:

I'm not sure how it could. It's example code, not part of numpy itself.

OK, maybe I should keep my 2 cents. They appear to be forged money, and worth nothing. :-). Gaël

On Tue, Apr 08, 2008 at 04:28:11PM -0700, Fernando Perez wrote:

I *could* make it pyrex/cython-valid, it's trivial but just adds noise IMHO... As Stefan said, pyrex is essentially unmaintained as far as publicly-visible development goes, while cython is very actively moving ahead and likely picking up better numpy support soon (thanks to Dag and other GSoC work), so why not just follow that?

Because Cython is not yet shipped be all the major distros (including not Enthought's EPD current Beta, AFAIK), so it rules out a fair amount of people who cannot/do not want to install something not shipped. Note that I don't care that much amount this issue. I am just explaining why I think it should be after 1.0.5 ships. Cheers, Gaël

Fernando Perez wrote:

On Tue, Apr 8, 2008 at 4:21 PM, Gael Varoquaux wrote:

...

On Tue, Apr 08, 2008 at 04:20:28PM -0700, Robert Kern wrote:

...

I'm not sure how it could. It's example code, not part of numpy itself.

OK, maybe I should keep my 2 cents. They appear to be forged money, and worth nothing.

:-).

I *could* make it pyrex/cython-valid, it's trivial but just adds noise IMHO... As Stefan said, pyrex is essentially unmaintained as far as publicly-visible development goes, while cython is very actively moving ahead and likely picking up better numpy support soon (thanks to Dag and other GSoC work), so why not just follow that?

I say just add it. We should move forward with Cython. More important is to see if random actually builds with Cython right now. There was an issue that I recall from a few weeks ago that Cython could not build the pyrex extension in NumPy. -Travis

This is off-topic and should be directed to the pyrex/cython list, but since we're on the subject: I suppose the following is true, but let me ask, since I have not used Cython. Please correct me if I'm wrong. I have a bunch of pyrex compiled .pyx code. If I start adding some Cython compiled code, all the extensions will live happily without colliding, even if I don't re-compile my old .pyx files with Cython. (I can't imagine how they'd conflict, since they both translate to .c extensions without anything to collide. Or are there linker-level symbols that collide or something difficult?) -Andrew Fernando Perez wrote:

On Tue, Apr 8, 2008 at 4:38 PM, Travis E. Oliphant wrote:

...

I say just add it. We should move forward with Cython. More important is to see if random actually builds with Cython right now. There was an issue that I recall from a few weeks ago that Cython could not build the pyrex extension in NumPy.

OK, I'll play with random for a bit.

BTW, the original 'bug' that started this thread is due to a change in Cython's casting behavior explained here:

http://wiki.cython.org/DifferencesFromPyrex

it's fixed with a simple extra (void *) cast as shown here:

print 'Printing array info for ndarray at 0x%0lx'% \ (arr,)

I just committed that code to the bzr branch.

In summary:

1. Do you want me to obliterate the old numpy/doc/pyrex and replace it with numpy/doc/cython? That would make it clear what the tool to use is...

2. I'll work on random and report shortly.

Cheers,

f _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion

A Wednesday 09 April 2008, Andrew Straw escrigué:

This is off-topic and should be directed to the pyrex/cython list, but since we're on the subject:

I suppose the following is true, but let me ask, since I have not used Cython. Please correct me if I'm wrong.

I have a bunch of pyrex compiled .pyx code. If I start adding some Cython compiled code, all the extensions will live happily without colliding, even if I don't re-compile my old .pyx files with Cython.

(I can't imagine how they'd conflict, since they both translate to .c extensions without anything to collide. Or are there linker-level symbols that collide or something difficult?)

I don't expect you having problems in that regard either. However, I've been having problems compiling perfectly valid Pyrex code with the Cython compiler. I just haven't had time to locate where the problem is and report it, but people using Pyrex and planning to migrate to Cython must be aware that these sort of things happen. Cheers, --

0,0< Francesc Altet     http://www.carabos.com/ V V Cárabos Coop. V.   Enjoy Data "-"

On Wed, Apr 9, 2008 at 12:25 AM, Francesc Altet wrote:

I don't expect you having problems in that regard either. However, I've been having problems compiling perfectly valid Pyrex code with the Cython compiler. I just haven't had time to locate where the problem is and report it, but people using Pyrex and planning to migrate to Cython must be aware that these sort of things happen.

It would be great to have some examples of this, so we can better understand whether they are bugs in cython or deliberate changes in behavior. There are some of the latter, described here: http://wiki.cython.org/DifferencesFromPyrex and in fact one of those already bit us in the tiny numpy/pyrex example, as reported originally in this thread. If something more serious arises we certainly need to know about it, but I know the cython devs are a pretty responsive bunch, so I hope we'll have a good chance of clean resolutions for those. Considering how pyrex has essentially no public development process to speak of, I find it a worrying tool to commit to for the long run, while cython is very actively developed, which is always a bonus. But I'm not trying to ram anything down people's throats, which is why I didn't even want to put in the mtrand.c changes before there was more feedback (even as a self-contained single commit). Here's the patch that commit would contain, against current SVN, so others can test it without putting anything into SVN itself: http://amath.colorado.edu/faculty/fperez/tmp/mtrand-cython.patch.gz I'll be happy to leave it as a new trac ticket with an attachment for it if it will make the testing/review process easier in the long run. Cheers, f

On Wed, Apr 9, 2008 at 12:49 AM, Robert Kern wrote:

On Wed, Apr 9, 2008 at 2:47 AM, Fernando Perez wrote:

...

I'll be happy to leave it as a new trac ticket with an attachment for it if it will make the testing/review process easier in the long run.

Please do. I don't want to change mtrand over until after 1.0.5 is released.

http://scipy.org/scipy/numpy/ticket/727 I assigned it to you for now, since I figured you'd have the most to say on the fate of mtrand. Cheers, f

On 09/04/2008, Francesc Altet wrote:

Well, I agree that Greg Ewing (the Pyrex creator) has possibly not been very speedy in adding the suggested patches (Greg has his own thoughts on what should be added to Pyrex and what not), which ultimately brought to the need of the Cython fork, but let me stress out that he has always been *very* responsive to the questions on the Pyrex list, and quick enough in fixing real problems in Pyrex. I'm personally very satisfied with Pyrex as does its job extremely well. And I'm specially grateful to Greg for his *huge* contribution to make the job of doing Python extensions a pretty simple job.

So, I don't really think that Pyrex should be considered a "worrying tool" at all (even in the "long run"), rather the contrary, it is a extremely useful tool.

Your first paragraph above served to convince me that this is indeed a worrying tool. a) The author is not quick (or willing?) to add patches b) The author wants to decide what goes in and what not (contrary to the community) (Sounds a bit like g77 vs gfortran.) Pyrex is therefore pretty much in maintenance mode. Cython has added some vast improvements; amongs other things - List comprehension - For i in range (instead of the hacked for i from ...) - Introspection of source into .pyx file I do agree with you that Pyrex is very useful, but Cython does exactly the same thing, and more, with the additional bonuses that it is being actively developed, and that the SAGE guys like NumPy :) Cheers Stéfan

On Wed, Apr 9, 2008 at 1:22 AM, Francesc Altet wrote:

Well, I agree that Greg Ewing (the Pyrex creator) has possibly not been very speedy in adding the suggested patches (Greg has his own thoughts on what should be added to Pyrex and what not), which ultimately brought to the need of the Cython fork, but let me stress out that he has always been *very* responsive to the questions on the Pyrex list, and quick enough in fixing real problems in Pyrex. I'm personally very satisfied with Pyrex as does its job extremely well. And I'm specially grateful to Greg for his *huge* contribution to make the job of doing Python extensions a pretty simple job.

So, I don't really think that Pyrex should be considered a "worrying tool" at all (even in the "long run"), rather the contrary, it is a extremely useful tool. Having said that, I'm all for pushing Cython forward (specially if the integration with NumPy becomes a reality :).

I certainly didn't mean to bash the developer, nor to diminish in any way his contribution. Obviously cython wouldn't exist in the first place if it weren't thanks to pyrex :) But for an open source project, in this day and age, to have this as its only public source of code history: http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/hg/ isn't really great IMO: public updates from 4 months ago? That's why I feel better about a project with an open development model as a core tool to depend on. Ultimately each developer can choose to run his personal project in any way he chooses to, but other projects are also free to choose what bases they build upon :) Cheers, f

On Tue, Apr 8, 2008 at 4:55 PM, Fernando Perez wrote:

2. I'll work on random and report shortly.

As far as I can tell, there are no problems at all. I'm running bic128[scipy]$ cython --version Cython version 0.9.6.12 This is both on 64-bit Fedora 8 and 32-bit Ubuntu 7.10. I made sure I was actually running off the newly generated C code written out by cython instead of the old pyrex one. If all agree, we can push the C sources and the generate_mtrand script as a single commit where I don't touch *anything* else. That way the buildbots can run the tests, and if that C commit changes anything, then it's easy to back off. I scanned the sources for a while, and the only other file that has significant amounts of pyrex mention is build_src.py: http://projects.scipy.org/scipy/numpy/browser/trunk/numpy/distutils/command/... I'd rather not touch that file yet, since distutils isn't exactly my favorite code to mess with :) So I'd like to know whether people would want to see this one isolated commit to happen, for the buildbots to test things out on other platforms or not. I can also make the patch available somewhere (it's huge, since it involves the auto-generated mtrand.c file) if you all prefer... Cheers, f