Hello Ian, On 29/06/11 15:16, Ian Ozsvald wrote:

Ian. ps. I posted the v0.1 PDF of my High Performance Python tutorial this morning (it is based on my EuroPython training session). It has a section on PyPy and I'd happily accept input if that section should be expanded:http://ianozsvald.com/2011/06/29/high-performance-python-tutorial-v0-1-from-...

I read your tutorial and tried some of the code, nice work. I tried to use the "bettermath" approach also on the pure python code, and run it on PyPy: on my machine, it takes 2.4 seconds instead of 6.2. For comparison, cython with bettermath takes 0.57 seconds, i.e. it's about 4 times faster than PyPy (with a trunk version of PyPy, did not try with pypy 1.5). For this kind of code, there is no fundamental reason why PyPy should be slower than cython, so I'll investigate a bit to see what is the problem. But still, I think that you should mention this "bettermath" implementation which is much faster on PyPy. ciao, Anto

On Sun, Jul 3, 2011 at 12:03 PM, Ian Ozsvald wrote:

Cool :-) Thanks Antonio. I'd left a note to myself in the report because I figured expanding the math to the primitive operations might help PyPy (seeing as it helped Cython and ShedSkin too). I'll redo the timings and add them to the v0.2 report (probably in two weeks time - I'm waiting for feedback from several people).

I've already tried a trunk version of PyPy and it was faster than PyPy, I'll take whichever build is the most recent and replace PyPy1.5 for the next updates to the report. If there is a trick to e.g. re-ordering the operations so it runs faster in PyPy, I'd be happy to accept a modification.

You can also use numpy arrays (or array from array module) on PyPy to get even more speedups. It's annoying because as of now pypy doesn't support much of numpy (1D float arrays only with ops please :) but it's a good start

Hi Maciej. I said that based on what was discussed in Armin+Antonio's talk at EuroPython. If it takes 6+months for a numpy re-implementation then I'm suggesting that we're into 2012. I'd be *very* happy to be proved wrong! Do Python 'array' objects run faster than lists in PyPy? I believe that in CPython they run at the same speed (i.e. they're just a convenient storage system, they don't offer any of numpy's efficient math benefits). Does the micronumpy library support doubles yet? If so I'd be happy to give it a go. If 'micronumpy' is the wrong name then tell me where I should look, I'm just going on what I've remembered from the PyPy-blog discussions about numpy support. Cheers, Ian. On 3 July 2011 18:38, Maciej Fijalkowski wrote:

On Sun, Jul 3, 2011 at 7:34 PM, Maciej Fijalkowski wrote:

...

On Sun, Jul 3, 2011 at 12:03 PM, Ian Ozsvald wrote:

...

Cool :-) Thanks Antonio. I'd left a note to myself in the report because I figured expanding the math to the primitive operations might help PyPy (seeing as it helped Cython and ShedSkin too). I'll redo the timings and add them to the v0.2 report (probably in two weeks time - I'm waiting for feedback from several people).

I've already tried a trunk version of PyPy and it was faster than PyPy, I'll take whichever build is the most recent and replace PyPy1.5 for the next updates to the report. If there is a trick to e.g. re-ordering the operations so it runs faster in PyPy, I'd be happy to accept a modification.

You can also use numpy arrays (or array from array module) on PyPy to get even more speedups. It's annoying because as of now pypy doesn't support much of numpy (1D float arrays only with ops please :) but it's a good start

"During 2011 at least it looks as though numpy integration will not happen." I personally challenge this statement :)

-- Ian Ozsvald (A.I. researcher, screencaster) ian@IanOzsvald.com http://IanOzsvald.com http://SocialTiesApp.com/ http://MorConsulting.com/ http://blog.AICookbook.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald

Re. micronumpy - all I'm doing in the Mandelbrot demo is multiplying and addition on doubles - that'll work now, right? If so I'll make the modification in a week or so when I next have some time. I'm guessing that there is no 'complex' support yet, even for basic operations? Adding micronumpy support would make for a nice addition to the v0.2 doc :-) Ian. re. the 6 month 'proper numpy' estimate - I'm just quoting some of the figures that were thrown around (with wide error margins depending on money raised, availability etc). I would *love* to see support come through sooner! On 3 July 2011 19:54, Maciej Fijalkowski wrote:

On Sun, Jul 3, 2011 at 8:35 PM, Ian Ozsvald wrote:

...

Hi Maciej. I said that based on what was discussed in Armin+Antonio's talk at EuroPython. If it takes 6+months for a numpy re-implementation then I'm suggesting that we're into 2012. I'd be *very* happy to be proved wrong!

I would estimate numpy for less than 6 months (numpy, not scipy and not matplotlib), but well that's me. There is also a lot of outside contribution, so it might take less wall time than manmonth time. From my perspective, if we can secure *some* funding for numpy, it should be ready sooner than 6 months from now.

...

Do Python 'array' objects run faster than lists in PyPy? I believe that in CPython they run at the same speed (i.e. they're just a convenient storage system, they don't offer any of numpy's efficient math benefits).

The situation between PyPy and CPython is quite different here. As of now, both PyPy and CPython store wrapped objects in lists (let's call them PyIntObject) and unwrapped (C level int) in numpy arrays. In case you have only interpreter, when reading a list you do:

py_x = list[index] do_something_with_py_x

when reading from numpy array (or array.array), you do:

py_x = new PyIntObject(array[index]) do_something_with_py_x

so you use less memory (cache is better), but you allocate a new object each iteration (bad)

But in case of the JIT, what happens is (list case)

py_x = list[index] x = py_x.value do_something_with_x # x as in integer value not py_x

array:

x = list[index] do_something_with_x

Hence, no allocation and memory saving - huge win!

...

Does the micronumpy library support doubles yet? If so I'd be happy to give it a go. If 'micronumpy' is the wrong name then tell me where I should look, I'm just going on what I've remembered from the PyPy-blog discussions about numpy support.

yes, doubles is pretty much all it supports as of now :) micronumpy is a good name, but it comes under a name "numpy", so you can do "import numpy". Not much will work though :-)

Note that we're also in the process of implementing faster vectorized operations, like numpyexpr, except better. The list of operations is very limited as of now though.

Cheers, fijal

PS. If it's unclear, I can try to explain more or pop up on IRC.

-- Ian Ozsvald (A.I. researcher, screencaster) ian@IanOzsvald.com http://IanOzsvald.com http://SocialTiesApp.com/ http://MorConsulting.com/ http://blog.AICookbook.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald

I'll re-open this earlier thread...Didrik Pinte of Enthought is offering to match my suggested £600 donation towards a numpy-pypy project. He's offering it in a personal capacity (i.e. not as an Enthought activity) out of his own pocket (and my donation would come out of my Consultancy). His offer came out of a discussion on pypy's merits at the London Financial Python Usergroup meet a few weeks back. So, you've got £1,200 via two individuals ready as a gift towards numpy integration, as/when someone can use the money. I won't promise that my gift will always be available as my personal situation may be changing in the next 6 weeks (so please - if you can use it - ask for it soon!). Regards, Ian (UK) On 29 June 2011 14:16, Ian Ozsvald wrote:

I'm glad this thread is up. Laura - I'm the chap from Armin's talk who offered a monthly retainer for a year towards numpy integration (in my mind I'm offering £50/month for 12 months). I spoke to you later and you mentioned the flattr site but having to do it each month is a bit of a pain (I know it is simple but I don't want to think about it...).

So, for the record, I have £600 sitting here with someone's name on it, I'll account for it as a marketing expense or something out of my company. I'm a one man consultancy, PyPy doesn't directly help me as I'm an A.I./science researcher (so I need numpy, IPython, matplotlib etc) but I believe strongly that it will help all of Python (and me in part) over time, so it is worth pledging some of my earnings towards the goal of eventual numpy integration.

If I can pledge it to someone or a project then that's cool, if I should just move the money to someone's account then that's cool too. I'm quite happy to have my name down as ContributorNumber1ForNumpy if it helps you spread the word.

Ian. ps. I posted the v0.1 PDF of my High Performance Python tutorial this morning (it is based on my EuroPython training session). It has a section on PyPy and I'd happily accept input if that section should be expanded: http://ianozsvald.com/2011/06/29/high-performance-pyethon-tutorial-v0-1-from...

On 29 June 2011 08:53, Laura Creighton wrote:

...

...

The idea was also to possibly attract new developers ... for example, if there would be "10 days in money" for adapting py2exe, I am sure many wou ld jump to solve this puzzle.

This is sort of a bad example.  Because py2exe embeds CPython, and we wouldn't want to do that.  So what we would probably want to do is to make some general tool that willmake a windows binary, or a mac one, and get rid of the need for bzfreeze and friends.  So now you are looking at a general embedding solution, and that is more than 10 days worth of work.

But I get  the idea.

<snip>

...

...

my dream was of a trustee service: after somebody commits to do the wor k, the pledgers have to pay to a trustee. then the work is done. then the trustee pays the worker.

This is one of the things I want to talk with fundedbyme about.  But having an explicit trustee is a new idea.  I think the pypy core developers are already rather well trusted in this community, but this may be important to new developers who aren't as well known. And it handles the problem' of 'I got sick and cannot do this  any more' more gracefully than other solutions.

...

Hmmm.... a structure could be:

- service provider does the technical stuff, as in:   # website   # collect pledges   # handle project description   # collect money   # distribute money after feature completion

fundedbyme has sort of indicated an interst in doing this (except they were talking about distribution before, and I was leaving project description to the project, not outsiders).  I will follow up on this when I get back home to Sweden.

...

- PSF / pypy-foundation / whateverfoundation provides the trust

Thanks for confirming the need for such a thing!

Harald

Thanks once again for seeing a marketing solution that nerds like us often miss.

Laura _______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev

-- Ian Ozsvald (A.I. researcher, screencaster) ian@IanOzsvald.com

http://IanOzsvald.com http://SocialTiesApp.com/ http://MorConsulting.com/ http://blog.AICookbook.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald

-- Ian Ozsvald (A.I. researcher, screencaster) ian@IanOzsvald.com http://IanOzsvald.com http://SocialTiesApp.com/ http://MorConsulting.com/ http://blog.AICookbook.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald

On Thu, Sep 15, 2011 at 1:59 PM, Lionel Barret De Nazaris wrote:

Add €200 to that.

Where do I pay ?

regards, -- Best regards, Lionel Barret de Nazaris Gamr7 - CEO

Hi We'll set up a way to donate to PyPy with certain proposal (like numpy) in mind really soon. Cheers, fijal