Re: [pypy-dev] bounties for pypy
This is sort of a bad example. Because py2exe embeds CPython, and we wouldn't want to do that. So what we would probably want to do is to make some general tool that willmake a windows binary, or a mac one, and get rid of the need for bzfreeze and friends. So now you are looking at a general embedding solution, and that is more than 10 days worth of work. But I get the idea. <snip>
This is one of the things I want to talk with fundedbyme about. But having an explicit trustee is a new idea. I think the pypy core developers are already rather well trusted in this community, but this may be important to new developers who aren't as well known. And it handles the problem' of 'I got sick and cannot do this any more' more gracefully than other solutions.
fundedbyme has sort of indicated an interst in doing this (except they were talking about distribution before, and I was leaving project description to the project, not outsiders). I will follow up on this when I get back home to Sweden.
Thanks once again for seeing a marketing solution that nerds like us often miss. Laura
I'm glad this thread is up. Laura - I'm the chap from Armin's talk who offered a monthly retainer for a year towards numpy integration (in my mind I'm offering £50/month for 12 months). I spoke to you later and you mentioned the flattr site but having to do it each month is a bit of a pain (I know it is simple but I don't want to think about it...). So, for the record, I have £600 sitting here with someone's name on it, I'll account for it as a marketing expense or something out of my company. I'm a one man consultancy, PyPy doesn't directly help me as I'm an A.I./science researcher (so I need numpy, IPython, matplotlib etc) but I believe strongly that it will help all of Python (and me in part) over time, so it is worth pledging some of my earnings towards the goal of eventual numpy integration. If I can pledge it to someone or a project then that's cool, if I should just move the money to someone's account then that's cool too. I'm quite happy to have my name down as ContributorNumber1ForNumpy if it helps you spread the word. Ian. ps. I posted the v0.1 PDF of my High Performance Python tutorial this morning (it is based on my EuroPython training session). It has a section on PyPy and I'd happily accept input if that section should be expanded: http://ianozsvald.com/2011/06/29/high-performance-python-tutorial-v0-1-from-... On 29 June 2011 08:53, Laura Creighton <lac@openend.se> wrote:
-- Ian Ozsvald (A.I. researcher, screencaster) ian@IanOzsvald.com http://IanOzsvald.com http://SocialTiesApp.com/ http://MorConsulting.com/ http://blog.AICookbook.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald
Hello Ian, On 29/06/11 15:16, Ian Ozsvald wrote:
I read your tutorial and tried some of the code, nice work. I tried to use the "bettermath" approach also on the pure python code, and run it on PyPy: on my machine, it takes 2.4 seconds instead of 6.2. For comparison, cython with bettermath takes 0.57 seconds, i.e. it's about 4 times faster than PyPy (with a trunk version of PyPy, did not try with pypy 1.5). For this kind of code, there is no fundamental reason why PyPy should be slower than cython, so I'll investigate a bit to see what is the problem. But still, I think that you should mention this "bettermath" implementation which is much faster on PyPy. ciao, Anto
Cool :-) Thanks Antonio. I'd left a note to myself in the report because I figured expanding the math to the primitive operations might help PyPy (seeing as it helped Cython and ShedSkin too). I'll redo the timings and add them to the v0.2 report (probably in two weeks time - I'm waiting for feedback from several people). I've already tried a trunk version of PyPy and it was faster than PyPy, I'll take whichever build is the most recent and replace PyPy1.5 for the next updates to the report. If there is a trick to e.g. re-ordering the operations so it runs faster in PyPy, I'd be happy to accept a modification. Hopefully there will also be a pyOpenCL version to add to the pyCUDA examples by then. Much obliged, Ian. On 2 July 2011 10:59, Antonio Cuni <anto.cuni@gmail.com> wrote:
-- Ian Ozsvald (A.I. researcher, screencaster) ian@IanOzsvald.com http://IanOzsvald.com http://SocialTiesApp.com/ http://MorConsulting.com/ http://blog.AICookbook.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald
On Sun, Jul 3, 2011 at 12:03 PM, Ian Ozsvald <ian@ianozsvald.com> wrote:
You can also use numpy arrays (or array from array module) on PyPy to get even more speedups. It's annoying because as of now pypy doesn't support much of numpy (1D float arrays only with ops please :) but it's a good start
Hi Maciej. I said that based on what was discussed in Armin+Antonio's talk at EuroPython. If it takes 6+months for a numpy re-implementation then I'm suggesting that we're into 2012. I'd be *very* happy to be proved wrong! Do Python 'array' objects run faster than lists in PyPy? I believe that in CPython they run at the same speed (i.e. they're just a convenient storage system, they don't offer any of numpy's efficient math benefits). Does the micronumpy library support doubles yet? If so I'd be happy to give it a go. If 'micronumpy' is the wrong name then tell me where I should look, I'm just going on what I've remembered from the PyPy-blog discussions about numpy support. Cheers, Ian. On 3 July 2011 18:38, Maciej Fijalkowski <fijall@gmail.com> wrote:
-- Ian Ozsvald (A.I. researcher, screencaster) ian@IanOzsvald.com http://IanOzsvald.com http://SocialTiesApp.com/ http://MorConsulting.com/ http://blog.AICookbook.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald
On Sun, Jul 3, 2011 at 8:35 PM, Ian Ozsvald <ian@ianozsvald.com> wrote:
I would estimate numpy for less than 6 months (numpy, not scipy and not matplotlib), but well that's me. There is also a lot of outside contribution, so it might take less wall time than manmonth time. From my perspective, if we can secure *some* funding for numpy, it should be ready sooner than 6 months from now.
The situation between PyPy and CPython is quite different here. As of now, both PyPy and CPython store wrapped objects in lists (let's call them PyIntObject) and unwrapped (C level int) in numpy arrays. In case you have only interpreter, when reading a list you do: py_x = list[index] do_something_with_py_x when reading from numpy array (or array.array), you do: py_x = new PyIntObject(array[index]) do_something_with_py_x so you use less memory (cache is better), but you allocate a new object each iteration (bad) But in case of the JIT, what happens is (list case) py_x = list[index] x = py_x.value do_something_with_x # x as in integer value not py_x array: x = list[index] do_something_with_x Hence, no allocation and memory saving - huge win!
yes, doubles is pretty much all it supports as of now :) micronumpy is a good name, but it comes under a name "numpy", so you can do "import numpy". Not much will work though :-) Note that we're also in the process of implementing faster vectorized operations, like numpyexpr, except better. The list of operations is very limited as of now though. Cheers, fijal PS. If it's unclear, I can try to explain more or pop up on IRC.
Re. micronumpy - all I'm doing in the Mandelbrot demo is multiplying and addition on doubles - that'll work now, right? If so I'll make the modification in a week or so when I next have some time. I'm guessing that there is no 'complex' support yet, even for basic operations? Adding micronumpy support would make for a nice addition to the v0.2 doc :-) Ian. re. the 6 month 'proper numpy' estimate - I'm just quoting some of the figures that were thrown around (with wide error margins depending on money raised, availability etc). I would *love* to see support come through sooner! On 3 July 2011 19:54, Maciej Fijalkowski <fijall@gmail.com> wrote:
-- Ian Ozsvald (A.I. researcher, screencaster) ian@IanOzsvald.com http://IanOzsvald.com http://SocialTiesApp.com/ http://MorConsulting.com/ http://blog.AICookbook.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald
Has anyone considered? http://www.fossfactory.org There are a lot of projects, and the money involved makes me believe that they are worth considering. http://www.fossfactory.org/browse.php
I'll re-open this earlier thread...Didrik Pinte of Enthought is offering to match my suggested £600 donation towards a numpy-pypy project. He's offering it in a personal capacity (i.e. not as an Enthought activity) out of his own pocket (and my donation would come out of my Consultancy). His offer came out of a discussion on pypy's merits at the London Financial Python Usergroup meet a few weeks back. So, you've got £1,200 via two individuals ready as a gift towards numpy integration, as/when someone can use the money. I won't promise that my gift will always be available as my personal situation may be changing in the next 6 weeks (so please - if you can use it - ask for it soon!). Regards, Ian (UK) On 29 June 2011 14:16, Ian Ozsvald <ian@ianozsvald.com> wrote:
-- Ian Ozsvald (A.I. researcher, screencaster) ian@IanOzsvald.com http://IanOzsvald.com http://SocialTiesApp.com/ http://MorConsulting.com/ http://blog.AICookbook.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald
Add EUR200 to that. Where do I pay ? regards, -- Best regards, Lionel Barret de Nazaris Gamr7 - CEO === Create bigger cities faster with Urban PAD <http://gamr7.com>. follow us : blog <http://urban-pad.posterous.com/rss.xml>, twitter <http://twitter.com/Gamr7>, facebook <http://www.facebook.com/pages/Urban-PAD/124363971359> Disclaimer: This message contains confidential and legally privileged information. This information is intended only for the addressee of this message. In case you have received this message in error, you should not disseminate, distribute or copy it. Please notify the sender immediately by e-mail and delete this message from your IT system. On 08/15/2011 03:03 PM, Ian Ozsvald wrote:
I'm glad this thread is up. Laura - I'm the chap from Armin's talk who offered a monthly retainer for a year towards numpy integration (in my mind I'm offering £50/month for 12 months). I spoke to you later and you mentioned the flattr site but having to do it each month is a bit of a pain (I know it is simple but I don't want to think about it...). So, for the record, I have £600 sitting here with someone's name on it, I'll account for it as a marketing expense or something out of my company. I'm a one man consultancy, PyPy doesn't directly help me as I'm an A.I./science researcher (so I need numpy, IPython, matplotlib etc) but I believe strongly that it will help all of Python (and me in part) over time, so it is worth pledging some of my earnings towards the goal of eventual numpy integration. If I can pledge it to someone or a project then that's cool, if I should just move the money to someone's account then that's cool too. I'm quite happy to have my name down as ContributorNumber1ForNumpy if it helps you spread the word. Ian. ps. I posted the v0.1 PDF of my High Performance Python tutorial this morning (it is based on my EuroPython training session). It has a section on PyPy and I'd happily accept input if that section should be expanded: http://ianozsvald.com/2011/06/29/high-performance-python-tutorial-v0-1-from-... On 29 June 2011 08:53, Laura Creighton <lac@openend.se> wrote:
-- Ian Ozsvald (A.I. researcher, screencaster) ian@IanOzsvald.com http://IanOzsvald.com http://SocialTiesApp.com/ http://MorConsulting.com/ http://blog.AICookbook.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald
Hello Ian, On 29/06/11 15:16, Ian Ozsvald wrote:
I read your tutorial and tried some of the code, nice work. I tried to use the "bettermath" approach also on the pure python code, and run it on PyPy: on my machine, it takes 2.4 seconds instead of 6.2. For comparison, cython with bettermath takes 0.57 seconds, i.e. it's about 4 times faster than PyPy (with a trunk version of PyPy, did not try with pypy 1.5). For this kind of code, there is no fundamental reason why PyPy should be slower than cython, so I'll investigate a bit to see what is the problem. But still, I think that you should mention this "bettermath" implementation which is much faster on PyPy. ciao, Anto
Cool :-) Thanks Antonio. I'd left a note to myself in the report because I figured expanding the math to the primitive operations might help PyPy (seeing as it helped Cython and ShedSkin too). I'll redo the timings and add them to the v0.2 report (probably in two weeks time - I'm waiting for feedback from several people). I've already tried a trunk version of PyPy and it was faster than PyPy, I'll take whichever build is the most recent and replace PyPy1.5 for the next updates to the report. If there is a trick to e.g. re-ordering the operations so it runs faster in PyPy, I'd be happy to accept a modification. Hopefully there will also be a pyOpenCL version to add to the pyCUDA examples by then. Much obliged, Ian. On 2 July 2011 10:59, Antonio Cuni <anto.cuni@gmail.com> wrote:
-- Ian Ozsvald (A.I. researcher, screencaster) ian@IanOzsvald.com http://IanOzsvald.com http://SocialTiesApp.com/ http://MorConsulting.com/ http://blog.AICookbook.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald
On Sun, Jul 3, 2011 at 12:03 PM, Ian Ozsvald <ian@ianozsvald.com> wrote:
You can also use numpy arrays (or array from array module) on PyPy to get even more speedups. It's annoying because as of now pypy doesn't support much of numpy (1D float arrays only with ops please :) but it's a good start
Hi Maciej. I said that based on what was discussed in Armin+Antonio's talk at EuroPython. If it takes 6+months for a numpy re-implementation then I'm suggesting that we're into 2012. I'd be *very* happy to be proved wrong! Do Python 'array' objects run faster than lists in PyPy? I believe that in CPython they run at the same speed (i.e. they're just a convenient storage system, they don't offer any of numpy's efficient math benefits). Does the micronumpy library support doubles yet? If so I'd be happy to give it a go. If 'micronumpy' is the wrong name then tell me where I should look, I'm just going on what I've remembered from the PyPy-blog discussions about numpy support. Cheers, Ian. On 3 July 2011 18:38, Maciej Fijalkowski <fijall@gmail.com> wrote:
-- Ian Ozsvald (A.I. researcher, screencaster) ian@IanOzsvald.com http://IanOzsvald.com http://SocialTiesApp.com/ http://MorConsulting.com/ http://blog.AICookbook.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald
On Sun, Jul 3, 2011 at 8:35 PM, Ian Ozsvald <ian@ianozsvald.com> wrote:
I would estimate numpy for less than 6 months (numpy, not scipy and not matplotlib), but well that's me. There is also a lot of outside contribution, so it might take less wall time than manmonth time. From my perspective, if we can secure *some* funding for numpy, it should be ready sooner than 6 months from now.
The situation between PyPy and CPython is quite different here. As of now, both PyPy and CPython store wrapped objects in lists (let's call them PyIntObject) and unwrapped (C level int) in numpy arrays. In case you have only interpreter, when reading a list you do: py_x = list[index] do_something_with_py_x when reading from numpy array (or array.array), you do: py_x = new PyIntObject(array[index]) do_something_with_py_x so you use less memory (cache is better), but you allocate a new object each iteration (bad) But in case of the JIT, what happens is (list case) py_x = list[index] x = py_x.value do_something_with_x # x as in integer value not py_x array: x = list[index] do_something_with_x Hence, no allocation and memory saving - huge win!
yes, doubles is pretty much all it supports as of now :) micronumpy is a good name, but it comes under a name "numpy", so you can do "import numpy". Not much will work though :-) Note that we're also in the process of implementing faster vectorized operations, like numpyexpr, except better. The list of operations is very limited as of now though. Cheers, fijal PS. If it's unclear, I can try to explain more or pop up on IRC.
Re. micronumpy - all I'm doing in the Mandelbrot demo is multiplying and addition on doubles - that'll work now, right? If so I'll make the modification in a week or so when I next have some time. I'm guessing that there is no 'complex' support yet, even for basic operations? Adding micronumpy support would make for a nice addition to the v0.2 doc :-) Ian. re. the 6 month 'proper numpy' estimate - I'm just quoting some of the figures that were thrown around (with wide error margins depending on money raised, availability etc). I would *love* to see support come through sooner! On 3 July 2011 19:54, Maciej Fijalkowski <fijall@gmail.com> wrote:
-- Ian Ozsvald (A.I. researcher, screencaster) ian@IanOzsvald.com http://IanOzsvald.com http://SocialTiesApp.com/ http://MorConsulting.com/ http://blog.AICookbook.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald
Has anyone considered? http://www.fossfactory.org There are a lot of projects, and the money involved makes me believe that they are worth considering. http://www.fossfactory.org/browse.php
I'll re-open this earlier thread...Didrik Pinte of Enthought is offering to match my suggested £600 donation towards a numpy-pypy project. He's offering it in a personal capacity (i.e. not as an Enthought activity) out of his own pocket (and my donation would come out of my Consultancy). His offer came out of a discussion on pypy's merits at the London Financial Python Usergroup meet a few weeks back. So, you've got £1,200 via two individuals ready as a gift towards numpy integration, as/when someone can use the money. I won't promise that my gift will always be available as my personal situation may be changing in the next 6 weeks (so please - if you can use it - ask for it soon!). Regards, Ian (UK) On 29 June 2011 14:16, Ian Ozsvald <ian@ianozsvald.com> wrote:
-- Ian Ozsvald (A.I. researcher, screencaster) ian@IanOzsvald.com http://IanOzsvald.com http://SocialTiesApp.com/ http://MorConsulting.com/ http://blog.AICookbook.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald
Add EUR200 to that. Where do I pay ? regards, -- Best regards, Lionel Barret de Nazaris Gamr7 - CEO === Create bigger cities faster with Urban PAD <http://gamr7.com>. follow us : blog <http://urban-pad.posterous.com/rss.xml>, twitter <http://twitter.com/Gamr7>, facebook <http://www.facebook.com/pages/Urban-PAD/124363971359> Disclaimer: This message contains confidential and legally privileged information. This information is intended only for the addressee of this message. In case you have received this message in error, you should not disseminate, distribute or copy it. Please notify the sender immediately by e-mail and delete this message from your IT system. On 08/15/2011 03:03 PM, Ian Ozsvald wrote:
participants (6)
-
Antonio Cuni -
Ian Ozsvald -
Laura Creighton -
Lionel Barret De Nazaris -
Maciej Fijalkowski -
Seung Soo, Ha