Dear PyPy'ers, First of all I would like to say that I think PyPy is an amazing project and that you have all done a really great job. Also the comments I have on the project are not aimed at any people in the project, more just at the general direction it appears to be going in. PyPy is on the edge of something great. A maintainable, powerful, flexible, fast interpreter is just what the python community needs. However just when it seems that PyPy can start to have some real significance in the Python world it seems like these benefits are being delayed for more research work which may take a long time. For instance a way of writing a rpython module that could be compiled to a Cpython extension or a PyPy extension would allow people to start using PyPy now, and at the same time make faster, powerful extensions for CPython while maintaining an upgrade path to PyPy. This would bring PyPy to the attention of a lot of people giving more testers/developers. Also, most people on #pypy seem to ask about using pypy to compile their simple python programs to c. Now, this doesn't seem like a great deal of work away (better error messages etc), but they are (politely) told that this is not what rpython is for. Now if rpython is not for this, why did you write PyPy in it? The same arguments could be applied to most programs (python is easier to read/maintain/write). I really can't see why something as useful as rpthon should remain an implementation detail, and again, exposing it would bring great exposure and benefits to the project. I don't want to come across like a moaner (and indeed, that's why I stop writing on #pypy as felt I couldn't be enough of a positive voice), and the only reason I'm writing this is because I think so much of the project and think it has so much potential. The last thing I want to see is for PyPy to become a great implemention with many powerful features, but then find that it had missed its time by not being "results driven" enough. The world doesn't need another powerful research/university language, it needs a great production language and with PyPy I think Python could be that language. Anyway, enough of my ranting. I'm sorry if I've offended anyone or completely missed the point. I'll go back to being a hopefull lurker again! Cheers, Ben
onsdagen den 7 december 2005 11.03 skrev Ben.Young@risk.sungard.com:
Dear PyPy'ers,
First of all I would like to say that I think PyPy is an amazing project and that you have all done a really great job. Also the comments I have on the project are not aimed at any people in the project, more just at the general direction it appears to be going in.
PyPy is on the edge of something great. A maintainable, powerful, flexible, fast interpreter is just what the python community needs. However just when it seems that PyPy can start to have some real significance in the Python world it seems like these benefits are being delayed for more research work which may take a long time.
For instance a way of writing a rpython module that could be compiled to a Cpython extension or a PyPy extension would allow people to start using PyPy now, and at the same time make faster, powerful extensions for CPython while maintaining an upgrade path to PyPy. This would bring PyPy to the attention of a lot of people giving more testers/developers.
Also, most people on #pypy seem to ask about using pypy to compile their simple python programs to c. Now, this doesn't seem like a great deal of work away (better error messages etc), but they are (politely) told that this is not what rpython is for. Now if rpython is not for this, why did you write PyPy in it? The same arguments could be applied to most programs (python is easier to read/maintain/write). I really can't see why something as useful as rpthon should remain an implementation detail, and again, exposing it would bring great exposure and benefits to the project.
I don't want to come across like a moaner (and indeed, that's why I stop writing on #pypy as felt I couldn't be enough of a positive voice), and the only reason I'm writing this is because I think so much of the project and think it has so much potential. The last thing I want to see is for PyPy to become a great implemention with many powerful features, but then find that it had missed its time by not being "results driven" enough. The world doesn't need another powerful research/university language, it needs a great production language and with PyPy I think Python could be that language.
Anyway, enough of my ranting. I'm sorry if I've offended anyone or completely missed the point. I'll go back to being a hopefull lurker again!
Thanks for your input Ben, I think you are quite right in everything you say, and there are people among the Pypy developers who would be very interested in working on making RPython directly useable. However, we are to a fairly large extent deadline driven. The EU financing comes with a large set of promises for what we are going to do and a fairly strict timeline to go with it. Currently this timeline says that we are to work on core optimisations, stacklessness and JIT, with the work to be finished by May 2006. Some people are also to do support for aspect oriented programming and constraints satisfaction. After May, there are other things promised until the official end of the EU project in November 2006. You can see the EU financing as being a fully commercial customer driven project. The only difference is the the customer hardly ever changes his mind. This means that (almost) everyone currently working on the project is very busy and doesn't have time to delve into interesting paths. Fortunately, we are in contact with a party that is very interested in doing exactly what you propose to do, and may be ready to pay for getting it done. However, this would require people not currently doing Pypy development to do the work. I'm not at liberty to discuss this in detail. I would just like to mention it so that you can see that there may be a way forward. Even though the EU financing is a straight-jacket, we should remember that we would be nowhere near what we have today without it. Best regards Jacob Hallén
onsdagen den 7 december 2005 11.03 skrev Ben.Young@risk.sungard.com:
Dear PyPy'ers,
First of all I would like to say that I think PyPy is an amazing
and that you have all done a really great job. Also the comments I have on the project are not aimed at any people in the project, more just at
general direction it appears to be going in.
PyPy is on the edge of something great. A maintainable, powerful, flexible, fast interpreter is just what the python community needs. However just when it seems that PyPy can start to have some real significance in the Python world it seems like these benefits are being delayed for more research work which may take a long time.
For instance a way of writing a rpython module that could be compiled to a Cpython extension or a PyPy extension would allow people to start using PyPy now, and at the same time make faster, powerful extensions for CPython while maintaining an upgrade path to PyPy. This would bring PyPy to the attention of a lot of people giving more testers/developers.
Also, most people on #pypy seem to ask about using pypy to compile
simple python programs to c. Now, this doesn't seem like a great deal of work away (better error messages etc), but they are (politely) told
this is not what rpython is for. Now if rpython is not for this, why did you write PyPy in it? The same arguments could be applied to most
(python is easier to read/maintain/write). I really can't see why something as useful as rpthon should remain an implementation detail, and again, exposing it would bring great exposure and benefits to the
I don't want to come across like a moaner (and indeed, that's why I
stop
writing on #pypy as felt I couldn't be enough of a positive voice), and the only reason I'm writing this is because I think so much of the
and think it has so much potential. The last thing I want to see is for PyPy to become a great implemention with many powerful features, but
find that it had missed its time by not being "results driven" enough. The world doesn't need another powerful research/university language, it needs a great production language and with PyPy I think Python could be that language.
Anyway, enough of my ranting. I'm sorry if I've offended anyone or completely missed the point. I'll go back to being a hopefull lurker again!
Thanks for your input Ben,
I think you are quite right in everything you say, and there are people among the Pypy developers who would be very interested in working on making RPython directly useable. However, we are to a fairly large extent deadline driven.
The EU financing comes with a large set of promises for what we are going to do and a fairly strict timeline to go with it. Currently this timeline says that we are to work on core optimisations, stacklessness and JIT, with
work to be finished by May 2006. Some people are also to do support for aspect oriented programming and constraints satisfaction. After May,
are other things promised until the official end of the EU project in November 2006. You can see the EU financing as being a fully commercial customer driven project. The only difference is the the customer hardly ever changes his mind.
This means that (almost) everyone currently working on the project is very busy and doesn't have time to delve into interesting paths.
Fortunately, we are in contact with a party that is very interested in doing exactly what you propose to do, and may be ready to pay for getting it done. However, this would require people not currently doing Pypy development to do the work. I'm not at liberty to discuss this in detail. I would just
mention it so that you can see that there may be a way forward.
Even though the EU financing is a straight-jacket, we should remember
Jacob Hallén <jacob@strakt.com> wrote on 07/12/2005 11:45:48: project the their that programs project. project then the there like to that we
would be nowhere near what we have today without it.
Best regards
Jacob Hallén
Hi Jacob, Thanks for the reply. I understand completely about the EU thing. Both a massive benefit and a minor curse. Just wanted to put my frustrations down in words! I guess it comes from wanting to contribute but having no time to do it at all. Cheers, Ben
Hey Ben, just one additional note: we did say sometimes that we will do our best to help someone working on such a tool ... it's not too far off and actually quite some work has been spend on improving and refining the translation process. It just needs someone with dedication and some time to think and experiment a bit, tackling some minor issues and discussing/promoting larger issues. Moreover, the project is evolving in more directions than are covered by the EU funding and the EU only partially funds development anyway. The current group cannot follow all interesting paths at the same time - although it sometimes may appear so :) cheers, holger On Wed, Dec 07, 2005 at 10:03 +0000, Ben.Young@risk.sungard.com wrote:
First of all I would like to say that I think PyPy is an amazing project and that you have all done a really great job. Also the comments I have on the project are not aimed at any people in the project, more just at the general direction it appears to be going in.
PyPy is on the edge of something great. A maintainable, powerful, flexible, fast interpreter is just what the python community needs. However just when it seems that PyPy can start to have some real significance in the Python world it seems like these benefits are being delayed for more research work which may take a long time.
For instance a way of writing a rpython module that could be compiled to a Cpython extension or a PyPy extension would allow people to start using PyPy now, and at the same time make faster, powerful extensions for CPython while maintaining an upgrade path to PyPy. This would bring PyPy to the attention of a lot of people giving more testers/developers.
Also, most people on #pypy seem to ask about using pypy to compile their simple python programs to c. Now, this doesn't seem like a great deal of work away (better error messages etc), but they are (politely) told that this is not what rpython is for. Now if rpython is not for this, why did you write PyPy in it? The same arguments could be applied to most programs (python is easier to read/maintain/write). I really can't see why something as useful as rpthon should remain an implementation detail, and again, exposing it would bring great exposure and benefits to the project.
I don't want to come across like a moaner (and indeed, that's why I stop writing on #pypy as felt I couldn't be enough of a positive voice), and the only reason I'm writing this is because I think so much of the project and think it has so much potential. The last thing I want to see is for PyPy to become a great implemention with many powerful features, but then find that it had missed its time by not being "results driven" enough. The world doesn't need another powerful research/university language, it needs a great production language and with PyPy I think Python could be that language.
Anyway, enough of my ranting. I'm sorry if I've offended anyone or completely missed the point. I'll go back to being a hopefull lurker again!
Cheers, Ben _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Hey Ben,
just one additional note: we did say sometimes that we will do our best to help someone working on such a tool ... it's not too far off and actually quite some work has been spend on improving and refining the translation process. It just needs someone with dedication and some time to think and experiment a bit, tackling some minor issues and discussing/promoting larger issues.
Moreover, the project is evolving in more directions than are covered by the EU funding and the EU only partially funds development anyway. The current group cannot follow all interesting paths at the same time - although it sometimes may appear so :)
cheers,
holger
On Wed, Dec 07, 2005 at 10:03 +0000, Ben.Young@risk.sungard.com wrote:
First of all I would like to say that I think PyPy is an amazing
and that you have all done a really great job. Also the comments I have on the project are not aimed at any people in the project, more just at
general direction it appears to be going in.
PyPy is on the edge of something great. A maintainable, powerful, flexible, fast interpreter is just what the python community needs. However just when it seems that PyPy can start to have some real significance in the Python world it seems like these benefits are being delayed for more research work which may take a long time.
For instance a way of writing a rpython module that could be compiled to a Cpython extension or a PyPy extension would allow people to start using PyPy now, and at the same time make faster, powerful extensions for CPython while maintaining an upgrade path to PyPy. This would bring PyPy to the attention of a lot of people giving more testers/developers.
Also, most people on #pypy seem to ask about using pypy to compile
simple python programs to c. Now, this doesn't seem like a great deal of work away (better error messages etc), but they are (politely) told
this is not what rpython is for. Now if rpython is not for this, why did you write PyPy in it? The same arguments could be applied to most
(python is easier to read/maintain/write). I really can't see why something as useful as rpthon should remain an implementation detail, and again, exposing it would bring great exposure and benefits to the
I don't want to come across like a moaner (and indeed, that's why I
stop
writing on #pypy as felt I couldn't be enough of a positive voice), and the only reason I'm writing this is because I think so much of the
and think it has so much potential. The last thing I want to see is for PyPy to become a great implemention with many powerful features, but
Hi Holger, I understand. I probably came across more pessimistic than I actually am. It's just very easy to get excited by a project like this, and see the endless possibilities (and not the endless hurdles)! Cheers, Ben hpk@trillke.net (holger krekel) wrote on 07/12/2005 16:28:14: project the their that programs project. project then
find that it had missed its time by not being "results driven" enough. The world doesn't need another powerful research/university language, it needs a great production language and with PyPy I think Python could be that
language.
Anyway, enough of my ranting. I'm sorry if I've offended anyone or completely missed the point. I'll go back to being a hopefull lurker again!
Cheers, Ben _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Hi Ben, On Wed, Dec 07, 2005 at 16:46 +0000, Ben.Young@risk.sungard.com wrote:
I understand. I probably came across more pessimistic than I actually am. It's just very easy to get excited by a project like this, and see the endless possibilities (and not the endless hurdles)!
hehe, indeed. i wasn't seeing your posting as pessimistic, btw. but maybe our little thread helps someone who is thinking of "how could i contribute something useful" :) holger
Ben.Young@risk.sungard.com wrote:
Hi Holger,
I understand. I probably came across more pessimistic than I actually am. It's just very easy to get excited by a project like this, and see the endless possibilities (and not the endless hurdles)!
Well, one hurdle is too few resources. I'd love to take that direction you proposed, partially, if I can find a PyPy core developer to help my company to fulfill its EU promises. Are you available? :-) -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
pypy-dev-bounces@codespeak.net wrote on 07/12/2005 17:21:18:
Ben.Young@risk.sungard.com wrote:
Hi Holger,
I understand. I probably came across more pessimistic than I actually am. It's just very easy to get excited by a project like this, and see the
endless possibilities (and not the endless hurdles)!
Well, one hurdle is too few resources. I'd love to take that direction you proposed, partially, if I can find a PyPy core developer to help my company to fulfill its EU promises.
Are you available? :-)
Believe me, there is nothing I would like more than to hack in and on python all day. Unfortunately I don't think my wife would like me doing all the travelling that seems to be involved. Plus, you'd have to beat a big financial software company on pay ;-) I really will try to find some time to help out. We've been having a very bad few weeks recently, but after Christmas things should quiten down and I may be able to help. I do still really want to come to a sprint. Maybe I can sell it to my company as some sort of training :) Cheers, Ben
-- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
This reply is solely to make a couple of points that I don't think have been made yet -- I don't want to give the impression that those points were less important that the ones I mention here. Ben.Young@risk.sungard.com writes:
Also, most people on #pypy seem to ask about using pypy to compile their simple python programs to c. Now, this doesn't seem like a great deal of work away (better error messages etc), but they are (politely) told that this is not what rpython is for. Now if rpython is not for this, why did you write PyPy in it?
Because we needed a description of the Python language that is amenable to analysis. I hope this isn't a new answer to you...
I don't want to come across like a moaner (and indeed, that's why I stop writing on #pypy as felt I couldn't be enough of a positive voice), and the only reason I'm writing this is because I think so much of the project and think it has so much potential. The last thing I want to see is for PyPy to become a great implemention with many powerful features, but then find that it had missed its time by not being "results driven" enough.
What results do you want?
The world doesn't need another powerful research/university language, it needs a great production language and with PyPy I think Python could be that language.
Yes, but I want *Python* to be that language, with its multitude of existing libraries and useful dyanmism and all the rest. Have you read this blog post: http://dirtsimple.org/2005/10/children-of-lesser-python.html ? I think I agree with his point that supporting 80% of the language is of much less than 80% of the value. If you have new code to write, then fine, writing it in RPython isn't that bad. But it's the people who want to, e.g., use urllib2 or some old code they wrote last year that I personally am interested in helping, i.e. every single user Python has today. This is why I'm most interested in the JIT and the standard interpreter end of things, not productizing an RPython compiler. Now I'm not and wouldn't want to be speaking for the project as a whole, and I agree that productizing RPython would be a very worthwhile project -- but I'm not going to do it, sorry. I hope that this has at least convinced you that I have no intention of PyPy being a research/university language, either. Cheers, mwh -- I never disputed the Perl hacking skill of the Slashdot creators. My objections are to the editors' taste, the site's ugly visual design, and the Slashdot community's raging stupidity. -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq
This reply is solely to make a couple of points that I don't think have been made yet -- I don't want to give the impression that those points were less important that the ones I mention here.
Ben.Young@risk.sungard.com writes:
Also, most people on #pypy seem to ask about using pypy to compile
simple python programs to c. Now, this doesn't seem like a great deal of work away (better error messages etc), but they are (politely) told
Hi Michael, pypy-dev-bounces@codespeak.net wrote on 08/12/2005 22:09:09: their that
this is not what rpython is for. Now if rpython is not for this, why did you write PyPy in it?
Because we needed a description of the Python language that is amenable to analysis. I hope this isn't a new answer to you...
I do understand that. It's just that as PyPy is a relatively complicated program it follows that rpython is good for making a lot of python programs amenable to analysis. (Yes as a by-product, but in my opinion an incredibly powerfull and usefull one)
I don't want to come across like a moaner (and indeed, that's why I
writing on #pypy as felt I couldn't be enough of a positive voice), and the only reason I'm writing this is because I think so much of the
and think it has so much potential. The last thing I want to see is for PyPy to become a great implemention with many powerful features, but
stop project then
find that it had missed its time by not being "results driven" enough.
What results do you want?
Sorry, I guess "results driven" came across slightly differently from how I meant it. I guess I meant that PyPy has many parts that with a bit of polish could be useable now in production based scenarios, as people keep asking about in IRC. I.e if people could write extensions, and PyPy was itself a bit faster and more polished then people could start using it now, and upgrade to a JIT version/different backend/thread model etc later.
The world doesn't need another powerful research/university language, it needs a great production language and with PyPy I think Python could be that language.
Yes, but I want *Python* to be that language, with its multitude of existing libraries and useful dyanmism and all the rest. Have you read this blog post:
http://dirtsimple.org/2005/10/children-of-lesser-python.html
? I think I agree with his point that supporting 80% of the language is of much less than 80% of the value.
If you have new code to write, then fine, writing it in RPython isn't that bad. But it's the people who want to, e.g., use urllib2 or some old code they wrote last year that I personally am interested in helping, i.e. every single user Python has today. This is why I'm most interested in the JIT and the standard interpreter end of things, not productizing an RPython compiler. Now I'm not and wouldn't want to be speaking for the project as a whole, and I agree that productizing RPython would be a very worthwhile project -- but I'm not going to do it, sorry.
I hope that this has at least convinced you that I have no intention of PyPy being a research/university language, either.
You have convinced me, and I'm glad that you are all so passionate about the project. Again, I didn't want to come across as moaning, and I want to thank you for all the work you have done so far. Cheers, Ben
Cheers, mwh
-- I never disputed the Perl hacking skill of the Slashdot creators. My objections are to the editors' taste, the site's ugly visual design, and the Slashdot community's raging stupidity. -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq
_______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Hi Ben! Ben.Young@risk.sungard.com wrote:
Because we needed a description of the Python language that is amenable to analysis. I hope this isn't a new answer to you...
I do understand that. It's just that as PyPy is a relatively complicated program it follows that rpython is good for making a lot of python programs amenable to analysis. (Yes as a by-product, but in my opinion an incredibly powerfull and usefull one)
No. RPython is not good for making a lot of Python programs amenable for analysis for several reasons. One of them is that although RPython is a rather nice and powerful language is *is not Python* (although it deceptively looks like it). That means (as Michael already pointed out) that it is really hard to convert an existing program to RPython since the style of programming is just totally different. So all the people saying "yay, I want RPython to speed up my program" will run into deep problems, no matter how much the RPython toolchain will be brushed up. After all, the PyPy standard interpreter was written in RPython from the ground up. The other problem is that you have nearly no standard library available in RPython, which is also something that is not easy to change. And it is easy to forget just how much most python programs are dependent on the stdlib :-). Cheers, Carl Friedrich Bolz
Carl Friedrich Bolz wrote:
RPython is not good for making a lot of Python programs amenable for analysis for several reasons. One of them is that although RPython is a rather nice and powerful language is *is not Python* (although it deceptively looks like it). That means (as Michael already pointed out) that it is really hard to convert an existing program to RPython since the style of programming is just totally different.
It is true that itis hard to concert an existing program to RPython, but it is not about being a totally different coding style. The kind of program one wants to convert to RPython is typically already written in a style towards optimization.Of course, some optimizations for Python are counter-productive in RPython, and the solution is to remove them. You will see what I mean when I do my presentation, tomorrow. Most what I had to do was to avoid type ambiguity, and to avoid features which we don't support, yet. But finally, the programs don't look so different. Partially, they even do look better, because RPython lets us use constructs, which you cannot use in Python, due to current speed limitations. For instance, for accessing certain pieces of data (talking of a simple parser I'm going to show), Python programmers tend to write string slices explicitly, using constants and optimizing their expressions by hand. This kind of optimizations makes Python programs much less readable and reallyraises the questions why they don't better write an extension. With RPython, I don't need to do any of this. Instead, I can afford to write tiny classes to be wrapped around the data I want to analyze, creating a much nicer, more readable and configurable source. This is due to the fact that with constant evaluation and inlining, all these tiny instances are melting away and producing the same high quality code as you would get by explicitly writing slices.
So all the people saying "yay, I want RPython to speed up my program" will run into deep problems, no matter how much the RPython toolchain will be brushed up. After all, the PyPy standard interpreter was written in RPython from the ground up.
I am addressing people who are already looking into writing extension modules. Giving them RPython as a tool to generate this extension module instead of hand-writing it is incredibly valuable. I do agree that this is not a cake walk, and I'm not going to try to make a tool that lets people do this with just any program, automatically. It makes sense to make RPython debugging easier and to get more help out of the tracebacks. Finally I'm not trying to compete with PyPy. Translating real Python is what PyPy is for. On the other hand, there is no point in forcing extension writers to continue their tedious work, after RPython exists.
The other problem is that you have nearly no standard library available in RPython, which is also something that is not easy to change. And it is easy to forget just how much most python programs are dependent on the stdlib :-).
Again this is not true. I can use all of the Python libraries from RPython,although I have to write some support code fir this. We wil end up by creating a new object space, which is able to give an abstraction of the existing CPython. There is quite some work to be doneto make this work smoothly. For sure it makes not much sense to use RPython stand-alone. Instead, I'm heading to build extension modules which can call back into any Python code, like CPython extensions are doing all the time. cheers - chris -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
participants (7)
-
Ben.Young＠risk.sungard.com -
Carl Friedrich Bolz -
Christian Tismer -
holger krekel -
hpk＠trillke.net
-
Jacob Hallén -
Michael Hudson