Re: native code compiler? (or, OCaml vs. Python)

Greg Ewing <greg@cosc.canterbury.ac.nz> wrote:
Really? It surprises me that after 10 years, this isn't something that has been given more priority. Is the problem simply too difficult?
There are some projects attacking parts of the problem, however, e.g. Psyco.
I've looked at Psyco, but it seems treated like an ad-hoc red-headed stepchild. Why isn't something this made part of Python? It would be nice to use Python for more serious projects, but it isn't fast enough currently. ===== Regards, Graham http://movies.yahoo.com.au - Yahoo! Movies - What's on at your local cinema?

Really? It surprises me that after 10 years, this isn't something that has been given more priority. Is the problem simply too difficult?
Yes. It surprises me that you think so lightly of something that if it was easy obviously would have been done already. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org> wrote: >
Yes. It surprises me that you think so lightly of something that if it was easy obviously would have been done already.
I think it's clear noone really knows why this hasn't been done yet. Even on this list, whose users are presumably well clued-in about Python, I've heard: ``you're welcome to volunteer.'' ``Nobody cared enough to do it.'' I'm sure writing a compiler isn't easy, and I wasn't suggesting that. However, head-way (Psycho) has been made in a very short period of time, so it hardly seems impossible given enough time and effort. I think such an effort would be well worth it. ===== Regards, Graham http://movies.yahoo.com.au - Yahoo! Movies - What's on at your local cinema?

I think it's clear noone really knows why this hasn't been done yet.
Bullshit. You're not getting a straight answer because it's a FAQ, but the answer can only be appreciated if you know a lot about dynamic languages *and* compilation. --Guido van Rossum (home page: http://www.python.org/~guido/)

Neil Schemenauer <nas@python.ca> wrote:
Talk is cheap.
What, so I can't inquire about or request a potential feature unless I can implement it myself? Please. ===== Regards, Graham http://movies.yahoo.com.au - Yahoo! Movies - What's on at your local cinema?

Graham, you have worn out your welcome here. Python is free -- if you don't like it, you can walk away, or pay somebody to add the feature you want. You can't just go about requesting features and expect them to be implemented if no-one else thinks they are needed. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org> wrote:
Graham, you have worn out your welcome here.
Well, I apologize for bringing up a subject you don't enjoy talking about. Has anyone ever told you guys that you don't handle constructive criticism well? I've seen it as a lurker on this list as well as c.l.p, and now firsthand. And Guido, the problem with having a crummy attitude like this is that it tends to infect those around you tainting the whole project. Don't turn Python into another OpenBSD or qmail.
Python is free -- if you don't like it, you can walk away, or pay somebody to add the feature you want.
Have you forgotten that without users and potential users Guido, Python would cease to exist?
So don't just go about requesting features and expect them to be implemented if no-one else thinks they are needed.
Sorry, but I don't remember ever requesting or expecting anything. In any case, thanks to those few who had the maturity to offer a constructive response to a question which is not well explained. ===== Regards, Graham http://movies.yahoo.com.au - Yahoo! Movies - What's on at your local cinema?

[Graham Guttocks]
OK, now I have been basically ignoring this thread and purposely staying out of this whole mini-flame war that is starting. But I have to disagree with the suggestion that this list, as a whole, cannot handle constructive criticism. I have to read every single piece of email that comes through this list and yes, some people can get a little snappy. But first of all most people here are volunteering their time here and when something that is stupid or has a hint of insult it flips a switch (I know it does in me). And second, we are all human (except Timbot, effbot, and Alexbot when he pops in from time to time). When this list gets a question about something that is asked constantly it just wears on some of us and we just want to ignore it. And yet people still responded to your question, which is more than some lists will do. But this list's integrity is very high. Saying the people here cannot take criticism is, to quote Guido, "bullshit". Every point here is argued and people still manage to keep it "clean" as long as you don't come in here making accusations. When someone comes in and suggests something hasn't been done because it might be "too difficult", most of us take it personally (I would suspect Guido especially). It is a suggestion that the knowledge of this list is not high enough to do this. As has been stated, though, it just has not been a priority of python-dev to worry about this and has nothing to do with ability. If you are that worried about performance, code it in C and hook into Python. It is going to take someone caring enough to actually write a solution and no one here cares enough. But you have been "trolling" this list, so you know all of this by now. I-can't-believe-I-managed-to-prevent-msyelf-from-yelling-ly yr's, Brett

Brett Cannon <bac@OCF.Berkeley.EDU> wrote:
and yes, some people can get a little snappy.
That's an understatement, I think.
But this list's integrity is very high.
Well, at times I just don't agree.
to quote Guido, "bullshit".
Yup, a great example of what I'm talking about. Interacting with some (not all mind you) of the Python bigwigs tends to leave a bad taste in my mouth, in the same manner as on the qmail or OpenBSD lists. The problem is not what, but how. Perception and cultural management of project mailinglists is extremely important in developing a project community. This is something that the haskell.org folks for example have really done right. Haskell mailinglists have a general warm, friendly tone that is highly conductive to discourse and community. I wish I could say the same for Python. ===== Regards, Graham http://movies.yahoo.com.au - Yahoo! Movies - What's on at your local cinema?

Graham Guttocks <graham_guttocks@yahoo.co.nz> writes:
This list has no problem with constructive criticism. The list has a problem with anything being posted to the list that doesn't have to do with the current development of Python. These posts fall in two major catagories: 1) Features or bugs that somebody wants written, but isn't willing to write. This is the Python Developers list not the Python Feature Request List. It is intended for you to discuss the changes you are making or proposing to make to the language, not to request that your pet project be written by the other readers. A bug reporting mechanism exists, and this isn't it. 2) Questions that have been asked a million times and are already on the web page or in the FAQ. You've managed to fall in both catagories. Merriam-Webster defines "constructive" as "promoting improvement or development" and "promoting" as "to contribute to the growth or prosperity of." It's hard to see how bringing up a question that was answered in the FAQ over a year ago is contributing to the growth of anything. Yet you got not one, but two replies from Guido who both told you exactly why native code generation hasn't been done and that the rest of the answer was in the FAQ. If it's so easy to do, and so worthwhile, do it. I have more tractable things to spend my time on.
Have you forgotten that without users and potential users Guido, Python would cease to exist?
Why? It certainly came into being with only developers. I think if only they ever used it, they would still consider it a worthwhile effort. -- Christopher A. Craig <list-python-dev@ccraig.org> ed is the standard text editor

list-python-dev@ccraig.org (Christopher A. Craig) wrote:
And you've managed to convince us that your message has no merit except to demonstrate that you are a sycophant trying to gain favour with the Python developers. I think it's clear given the long, lively, and opinionated discussion that ensued as result of my message, that the issue is not as cut and dry as you indicated. A closed mouth says nothing wrong; a closed mind does nothing right. ===== Regards, Graham http://movies.yahoo.com.au - Yahoo! Movies - What's on at your local cinema?

At 9:45 AM +1300 1/31/03, Graham Guttocks wrote:
I think such an effort would be well worth it.
Then go for it. It's actually not all that tough, but badgering people who've decided not to for a variety of reasons won't get you that far... -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

Dan Sugalski <dan@sidhe.org> wrote:
It's actually not all that tough
Oh? Guido seems to disagree with you.
but badgering people who've decided not to for a variety of reasons won't get you that far...
Badgering? Look, it's obvious that I've hit a nerve here, but that wasn't my intention. The FAQ is not clear as to what those ``variety of reasons'' are, and I guess I'm learning why this is. ===== Regards, Graham http://movies.yahoo.com.au - Yahoo! Movies - What's on at your local cinema?

"Graham Guttocks" <graham_guttocks@yahoo.co.nz> wrote in message news:20030130211504.810.qmail@web10302.mail.yahoo.com...
No contradiction or disagreement. It is not all that tough to do something of limited benefit, such as pre-interpreting Python byte codes into a series of function calls in some compilable-to-machine-code language. It is *must* harder to do something much more useful, which is all Guido would be interested in. A a grad student, Armin Rigo conceived and implemented a clever idea which works with a large subset of Python. Now that he has graduated to the real world, I am sure all of us hope that he finds the means to continue. PyRex and Weave are other projects that help certain Python applications. Terry J. Reedy

At 5:55 PM -0500 1/30/03, Terry Reedy wrote:
You underestimate the benefit in the first case, and the second case isn't really much more difficult. I expect one of the several reasons that nobody's pursued this is that the current interpreter is deemed fast enough, which is a perfectly good reason to not bother going further. -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

At 10:15 AM +1300 1/31/03, Graham Guttocks wrote:
That's fine. Guido and I have a different set of skills. It's not that tough with the right set. (In the same way that designing languages isn't that tough with the right set of skills)
Because it freaks everyone out. :) -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

At 12:58 AM +0100 1/31/03, Fredrik Lundh wrote:
It's not done yet, no. Doesn't mean that it isn't sufficiently complete to draw some conclusions, and we do have a working cross-platform JIT, which is the part that has the most direct bearing on the original subject of this little firestorm. Native code compilation isn't difficult, nor is expecting a gain of at least 20% unreasonable over straight interpretation. (Expecting a factor of 10-12 speedup not unreasonable in some circumstances) Ignoring parrot, I'll point you at some of the Scheme and Smalltalk setups. (Andyou'd be hard-pressed to find something to beat smalltalk for runtime dynamism) If you want to bet, I'll put up $10 and a round of beer (or other beverage of your choice) at OSCON 2004 for all the python labs & zope folks that says parrot beats the current python interpreter when running all the python benchmark suite programs (that don't depend on extensions written in C, since we're shooting for pure engine performance) from bytecode. Game? -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

Dan Sugalski wrote:
Not on python-dev.
If you want to bet
Just show me the code.
If you're talking about simple stuff like pystone, people have already done *interpreters* that can run that kind of bench- mark 5-25 times faster than CPython. All it takes is a fast GC, call-site caching, some low-overhead primitives, and a fast calling mechanism. But if you want to run real Python programs, you still have to solve all the hard problems. Judging from the Parrot material I've seen, it's not obvious that you're even aware of what they are. After all, if you know how to solve them, wouldn't it be easier to tell us what you've done (or plan to do), instead of pointing to non-Parrot implementations of non-Python languages done by others? But let's hope I'm wrong. The only way to *prove* me wrong is to ship the code. </F>

At 9:12 AM +0100 1/31/03, Fredrik Lundh wrote:
The original topic was native-code compilers, and it was put forth that writing one for a dynamic language was either very hard or impossble, and if you did there wasn't much win. The counterexamples of the Scheme and Smalltalk compilers are relevant--both languages are more dynamic at runtime, and both languages have native compilers that provide significant speed benefits, and both languages have implementations with relatively little man-time and brainpower that provided significant speed benefits without semantic changes. Unless I'm misreading you, and you're positing that python's more dynamic at runtime than smalltalk, but I presume that's not the case.
If you want to bet
Just show me the code.
That was the point of the bet.
Then pick one you consider more indicative. I don't much care, honestly--as long as it doesn't depend on C extensions it's fine.
I have a list of hard problems. Luckily for me, python doesn't hit any of them, but that may as much be because of what I consider a "hard problem" than anything else.
If parrot was finished, sure. It's not, hence the 2004 date on the challenge (while I could meet the deadline for this year's OSCON, enough other things would get shuffled aside that it wouldn't be a good allocation of resources) and the Scheme and Smalltalk examples, which *do* exist, and have for quite some time. C'mon Fred, you *know* that if I just told you how I planned to solve the problems I'd get ripped to shreds (and rightly so) for handwaving past the implementation issues, and nothing I have now will satisfy, as it's not finished so there's always that "well, it isn't done yet" rejoinder. (Which, again, is both fine and appropriate) Still, the challenge stands if you or anyone else wants to take me up on it. Catch me before OSCON 2003 and we'll go from there. -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

Dan Sugalski <dan@sidhe.org> writes:
It's unclear to me that scheme is more dynamic than Python. Are you thinking of call/cc or something? What I know of Smalltalk (i.e. almost nothing) lends me to believe you on that one. Cheers, M. -- FORD: Just pust the fish in your ear, come on, it's only a little one. ARTHUR: Uuuuuuuuggh! -- The Hitch-Hikers Guide to the Galaxy, Episode 1

[Michael Hudson]
It's unclear to me that scheme is more dynamic than Python.
I don't think Scheme is more dynamic either. The language has set types that you just don't happen to have access to or see directly. But there is no clear cut way of defining new types short of writing it in C. It was bad enough reading Paul Graham calling Python "water-down Lisp with infix syntax and no macros", but having Scheme being considered more dynamic is blasphemy in my book (but that might be because Scheme was forced upon me in school =). -Brett

On Fri, 31 Jan 2003, Brett Cannon wrote:
It was bad enough reading Paul Graham calling Python "water-down Lisp with infix syntax and no macros"
Python *does* have infix syntax and no macros. I happen to consider both of these things significant advantages! -- ?!ng

Brett Cannon <bac@OCF.Berkeley.EDU>:
Yo. Don't you be dissin' my homeboy Paul. He is flyyyy. Actually, I had a long lunch with Paul recently and am in a position to state confidently that he thinks Python is pretty nifty. Yes, Paul has some LISP-inspired criticisms. I agree with some (yes, Python would be a better language if it were a little more expression-oriented) and I disgree with others (I have regretfully concluded that macros are a bad idea). On the other hand, he concedes that dict as a first-class type is a very good idea, and that the Python OO system is right in style even if arguable in detail. Paul's website recommends Python as the best alternative if you can't do LISP; see <http://www.paulgraham.com/faq.html>. Under the circumstances (which include the fact that he's one of the two or three influence leaders of the LISP community) that's about the strongest endorsement one could expect. Guido, if you ever get a chance to hang with Paul, take it. He's an very intelligent and capable person, well-spoken, a natural leader, and no mean language designer himself. Knowing both of you, I predict you would get along famously and learn a lot from each other. In fact, you could do a lot worse for a keynote speaker at a future Python conference than to invite him. And then listen to his talk carefully. Very few people would be better qualified than Paul as a friendly critic from a different tradition. -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Brett Cannon wrote:
It's unclear to me that scheme is more dynamic than Python.
I don't think Scheme is more dynamic either.
from a language runtime perspective, I don't think Smalltalk is more dynamic either. I'm still waiting for the Parroteers to prove me wrong. </F>

At 8:44 AM +0100 2/1/03, Fredrik Lundh wrote:
You'll probably be waiting a long time--it's generally off topic for the list, most people who aren't convinced won't believe me anyway, and I don't really care enough about this to put in the effort to convince you. -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

At 12:00 PM +0100 2/3/03, Christian Tismer wrote:
While it could be, that's unlikely. Still, that'd be part of the ground rules we'd work out if someone takes me up on the challenge. -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

I'm in. Given how low you set your stakes, I don't think you're very confident, so I'd like to call your bluff. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

At 1:15 PM -0500 2/3/03, Guido van Rossum wrote:
The stakes are low because I have to pony up the cash if I lose--my budget's limited, and regardless of how confident I am of winning, if I put up a lot, my wife will kill me dead for making the challenge and, well I think I'd rather avoid that. :) Shall we, then, arrange the details at OSCON 2003, if you'll be there? I expect we can twist the conference organizer's arm into letting us make a tiny announcement near the end... -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

Especially as you'd automatically lose if your wife were to kill you. :)
Sure. We should do this as a lightning talk in the Python lightning track. --Guido van Rossum (home page: http://www.python.org/~guido/)

At 1:41 PM -0500 2/3/03, Guido van Rossum wrote:
Well, we might still beat you, though I'd *definitely* lose personally. :)
Works for me. I doubt we'll need that much time to work things out. OTOH, as conferences love animosity, real or imagined, it wouldn't surprise me if Tim O'Reilly'd be happy to announce this if we can work the details out ahead of time. Fine with me either way... -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

Guido van Rossum wrote:
I'm in. Given how low you set your stakes, I don't think you're very confident, so I'd like to call your bluff. :-)
Any opinion on where to spend effort? I was thinking of dusting off the register VM code (aka rattlesnake). OTOH, perhaps the non-local namespace optimizations would give more bang for the buck. Neil

I was thinking it might be most effective to have a little conversation with Dan's potential sponsors. But maybe that's because my first name is Guido... ;-) Seriously, I expect Dan to lose without any effort on our part, but if we want to make an effort, I think that there could be some low-hanging fruit in inlining certain builtins (e.g. len, range, xrange) that as far as we know aren't shadowed by globals. Is that what you call non-local namespace optimizations? A new VM design would be a major upheaval, but if we can pull it off it would certainly not be a bad idea. John Aycock and some of his students have a design of a new VM that might be worth looking into. --Guido van Rossum (home page: http://www.python.org/~guido/)

At 2:18 PM -0500 2/3/03, Guido van Rossum wrote:
Seriously, I expect Dan to lose without any effort on our part,
Really! Well, then, shall we raise the stakes, and add a cream pie (of the loser's choice) at ten paces to the bet? :) -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

I'd be delighted to deliver it in person. :) --Guido van Rossum (home page: http://www.python.org/~guido/)

At 2:31 PM -0500 2/3/03, Guido van Rossum wrote:
Absolutely--it'd be no fun otherwise. :) -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

Dan> Really! Well, then, shall we raise the stakes, and add a cream pie Dan> (of the loser's choice) at ten paces to the bet? :) Having had personal experience with the inverse relationship between geek quotient and athletic prowess I suspect you might want to make that five paces. ;-) Skip

At 1:36 PM -0600 2/3/03, Skip Montanaro wrote:
Nah, that's OK--I'll be practicing my aim before then. :-P -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

That was my line. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

Dan Sugalski <dan@sidhe.org>:
Really! Well, then, shall we raise the stakes, and add a cream pie (of the loser's choice) at ten paces to the bet? :)
Oh, *goddess*. I want to emcee this event in the worst way. And I know exactly how to do it -- like a herald reading cartels of defiance at a joust. Gentlemen, you've got yourself a master of ceremonies and a frame of comic patter, if you want it. For this I might even put on a tuxedo... -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

At 3:25 PM -0500 2/3/03, Eric S. Raymond wrote:
Argh! The very thought of that is making me want to gouge my eyes out. :) Seriously, this is something we'll have to work out with the conference folks. Wouldn't surprise me if they may have someone in mind, or if not we go with some semi-independent person. (Perhaps some bitter Lisp hacker that hates all of us because everyone's more successful than Lisp... :) -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

Dan Sugalski <dan@sidhe.org>:
If you suggest Paul Graham I'm going to have to hurt you. -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

At 5:41 PM -0500 2/3/03, Eric S. Raymond wrote:
Hey, if we can pitch a pie at him regardless of who wins, I'm OK with it. Come to think of it, if you want to host it...: ) -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

Dan Sugalski <dan@sidhe.org>:
Huh. You know, it didn't occur to me that bias could be an issue here. Not to worry; if Guido loses, I will be just as snarky at him as I would have been at you. One of the reasons I like Guido is that I am confident he would expect no less of me :-). -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Ah, you mean these: - PEP 266 Optimizing Global Variable/Attribute Access Montanaro PEP 267 Optimized Access to Module Namespaces Hylton PEP 280 Optimizing access to globals van Rossum Yes, that would be a good idea too. And I suggest that we analyze pystone (or whatever benchmark we decide to use). --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido> And I suggest that we analyze pystone (or whatever benchmark we Guido> decide to use). If you decide to use pystone, I'd argue you incorporate psyco into the distribution and stick the following code right after the definition of Proc8: try: import psyco except ImportError: pass else: psyco.bind(Proc8) (I believe that's the correct psyco call and the correct function to optimize. It's been awhile.) ;-) Skip

Don't forget Marc-Andre Lemburg's pybench http://www.egenix.com/files/python/ http://www.egenix.com/files/python/pybench-1.0.zip ka

Neil> Any opinion on where to spend effort? I was thinking of dusting Neil> off the register VM code (aka rattlesnake). OTOH, perhaps the Neil> non-local namespace optimizations would give more bang for the Neil> buck. I'd help play around with rattlesnake. Skip

>> I'd help play around with rattlesnake. Brett> Any links on rattlesnake? I tried googling and came up with only Brett> a brief mention by Ping and Skip about it. You just didn't look far enough back. ;-) Rattlesnake was something I started in 1998 (based on 1.5.1 as I recall) to come up with an alternate, register-based virtual machine for Python. The observation that Tim made which got me headed in that direction was that three-register machines are much easier to optimize than stack-based architectures. The plan was to mechanically translate the bytecode for the stack architecture to the bytecode for the register architecture. (I used my peephole optimizer as the basis for that translator. Conversion to the new architecture was thus a rather substantial "peephole" operation.) The result should be relatively easy to optimize (just by eliding many of stack motion, if nothing else). A few of us got a reasonable start on it, but it eventually fizzled. There were a couple things that didn't ever work right, and starting with 1.5.2 the virtual machine started moving faster than I could keep up in the odd spare moment. It sat idle until a little over a year ago when Neil began working on it. To the best of my knowledge, he's never released anything, but whatever he's got laying about is bound to be more current than anything I could point you at. Still, here are some pointers for those interested: * Here's a pointer to a paper about the peephole optimizer: http://www.musi-cal.com/~skip/python/spam7/optimizer.html * Here's the latest stuff I bundled up for Neil in 2001: http://www.musi-cal.com/~skip/python/rattlesnake20010813.tar.gz * Here's the long inactive mailing list: http://groups.yahoo.com/group/rattlesnake * There's a python-dev thread from February 2002: http://mail.python.org/pipermail/python-dev/2002-February/thread.html (search for "rattlesnake"). * Finally, here's a python-dev thread about rattlesnake from 1998: http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&threadm=7399hf%24scb%241%40nnrp1.dejanews.com&rnum=1&prev=/groups%3Fq%3Drattlesnake%2Bmore%2Bbite%2Bfor%2Bpython%26hl%3Den%26lr%3D%26ie%3DUTF-8%26oe%3DUTF-8%26selm%3D7399hf%2524scb%25241%2540nnrp1.dejanews.com%26rnum%3D1 Skip

On Mon, Feb 03, 2003 at 11:17:11AM -0800, Neil Schemenauer wrote:
As Guido mentioned, inline builtins. I have a real simple patch which does this. ie, LOAD_GLOBAL(name) -> LOAD_CONST(builtin_function). However, there is a problem with the patch: the resulting code can't be marshalled properly. I haven't tried to fix that yet. Right now JUMP_IF_(TRUE, FALSE) keep their computed value on the stack. They are always followed by POP_TOP. If the JUMP_IF_* removed the value, it would be one less trip through eval_frame loop (no POP_TOP). I've got a patch for this which fixes 5 of the 8 cases where JUMP_IF_* are generated. The problem with the remaining 3 cases is that a jump to a jump occurs. By removing the jump to a jump, that should also help performance. I have a crazy idea that removing the switch and making our own jump table in the eval_frame loop could improve performance. But I've never tried this because it's a lot of work. And it could hurt, not help performance. I can mail or post the partial patches if anyone is interested. Neal

That's because you can't marshal references to built-in function objects. I was thinking of adding appropriate new opcodes for a few builtins that are called a lot, like len. This would be implemented using something like this: case BUILTIN_LEN: { long n; v = POP(); n = PyObject_Size(v); Py_DECREF(v); if (n >= 0) { x = PyInt_FromLong(n); if (x != NULL) { PUSH(x); continue; } } else { err = n; } break; }
Yes. I think there's also an inefficiency in the bytecode generated for 'not': instead of generating a UNARY_NOT opcode, it could switch the sense of the test (changing JUMP_IF_FALSE into JUMP_IF_TRUE and vice versa).
The only way to find out is to try it. ;-) --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido> I was thinking of adding appropriate new opcodes for a few Guido> builtins that are called a lot, like len. This would be Guido> implemented using something like this: Guido> case BUILTIN_LEN: ... Would you special case those calls so that, in effect, __builtin__.len couldn't be overridden by a "len" object in the globals or locals? Skip

No, I would only generate a BUILTIN_LEN opcode if there was no local or global variable 'len'. A few days ago I proposed a restriction on assigning to an attribute of a module that stores a new name in that module that is the same as the name of a built-in; that was meant to outlaw people doing ugly stuff like import random random.len = lambda x: len(x)-1 print random.choice([1,2,3]) and expecting to affect the implementation of choice(). I don't think anybody would do this on purpose (or even by accident), but the possibility exists, and I'd prefer to live without lying awake about that case. (We could make random.__dict__ read-only, like the new-style class __dict__, if you worry about other ways of stuffing unexpected variables inside it. We could also avoid the optimization if there's an exec statement anywhere in a module, since one could exec code that says import __builtin__ global len len = lambda x: __builtin__.len(x)-1 or one could find another way of outlawing such abominations (including simply declaring that one shouldn't do that). --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
do you mean specifically random.__dict__ or any modules dict? If the latter there would be quite some breakage. It is at least used for monkey patching modules to make them "unittestable" which is a valid use case IMO. Maybe module's dicts could be "frozen" by default and this could be explicitely turned off (via a sys-hook). holger

Any module's dict.
Why would this be done by patching the module's __dict__ rather than assigning to attributes of the module?
Maybe module's dicts could be "frozen" by default and this could be explicitely turned off (via a sys-hook).
What I proposed was only closing off write access to the __dict__ so that the setattr code for type module can implement certain restrictions (e.g. no assignment to attributes named "len"). --Guido van Rossum (home page: http://www.python.org/~guido/)

"GvR" == Guido van Rossum <guido@python.org> writes:
do you mean specifically random.__dict__ or any modules dict?
GvR> Any module's dict.
GvR> Why would this be done by patching the module's __dict__ rather GvR> than assigning to attributes of the module?
Maybe module's dicts could be "frozen" by default and this could be explicitely turned off (via a sys-hook).
GvR> What I proposed was only closing off write access to the GvR> __dict__ so that the setattr code for type module can implement GvR> certain restrictions (e.g. no assignment to attributes named GvR> "len"). I'd certainly like that feature, except for the times I'd want to circumvent it. On many occasions I have externally patched a module's namespace for debugging. It's been really convenient to replace some function with a wrapper that does some logging or updates a table tracking currently used resources. I've done the same with builtin open/file. As you noted, it seems less useful to override other builtins for this purpose. I'd suggest that frozen module namespaces by a feature of -O, except that I never use -O. Jeremy

"JH" == Jeremy Hylton <jeremy@zope.com> writes:
JH> On many occasions I have externally patched a module's JH> namespace for debugging. It's been really convenient to JH> replace some function with a wrapper that does some logging or JH> updates a table tracking currently used resources. I've done JH> the same with builtin open/file. As you noted, it seems less JH> useful to override other builtins for this purpose. I've done the same, and when I've needed, it's been incredibly handy. Rare, but useful. Then again, I've also only done this with open(), so maybe something like an open() hook would suffice. I can imagine some of the other built-ins /might/ be useful to debuggingly replace, but that's just speculation on my part, and may be yagni. -Barry

"BAW" == Barry A Warsaw <barry@python.org> writes:
"JH" == Jeremy Hylton <jeremy@zope.com> writes:
JH> On many occasions I have externally patched a module's namespace JH> for debugging. It's been really convenient to replace some JH> function with a wrapper that does some logging or updates a JH> table tracking currently used resources. I've done the same JH> with builtin open/file. As you noted, it seems less useful to JH> override other builtins for this purpose. BAW> I've done the same, and when I've needed, it's been incredibly BAW> handy. Rare, but useful. Then again, I've also only done this BAW> with open(), so maybe something like an open() hook would BAW> suffice. I can imagine some of the other built-ins /might/ be BAW> useful to debuggingly replace, but that's just speculation on BAW> my part, and may be yagni. But it is useful for to replace modules. For example, if you want to debug a module that uses the thread module, you can replace it's "thread" global with something that hooks "start_new_thread." I think that pattern is not uncommon. Jeremy

My proposal is to allow rebinding existing globals, just not to allow binding *new* globals *if* they shadow certain builtins. --Guido van Rossum (home page: http://www.python.org/~guido/)

"GvR" == Guido van Rossum <guido@python.org> writes:
GvR> My proposal is to allow rebinding existing globals, just not to GvR> allow binding *new* globals *if* they shadow certain builtins. Oh! +1 on that. just-add-a-future-statement-ly y'rs, Jeremy

We really only need to disallow assignment to names that would shadow builtins that are in fact inlined by the compiler. open/file wouldn't be one of these (specifically open/file would be exempted from inlining, and so would __import__, because these are known special cases). --Guido van Rossum (home page: http://www.python.org/~guido/)

My recollection from Skip's rattlesnake paper and from a similar paper about peephole optimizations in Interlisp is that we'll see perhaps a 5-10% speedup by focusing on the small stuff. I expect that the builtin-specific opcodes will do more than that, but the module-level namespace still ends up being expensive to use and arbitrary module globals are probably called as often as any other global. It'd be interesting to measure how many of the LOAD_GLOBAL calls are for builtin names using some simple benchmarks. Jeremy

[Guido van Rossum]
Wasn't there also a suggestion (I think Christian made just before Minimal Python was announced) to have the parser know what built-ins were and thus just inline the code? I think it's time the eval loop and I to get acquianted... -Brett

Most builtins don't have an inline version unless you add new opcodes that invoke e.g. PyObject_Size. --Guido van Rossum (home page: http://www.python.org/~guido/)

Neal> Right now JUMP_IF_(TRUE, FALSE) keep their computed value on the Neal> stack. They are always followed by POP_TOP. If the JUMP_IF_* Neal> removed the value, it would be one less trip through eval_frame Neal> loop (no POP_TOP). I've got a patch for this which fixes 5 of the Neal> 8 cases where JUMP_IF_* are generated. The problem with the Neal> remaining 3 cases is that a jump to a jump occurs. By removing Neal> the jump to a jump, that should also help performance. See my peephole optimizer reference in an earlier note which (used to) handle such stuff. Perhaps it should be dusted off. Skip

[Neil Schemenauer]
Any opinion on where to spend effort?
For real life boosts, function/method calls. Making "len" a builtin wouldn't help so much for saving a couple dict lookups as it would for avoiding the call machinery, and for callees coded in Python that's heavy work.

"TP" == Tim Peters <tim.one@comcast.net> writes:
TP> [Neil Schemenauer]
Any opinion on where to spend effort?
TP> For real life boosts, function/method calls. Making "len" a TP> builtin wouldn't help so much for saving a couple dict lookups TP> as it would for avoiding the call machinery, and for callees TP> coded in Python that's heavy work. Do you have any ideas? IIRC the largest expense in function calls is setting up the frame object for the new function to use. I don't think there are any low-hanging fruit here, because the frame model is pretty deeply embedded in the interpreter. It seems like all the work in PyEval_EvalCodeEx() has to get done sometime. Jeremy

Then that's where we should have a clean fresh look. --Guido van Rossum (home page: http://www.python.org/~guido/)

Neil Schemenauer <nas@python.ca> writes:
There's this field called tp_cache in type objects that's not currently used... Also Oren's ideas about a lookdict_namespace sounded interesting (always interning keys, inserting failure markers, bouncing succeeding lookups into the first slot examined, IIRC). Finally, anyone planning on a new VM would be well advised to finish off the ast-branch work, I'd suggest. Cheers, M. -- Darned confusing, unless you have that magic ingredient coffee, of which I can pay you Tuesday for a couple pounds of extra-special grind today. -- John Mitchell, 11 Jan 1999

[Graham Guttocks]
... I think such an effort would be well worth it.
Indeed, I spent the first 15 years of my computer career writing optimizing compilers for a living, and it's *so* worth it that in 10+ years of Python development, nobody has offered to pay me so much as an insignificant fraction of how much it's worth <0.5 wink>. These kinds of things get done by legions of grad students in search of a thesis, or by serious funding; they don't get done by part-time volunteers with demanding jobs.

From: "Tim Peters" <tim.one@comcast.net>
These kinds of things get done by legions of grad students in search of a thesis
in which case then typically portability, long term maintainability and absence of rough edges are quite irrelevant.

[Tim]
These kinds of things get done by legions of grad students in search of a thesis ...
[Samuele Pedroni]
in which case then typically portability, long term maintainability and absence of rough edges are quite irrelevant.
At first, yes, but it's not irrelevant that *something* gets done. For example, countless now-defunct Lisp companies were based on the founder(s)'s grad work, and a huge (relative to the size of the community) amount of Lisp compiler knowledge is still available for the grabbing. Even if you don't look at a lick of that code, the heritage of published papers is priceless. Or worthless, to judge from how often it's referenced <0.1 wink>.

Tim Peters <tim.one@comcast.net> wrote:
I see. Again, I didn't realize the enormity of this, because of how fast Psycho has progressed (from a part-time volunteer). You might consider adding the above to the FAQ to make it more clear that it is theoretically possible, but not without an enormous amount of work and/or funding. ===== Regards, Graham http://movies.yahoo.com.au - Yahoo! Movies - What's on at your local cinema?

[Graham Guttocks]
I see. Again, I didn't realize the enormity of this, because of how fast Psycho has progressed (from a part-time volunteer).
Not any part-time volunteer. Armin did Psycho while finishing his doctoral dissertation, and applied a brain the size of a planet to the job, with passion. That's the way intriguing compilers for dynamic languages get written in the absence of realistic funding (Dan notwithstanding <wink>). For a recent idea of what "realistic funding" means, dig thru now-old but not-yet-ancient stories about Sun's efforts on Java JIT technology. Of course everyone's an expert on everything they don't do, so it's natural to assume it's just a lack of will, or foresight, or something. Since O'Caml is in the Subject line here, note that one person reimplemented Python in O'Caml, with the goal of running much faster than CPython: http://vyper.sourceforge.net/ They gave up some time ago, with the last word on the topic there being: The interpreter Pystone benchmark is currently about 10 times slower than CPython 1.5.2. Considerable improvement is expected in the future. At least CPython (what we call the C implementation of Python) has gotten faster since then, but I'm not sure that's the considerable improvement the author had in mind <wink>.
The question doesn't upset me, but I prefer to limit the FAQ to frequently asked questions.

At 7:36 PM -0500 1/30/03, Tim Peters wrote:
Heh. I still need to finish, and while I'm not sure I have any planet-sized brains to bring to bear, I do now have defense of an ego the size of a planet, which may suffice. :)
For a recent idea of what "realistic funding" means, dig thru now-old but not-yet-ancient stories about Sun's efforts on Java JIT technology.
Java is something of a special case. The task of optimizing JVM code is fairly non-trivial just given how much information is thrown out by the compiler. (Stack-based intermediate targets are suboptimal when you're looking to then transform to a register based target such as hardware unless a lot of notation goes along with it which, alas, isn't generally the case with JVM code) -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

Tim Peters <tim.one@comcast.net> writes:
dissertation, and applied a brain the size of a planet to the job, with passion.
It could be pointed out that Armin's thesis was in purest-of-the-pure mathematics and had nothing to with Python. Last I heard, he'd got a postdoc in something that could related to dynamic compilation, but I don't know if that amounts to being paid to work on psyco. Cheers, M. -- Screaming 14-year-old boys attempting to prove to each other that they are more 3133t than j00. -- Reason #8 for quitting slashdot today, from http://www.cs.washington.edu/homes/klee/misc/slashdot.html

Hello Michael, On Fri, Jan 31, 2003 at 01:32:32PM +0000, Michael Hudson wrote:
Yes, it does. I will certainly include my share of the "Minimal Python" project (announced here some time ago) into my postdoc work, and the project is not at all unlinked with Psyco. Oh yes, I could use this mail to confirm I will soon release 1.0 in beta form. Armin.

From: "Guido van Rossum" <guido@python.org>
a "data" point from Towards a Universal Implementation Substrate for Object-Oriented Languages, Mario Wolczko, Ole Agesen, David Ungar, paper presented at the OOPSLA 99 workshop on Simplicity, Performance and Portability in Virtual Machine Design, OOPSLA '99, Denver, CO, Nov 2, 1999. http://research.sun.com/kanban/oopsla-vm-wkshp.pdf 1 Building virtual machines is hard Building a high-performance virtual machine (VM) for an object-oriented language such as Smalltalk or JavaT 2 is a major undertaking. ... In summary, building a high-performance VM typically requires several man-years of effort by those experienced in the field (e.g., the Self VM, in its present form, is the result of some 25 man-years of work), and results in a large, complex program, which is difficult to modify and adapt.

Graham Guttocks wrote:
Nobody cared enough to do it.
ASFAIK there will soon be a 1.0 release.
Why isn't something this made part of Python?
I guess it's still far too experimental and platform specific to be even considered.
It would be nice to use Python for more serious projects, but it isn't fast enough currently.
Python has shown to be pretty appropriate for a lot of tasks. If you like to discuss or comment on this i am ready but let's please move it to comp.lang.python then. holger

"GG" == Graham Guttocks <graham_guttocks@yahoo.co.nz> writes:
GG> It would be nice to use Python for more serious projects, but GG> it isn't fast enough currently. Really? How many of these 2100+ projects (just to pick a few at random <wink>) aren't serious? http://sourceforge.net/softwaremap/trove_list.php?form_cat=178 -Barry

"Barry A. Warsaw" <barry@python.org> wrote:
Really? How many of these 2100+ projects (just to pick a few at random <wink>) aren't serious?
Serious wasn't a good choice of word, but I think you know what I mean. e.g, what if I wanted to write an httpd daemon that could scale as well as Apache (millions of hits/day)? Python would not be up to the task. A language like CMUCL or OCaml would be, because they offer a native code compiler. ===== Regards, Graham http://movies.yahoo.com.au - Yahoo! Movies - What's on at your local cinema?

"GG" == Graham Guttocks <graham_guttocks@yahoo.co.nz> writes:
GG> Serious wasn't a good choice of word, but I think you know GG> what I mean. Only that Python isn't the appropriate tool for every programming task. No news there. But it is perfectly appropriate for a vast array of programming tasks. GG> e.g, what if I wanted to write an httpd daemon that could GG> scale as well as Apache (millions of hits/day)? Python would GG> not be up to the task. A language like CMUCL or OCaml would GG> be, because they offer a native code compiler. You mean like Twisted? http://twistedmatrix.com/documents/howto/faq#auto8 -Barry

Graham Guttocks wrote:
For the record, I just ran a quick benchmark against Apache and Twisted (pure Python) using 'ab' (the Apache benchmark tool) and Twisted wins hands down. Here are the results of ab -n 1000 and ab -n 10000 for each: http://just.letterror.com/~just/apachebench.txt http://just.letterror.com/~just/twistedbench.txt The Twisted-based server I set up is very basic, Apache uses the default configuration as it came with my OS (MacOSX.2.3). Just

[Graham Guttocks]
e.g, what if I wanted to write an httpd daemon that could scale as well as Apache (millions of hits/day)?
More than 625 hits per second average? If you really have this need, and see Python developments that could help you, I would guess that Python developers might be interested in them. If you do not really have this need, than this discussion is merely discursive and academical, and might be better held elsewhere than on `python-dev'.
Python would not be up to the task.
I would be tempted to consider many architectural avenues, before bluntly deciding that the choice of Python as an implementation language is _the_ problem. With millions of hits per day, I would most probably be big enough to have a flurry of other problems of all kinds, and then, Python would be more on the side of many solutions than the source of my problem. -- François Pinard http://www.iro.umontreal.ca/~pinard

Francois Pinard <pinard@iro.umontreal.ca> wrote:
Yes, I really do have this need. Not specifically for a httpd daemon, but I think it would be nice to see Python as a comparable alternative to C++ let's say for performance oriented applications.
Well, architectural approaches take much more work and much better programmers. I'd be nice to have some more firepower as to not have to lean so heavily in this direction every time. I'm sure the world's top tennis players could beat me quite handily with a frying plan, but they'd have to work so much harder than if they just had a real racquet. ===== Regards, Graham http://movies.yahoo.com.au - Yahoo! Movies - What's on at your local cinema?

On Fri, 31 Jan 2003, [iso-8859-1] Graham Guttocks wrote:
Depending on your type of traffic, a server may be network bound long before it's CPU bound, so compiling doesn't really help much. Millions of hits per day isn't a big deal, BTW. At the company I work for we have a Python server that easily handles 200 req/sec on sub-1GHz Intel boxes. At constant load that's about 20 million per day. While not as fast as Apache, it's far higher performance than we or many other sites need. -Dave

Graham Guttocks:
Almost always, only a small part of any program is computationally intensive. Many people have had great success in such cases by coding that small part in C, making an extension module out of it, and using Python for all the rest. This technique has proved suffiently successful that, so far, nobody has had both enough motivation and enough ability to attempt the creation of an optimising compiler for pure Python. And yes, it *is* a very difficult problem, due to the complete absence of explicit type information, and the extremely dynamic nature of Python's semantics. In summary, ``Nobody cared enough to do it.'' But there are good *reasons* for that.
``you're welcome to volunteer.''
I suppose that may have sounded like a brush-off, but it wasn't mean that way. If you have the necessary knowledge and ability to attempt anything in this area, any contribution you could make really would be welcome. If not, well, you'll just have to take our word for it that what you're asking for is very non-trivial. If it weren't, someone would probably have done it by now.
As I understand it, Psyco isn't a compiler in the conventional sense of something which statically analyses the program and generates code ahead of time. Rather, it notices at run time that certain functions tend to be called with certain types as parameters, and compiles specialised versions of the function for those types. A sort of "empirical type analysis". Psyco is still in the early stages. If it proves successful enough, maybe it or something like it will become a part of the standard Python implementation. Or maybe not -- the current Python implementation is exceptionally straightforward and easy to understand, and Guido may want to keep it that way. By the way, you might also like to take a look at Pyrex: http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/ which is my attempt at making it easier to bridge the gap between Python and other languages. As a side effect, it can be used to "compile" Python code, although the speed gain that way is small (only about 2 or 3 times at best). It's main purpose is for interfacing with C code, which is where the real speed gains are to be had. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

[Graham Guttocks]
It would be nice to use Python for more serious projects, but it isn't fast enough currently.
It might well depend on projects. We have fairly serious, production-level projects, and Python is quite fast enough for them. And very dependable too, in our experience. A real blessing. Oh, we got a bit of profiling and tuning to do here and there, once in a while. Then, a few thoughtful algorithmic changes give us great rewards. Extensions are also possible. Of course, speed increases in Python are welcome, but we should not let ourselves go frantic about speed. I guess that serious projects are often more interested in good and dependable results, and ease in development and maintenance, than in saving a few minutes or hours on a production run. So, for one at least, I think I like the current orientation, priorities and attitudes of Python developers about speed. They undoubtedly care about it, but intelligently, by protecting dependability and usability. -- François Pinard http://www.iro.umontreal.ca/~pinard

On 31 January 2003, Graham Guttocks said:
It would be nice to use Python for more serious projects, but it isn't fast enough currently.
The usual solution is to take the bits that are too slow in Python and recode them in C. This may not be as far-sighted as a spiffy native-code Python compiler, but it's a hell of a lot less work for each individual project. A good middle-ground is to figure out where Python is slow and make it faster. Eg. I recently discovered that certain operations in ZODB -- specifically, converting back and forth between 8-byte strings and 64-bit integers -- are a significant bottleneck. Those operations are currently implemented as tiny Python functions that use the struct module to do most of their work. Through discussion on the zodb-dev@zope.org list, I learned that the struct module has never been heavily optimized (aka "passed through the timbot"). I chose to reimplement those ZODB operations in C, because I wanted maximum bang for my (employer's) buck. But a more far-sighted approach -- ie. something that would benefit more projects for a longer time -- would be to spend some time tightening up the struct module. In all that time, it never once occurred to me that compiling ZODB to machine code was the answer. Dynamic typing is boatloads of fun, but it can really kill performance to do all that work at runtime. Doesn't matter if your runtime is Python bytecode interpreted by a C program or just plain machine code. Unless, of course, you have a super-sophisticated compiler that does ML-style type inference ... but without ML's bondage and discipline. I suspect that compiling Python properly (ie. with real performance gains) would be roughly as difficult as compiling ML -- and, because Python always gives you a way to sneak around whatever informal type safety mechanisms you might use, you wouldn't always benefit from type inference. Greg -- Greg Ward <gward@python.net> http://www.gerg.ca/ No animals were harmed in transmitting this message.

Really? It surprises me that after 10 years, this isn't something that has been given more priority. Is the problem simply too difficult?
Yes. It surprises me that you think so lightly of something that if it was easy obviously would have been done already. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org> wrote: >
Yes. It surprises me that you think so lightly of something that if it was easy obviously would have been done already.
I think it's clear noone really knows why this hasn't been done yet. Even on this list, whose users are presumably well clued-in about Python, I've heard: ``you're welcome to volunteer.'' ``Nobody cared enough to do it.'' I'm sure writing a compiler isn't easy, and I wasn't suggesting that. However, head-way (Psycho) has been made in a very short period of time, so it hardly seems impossible given enough time and effort. I think such an effort would be well worth it. ===== Regards, Graham http://movies.yahoo.com.au - Yahoo! Movies - What's on at your local cinema?

I think it's clear noone really knows why this hasn't been done yet.
Bullshit. You're not getting a straight answer because it's a FAQ, but the answer can only be appreciated if you know a lot about dynamic languages *and* compilation. --Guido van Rossum (home page: http://www.python.org/~guido/)

Neil Schemenauer <nas@python.ca> wrote:
Talk is cheap.
What, so I can't inquire about or request a potential feature unless I can implement it myself? Please. ===== Regards, Graham http://movies.yahoo.com.au - Yahoo! Movies - What's on at your local cinema?

Graham, you have worn out your welcome here. Python is free -- if you don't like it, you can walk away, or pay somebody to add the feature you want. You can't just go about requesting features and expect them to be implemented if no-one else thinks they are needed. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org> wrote:
Graham, you have worn out your welcome here.
Well, I apologize for bringing up a subject you don't enjoy talking about. Has anyone ever told you guys that you don't handle constructive criticism well? I've seen it as a lurker on this list as well as c.l.p, and now firsthand. And Guido, the problem with having a crummy attitude like this is that it tends to infect those around you tainting the whole project. Don't turn Python into another OpenBSD or qmail.
Python is free -- if you don't like it, you can walk away, or pay somebody to add the feature you want.
Have you forgotten that without users and potential users Guido, Python would cease to exist?
So don't just go about requesting features and expect them to be implemented if no-one else thinks they are needed.
Sorry, but I don't remember ever requesting or expecting anything. In any case, thanks to those few who had the maturity to offer a constructive response to a question which is not well explained. ===== Regards, Graham http://movies.yahoo.com.au - Yahoo! Movies - What's on at your local cinema?

[Graham Guttocks]
OK, now I have been basically ignoring this thread and purposely staying out of this whole mini-flame war that is starting. But I have to disagree with the suggestion that this list, as a whole, cannot handle constructive criticism. I have to read every single piece of email that comes through this list and yes, some people can get a little snappy. But first of all most people here are volunteering their time here and when something that is stupid or has a hint of insult it flips a switch (I know it does in me). And second, we are all human (except Timbot, effbot, and Alexbot when he pops in from time to time). When this list gets a question about something that is asked constantly it just wears on some of us and we just want to ignore it. And yet people still responded to your question, which is more than some lists will do. But this list's integrity is very high. Saying the people here cannot take criticism is, to quote Guido, "bullshit". Every point here is argued and people still manage to keep it "clean" as long as you don't come in here making accusations. When someone comes in and suggests something hasn't been done because it might be "too difficult", most of us take it personally (I would suspect Guido especially). It is a suggestion that the knowledge of this list is not high enough to do this. As has been stated, though, it just has not been a priority of python-dev to worry about this and has nothing to do with ability. If you are that worried about performance, code it in C and hook into Python. It is going to take someone caring enough to actually write a solution and no one here cares enough. But you have been "trolling" this list, so you know all of this by now. I-can't-believe-I-managed-to-prevent-msyelf-from-yelling-ly yr's, Brett

Brett Cannon <bac@OCF.Berkeley.EDU> wrote:
and yes, some people can get a little snappy.
That's an understatement, I think.
But this list's integrity is very high.
Well, at times I just don't agree.
to quote Guido, "bullshit".
Yup, a great example of what I'm talking about. Interacting with some (not all mind you) of the Python bigwigs tends to leave a bad taste in my mouth, in the same manner as on the qmail or OpenBSD lists. The problem is not what, but how. Perception and cultural management of project mailinglists is extremely important in developing a project community. This is something that the haskell.org folks for example have really done right. Haskell mailinglists have a general warm, friendly tone that is highly conductive to discourse and community. I wish I could say the same for Python. ===== Regards, Graham http://movies.yahoo.com.au - Yahoo! Movies - What's on at your local cinema?

Graham Guttocks <graham_guttocks@yahoo.co.nz> writes:
This list has no problem with constructive criticism. The list has a problem with anything being posted to the list that doesn't have to do with the current development of Python. These posts fall in two major catagories: 1) Features or bugs that somebody wants written, but isn't willing to write. This is the Python Developers list not the Python Feature Request List. It is intended for you to discuss the changes you are making or proposing to make to the language, not to request that your pet project be written by the other readers. A bug reporting mechanism exists, and this isn't it. 2) Questions that have been asked a million times and are already on the web page or in the FAQ. You've managed to fall in both catagories. Merriam-Webster defines "constructive" as "promoting improvement or development" and "promoting" as "to contribute to the growth or prosperity of." It's hard to see how bringing up a question that was answered in the FAQ over a year ago is contributing to the growth of anything. Yet you got not one, but two replies from Guido who both told you exactly why native code generation hasn't been done and that the rest of the answer was in the FAQ. If it's so easy to do, and so worthwhile, do it. I have more tractable things to spend my time on.
Have you forgotten that without users and potential users Guido, Python would cease to exist?
Why? It certainly came into being with only developers. I think if only they ever used it, they would still consider it a worthwhile effort. -- Christopher A. Craig <list-python-dev@ccraig.org> ed is the standard text editor

list-python-dev@ccraig.org (Christopher A. Craig) wrote:
And you've managed to convince us that your message has no merit except to demonstrate that you are a sycophant trying to gain favour with the Python developers. I think it's clear given the long, lively, and opinionated discussion that ensued as result of my message, that the issue is not as cut and dry as you indicated. A closed mouth says nothing wrong; a closed mind does nothing right. ===== Regards, Graham http://movies.yahoo.com.au - Yahoo! Movies - What's on at your local cinema?

At 9:45 AM +1300 1/31/03, Graham Guttocks wrote:
I think such an effort would be well worth it.
Then go for it. It's actually not all that tough, but badgering people who've decided not to for a variety of reasons won't get you that far... -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

Dan Sugalski <dan@sidhe.org> wrote:
It's actually not all that tough
Oh? Guido seems to disagree with you.
but badgering people who've decided not to for a variety of reasons won't get you that far...
Badgering? Look, it's obvious that I've hit a nerve here, but that wasn't my intention. The FAQ is not clear as to what those ``variety of reasons'' are, and I guess I'm learning why this is. ===== Regards, Graham http://movies.yahoo.com.au - Yahoo! Movies - What's on at your local cinema?

"Graham Guttocks" <graham_guttocks@yahoo.co.nz> wrote in message news:20030130211504.810.qmail@web10302.mail.yahoo.com...
No contradiction or disagreement. It is not all that tough to do something of limited benefit, such as pre-interpreting Python byte codes into a series of function calls in some compilable-to-machine-code language. It is *must* harder to do something much more useful, which is all Guido would be interested in. A a grad student, Armin Rigo conceived and implemented a clever idea which works with a large subset of Python. Now that he has graduated to the real world, I am sure all of us hope that he finds the means to continue. PyRex and Weave are other projects that help certain Python applications. Terry J. Reedy

At 5:55 PM -0500 1/30/03, Terry Reedy wrote:
You underestimate the benefit in the first case, and the second case isn't really much more difficult. I expect one of the several reasons that nobody's pursued this is that the current interpreter is deemed fast enough, which is a perfectly good reason to not bother going further. -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

At 10:15 AM +1300 1/31/03, Graham Guttocks wrote:
That's fine. Guido and I have a different set of skills. It's not that tough with the right set. (In the same way that designing languages isn't that tough with the right set of skills)
Because it freaks everyone out. :) -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

At 12:58 AM +0100 1/31/03, Fredrik Lundh wrote:
It's not done yet, no. Doesn't mean that it isn't sufficiently complete to draw some conclusions, and we do have a working cross-platform JIT, which is the part that has the most direct bearing on the original subject of this little firestorm. Native code compilation isn't difficult, nor is expecting a gain of at least 20% unreasonable over straight interpretation. (Expecting a factor of 10-12 speedup not unreasonable in some circumstances) Ignoring parrot, I'll point you at some of the Scheme and Smalltalk setups. (Andyou'd be hard-pressed to find something to beat smalltalk for runtime dynamism) If you want to bet, I'll put up $10 and a round of beer (or other beverage of your choice) at OSCON 2004 for all the python labs & zope folks that says parrot beats the current python interpreter when running all the python benchmark suite programs (that don't depend on extensions written in C, since we're shooting for pure engine performance) from bytecode. Game? -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

Dan Sugalski wrote:
Not on python-dev.
If you want to bet
Just show me the code.
If you're talking about simple stuff like pystone, people have already done *interpreters* that can run that kind of bench- mark 5-25 times faster than CPython. All it takes is a fast GC, call-site caching, some low-overhead primitives, and a fast calling mechanism. But if you want to run real Python programs, you still have to solve all the hard problems. Judging from the Parrot material I've seen, it's not obvious that you're even aware of what they are. After all, if you know how to solve them, wouldn't it be easier to tell us what you've done (or plan to do), instead of pointing to non-Parrot implementations of non-Python languages done by others? But let's hope I'm wrong. The only way to *prove* me wrong is to ship the code. </F>

At 9:12 AM +0100 1/31/03, Fredrik Lundh wrote:
The original topic was native-code compilers, and it was put forth that writing one for a dynamic language was either very hard or impossble, and if you did there wasn't much win. The counterexamples of the Scheme and Smalltalk compilers are relevant--both languages are more dynamic at runtime, and both languages have native compilers that provide significant speed benefits, and both languages have implementations with relatively little man-time and brainpower that provided significant speed benefits without semantic changes. Unless I'm misreading you, and you're positing that python's more dynamic at runtime than smalltalk, but I presume that's not the case.
If you want to bet
Just show me the code.
That was the point of the bet.
Then pick one you consider more indicative. I don't much care, honestly--as long as it doesn't depend on C extensions it's fine.
I have a list of hard problems. Luckily for me, python doesn't hit any of them, but that may as much be because of what I consider a "hard problem" than anything else.
If parrot was finished, sure. It's not, hence the 2004 date on the challenge (while I could meet the deadline for this year's OSCON, enough other things would get shuffled aside that it wouldn't be a good allocation of resources) and the Scheme and Smalltalk examples, which *do* exist, and have for quite some time. C'mon Fred, you *know* that if I just told you how I planned to solve the problems I'd get ripped to shreds (and rightly so) for handwaving past the implementation issues, and nothing I have now will satisfy, as it's not finished so there's always that "well, it isn't done yet" rejoinder. (Which, again, is both fine and appropriate) Still, the challenge stands if you or anyone else wants to take me up on it. Catch me before OSCON 2003 and we'll go from there. -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

Dan Sugalski <dan@sidhe.org> writes:
It's unclear to me that scheme is more dynamic than Python. Are you thinking of call/cc or something? What I know of Smalltalk (i.e. almost nothing) lends me to believe you on that one. Cheers, M. -- FORD: Just pust the fish in your ear, come on, it's only a little one. ARTHUR: Uuuuuuuuggh! -- The Hitch-Hikers Guide to the Galaxy, Episode 1

[Michael Hudson]
It's unclear to me that scheme is more dynamic than Python.
I don't think Scheme is more dynamic either. The language has set types that you just don't happen to have access to or see directly. But there is no clear cut way of defining new types short of writing it in C. It was bad enough reading Paul Graham calling Python "water-down Lisp with infix syntax and no macros", but having Scheme being considered more dynamic is blasphemy in my book (but that might be because Scheme was forced upon me in school =). -Brett

On Fri, 31 Jan 2003, Brett Cannon wrote:
It was bad enough reading Paul Graham calling Python "water-down Lisp with infix syntax and no macros"
Python *does* have infix syntax and no macros. I happen to consider both of these things significant advantages! -- ?!ng

Brett Cannon <bac@OCF.Berkeley.EDU>:
Yo. Don't you be dissin' my homeboy Paul. He is flyyyy. Actually, I had a long lunch with Paul recently and am in a position to state confidently that he thinks Python is pretty nifty. Yes, Paul has some LISP-inspired criticisms. I agree with some (yes, Python would be a better language if it were a little more expression-oriented) and I disgree with others (I have regretfully concluded that macros are a bad idea). On the other hand, he concedes that dict as a first-class type is a very good idea, and that the Python OO system is right in style even if arguable in detail. Paul's website recommends Python as the best alternative if you can't do LISP; see <http://www.paulgraham.com/faq.html>. Under the circumstances (which include the fact that he's one of the two or three influence leaders of the LISP community) that's about the strongest endorsement one could expect. Guido, if you ever get a chance to hang with Paul, take it. He's an very intelligent and capable person, well-spoken, a natural leader, and no mean language designer himself. Knowing both of you, I predict you would get along famously and learn a lot from each other. In fact, you could do a lot worse for a keynote speaker at a future Python conference than to invite him. And then listen to his talk carefully. Very few people would be better qualified than Paul as a friendly critic from a different tradition. -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Brett Cannon wrote:
It's unclear to me that scheme is more dynamic than Python.
I don't think Scheme is more dynamic either.
from a language runtime perspective, I don't think Smalltalk is more dynamic either. I'm still waiting for the Parroteers to prove me wrong. </F>

At 8:44 AM +0100 2/1/03, Fredrik Lundh wrote:
You'll probably be waiting a long time--it's generally off topic for the list, most people who aren't convinced won't believe me anyway, and I don't really care enough about this to put in the effort to convince you. -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

At 12:00 PM +0100 2/3/03, Christian Tismer wrote:
While it could be, that's unlikely. Still, that'd be part of the ground rules we'd work out if someone takes me up on the challenge. -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

I'm in. Given how low you set your stakes, I don't think you're very confident, so I'd like to call your bluff. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

At 1:15 PM -0500 2/3/03, Guido van Rossum wrote:
The stakes are low because I have to pony up the cash if I lose--my budget's limited, and regardless of how confident I am of winning, if I put up a lot, my wife will kill me dead for making the challenge and, well I think I'd rather avoid that. :) Shall we, then, arrange the details at OSCON 2003, if you'll be there? I expect we can twist the conference organizer's arm into letting us make a tiny announcement near the end... -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

Especially as you'd automatically lose if your wife were to kill you. :)
Sure. We should do this as a lightning talk in the Python lightning track. --Guido van Rossum (home page: http://www.python.org/~guido/)

At 1:41 PM -0500 2/3/03, Guido van Rossum wrote:
Well, we might still beat you, though I'd *definitely* lose personally. :)
Works for me. I doubt we'll need that much time to work things out. OTOH, as conferences love animosity, real or imagined, it wouldn't surprise me if Tim O'Reilly'd be happy to announce this if we can work the details out ahead of time. Fine with me either way... -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

Guido van Rossum wrote:
I'm in. Given how low you set your stakes, I don't think you're very confident, so I'd like to call your bluff. :-)
Any opinion on where to spend effort? I was thinking of dusting off the register VM code (aka rattlesnake). OTOH, perhaps the non-local namespace optimizations would give more bang for the buck. Neil

I was thinking it might be most effective to have a little conversation with Dan's potential sponsors. But maybe that's because my first name is Guido... ;-) Seriously, I expect Dan to lose without any effort on our part, but if we want to make an effort, I think that there could be some low-hanging fruit in inlining certain builtins (e.g. len, range, xrange) that as far as we know aren't shadowed by globals. Is that what you call non-local namespace optimizations? A new VM design would be a major upheaval, but if we can pull it off it would certainly not be a bad idea. John Aycock and some of his students have a design of a new VM that might be worth looking into. --Guido van Rossum (home page: http://www.python.org/~guido/)

At 2:18 PM -0500 2/3/03, Guido van Rossum wrote:
Seriously, I expect Dan to lose without any effort on our part,
Really! Well, then, shall we raise the stakes, and add a cream pie (of the loser's choice) at ten paces to the bet? :) -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

I'd be delighted to deliver it in person. :) --Guido van Rossum (home page: http://www.python.org/~guido/)

At 2:31 PM -0500 2/3/03, Guido van Rossum wrote:
Absolutely--it'd be no fun otherwise. :) -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

Dan> Really! Well, then, shall we raise the stakes, and add a cream pie Dan> (of the loser's choice) at ten paces to the bet? :) Having had personal experience with the inverse relationship between geek quotient and athletic prowess I suspect you might want to make that five paces. ;-) Skip

At 1:36 PM -0600 2/3/03, Skip Montanaro wrote:
Nah, that's OK--I'll be practicing my aim before then. :-P -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

That was my line. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

Dan Sugalski <dan@sidhe.org>:
Really! Well, then, shall we raise the stakes, and add a cream pie (of the loser's choice) at ten paces to the bet? :)
Oh, *goddess*. I want to emcee this event in the worst way. And I know exactly how to do it -- like a herald reading cartels of defiance at a joust. Gentlemen, you've got yourself a master of ceremonies and a frame of comic patter, if you want it. For this I might even put on a tuxedo... -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

At 3:25 PM -0500 2/3/03, Eric S. Raymond wrote:
Argh! The very thought of that is making me want to gouge my eyes out. :) Seriously, this is something we'll have to work out with the conference folks. Wouldn't surprise me if they may have someone in mind, or if not we go with some semi-independent person. (Perhaps some bitter Lisp hacker that hates all of us because everyone's more successful than Lisp... :) -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

Dan Sugalski <dan@sidhe.org>:
If you suggest Paul Graham I'm going to have to hurt you. -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

At 5:41 PM -0500 2/3/03, Eric S. Raymond wrote:
Hey, if we can pitch a pie at him regardless of who wins, I'm OK with it. Come to think of it, if you want to host it...: ) -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk

Dan Sugalski <dan@sidhe.org>:
Huh. You know, it didn't occur to me that bias could be an issue here. Not to worry; if Guido loses, I will be just as snarky at him as I would have been at you. One of the reasons I like Guido is that I am confident he would expect no less of me :-). -- <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
participants (27)
-
Armin Rigo
-
barry@python.org
-
Brett Cannon
-
Christian Tismer
-
Dan Sugalski
-
Dave Brueck
-
Eric S. Raymond
-
Francois Pinard
-
Fredrik Lundh
-
Graham Guttocks
-
Greg Ewing
-
Greg Ward
-
Guido van Rossum
-
holger krekel
-
Jeremy Hylton
-
jeremy@zope.com
-
Just van Rossum
-
Ka-Ping Yee
-
Kevin Altis
-
list-python-dev@ccraig.org
-
Michael Hudson
-
Neal Norwitz
-
Neil Schemenauer
-
Samuele Pedroni
-
Skip Montanaro
-
Terry Reedy
-
Tim Peters