Mailman 3 May 2012 - pypy-dev

alioth tests?
by xancorreu May 7, 2012

May 7, 2012

Hi, I'm just curious: can you make the tests of alioth [http://shootout.alioth.debian.org/u32q/which-programming-languages-are-fast… If you do, you can compare pypy to cpython and also to other programming languages. What do you think about this idea? Thanks in advance, Xan.

5 9

feedback on PEP 421
by Eric Snow May 6, 2012

May 6, 2012

Hi, I'm looking for feedback on PEP 421: "Adding sys.implementation". The idea came up in 2009 and garnered positive feedback, but didn't go anywhere. I've revived it and am hoping to get it worked out in time for Python 3.3. Any feedback would be very helpful, particularly with regard to the decision on the type of sys.implementation and the constraints on sys.implementation.version. Thanks. -eric http://www.python.org/dev/peps/pep-0421/

2 1

github pypy repo mirror
by Yury Selivanov May 5, 2012

May 5, 2012

Hello, It looks like https://github.com/pypy/pypy is out of sync. Can somebody take a look at that? Thanks! - Yury

4 11

win32-stdlib branch
by Matti Picus May 5, 2012

May 5, 2012

I have modified files in lib-python/modified-2.7 to close all open files so that more of the lib-python tests pass in windows. The modifications live on the win32-stdlib branch, a recent diff to default shows: hg diff -r default --stat lib-python/modified-2.7/mailbox.py | 2179 +++++++++++++++++++++ lib-python/modified-2.7/tarfile.py | 60 +- lib-python/modified-2.7/test/test_mailbox.py | 1987 +++++++++++++++++++ lib-python/modified-2.7/test/… [View More]

2 2

Re: [pypy-dev] Syntax for the 'transaction' module
by Laura Creighton May 4, 2012

May 4, 2012

Listening to the whispers of your own creativity is always hard and fraught with error, but I am getting the strong sense that what we are up against is the idea that _processes_ are _things_ that are made up of _lines of code_ which _run_ on a _core_. Well, a sheep is a _thing_ in a very real sense that a flock is not 'a thing' but only 'a way of looking'. And the 'running' part can be thought of as 'happening all the time' -- it is just from time to time we need to provide a coherant way of … [View More]

3 2

Re: [pypy-dev] Syntax for the 'transaction' module
by Laura Creighton May 3, 2012

May 3, 2012

In a message of Wed, 02 May 2012 09:22:03 +0200, Armin Rigo writes: >Hi Laura, > >A note on my two previous messages: in the first one I argued about >@breakable, while the second one I suggested to use threads and "with >atomic". The relationship between the two is: in the first model you >would put @breakable on some outer functions until it calls things >that are supposed to be atomic; in the second model you would identify >the call in question and put "with atomic" … [View More]there. I would personally >prefer @breakable because it forces the "atomic by default" idea a bit >further, but I fear that the second model is better because it is >fully compatible with existing programs. Moreover, as I argue >repeatedly, regular apps should use neither, and only library >implementors should have to worry. > > >A bientÃ´t, > >Armin. As promised, I have been doing more thinking while paddling. We camped out in GÃ¶teborg's archipelago and the island, as is common, has sheep. It occured to me that what we have is very like a shepherding problem. Since my uncle raised sheep, I have some personal experience with these problems from my childhood. 1. Assume you have 2 pastures, with a gate between them, and a flock of sheep. Your flock is in one pasture, and you want to get them to the other pasture through the gate. 2. Sheep are, to use the terms we have been using, 'unbreakable', and 'atomic'. (If you slice them into pieces to try to get them to fit through the gate, you will end up with non-functioning sheep on the other side). :-) 3. If your gate is narrow, you will have to serialise your sheep, and have them pass through one at a time. 4. If it is wider then parts of different sheep will pass through the gate intermingled from the perspective of a camera mounted on the gate. "Nose of sheep 1, nose of sheep 2, head of sheep 1, nose of sheep 3, head of sheep 3, body of sheep 3, tail of sheep 3, body of sheep 2, body of sheep 1, tail of sheep 2, tail of sheep 1" is what the camera might report as 'having gone passed', and we might conclude that sheep 3 is a small lamb, and that sheep 1 is its mother who slowed down going through the gate so that the lamb could keep up with her -- but all of this doesn't matter because, as long as you do not try to break them, the flock will function perfectly on the other side of the gate without any attention being paid to them by you. The main problem you have in moving pastures like this, as a shepherd, is that ewes (mother sheep) normally have twins or triplets (at least with the breed that my Uncle raised). Hungry lambs want to find their mothers right away, but the rest of the time they are busy frolicking with other lambs, butting heads, chasing each other, and doing other jeuvenile sheep things. They don't play with their siblings any more than with the other lambs. Ewes, on the other hand, want their own offspring close together, and with them all the time. The mechanism that the sheep use to keep track of each other is to bleat. Each lamb knows the sound of its mother's voice, and each ewe knows the sound of its own lambs'. They bleat more or less constantly when then are moving from a part of the pasture to another, and of course when they are being moved -- and they aren't all that quiet when grazing, either. If you are moving sheep through a gate, and it turns out that you have one lamb in the new pasture while it's sibling(s) are in the old pasture you can freeze your whole operation. The mother ewe will stand in your gate and refuse to move until she is reunited with all her lambs, which might be impossible if your gate is narrow. I don't think we have anything like this in our model, though maybe there is something analogous to bleating. Do we need to handle the case of bits of code that are themselves atomic that depend on being run with other bits of code in a predictable sequence, or one of several possible sequences? But you don't 'lock sheep', or sheep families. All you can do is try to submit them to the gate again. And from personal experience, it would be *very* *very* useful if you could magically restore the lamb that is early into the new pasture into the old pasture, right beside its mother. (Of course then you would just be able to teleport the whole lot and go away with gates altogether). At any rate, if we want to discourage people from using what they think they already know about concurrency and locking and whatnot, we might want to talk about what we are doing as herding a flock of <whatever we call the atomic series of instructions>. I'm pretty sure that this language isn't already being used somewhere to mean something else, at any rate. :-) You (or mostly your dogs) 'drive' and 'fetch' sheep through gates -- driving is away from the shepherd and fetching is towards the shepherd -- and you gather individual sheep into a flock. www.herding-dog-training-border-collie-sheepdog-dvd.com/herding_sheepdog_co… has some other interesting -- but I doubt useful words. But maybe, instead of scheduling, we are gathering? Not sure any of this helps at all, Laura [View Less]

2 1

Re: [pypy-dev] Syntax for the 'transaction' module
by Laura Creighton May 2, 2012

May 2, 2012

In a message of Tue, 01 May 2012 11:27:56 +0200, Armin Rigo writes: >* First question: 'transaction'. The name is kind of bogus, because >it implies that it must be based on transactional memory. Such a name >doesn't make sense if, say, you are running the single-core emulator >version. What the module is about is to give a way to schedule tasks >and run them in some unspecified order. The problem is that the csc namespace is overloaded with terms that already mean 'make-a-… [View More]noun out of the verb I want to do', and the verb list that means 'do something' is a rather full namespace as well. Often you can get what you want by dropping the noun-ification -- so you would replace 'transaction' with 'transact'. This will not help the 'tranactional memory' association. So how about 'schedule'? My experience says that if you pick a verb, and not a noun, the code often comes out cleaner, because otherwise you tend to write -- or the people who use your library tend to write -- state-full things by habit even when they are not needed because the language points them that way. It looks to me as if your design is rather stateless by design, so maybe you want a verb and not a noun in any case. >* How about replacing the global functions 'transaction.add()' and >'transaction.run()' with a class, like 'transaction.Runner', that you >need to instantiate and on which you call the methods add() and run(). > If moreover the class has an '__exit__()' that redirects to run(), >then you can use it like this: > > with transaction.Runner() as t: > for block in blocks: > t.add(do_stuff, block) > >And maybe, like concurrent.futures, a map() method --- although it >seems to me like CPython was happily relegating the built-in map() in >the corner of "advanced users only"; then adding a map() again seems a >bit counter-productive --- it would look like that: > > with transaction.Runner() as t: > t.map(do_stuff, blocks) > >* Note that the added transactions are only run on __exit__(), not >when you call add() or map(). This might be another argument against >map(). Or an argument to build a different sort of 'add me lots of things' function?? I am not sure this makes sense. I am going kayaking overnight now, I will think of this more while paddling. >* The point of the examples above is that "t" can also be passed >around in the transactions, and t.add() called on it again from there. > Also, such a syntax nicely removes the need for any global state, and >so it opens the door to nesting: you can write one of the above >examples inside code that happens to run itself as a transaction from >some unrelated outer Runner(). Right now, you cannot call >transaction.run() from within a transaction --- and it doesn't make >sense because we cannot tell if the transaction.add() that you just >did were meant to schedule transactions for the outer or the future >inner run(). That's what is better with this proposed new API. >(Supporting it requires more work in the current implementation, >though.) But, for what it is worth, it pleases my aesthetic sense to no end. Which is why I want an 'add me lots of things' that works with this. :-) >* Another issue. I think by now that we need special support to mean: >"I want to end a transaction, then non-transactionally call this C >function that will likely block for some time, and when it returns, I >want to start the next transaction". This seem general enough to >support various kinds of things, like calling select() or >epoll_wait(). The question is what kind of support we want. > >I played with various ideas and I'll present the combination that >satisfies me the most, but I'm open to any other suggestion. > >We could in theory support calling in-line the function, i.e. just >call select() and it will break the current transaction in two. This >is similar to the fact that select() in CPython releases and >re-acquires the GIL. But it breaks the abstraction that every add() >gives *one* transaction. transaction == 'set of things to schedule, atomically'? or something else? Right now it is not clear to me why 'one' is important, probably because I do not understand something. >It kind of goes against the general design >of the API so far, which is that you add() things to do, but don't do >them right now --- they will be done later. To voice it differently, >I dislike this solution because you can break a working program just >by adding a debugging "print" in the middle (assuming that "print" >would also break the current transaction in two, like it releases the >GIL in CPython). It would break the program because what used to be >in the same transaction, no longer is: random things (done by other >unrelated transactions) can suddenly have happened because you added a >"print". I can agree that any program that breaks when you add a print is a bear to debig -- I used to get this quite often with Borland's C++ and it made me pull out hair. >The idea I'm playing with is two running modes: "breakable" vs >"non-breakable". Say you have a "@breakable" decorator that you have >to add explicitly on some of your Python functions. The transaction >is breakable only when all functions in the call stack are @breakable. > As soon as one non-breakable function is in the call stack, then the >transaction is not breakable (to err on the side of safety). No clue >if this would make any sense to the user, though. In the end a call >to select() would either break the transaction in two (if the current >mode is "breakable"), or, like now, in non-breakable mode it would >turn the transaction inevitable (which is bad if the C call is >blocking, because it blocks all other transactions too, but which is >at least correct). This seems up-side down to me. Do you want to decorate them as breakable? Or assume that they are breakable and decorate them as unbreakable? Maybe I am missing something crucial, but I think the other way may be easier for newly written code. Your way may be cooler for porting exisitng things, though, I need to think more. > >Thanks for reading all my ranting. Ideas welcome... > > >A bientÃ´t > >Armin. Laura [View Less]

8 12

Syntax for the 'transaction' module
by Armin Rigo May 1, 2012

May 1, 2012

Hi all, Following the blog post about STM, I would like to sollicitate your attention to come up with some better syntax for the 'transaction' module. * First question: 'transaction'. The name is kind of bogus, because it implies that it must be based on transactional memory. Such a name doesn't make sense if, say, you are running the single-core emulator version. What the module is about is to give a way to schedule tasks and run them in some unspecified order. * How about replacing the … [View More]global functions 'transaction.add()' and 'transaction.run()' with a class, like 'transaction.Runner', that you need to instantiate and on which you call the methods add() and run(). If moreover the class has an '__exit__()' that redirects to run(), then you can use it like this: with transaction.Runner() as t: for block in blocks: t.add(do_stuff, block) And maybe, like concurrent.futures, a map() method --- although it seems to me like CPython was happily relegating the built-in map() in the corner of "advanced users only"; then adding a map() again seems a bit counter-productive --- it would look like that: with transaction.Runner() as t: t.map(do_stuff, blocks) * Note that the added transactions are only run on __exit__(), not when you call add() or map(). This might be another argument against map(). * The point of the examples above is that "t" can also be passed around in the transactions, and t.add() called on it again from there. Also, such a syntax nicely removes the need for any global state, and so it opens the door to nesting: you can write one of the above examples inside code that happens to run itself as a transaction from some unrelated outer Runner(). Right now, you cannot call transaction.run() from within a transaction --- and it doesn't make sense because we cannot tell if the transaction.add() that you just did were meant to schedule transactions for the outer or the future inner run(). That's what is better with this proposed new API. (Supporting it requires more work in the current implementation, though.) * Another issue. I think by now that we need special support to mean: "I want to end a transaction, then non-transactionally call this C function that will likely block for some time, and when it returns, I want to start the next transaction". This seem general enough to support various kinds of things, like calling select() or epoll_wait(). The question is what kind of support we want. I played with various ideas and I'll present the combination that satisfies me the most, but I'm open to any other suggestion. We could in theory support calling in-line the function, i.e. just call select() and it will break the current transaction in two. This is similar to the fact that select() in CPython releases and re-acquires the GIL. But it breaks the abstraction that every add() gives *one* transaction. It kind of goes against the general design of the API so far, which is that you add() things to do, but don't do them right now --- they will be done later. To voice it differently, I dislike this solution because you can break a working program just by adding a debugging "print" in the middle (assuming that "print" would also break the current transaction in two, like it releases the GIL in CPython). It would break the program because what used to be in the same transaction, no longer is: random things (done by other unrelated transactions) can suddenly have happened because you added a "print". The idea I'm playing with is two running modes: "breakable" vs "non-breakable". Say you have a "@breakable" decorator that you have to add explicitly on some of your Python functions. The transaction is breakable only when all functions in the call stack are @breakable. As soon as one non-breakable function is in the call stack, then the transaction is not breakable (to err on the side of safety). No clue if this would make any sense to the user, though. In the end a call to select() would either break the transaction in two (if the current mode is "breakable"), or, like now, in non-breakable mode it would turn the transaction inevitable (which is bad if the C call is blocking, because it blocks all other transactions too, but which is at least correct). Thanks for reading all my ranting. Ideas welcome... A bientôt, Armin. [View Less]

2 1

HONG LEONG BANK (Malaysia)
by Edward Lee May 1, 2012

May 1, 2012

Hello, I contacted you to assist in distributing the money left behind by my late client, Regards, Mr Edward Lee +60146308549 Fax:+60(0)321784290

1 0