Mailman 3 July 1999 - Python-Dev

Fancy control flow
by Tim Peters 12 Jul '99

12 Jul '99

Responding to a msg of Guido's that shows up in the archives but didn't come across the mail link (the proper authorities have been notified, and I'm sure appropriate heads will roll at the appropriate pace <wink> ...). > From guido(a)CNRI.Reston.VA.US Mon, 05 Jul 1999 08:06:03 -0400 > Date: Mon, 05 Jul 1999 08:06:03 -0400 > From: Guido van Rossum guido(a)CNRI.Reston.VA.US > Subject: Fake threads (was [Python-Dev] ActiveState & fork & Perl) [generators, coroutines, continuations] > I still don't understand all of this (I have not much of an idea of > what Christian's search for hidden registers is about and what kind of > analysis he needs) but I think of continuations as requiring > (theoretically) coping the current stack (to and from), while > generators and coroutines just need their own piece of stack set aside. A generator needs one frame, period (its own!). "Modern" coroutines can get more involved, mixing regular calls with coroutine transfers in arbitrary ways. About Christian's mysterious quest, we've been pursuing it offline. By "hidden registers" I think he means stuff on the eval stack that should *not* be saved/restored as part of a continuation's state. It's not clear to me that this isn't the empty set. The most debatable case I've seen is the Python "for" loop, which hides an anonymous loop counter on the stack. for i in seq: if func1(i): func2(i) This is more elaborate than necessary <wink>, but since it's the one we've discussed offline I'll just stick with it. Suppose func1 saves a continuation on the first iteration, and func2 invokes that continuation on the fifth. Does the resumed continuation "see" the loop as being on its first iteration or as being on its fifth? In favor of the latter is that the loop above "should be" equivalent to this: hidden = 0 while 1: try: temp = seq[hidden] except IndexError: break hidden = hidden + 1 i = temp if func1(i): func2(i) since that's what "for" *does* in Python. With the latter spelling, it's clear that the continuation should see the loop as being on its fifth iteration (continuations see changes in bindings, and making the loop counter a named local exposes it to that rule). But if the entire eval stack is (conceptually) saved/restored, the loop counter is part of it, so the continuation will see the loop counter at its old value. I think it's arguable either way, and argued in favor of "fifth" initially. Now I'm uncertain, but leaning toward "first". > The difference between any of these and threads (fake or real) is that > they pass control explicitly, while threads (typically) presume > pre-emptive scheduling, i.e. they make independent parallel progress > without explicit synchronization. Yes. > (Hmm, how do you do this with fake threads? Or are these only required > to switch whenever you touch a mutex?) I'd say they're only *required* to switch when one tries to acquire a mutex that's already locked. It would be nicer to switch them as ceval already switches "real threads", that is give another one a shot every N bytecodes. > I'm not sure if there's much of a difference between generators and > coroutines -- it seems just the termination convention. A generator is a semi-coroutine, but is the easier half <wink>. > (Hmm... would/should a generator be able to raise an exception in its > caller? Definitely. This is all perfectly clear for a generator -- it has a unique & guaranteed still-active place to return *to*. Years ago I tried to rename them "resumable functions" to get across what a trivial variation of plain functions they really are ... > A coroutine?) This one is muddier. A (at line A1) transfers to B (at line B1), which transfers at line B2 to A (at line A2), which at line A3 transfers to B (at line B3), and B raises an exception at line B4. The obvious thing to do is to pass it on to line A3+1, but what if that doesn't catch it either? We got to A3 from A2 from B2 from B1, but B1 is long gone. That's a real difference with generators: resuming a generator is stack-like, while a co-transfer is just moving control around a flat graph, like pushing a pawn around a chessboard. The coroutine implementation I posted 5 years ago almost punted on this one: if any coroutine suffered an unhandled exception, all coroutines were killed and an EarlyExit exception was raised in "the main coroutine" (the name given to the thread of your code that created the coroutine objects to begin with). Deserves more thought than that, though. or-maybe-it-doesn't<wink>-ly y'rs - tim

4 12

Win32 Extensions Registered Users
by Mark Hammond 10 Jul '99

10 Jul '99

Hi all, As you may or may not have noticed, I have recently begun offering a Registered Users" program where people who use my Windows extensions can pay $50.00 per 2 years, and get a range of benefits. The primary benefits are: * Early access to binary versions. * Registered Users only mailing list (very low volume to date) * Better support from me. The last benefit really isnt to this list - anyone here will obviously get (and hopefully does get) a pretty good response should they need to mail me. The early access to binary versions may be of interest. As everyone on this list spends considerable and worthwhile effort helping Python, I would like to offer everyone here a free registration. If you would like to take advantage, just send me a quick email. I will email you the "top secret" <wink> location of the Registered Users page (where the very slick and very new Pythonwin can be found). Also, feel free to join the registered users mailing list at http://mailman.pythonpros.com/mailman/listinfo/win32-reg-users. This is low volume, and once volume does increase an announce list will be created, so you can join without fear of more swamping of your mailbox. And just FYI, I am very pleased with the registration process to date. In about 3 weeks I have around 20 paid users! If I can keep that rate up I will be very impressed (although that already looks highly unlikely :-) Even still, I consider it going well. Mark.

1 0

RE: Fake threads (was [Python-Dev] ActiveState & fork & Perl)
by Ken Manheimer 08 Jul '99

08 Jul '99

Christian wrote: > Really, no concern necessary. The state is not saved at all > (despite one frame), it is just not dropped. :-) I have to say, that's not completely reassuring.-) While little or nothing additional is created, stuff that normally would be quite transient remains around. > To shorten this: The whole story is nothing more than a tree, > where exactly one leaf is active at any time, and its view > of the call chain is always linear. That's wonderful - i particularly like that multiple continuations from the same frame only amount to a single retention of the stack for that frame. My concern is not alleviated, however. My concern is the potential, but often-realized hairiness of computation trees. Eg, looped calls to a function amount to nodes with myriad branches - one for each iteration - and each branch can be an arbitrary computation. If there were a continuation retained each time around the loop, worse, somewhere down the call stack within the loop, you could quickly amass a lot of stuff that would otherwise be reaped immediately. So it seems like use of continuations *can* be surprisingly expensive, with the expense commensurate with, and as hard (or easy) to predict as the call dynamics of the call tree. (Boy, i can see how continuations would be useful for backtracking-style chess algorithms and such. Of course, discretion about what parts of the computation is retained at each branch would probably be an important economy for large computations, while stashing the continuation retains everything...) (It's quite possible that i'm missing something - i hope i'm not being thick headed.) Note that i do not raise this to argue against continuations. In fact, they seem to me to be at least the right conceptual foundation for these advanced control structures (i happen to "like" stream abstractions, which i gather is what generators are). It just seems like it may a concern, something about which people experience with continuations experience (eg, the scheme community) would have some lore - accumulated wisdom... ken klm(a)digicool.com

2 1

Welcome to Chris Petrilli
by Guido van Rossum 06 Jul '99

06 Jul '99

Chris, would you mind posting a few bits about yourself? Most of the people on this list have met each other at one point or another (with the big exception of the elusive Tim Peters :-); it's nice to know more than a name... --Guido van Rossum (home page: http://www.python.org/~guido/)

2 1

Re: Welcome Jean-Claude Wippler
by Jean-Claude Wippler 06 Jul '99

06 Jul '99

Thank you Guido, for admitting this newbie to Python-dev :) [Guido: ... you are mostly interested in lurking ... digest mode ...] Fear of being flooded by email, a little shy (who, me?), and yes, a bit of curiosity. Gosh, I got to watch my steps, you figured it all out :) Thanks again. I went through the last month or so of discussion, and am fascinated by the topics and issues you guys are dealing with. And now, seeing Tim's generator/coroutine/continuations description is fantastic. Makes it obvious that I'm already wasting way too much bandwidth. When others come to mind, I'll let them know about this list. But so far, everyone I can come up with already is a member, it seems. -- Jean-Claude

1 0

Welcome Jean-Claude Wippler
by Guido van Rossum 05 Jul '99

05 Jul '99

We have a new python-dev member. Welcome, Jean-Claude! (It seems you are mostly interested in lurking, since you turned on digest mode :-) Remember, the list's archives and member list are public; noth are accessible via http://www.python.org/mailman/listinfo/python-dev I would welcome more members -- please suggest names and addresses to me! --Guido van Rossum (home page: http://www.python.org/~guido/)

1 0

Re: [Python-Dev] Paul Prescod: add Expat to 1.6
by Jack Jansen 01 Jul '99

01 Jul '99

Recently, Skip Montanaro <skip(a)mojam.com> said: > > Andrew> My personal leaning is that we can get more bang for the buck by > Andrew> working on the Distutils effort, so that installing a package > Andrew> like PyExpat becomes much easier, rather than piling more things > Andrew> into the core distribution. > > Amen to that. See Guido's note and my response regarding soundex in the > Doc-SIG. Perhaps you could get away with a very small core distribution > that only contained the stuff necessary to pull everything else from the net > via http or ftp... I don't know whether this subject belongs on the python-dev list (is there a separate distutils list?), but let's please be very careful with this. The Perl people apparently think that their auto-install stuff is so easy to use that if you find a tool on the net that needs Perl they'll just give you a few incantations you need to build the "correct" perl to run the tool, but I've never managed to do so. My last try was when I spent 2 days to try and get the perl-based Palm software for unix up and running. With various incompatilble versions of perl installed in /usr/local by the systems staff and knowing nothing about perl I had to give up at some point, because it was costing far more time (and diskspace:-) than the whole thing was worth. Something like mailman is (afaik) easy to install for non-pythoneers because it only depends on a single, well-defined Python distribution. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm

3 2

ob_refcnt access
by Vladimir Marangozov 01 Jul '99

01 Jul '99

How about introducing internal macros for explicit ob_refcnt accesses in the core? Actually, there are a number of places where one can see "op->ob_refcnt" logic, which could be replaced with _Py_GETREF(op), _Py_SETREF(op, n) thus decoupling completely the low level refcount management defined in object.h: #define _Py_GETREF(op) (((PyObject *)op)->ob_refcnt) #define _Py_SETREF(op, n) (((PyObject *)op)->ob_refcnt = (n)) Comments? I've contributed myself to the mess in intobject.c & floatobject.c, so I thought that such macros would make the code cleaner. Here's the current state of affairs: python/dist/src>find . -name "*.[c]" -exec grep ob_refcnt {} \; -print (void *) v, ((PyObject *) v)->ob_refcnt)) ./Modules/_tkinter.c if (self->arg->ob_refcnt > 1) { \ if (ob->ob_refcnt < 2 || self->fast) if (args->ob_refcnt > 1) { ./Modules/cPickle.c if (--inst->ob_refcnt > 0) { ./Objects/classobject.c if (result->ob_refcnt == 1) ./Objects/fileobject.c if (PyFloat_Check(p) && p->ob_refcnt != 0) if (!PyFloat_Check(p) || p->ob_refcnt == 0) { if (PyFloat_Check(p) && p->ob_refcnt != 0) { p, p->ob_refcnt, buf); ./Objects/floatobject.c if (PyInt_Check(p) && p->ob_refcnt != 0) if (!PyInt_Check(p) || p->ob_refcnt == 0) { if (PyInt_Check(p) && p->ob_refcnt != 0) p, p->ob_refcnt, p->ob_ival); ./Objects/intobject.c assert(v->ob_refcnt == 1); /* Since v will be used as accumulator! */ ./Objects/longobject.c if (op->ob_refcnt <= 0) op->ob_refcnt, (long)op); op->ob_refcnt = 1; if (op->ob_refcnt < 0) fprintf(fp, "[%d] ", op->ob_refcnt); ./Objects/object.c if (!PyString_Check(v) || v->ob_refcnt != 1) { if (key->ob_refcnt == 2 && key == value) { ./Objects/stringobject.c if (!PyTuple_Check(op) || op->ob_refcnt != 1) { if (v == NULL || !PyTuple_Check(v) || v->ob_refcnt != 1) { ./Objects/tupleobject.c if (PyList_Check(seq) && seq->ob_refcnt == 1) { if (args->ob_refcnt > 1) { ./Python/bltinmodule.c if (value->ob_refcnt != 1) ./Python/import.c return PyInt_FromLong((long) arg->ob_refcnt); ./Python/sysmodule.c -- Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252

6 11