Mailman 3 June 1999 - Python-Dev

Re: [Python-Dev] ActiveState & fork & Perl
by Tim Peters 14 Jul '99

14 Jul '99

Moving back in time ... [GordonM] > Perhaps Christian's stackless Python would enable green threads... [Guido] > This has been suggested before... While this seems possible at first, > all blocking I/O calls would have to be redone to pass control to the > thread scheduler, before this would be useful -- a huge task! I didn't understand this. If I/O calls are left alone, and a green thread hit one, the whole program just sits there waiting for the call to complete, right? But if the same thing happens using "real threads" today, the same thing happens today anyway <wink>. That is, if a thread doesn't release the global lock before a blocking call today, the whole program just sits there etc. Or do you have some other kind of problem in mind here? unconvincedly y'rs - tim

11 42

ob_refcnt access
by Vladimir Marangozov 01 Jul '99

01 Jul '99

How about introducing internal macros for explicit ob_refcnt accesses in the core? Actually, there are a number of places where one can see "op->ob_refcnt" logic, which could be replaced with _Py_GETREF(op), _Py_SETREF(op, n) thus decoupling completely the low level refcount management defined in object.h: #define _Py_GETREF(op) (((PyObject *)op)->ob_refcnt) #define _Py_SETREF(op, n) (((PyObject *)op)->ob_refcnt = (n)) Comments? I've contributed myself to the mess in intobject.c & floatobject.c, so I thought that such macros would make the code cleaner. Here's the current state of affairs: python/dist/src>find . -name "*.[c]" -exec grep ob_refcnt {} \; -print (void *) v, ((PyObject *) v)->ob_refcnt)) ./Modules/_tkinter.c if (self->arg->ob_refcnt > 1) { \ if (ob->ob_refcnt < 2 || self->fast) if (args->ob_refcnt > 1) { ./Modules/cPickle.c if (--inst->ob_refcnt > 0) { ./Objects/classobject.c if (result->ob_refcnt == 1) ./Objects/fileobject.c if (PyFloat_Check(p) && p->ob_refcnt != 0) if (!PyFloat_Check(p) || p->ob_refcnt == 0) { if (PyFloat_Check(p) && p->ob_refcnt != 0) { p, p->ob_refcnt, buf); ./Objects/floatobject.c if (PyInt_Check(p) && p->ob_refcnt != 0) if (!PyInt_Check(p) || p->ob_refcnt == 0) { if (PyInt_Check(p) && p->ob_refcnt != 0) p, p->ob_refcnt, p->ob_ival); ./Objects/intobject.c assert(v->ob_refcnt == 1); /* Since v will be used as accumulator! */ ./Objects/longobject.c if (op->ob_refcnt <= 0) op->ob_refcnt, (long)op); op->ob_refcnt = 1; if (op->ob_refcnt < 0) fprintf(fp, "[%d] ", op->ob_refcnt); ./Objects/object.c if (!PyString_Check(v) || v->ob_refcnt != 1) { if (key->ob_refcnt == 2 && key == value) { ./Objects/stringobject.c if (!PyTuple_Check(op) || op->ob_refcnt != 1) { if (v == NULL || !PyTuple_Check(v) || v->ob_refcnt != 1) { ./Objects/tupleobject.c if (PyList_Check(seq) && seq->ob_refcnt == 1) { if (args->ob_refcnt > 1) { ./Python/bltinmodule.c if (value->ob_refcnt != 1) ./Python/import.c return PyInt_FromLong((long) arg->ob_refcnt); ./Python/sysmodule.c -- Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252

6 11

Is Python Free Software, free software, Open Source, open source, etc?
by Mark Hammond 30 Jun '99

30 Jun '99

This probably isnt the correct list, but I really dont want to start a philosophical discussion - hopefully people here are both "in the know" and able to resist a huge thread :-) Especially given the recent slashdot flamefest between RMS and ESR, I thought it worth getting correct. I just read a statement early in our book - "Python is an Open Source tool, ...". Is this "near enough"? Should I avoid this term in preference for something more generic (ie, even simply dropping the caps?) - but the OS(tm) idea seems doomed anyway... Just-hoping-to-avoid-flame-mail-from-rabid-devotees-of-either-religion :-) Mark.

5 5

Merge the string_methods tag?
by Mark Hammond 28 Jun '99

28 Jun '99

Ive been running the string_methods tag (term?) under CVS for quite some time now, and it seems to work perfectly. I admit that I havent stressed the string methods much, but I feel confident that Barry's patches havent broken existing string code. Also, I find using that tag with CVS a bit of a pain. A few updates have been checked into the main branch, and you tend to miss these (its a pity CVS can't be told "only these files are affected by this tag, so the rest should follow the main branch." I know I can do that personally, but that means I personally need to know all files possibly affected by the branch.) Anyway, I digress... I propose that these extensions be merged into the main branch. The main advantage is that we force more people to bash on it, rather than allowing them to make that choice <wink>. If the Unicode type is also considered highly experimental, we can make a new tag for that change, but that is really quite independant of the string methods. Mark.

6 10

Paul Prescod: add Expat to 1.6
by Andrew M. Kuchling 28 Jun '99

28 Jun '99

Paul Prescod sent the following note to the XML-SIG mailing list. Thoughts? --amk

4 4

RE: [Python-Dev] ob_refcnt access
by Sam Rushing 28 Jun '99

28 Jun '99

Mark Hammond writes: > I tried, and look what happened :-) Seriously, some if this stuff > would be way cool. > > Bit I also understand completely the silence on this issue. When > the thread started, there was much discussion about exactly what > the hell these continuation/coroutine thingies even were. However, > there were precious few real-world examples where they could be > used. A few acedemic, theoretical places, but the only real > contender I have seen brought up was Medusa. There were certainly > no clear examples of "as soon as we have this, I could change abc > to take advantage, and this would give us the very cool xyz" Part of the problem is that we didn't have the feature to play with. Many of the possibilities are showing up now that it's here... The basic advantage to coroutines is they allow you to turn any event-driven/state-machine problem into one that is managed with 'normal' control state; i.e., for loops, while loops, nested procedure calls, etc... Here are a few possible real-world uses: ================================================== Parsing. I remember a discussion from a few years back about the distinction between 'push' and 'pull' model parsers. Coroutines let you have it both ways; you can write a parser in the most natural way (pull), but use it as a 'push'; i.e. for a web browser. ================================================== "http sessions". A single 'thread' of control that is re-entered whenever a hit from a particular user ('session') comes in to the web server: [Apologies to those that have already seen this cheezy example] def ecommerce (session): session.login() # sends a login form, waits for it to return basket = [] while 1: item = session.shop_for_item() if item: basket.append (item) else: break if basket: session.get_shipping_info() session.get_payment_info() session.transact() 'session.shop_for_item()' will resume the main coroutine, which will resume this coroutine only when a new hit comes in from that session/user, and 'return' this hit to the while loop. I have a little web server that uses this idea to play blackjack: http://www.nightmare.com:7777/ http://www.nightmare.com/stuff/blackjack_httpd.py [though I'm a little fuzzy on the rules]. Rather than building a state machine that keeps track of where the user has been, and what they're doing, you can keep all the state in local variables (like 'basket' above) - in other words, it's a much more natural style of programming. ================================================== One of the areas I'm most excited about is GUI coding. All GUI's are event driven. All GUI code is therefore written in a really twisted, state-machine fashion; interactions are very complex. OO helps a bit, but doesn't change the basic difficulty - past a certain point interesting things become too complex to try... Mr. Fuchs' paper ("Escaping the event loop: an alternative control structure for multi-threaded GUIs") does a much better job of describing this than I can: http://cs.nyu.edu/phd_students/fuchs/ http://cs.nyu.edu/phd_students/fuchs/gui.ps ================================================== Tim's example of 'dumping' a computation in the middle and storing it on disk (or sending it over a network), is not a fantasy... I have a 'stackless' Scheme system that does this right now. ================================================== Ok, final example. Isn't there an interface in Python to call a certain function after every so many vm insns? Using coroutines you could hook into this and provide non-preemptive 'threads' for those platforms that don't have them. [And the whole thing would be written in Python, not in C!] ================================================== > So, if anyone else if feeling at all like me about this issue, they > are feeling all warm and fuzzy knowing that a few smart people are > giving us the facility to do something we hope we never, ever have > to do. :-) "When the only tool you have is a hammer, everything looks like a nail". I saw the guys over in the Scheme shop cutting wood with a power saw; now I feel like a schmuck with my hand saw. You are right to be frightened by the strangeness of the underlying machinery; hopefully a simple and easy-to-understand interface can be built for the C level as well as Python. I think Christian's 'frame dispatcher' is fairly clear, and not *that* much of a departure from the current VM; it's amazing to me how little work really had to be done! -Sam

2 1

Re: ob_refcnt access (fwd)
by Vladimir Marangozov 25 Jun '99

25 Jun '99

FYI, my second message on this issue didn't reach the list because of a stupid error of mine, so Guido and I exchanged two mails in private. His response to the msg below was that he thinks that tweaking the refcount scheme at this level wouldn't contribute much and that he doesn't intend to change anything on this until 2.0 which will be rewritten from scratch. Besides, if I want to satisfy my curiosity in hacking the refcounts I can do it with a small patch because I've already located the places where the ob_refcnt slot is accessed directly. ----- Forwarded message -----

2 1

Re: ob_refcnt access
by Vladimir Marangozov 24 Jun '99

24 Jun '99

marangoz wrote: > > > How about introducing internal macros for explicit ob_refcnt accesses > in the core? Actually, there are a number of places where one can see > "op->ob_refcnt" logic, which could be replaced with _Py_GETREF(op), > _Py_SETREF(op, n) thus decoupling completely the low level refcount > management defined in object.h: > > #define _Py_GETREF(op) (((PyObject *)op)->ob_refcnt) > #define _Py_SETREF(op, n) (((PyObject *)op)->ob_refcnt = (n)) > > Comments? Of course, the above should be (PyObject *)(op)->ob_refcnt. Also, I forgot to mention that if this detail doesn't hurt code aesthetics, one (I) could experiment more easily all sort of weird things with refcounting... I formulated the same wish for malloc & friends some time ago, that is, use everywhere in the core PyMem_MALLOC, PyMem_FREE etc, which would be defined for now as malloc, free, but nobody seems to be very excited about a smooth transition to other kinds of malloc. Hence, I reiterate this wish, 'cause switching to macros means preparing the code for the future, even if in the future it remains intact ;-). Defining these basic interfaces is clearly Guido's job :-) as he points out in his summary of the last Open Source summit, but nevertheless, I'm raising the issue to let him see what other people think about this and allow him to make decisions easier :-) -- Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252

1 0

Re: ob_refcnt access
by None 24 Jun '99

24 Jun '99

marangoz wrote: > > > How about introducing internal macros for explicit ob_refcnt accesses > in the core? Actually, there are a number of places where one can see > "op->ob_refcnt" logic, which could be replaced with _Py_GETREF(op), > _Py_SETREF(op, n) thus decoupling completely the low level refcount > management defined in object.h: > > #define _Py_GETREF(op) (((PyObject *)op)->ob_refcnt) > #define _Py_SETREF(op, n) (((PyObject *)op)->ob_refcnt = (n)) > > Comments? Of course, the above should be (PyObject *)(op)->ob_refcnt. Also, I forgot to mention that if this detail doesn't hurt code aesthetics, one (I) could experiment more easily all sort of weird things with refcounting... I formulated the same wish for malloc & friends some time ago, that is, use everywhere in the core PyMem_MALLOC, PyMem_FREE etc, which would be defined for now as malloc, free, but nobody seems to be very excited about a smooth transition to other kinds of malloc. Hence, I reiterate this wish, 'cause switching to macros means preparing the code for the future, even if in the future it remains intact ;-). Defining these basic interfaces is clearly Guido's job :-) as he points out in his summary of the last Open Source summit, but nevertheless, I'm raising the issue to let him see what other people think about this and allow him to make decisions easier :-) -- Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 ----- End of forwarded message ----- -- Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252

1 0

Re: ob_refcnt access
by None 24 Jun '99

24 Jun '99

marangoz wrote: > > > How about introducing internal macros for explicit ob_refcnt accesses > in the core? Actually, there are a number of places where one can see > "op->ob_refcnt" logic, which could be replaced with _Py_GETREF(op), > _Py_SETREF(op, n) thus decoupling completely the low level refcount > management defined in object.h: > > #define _Py_GETREF(op) (((PyObject *)op)->ob_refcnt) > #define _Py_SETREF(op, n) (((PyObject *)op)->ob_refcnt = (n)) > > Comments? Of course, the above should be (PyObject *)(op)->ob_refcnt. Also, I forgot to mention that if this detail doesn't hurt code aesthetics, one (I) could experiment more easily all sort of weird things with refcounting... I formulated the same wish for malloc & friends some time ago, that is, use everywhere in the core PyMem_MALLOC, PyMem_FREE etc, which would be defined for now as malloc, free, but nobody seems to be very excited about a smooth transition to other kinds of malloc. Hence, I reiterate this wish, 'cause switching to macros means preparing the code for the future, even if in the future it remains intact ;-). Defining these basic interfaces is clearly Guido's job :-) as he points out in his summary of the last Open Source summit, but nevertheless, I'm raising the issue to let him see what other people think about this and allow him to make decisions easier :-) -- Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 ----- End of forwarded message ----- -- Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252

1 0

Python-Dev June 1999