Moving back in time ...
[GordonM]
> Perhaps Christian's stackless Python would enable green threads...
[Guido]
> This has been suggested before... While this seems possible at first,
> all blocking I/O calls would have to be redone to pass control to the
> thread scheduler, before this would be useful -- a huge task!
I didn't understand this. If I/O calls are left alone, and a green thread
hit one, the whole program just sits there waiting for the call to complete,
right?
But if the same thing happens using "real threads" today, the same thing
happens today anyway <wink>. That is, if a thread doesn't release the
global lock before a blocking call today, the whole program just sits there
etc.
Or do you have some other kind of problem in mind here?
unconvincedly y'rs - tim
How about introducing internal macros for explicit ob_refcnt accesses
in the core? Actually, there are a number of places where one can see
"op->ob_refcnt" logic, which could be replaced with _Py_GETREF(op),
_Py_SETREF(op, n) thus decoupling completely the low level refcount
management defined in object.h:
#define _Py_GETREF(op) (((PyObject *)op)->ob_refcnt)
#define _Py_SETREF(op, n) (((PyObject *)op)->ob_refcnt = (n))
Comments?
I've contributed myself to the mess in intobject.c & floatobject.c, so
I thought that such macros would make the code cleaner.
Here's the current state of affairs:
python/dist/src>find . -name "*.[c]" -exec grep ob_refcnt {} \; -print
(void *) v, ((PyObject *) v)->ob_refcnt))
./Modules/_tkinter.c
if (self->arg->ob_refcnt > 1) { \
if (ob->ob_refcnt < 2 || self->fast)
if (args->ob_refcnt > 1) {
./Modules/cPickle.c
if (--inst->ob_refcnt > 0) {
./Objects/classobject.c
if (result->ob_refcnt == 1)
./Objects/fileobject.c
if (PyFloat_Check(p) && p->ob_refcnt != 0)
if (!PyFloat_Check(p) || p->ob_refcnt == 0) {
if (PyFloat_Check(p) && p->ob_refcnt != 0) {
p, p->ob_refcnt, buf);
./Objects/floatobject.c
if (PyInt_Check(p) && p->ob_refcnt != 0)
if (!PyInt_Check(p) || p->ob_refcnt == 0) {
if (PyInt_Check(p) && p->ob_refcnt != 0)
p, p->ob_refcnt, p->ob_ival);
./Objects/intobject.c
assert(v->ob_refcnt == 1); /* Since v will be used as accumulator! */
./Objects/longobject.c
if (op->ob_refcnt <= 0)
op->ob_refcnt, (long)op);
op->ob_refcnt = 1;
if (op->ob_refcnt < 0)
fprintf(fp, "[%d] ", op->ob_refcnt);
./Objects/object.c
if (!PyString_Check(v) || v->ob_refcnt != 1) {
if (key->ob_refcnt == 2 && key == value) {
./Objects/stringobject.c
if (!PyTuple_Check(op) || op->ob_refcnt != 1) {
if (v == NULL || !PyTuple_Check(v) || v->ob_refcnt != 1) {
./Objects/tupleobject.c
if (PyList_Check(seq) && seq->ob_refcnt == 1) {
if (args->ob_refcnt > 1) {
./Python/bltinmodule.c
if (value->ob_refcnt != 1)
./Python/import.c
return PyInt_FromLong((long) arg->ob_refcnt);
./Python/sysmodule.c
--
Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr
http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252
This probably isnt the correct list, but I really dont want to start a
philosophical discussion - hopefully people here are both "in the know" and
able to resist a huge thread :-)
Especially given the recent slashdot flamefest between RMS and ESR, I
thought it worth getting correct.
I just read a statement early in our book - "Python is an Open Source tool,
...".
Is this "near enough"? Should I avoid this term in preference for
something more generic (ie, even simply dropping the caps?) - but the
OS(tm) idea seems doomed anyway...
Just-hoping-to-avoid-flame-mail-from-rabid-devotees-of-either-religion :-)
Mark.
Ive been running the string_methods tag (term?) under CVS for quite some
time now, and it seems to work perfectly. I admit that I havent stressed
the string methods much, but I feel confident that Barry's patches havent
broken existing string code.
Also, I find using that tag with CVS a bit of a pain. A few updates have
been checked into the main branch, and you tend to miss these (its a pity
CVS can't be told "only these files are affected by this tag, so the rest
should follow the main branch." I know I can do that personally, but that
means I personally need to know all files possibly affected by the branch.)
Anyway, I digress...
I propose that these extensions be merged into the main branch. The main
advantage is that we force more people to bash on it, rather than allowing
them to make that choice <wink>. If the Unicode type is also considered
highly experimental, we can make a new tag for that change, but that is
really quite independant of the string methods.
Mark.
Mark Hammond writes:
> I tried, and look what happened :-) Seriously, some if this stuff
> would be way cool.
>
> Bit I also understand completely the silence on this issue. When
> the thread started, there was much discussion about exactly what
> the hell these continuation/coroutine thingies even were. However,
> there were precious few real-world examples where they could be
> used. A few acedemic, theoretical places, but the only real
> contender I have seen brought up was Medusa. There were certainly
> no clear examples of "as soon as we have this, I could change abc
> to take advantage, and this would give us the very cool xyz"
Part of the problem is that we didn't have the feature to play with.
Many of the possibilities are showing up now that it's here...
The basic advantage to coroutines is they allow you to turn any
event-driven/state-machine problem into one that is managed with
'normal' control state; i.e., for loops, while loops, nested procedure
calls, etc...
Here are a few possible real-world uses:
==================================================
Parsing. I remember a discussion from a few years back about the
distinction between 'push' and 'pull' model parsers. Coroutines let
you have it both ways; you can write a parser in the most natural way
(pull), but use it as a 'push'; i.e. for a web browser.
==================================================
"http sessions". A single 'thread' of control that is re-entered
whenever a hit from a particular user ('session') comes in to the web
server:
[Apologies to those that have already seen this cheezy example]
def ecommerce (session):
session.login() # sends a login form, waits for it to return
basket = []
while 1:
item = session.shop_for_item()
if item:
basket.append (item)
else:
break
if basket:
session.get_shipping_info()
session.get_payment_info()
session.transact()
'session.shop_for_item()' will resume the main coroutine, which will
resume this coroutine only when a new hit comes in from that
session/user, and 'return' this hit to the while loop.
I have a little web server that uses this idea to play blackjack:
http://www.nightmare.com:7777/http://www.nightmare.com/stuff/blackjack_httpd.py
[though I'm a little fuzzy on the rules].
Rather than building a state machine that keeps track of where the
user has been, and what they're doing, you can keep all the state in
local variables (like 'basket' above) - in other words, it's a much
more natural style of programming.
==================================================
One of the areas I'm most excited about is GUI coding. All GUI's are
event driven. All GUI code is therefore written in a really twisted,
state-machine fashion; interactions are very complex. OO helps a bit,
but doesn't change the basic difficulty - past a certain point
interesting things become too complex to try...
Mr. Fuchs' paper ("Escaping the event loop: an alternative control
structure for multi-threaded GUIs") does a much better job of
describing this than I can:
http://cs.nyu.edu/phd_students/fuchs/http://cs.nyu.edu/phd_students/fuchs/gui.ps
==================================================
Tim's example of 'dumping' a computation in the middle and storing it
on disk (or sending it over a network), is not a fantasy... I have a
'stackless' Scheme system that does this right now.
==================================================
Ok, final example. Isn't there an interface in Python to call a
certain function after every so many vm insns? Using coroutines you
could hook into this and provide non-preemptive 'threads' for those
platforms that don't have them. [And the whole thing would be written
in Python, not in C!]
==================================================
> So, if anyone else if feeling at all like me about this issue, they
> are feeling all warm and fuzzy knowing that a few smart people are
> giving us the facility to do something we hope we never, ever have
> to do. :-)
"When the only tool you have is a hammer, everything looks like a
nail". I saw the guys over in the Scheme shop cutting wood with a
power saw; now I feel like a schmuck with my hand saw.
You are right to be frightened by the strangeness of the underlying
machinery; hopefully a simple and easy-to-understand interface can be
built for the C level as well as Python. I think Christian's 'frame
dispatcher' is fairly clear, and not *that* much of a departure from
the current VM; it's amazing to me how little work really had to be
done!
-Sam
FYI, my second message on this issue didn't reach the list because
of a stupid error of mine, so Guido and I exchanged two mails
in private. His response to the msg below was that he thinks
that tweaking the refcount scheme at this level wouldn't contribute
much and that he doesn't intend to change anything on this until 2.0
which will be rewritten from scratch.
Besides, if I want to satisfy my curiosity in hacking the refcounts
I can do it with a small patch because I've already located the places
where the ob_refcnt slot is accessed directly.
----- Forwarded message -----
marangoz wrote:
>
>
> How about introducing internal macros for explicit ob_refcnt accesses
> in the core? Actually, there are a number of places where one can see
> "op->ob_refcnt" logic, which could be replaced with _Py_GETREF(op),
> _Py_SETREF(op, n) thus decoupling completely the low level refcount
> management defined in object.h:
>
> #define _Py_GETREF(op) (((PyObject *)op)->ob_refcnt)
> #define _Py_SETREF(op, n) (((PyObject *)op)->ob_refcnt = (n))
>
> Comments?
Of course, the above should be (PyObject *)(op)->ob_refcnt. Also, I forgot
to mention that if this detail doesn't hurt code aesthetics, one (I) could
experiment more easily all sort of weird things with refcounting...
I formulated the same wish for malloc & friends some time ago, that is,
use everywhere in the core PyMem_MALLOC, PyMem_FREE etc, which would be
defined for now as malloc, free, but nobody seems to be very excited
about a smooth transition to other kinds of malloc. Hence, I reiterate
this wish, 'cause switching to macros means preparing the code for the
future, even if in the future it remains intact ;-).
Defining these basic interfaces is clearly Guido's job :-) as he points
out in his summary of the last Open Source summit, but nevertheless,
I'm raising the issue to let him see what other people think about this
and allow him to make decisions easier :-)
--
Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr
http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252
----- End of forwarded message -----
--
Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr
http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252
marangoz wrote:
>
>
> How about introducing internal macros for explicit ob_refcnt accesses
> in the core? Actually, there are a number of places where one can see
> "op->ob_refcnt" logic, which could be replaced with _Py_GETREF(op),
> _Py_SETREF(op, n) thus decoupling completely the low level refcount
> management defined in object.h:
>
> #define _Py_GETREF(op) (((PyObject *)op)->ob_refcnt)
> #define _Py_SETREF(op, n) (((PyObject *)op)->ob_refcnt = (n))
>
> Comments?
Of course, the above should be (PyObject *)(op)->ob_refcnt. Also, I forgot
to mention that if this detail doesn't hurt code aesthetics, one (I) could
experiment more easily all sort of weird things with refcounting...
I formulated the same wish for malloc & friends some time ago, that is,
use everywhere in the core PyMem_MALLOC, PyMem_FREE etc, which would be
defined for now as malloc, free, but nobody seems to be very excited
about a smooth transition to other kinds of malloc. Hence, I reiterate
this wish, 'cause switching to macros means preparing the code for the
future, even if in the future it remains intact ;-).
Defining these basic interfaces is clearly Guido's job :-) as he points
out in his summary of the last Open Source summit, but nevertheless,
I'm raising the issue to let him see what other people think about this
and allow him to make decisions easier :-)
--
Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr
http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252
----- End of forwarded message -----
--
Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr
http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252
marangoz wrote:
>
>
> How about introducing internal macros for explicit ob_refcnt accesses
> in the core? Actually, there are a number of places where one can see
> "op->ob_refcnt" logic, which could be replaced with _Py_GETREF(op),
> _Py_SETREF(op, n) thus decoupling completely the low level refcount
> management defined in object.h:
>
> #define _Py_GETREF(op) (((PyObject *)op)->ob_refcnt)
> #define _Py_SETREF(op, n) (((PyObject *)op)->ob_refcnt = (n))
>
> Comments?
Of course, the above should be (PyObject *)(op)->ob_refcnt. Also, I forgot
to mention that if this detail doesn't hurt code aesthetics, one (I) could
experiment more easily all sort of weird things with refcounting...
I formulated the same wish for malloc & friends some time ago, that is,
use everywhere in the core PyMem_MALLOC, PyMem_FREE etc, which would be
defined for now as malloc, free, but nobody seems to be very excited
about a smooth transition to other kinds of malloc. Hence, I reiterate
this wish, 'cause switching to macros means preparing the code for the
future, even if in the future it remains intact ;-).
Defining these basic interfaces is clearly Guido's job :-) as he points
out in his summary of the last Open Source summit, but nevertheless,
I'm raising the issue to let him see what other people think about this
and allow him to make decisions easier :-)
--
Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr
http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252
