From guido@zope.com  Wed Aug  1 00:12:32 2001
From: guido@zope.com (Guido van Rossum)
Date: Tue, 31 Jul 2001 19:12:32 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: Your message of "Tue, 31 Jul 2001 13:31:01 MDT."
 <15207.1909.395000.123189@gargle.gargle.HOWL>
References: <20010730051831.B1122@thyrsus.com> <Pine.NXT.4.21.0107301836260.258-100000@localhost.virginia.edu> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com>
 <15207.1909.395000.123189@gargle.gargle.HOWL>
Message-ID: <200107312312.TAA17483@cj20424-a.reston1.va.home.com>

> Guido van Rossum writes:
> > Yeah, but the runtime could offer a choice of data types -- for Python
> > code the constants table would contain Python ints and strings etc., for
> > Perl code it would contain Perl string-number objects.  Maybe.
> 
> A perl6 value have a vtable, essentially an array of function pointers
> which comprises the standard operations on that value.  I talked to
> Dan (the perl6 internals guy, dan@sidhe.org) about an impedence
> mismatch between Perl and Python data types, and he pointed out that
> you can have Perl values and Python values, each with their own
> semantics, simply by having separate vtables (and thus separate
> functions to implement the behaviour of those types).  Code can work
> with either type because the type carries around (in its vtable) the
> knowledge of how it should behave.

The vtable looks a lot like Python's type object.  Is Perl's vtable an
object in its own right?  In Python, it is: in C, obj->ob_type is the
type object, in Python type(obj) or (in 2.2) obj.__class__ gives the
type object.

> Feel free to grill Dan about these things if you want.

No thanks. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@zope.com  Wed Aug  1 00:15:03 2001
From: guido@zope.com (Guido van Rossum)
Date: Tue, 31 Jul 2001 19:15:03 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: Your message of "Tue, 31 Jul 2001 04:14:43 EDT."
 <20010731041443.A26075@thyrsus.com>
References: <20010730051831.B1122@thyrsus.com> <Pine.NXT.4.21.0107301836260.258-100000@localhost.virginia.edu> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1393.232974.785433@beluga.mojam.com>
 <20010731041443.A26075@thyrsus.com>
Message-ID: <200107312315.TAA17507@cj20424-a.reston1.va.home.com>

> In the conservative version of this vision, Perl and Python have
> different runtimes dynamically linked to an instance of the same
> opcode interpreter.  Memory allocation/GC and scheduling/threading are
> handled inside the opcode interpreter but the OS and environment
> binding is (mostly) in the libraries.

Hm, the Python bytecode interpreter doesn't do the memory
allocation/GC, and only does the scheduling/threading indirectly (by
releasing and immediately reacquiring the global interpreter lock).

> Things Python would bring to this party: our serious-cool GC, our 
> C extension/embedding system (*much* nicer than XS).  Things Perl would
> bring: blazingly fast regexps, taint, flexitypes, references. 

I'm not sure I want references, and I don't know what flexitypes are.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@zope.com  Wed Aug  1 00:26:13 2001
From: guido@zope.com (Guido van Rossum)
Date: Tue, 31 Jul 2001 19:26:13 -0400
Subject: [Python-Dev] Revised decimal type PEP
In-Reply-To: Your message of "Tue, 31 Jul 2001 03:27:11 EDT."
 <01073103271101.02004@fermi.eeel.nist.gov>
References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073023115207.02466@fermi.eeel.nist.gov> <3B66DE8C.C9C62012@lemburg.com>
 <01073103271101.02004@fermi.eeel.nist.gov>
Message-ID: <200107312326.TAA17540@cj20424-a.reston1.va.home.com>

> Also, why do you consider a float to be a "larger" value type than decimal?  
> Do you mean that a float is less precise?

(Warning: I think the following is a sound model, but I'm still
practicing how to explain it right.)


I have this ordering of the types in mind:

int/long < decimal < rational < float < complex
\---------------------------/   \-------------/
            exact                  inexact

This is different from the Scheme numeric "tower" -- I no longer agree
with the Scheme model any more.

The ordering is only to determine what happens on mixed arithmetic:
the result has the rightmost type in the diagram (or a type further on
the right in some cases).

The ints are a subset of the decimal numbers, and the decimal numbers
(in this view) are a subset of the rational numbers.  Ints and
decimals aren't closed under division -- the result of division on
these (in general) is a rational.  While the exact values of floats
are a subset of the rationals, the inexactness property (which I give
all floats) makes that each float stands for an infinite set of
numbers *including* the exact value.  When a binary operation involves
an exact and an inexact operand, the result is inexact.

Tim's "numeric context" contains a bunch of flags controlling detailed
behavior of numeric operations.  It could specify that mixing exact
and inexact numbers is illegal, and that would be Michael's pedantic
mode.  It could also specify warnings.  (I would never call a mode
that issues warnings "safe" :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@zope.com  Wed Aug  1 00:32:07 2001
From: guido@zope.com (Guido van Rossum)
Date: Tue, 31 Jul 2001 19:32:07 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: Your message of "Tue, 31 Jul 2001 05:18:08 EDT."
 <20010731051808.A27187@thyrsus.com>
References: <20010730051831.B1122@thyrsus.com> <Pine.NXT.4.21.0107301836260.258-100000@localhost.virginia.edu> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1909.395000.123189@gargle.gargle.HOWL> <20010731225445.B626@xs4all.nl>
 <20010731051808.A27187@thyrsus.com>
Message-ID: <200107312332.TAA17611@cj20424-a.reston1.va.home.com>

> Making this work is going to take strong leadership from Larry and
> Guido.  We're laying some of the technical groundwork now.  More will
> have to be done.  But I think the key moment, if it happens, will be
> the one at which Guido and Larry, each flanked by their three or four
> chief lieutenants, shake hands for the cameras and issue a joint ukase
> to their tribes.

Larry, are you following this?  What's *your* view?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dan@sidhe.org  Wed Aug  1 00:34:00 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Tue, 31 Jul 2001 19:34:00 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <200107312315.TAA17507@cj20424-a.reston1.va.home.com>
References: <Your message of "Tue, 31 Jul 2001 04:14:43 EDT." <20010731041443.A26075@thyrsus.com>
 <20010730051831.B1122@thyrsus.com>
 <Pine.NXT.4.21.0107301836260.258-100000@localhost.virginia.edu>
 <15206.51329.561652.565480@beluga.mojam.com>
 <200107311708.NAA16497@cj20424-a.reston1.va.home.com>
 <15206.61296.958360.72700@beluga.mojam.com>
 <200107311900.PAA17062@cj20424-a.reston1.va.home.com>
 <15207.1393.232974.785433@beluga.mojam.com>
 <20010731041443.A26075@thyrsus.com>
Message-ID: <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48>

At 07:15 PM 7/31/2001 -0400, Guido van Rossum wrote:
> > In the conservative version of this vision, Perl and Python have
> > different runtimes dynamically linked to an instance of the same
> > opcode interpreter.  Memory allocation/GC and scheduling/threading are
> > handled inside the opcode interpreter but the OS and environment
> > binding is (mostly) in the libraries.
>
>Hm, the Python bytecode interpreter doesn't do the memory
>allocation/GC, and only does the scheduling/threading indirectly (by
>releasing and immediately reacquiring the global interpreter lock).

The Parrot interpreter does (or will at least, as there's not much code 
behind it yet) handle GC and intercept memory allocation, at least if you 
want to use managed memory. (Though that's only of interest to people who 
are writing C/C++/Fortran/Whatever extensions to the interpreter) GC & 
object destruction are going to be dealt with separately by the 
interpreter, FWIW. Which might not be much.

As for threading, well, that's where things get interesting. Perl's tried 
it two ways (multiple threads in the same interpreter, and one thread per 
interpreter, with cloned interpreters) both of which aren't very good. And 
the global lock thing's not that keen either. The plan for parrot is to 
allow multiple interpreters to run simultaneously, with shared data 
protected on access time. (And *only* shared data protected, so there's no 
penalty to access non-shared data) Along with a scheme to allow only 
partial (or no) visibility of the spawning thread's variables. We plan on 
basing the secure sandbox scheme around this as well.

> > Things Python would bring to this party: our serious-cool GC, our
> > C extension/embedding system (*much* nicer than XS).  Things Perl would
> > bring: blazingly fast regexps, taint, flexitypes, references.
>
>I'm not sure I want references,

You certainly don't have to have them. The parser doesn't have to 
necessarily expose all the features of the interpreter. What you'd expose 
to the programmer would depend on the language grammar--if you can't write 
valid Python code to take references, well, you don't get them. :)

>  and I don't know what flexitypes are.

I presume this is perl's "I'll make it any type I darned well need" basic 
scalar, or possibly it's auto-resizing arrays and hashes. Neither of which 
you have to have either. Since that behaviour's a function of the type of a 
variable, we'd just not give python variables perl variable types. (Or the 
vtables that go with them)

Now, what happens when a Python routine is passed a perl variable is an 
interesting question. Personally I'd presume that it Did The Right Thing, 
but I'm not 100% sure what that is. Probably maintain its core behavior. 
(So if I passed in a Magic Morphing Perl Scalar to a Python routine that 
needed an integer value, it'd get one, but if I tried to use a Python 
integer in string context in a perl sub, I'd get yelled at. (Assuming 
that's the correct behaviour, of course. If not, well, it'd do something else))

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From paulp@ActiveState.com  Wed Aug  1 00:42:57 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Tue, 31 Jul 2001 16:42:57 -0700
Subject: [Python-Dev] Parrot -- should life imitate satire?
References: <Your message of "Tue, 31 Jul 2001 04:14:43 EDT." <20010731041443.A26075@thyrsus.com>
 <20010730051831.B1122@thyrsus.com>
 <Pine.NXT.4.21.0107301836260.258-100000@localhost.virginia.edu>
 <15206.51329.561652.565480@beluga.mojam.com>
 <200107311708.NAA16497@cj20424-a.reston1.va.home.com>
 <15206.61296.958360.72700@beluga.mojam.com>
 <200107311900.PAA17062@cj20424-a.reston1.va.home.com>
 <15207.1393.232974.785433@beluga.mojam.com>
 <20010731041443.A26075@thyrsus.com> <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48>
Message-ID: <3B674281.6962A99D@ActiveState.com>

Dan Sugalski wrote:
> 
>...
> 
> As for threading, well, that's where things get interesting. Perl's tried
> it two ways (multiple threads in the same interpreter, and one thread per
> interpreter, with cloned interpreters) both of which aren't very good. And
> the global lock thing's not that keen either. 

What is the downside of the global lock on the average single processor
machine? I tend to think that the "default" threading model should allow
simple and easy, everything-shared multi-threading on ordinary machines.
Having a multi-processor-friendly advanced mode is a great extension for
the wizards.
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From aahz@rahul.net  Wed Aug  1 00:55:26 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Tue, 31 Jul 2001 16:55:26 -0700 (PDT)
Subject: [Python-Dev] Revised decimal type PEP
In-Reply-To: <200107312326.TAA17540@cj20424-a.reston1.va.home.com> from "Guido van Rossum" at Jul 31, 2001 07:26:13 PM
Message-ID: <20010731235526.8E8ED99C82@waltz.rahul.net>

Guido van Rossum wrote:
> 
> int/long < decimal < rational < float < complex
> \---------------------------/   \-------------/
>             exact                  inexact

Note that in Cowlishaw's implementation of decimal numbers, decimals are
*not* exact.  Truncation (rounding), overflow, and underflow errors can
occur under addition, subtraction, and multiplication.  It's trivial to
set them to be unbounded, but then Cowlishaw provides no mechanism for
determining the truncation of division.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From aahz@rahul.net  Wed Aug  1 01:04:52 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Tue, 31 Jul 2001 17:04:52 -0700 (PDT)
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48> from "Dan Sugalski" at Jul 31, 2001 07:34:00 PM
Message-ID: <20010801000452.560D999C82@waltz.rahul.net>

Dan Sugalski wrote:
> 
> As for threading, well, that's where things get interesting. Perl's
> tried it two ways (multiple threads in the same interpreter, and one
> thread per interpreter, with cloned interpreters) both of which aren't
> very good. And the global lock thing's not that keen either. The plan
> for parrot is to allow multiple interpreters to run simultaneously,
> with shared data protected on access time. (And *only* shared data
> protected, so there's no penalty to access non-shared data) Along with
> a scheme to allow only partial (or no) visibility of the spawning
> thread's variables. We plan on basing the secure sandbox scheme around
> this as well.

This is where calling what Python uses "variables" gets you into real
problems.  Many people think that the terms for "variable" and
"reference" should be "name" and "binding"; it makes things far less
confusing.  See, Python objects are *global*.  Period.  No ifs, ands, or
buts.  Any object can be shared at any time.  More than that, most of
the time Python threads are bound to class instances, so causing another
thread to get a binding to an object is as simple as setting an
attribute of that thread's instance.

And, as Paul pointed out, the global interpreter lock only kills you if
you're running SMP in the first place.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From DavidA@ActiveState.com  Wed Aug  1 01:03:41 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Tue, 31 Jul 2001 17:03:41 -0700
Subject: [Python-Dev] Parrot -- should life imitate satire?
References: <20010731223259.A626@xs4all.nl>
Message-ID: <3B67475D.76FDBE4F@ActiveState.com>

Thomas Wouters wrote:

> The Perl version of the compiled code could of course be
> 
> PUSH "1"
> COERCE_INT
> PUSH "2"
> COERCE_INT
> BINARY_ADD
> 
> for Perl's
> 
> "1" + "2"
> 
> and
> 
> PUSH "1"
> PUSH "2"
> BINARY_ADD
> 
> for it's
> 
> "1" . "2"

I hate to bring up the fact that in Perl6 the . will be used for
attribute access (or at least method dereferencing), and some other
operator will be used for string concatenation =).

=)


From greg@cosc.canterbury.ac.nz  Wed Aug  1 01:17:14 2001
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed, 01 Aug 2001 12:17:14 +1200 (NZST)
Subject: [Python-Dev] Revised decimal type PEP
In-Reply-To: <20010731235526.8E8ED99C82@waltz.rahul.net>
Message-ID: <200108010017.MAA00798@s454.cosc.canterbury.ac.nz>

aahz@rahul.net (Aahz Maruch):

> Truncation (rounding), overflow, and underflow errors can
> occur under addition, subtraction, and multiplication.  It's trivial to
> set them to be unbounded, but then Cowlishaw provides no mechanism for
> determining the truncation of division.

If you allow for the representation of repeating parts
in your unbounded decimals, they could be closed under
division. (I think -- does the division of one repeating
decimal by another always lead to a third repeating decimal?
Yes, it must, because every rational can be expressed as a
repeating decimal and vice versa, IIRC. Hmmm, that means
we'd just be implementing rationals another way...)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From dan@sidhe.org  Wed Aug  1 01:13:33 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Tue, 31 Jul 2001 20:13:33 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <3B674281.6962A99D@ActiveState.com>
References: <Your message of "Tue, 31 Jul 2001 04:14:43 EDT." <20010731041443.A26075@thyrsus.com>
 <20010730051831.B1122@thyrsus.com>
 <Pine.NXT.4.21.0107301836260.258-100000@localhost.virginia.edu>
 <15206.51329.561652.565480@beluga.mojam.com>
 <200107311708.NAA16497@cj20424-a.reston1.va.home.com>
 <15206.61296.958360.72700@beluga.mojam.com>
 <200107311900.PAA17062@cj20424-a.reston1.va.home.com>
 <15207.1393.232974.785433@beluga.mojam.com>
 <20010731041443.A26075@thyrsus.com>
 <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48>
Message-ID: <5.1.0.14.0.20010731200907.02747f58@24.8.96.48>

At 04:42 PM 7/31/2001 -0700, Paul Prescod wrote:
>Dan Sugalski wrote:
> >
> >...
> >
> > As for threading, well, that's where things get interesting. Perl's tried
> > it two ways (multiple threads in the same interpreter, and one thread per
> > interpreter, with cloned interpreters) both of which aren't very good. And
> > the global lock thing's not that keen either.
>
>What is the downside of the global lock on the average single processor
>machine? I tend to think that the "default" threading model should allow
>simple and easy, everything-shared multi-threading on ordinary machines.
>Having a multi-processor-friendly advanced mode is a great extension for
>the wizards.

If you hold the lock during an I/O operation, you'll lose time you could 
have otherwise used. Getting and releasing a global lock frequently also 
costs performance you might otherwise have used in other places. Mutex 
releases require memory coherency, which will force your CPU to flush any 
pending writes that might be hanging about, which will tend to drop it's 
efficiency, especially on heavily out-of-order machines like the Alpha.

Also, that is a zillion and a half mutex aquisition and releases, most of 
which you probably have no need of. Even cutting out, say 5-10% of them 
will be noticeable. (I do work on SMP machines as a rule, so I am a little 
biased against things that single-thread me when I don't need it--what's 
the point of 500% idle time?)

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From dan@sidhe.org  Wed Aug  1 01:19:41 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Tue, 31 Jul 2001 20:19:41 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <20010801000452.560D999C82@waltz.rahul.net>
References: <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48>
Message-ID: <5.1.0.14.0.20010731201359.02719598@24.8.96.48>

At 05:04 PM 7/31/2001 -0700, Aahz Maruch wrote:
>Dan Sugalski wrote:
> >
> > As for threading, well, that's where things get interesting. Perl's
> > tried it two ways (multiple threads in the same interpreter, and one
> > thread per interpreter, with cloned interpreters) both of which aren't
> > very good. And the global lock thing's not that keen either. The plan
> > for parrot is to allow multiple interpreters to run simultaneously,
> > with shared data protected on access time. (And *only* shared data
> > protected, so there's no penalty to access non-shared data) Along with
> > a scheme to allow only partial (or no) visibility of the spawning
> > thread's variables. We plan on basing the secure sandbox scheme around
> > this as well.
>
>This is where calling what Python uses "variables" gets you into real
>problems.  Many people think that the terms for "variable" and
>"reference" should be "name" and "binding"; it makes things far less
>confusing.

I'm not sure that's the appropriate translation for what perl considers 
variables and references, but that's OK.

>See, Python objects are *global*.  Period.  No ifs, ands, or
>buts.

Is this global in the "all variables are in one big pool and can be looked 
up by name", more or less the equivalent of externing all your C variables, 
or global in the "Any thread might be able to see it" sense? (I thought 
Python had lexically scoped variables, but I might be wrong here. I'm an 
internals guy--the actual language being run is just an implementation 
detail... :)

If it's "All objects are visible to all threads" thing, that's fine, you 
can still handle threading with some care. Perl's done it wrong once, and I 
know how to do it much less wrong with Parrot. (It's not, strictly 
speaking, possible to do it right without the programmer taking an active hand)

>Any object can be shared at any time.  More than that, most of
>the time Python threads are bound to class instances, so causing another
>thread to get a binding to an object is as simple as setting an
>attribute of that thread's instance.

Ah. Doable, and not much of a problem. It'll require tweaking the thread 
model a touch (for which I appreciate you bringing this up) but better now 
than later.

>And, as Paul pointed out, the global interpreter lock only kills you if
>you're running SMP in the first place.

Yep, well, I do. :)

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From paulp@ActiveState.com  Wed Aug  1 01:27:48 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Tue, 31 Jul 2001 17:27:48 -0700
Subject: [Python-Dev] Parrot -- should life imitate satire?
References: <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48> <5.1.0.14.0.20010731201359.02719598@24.8.96.48>
Message-ID: <3B674D04.54F2E83E@ActiveState.com>

Dan Sugalski wrote:
> 
>...
> 
> Is this global in the "all variables are in one big pool and can be looked
> up by name", more or less the equivalent of externing all your C variables,
> or global in the "Any thread might be able to see it" sense? (I thought
> Python had lexically scoped variables, but I might be wrong here. I'm an
> internals guy--the actual language being run is just an implementation
> detail... :)

Python certainly has HIGHLY SCOPED variables. Hardly anything is global
in that sense. Even a two-line script will not really have truly global,
global variables. I wasn't so happy with Aahz's choice of wording there.
:)

Only the namespace of module names is truly global. i.e. "import sys"
gets you the same module object everywhere so you can use modules as
containers for other kinds of globals.

> If it's "All objects are visible to all threads" thing, that's fine, you
> can still handle threading with some care. Perl's done it wrong once, and I
> know how to do it much less wrong with Parrot. (It's not, strictly
> speaking, possible to do it right without the programmer taking an active hand)

Agreed.

>...
> >And, as Paul pointed out, the global interpreter lock only kills you if
> >you're running SMP in the first place.
> 
> Yep, well, I do. :)

Still, I would encourage you to optimize for the common case. People
have been predicting multiprocessor would be the common case for years
but Moore's law keeps delaying the need to pump up all of our servers
with a dozen processors.

The Python threading model is one interpreter for all threads unless you
really need multiple interpreters in which case you are sort of on your
own.
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From paul@pfdubois.com  Wed Aug  1 01:19:45 2001
From: paul@pfdubois.com (Paul F. Dubois)
Date: Tue, 31 Jul 2001 17:19:45 -0700
Subject: [Python-Dev] numerical models and numpy
Message-ID: <ADEOIFHFONCLEEPKCACCCEIMCLAA.paul@pfdubois.com>

I'm a little worried that you all may be assuming that I'm keeping up with
this flood of stuff on python-dev about the numerical model. I have tried
but I just can't manage that much volume. My silence doesn't mean approval.

Whatever "native" types you decide on there will be considerable pressure on
us to include corresponding types in Numerical Python, but I don't think
that can happen. At the moment there seem to be few people with any time to
spare to help me. I won't be able to do such work myself as my job just has
no use for decimals, rationals, etc.

My community would like a native Python type corresponding to a C float (and
a corresponding complex made out of two floats). Numpy already has those. I
was considering just hacking the source file for the native Python type
corresponding to a C double -- until I looked at it. What a complicated
beast it is. I have room in my draft implementation of PEP 242 for the beast
and I was going to do it in there.

To those of you thinking about this that haven't read PEP 242, please do so.
The Fortran committee thought hard about these issues and concluded that
some numeric "types" are not really different types but different "kinds" of
the same basic type.

-- Paul





From DavidA@ActiveState.com  Wed Aug  1 01:33:15 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Tue, 31 Jul 2001 17:33:15 -0700
Subject: [Python-Dev] Parrot -- should life imitate satire?
References: <Your message of "Tue, 31 Jul 2001 04:14:43 EDT." <20010731041443.A26075@thyrsus.com>
 <20010730051831.B1122@thyrsus.com>
 <Pine.NXT.4.21.0107301836260.258-100000@localhost.virginia.edu>
 <15206.51329.561652.565480@beluga.mojam.com>
 <200107311708.NAA16497@cj20424-a.reston1.va.home.com>
 <15206.61296.958360.72700@beluga.mojam.com>
 <200107311900.PAA17062@cj20424-a.reston1.va.home.com>
 <15207.1393.232974.785433@beluga.mojam.com>
 <20010731041443.A26075@thyrsus.com>
 <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48> <5.1.0.14.0.20010731200907.02747f58@24.8.96.48>
Message-ID: <3B674E4B.1B849A06@ActiveState.com>

To bring in yet someone else to this discussion, Mark Hammond and I were
chatting with Brendan Eich (of JavaScript and Mozilla fame) last week
about interpreter locks and the like, and Brendan mentioned that he had
done a lot of possibly reusable work in optimizing the various kinds of
thread lock strategies in JavaScript, as JS is used a lot server-side. 
Specifically he was talking about some 'multi-level' locking stuff which
sounded impressive to me at least.

I doubt if Brendan has time to dig into either of Perl or Python's
internals, but I'm sure he'll provide general pointers if folks ask
nicely. =)  Brendan, do you care to give an executive overview of the
locking strategies used in JS?

--david ascher

PS: Brendan: If you want to know the context of this discussion, see:

	http://aspn.activestate.com/ASPN/Mail/Browse/Threaded/python-dev


From paulp@ActiveState.com  Wed Aug  1 01:36:43 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Tue, 31 Jul 2001 17:36:43 -0700
Subject: [Python-Dev] Parrot -- should life imitate satire?
References: <Your message of "Tue, 31 Jul 2001 04:14:43 EDT." <20010731041443.A26075@thyrsus.com>
 <20010730051831.B1122@thyrsus.com>
 <Pine.NXT.4.21.0107301836260.258-100000@localhost.virginia.edu>
 <15206.51329.561652.565480@beluga.mojam.com>
 <200107311708.NAA16497@cj20424-a.reston1.va.home.com>
 <15206.61296.958360.72700@beluga.mojam.com>
 <200107311900.PAA17062@cj20424-a.reston1.va.home.com>
 <15207.1393.232974.785433@beluga.mojam.com>
 <20010731041443.A26075@thyrsus.com>
 <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48> <5.1.0.14.0.20010731200907.02747f58@24.8.96.48>
Message-ID: <3B674F1B.4CD32765@ActiveState.com>

Dan Sugalski wrote:
> 
>...
> 
> If you hold the lock during an I/O operation, you'll lose time you could
> have otherwise used. 

"Doctor it hurts when I do that." :) We just don't hold the lock during
I/O or other computations that do not affect Python interpreter data
structures.

-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From aahz@rahul.net  Wed Aug  1 01:49:38 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Tue, 31 Jul 2001 17:49:38 -0700 (PDT)
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <5.1.0.14.0.20010731200907.02747f58@24.8.96.48> from "Dan Sugalski" at Jul 31, 2001 08:13:33 PM
Message-ID: <20010801004938.67EA499C80@waltz.rahul.net>

Dan Sugalski wrote:
> 
> If you hold the lock during an I/O operation, you'll lose time you could 
> have otherwise used. 

Python extensions can (and do, for all standard I/O operations) release
the Global Interpreter Lock.  For more information, check out
http://starship.python.net/crew/aahz/
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From aahz@rahul.net  Wed Aug  1 01:59:12 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Tue, 31 Jul 2001 17:59:12 -0700 (PDT)
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <5.1.0.14.0.20010731201359.02719598@24.8.96.48> from "Dan Sugalski" at Jul 31, 2001 08:19:41 PM
Message-ID: <20010801005913.3B5F999C80@waltz.rahul.net>

Dan Sugalski wrote:
> At 05:04 PM 7/31/2001 -0700, Aahz Maruch wrote:
>>Dan Sugalski wrote:
>>>
>>> As for threading, well, that's where things get interesting. Perl's
>>> tried it two ways (multiple threads in the same interpreter, and one
>>> thread per interpreter, with cloned interpreters) both of which aren't
>>> very good. And the global lock thing's not that keen either. The plan
>>> for parrot is to allow multiple interpreters to run simultaneously,
>>> with shared data protected on access time. (And *only* shared data
>>> protected, so there's no penalty to access non-shared data) Along with
>>> a scheme to allow only partial (or no) visibility of the spawning
>>> thread's variables. We plan on basing the secure sandbox scheme around
>>> this as well.
>>
>>This is where calling what Python uses "variables" gets you into real
>>problems.  Many people think that the terms for "variable" and
>>"reference" should be "name" and "binding"; it makes things far less
>>confusing.
> 
> I'm not sure that's the appropriate translation for what perl considers 
> variables and references, but that's OK.

Paul evidently disagrees with me, but I'll write a rebuttal to that
right after I finish this.  ;-)

>>See, Python objects are *global*.  Period.  No ifs, ands, or
>>buts.
> 
> Is this global in the "all variables are in one big pool and can be
> looked up by name", more or less the equivalent of externing all your
> C variables, or global in the "Any thread might be able to see it"
> sense? (I thought Python had lexically scoped variables, but I might
> be wrong here. I'm an internals guy--the actual language being run is
> just an implementation detail... :)

Somewhere in between.  Python names are scoped; Python objects are not.
Here's a trival example: given a global list "l" (and "global" in this
phrase almost always means "module global"), and the following code
fragment running in one thread

    def foo():
        x = 1
        l.append(x)

any other threads created in that module will immediately have access to
the locally-created object "1".

Beyond that, though, given Python's introspection capabilities, it is
theoretically correct to say that all variables can be looked up by
name.

> If it's "All objects are visible to all threads" thing, that's fine,
> you can still handle threading with some care. Perl's done it wrong
> once, and I know how to do it much less wrong with Parrot. (It's not,
> strictly speaking, possible to do it right without the programmer
> taking an active hand)

Yup.  Which is why the GIL represents a pretty good compromise, most of
the time, particularly the way it's implemented in Python.

>>And, as Paul pointed out, the global interpreter lock only kills you if
>>you're running SMP in the first place.
> 
> Yep, well, I do. :)

Should have added, "...and only if you're running pure Python code."
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From aahz@rahul.net  Wed Aug  1 02:06:00 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Tue, 31 Jul 2001 18:06:00 -0700 (PDT)
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <3B674D04.54F2E83E@ActiveState.com> from "Paul Prescod" at Jul 31, 2001 05:27:48 PM
Message-ID: <20010801010600.2764699C80@waltz.rahul.net>

Paul Prescod wrote:
> Dan Sugalski wrote:
>> 
>> Is this global in the "all variables are in one big pool and can be looked
>> up by name", more or less the equivalent of externing all your C variables,
>> or global in the "Any thread might be able to see it" sense? (I thought
>> Python had lexically scoped variables, but I might be wrong here. I'm an
>> internals guy--the actual language being run is just an implementation
>> detail... :)
> 
> Python certainly has HIGHLY SCOPED variables. Hardly anything is global
> in that sense. Even a two-line script will not really have truly global,
> global variables. I wasn't so happy with Aahz's choice of wording there.
> :)

No, but it will have global, global objects.  Any time one creates an
object in Python, it has the potential to be bound to any name.
Consider the module foo with attribute bar; foo.bar can be set at any
time in any scope to any other object existing somewhere in Python's
heap.  And that change will be visible everwhere that the name foo.bar
is visible.  It gets even trickier with multiple threads created in a
single module namespace combined with mutable objects and global names
(as I illustrated in my response to Dan).
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From aahz@rahul.net  Wed Aug  1 02:07:50 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Tue, 31 Jul 2001 18:07:50 -0700 (PDT)
Subject: [Python-Dev] Revised decimal type PEP
In-Reply-To: <200108010017.MAA00798@s454.cosc.canterbury.ac.nz> from "Greg Ewing" at Aug 01, 2001 12:17:14 PM
Message-ID: <20010801010750.E045299C80@waltz.rahul.net>

Greg Ewing wrote:
> aahz@rahul.net (Aahz Maruch):
>> 
>> Truncation (rounding), overflow, and underflow errors can
>> occur under addition, subtraction, and multiplication.  It's trivial to
>> set them to be unbounded, but then Cowlishaw provides no mechanism for
>> determining the truncation of division.
> 
> If you allow for the representation of repeating parts in your
> unbounded decimals, they could be closed under division. (I think --
> does the division of one repeating decimal by another always lead to
> a third repeating decimal?  Yes, it must, because every rational can
> be expressed as a repeating decimal and vice versa, IIRC. Hmmm, that
> means we'd just be implementing rationals another way...)

<shudder>  Tell ya what, *you* write the algorithm and I'll think about
it sometime in the next five years.  ;-)
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From ping@lfw.org  Wed Aug  1 02:12:55 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Tue, 31 Jul 2001 18:12:55 -0700 (PDT)
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <20010731041443.A26075@thyrsus.com>
Message-ID: <Pine.LNX.4.32.0107311809160.19357-100000@ziggy.localdomain.fake>

On Tue, 31 Jul 2001, Eric S. Raymond wrote:
> Things Python would bring to this party: our serious-cool GC, our
> C extension/embedding system (*much* nicer than XS).  Things Perl would
> bring: blazingly fast regexps, taint, flexitypes, references.

I don't really understand the motivation.  Do we want any of those things?

Are our current regexps not good enough?
What are flexitypes, and why would we really want them?
Don't we already have references?

Taint might be worth something, and i would see the regexp engine as
incrementally better perhaps, but that doesn't seem enough to justify
a massive re-engineering of both languages.


-- ?!ng



From paulp@ActiveState.com  Wed Aug  1 02:11:34 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Tue, 31 Jul 2001 18:11:34 -0700
Subject: [Python-Dev] Parrot -- should life imitate satire?
References: <20010801010600.2764699C80@waltz.rahul.net>
Message-ID: <3B675746.F060EAEC@ActiveState.com>

Aahz Maruch wrote:
> 
> ...
> 
> No, but it will have global, global objects.  Any time one creates an
> object in Python, it has the potential to be bound to any name.
> Consider the module foo with attribute bar; foo.bar can be set at any
> time in any scope to any other object existing somewhere in Python's
> heap.  

Right, but what language is otherwise? In what language are variables in
one module firewalled from others so that references can't go back and
forth? Only functional languages and perhaps security-constrained
languages like JavaScript.

> .... And that change will be visible everwhere that the name foo.bar
> is visible.  It gets even trickier with multiple threads created in a
> single module namespace combined with mutable objects and global names
> (as I illustrated in my response to Dan).
> ...

I don't see that as having much to do with "globality." Python objects
are shared between threads including the module object and the
dictionary of module objects. But variables in Python are seldom global
in the sense of C's extern or Perl's default scope.
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From ping@lfw.org  Wed Aug  1 02:23:39 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Tue, 31 Jul 2001 18:23:39 -0700 (PDT)
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: <200107310622.CAA11742@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.LNX.4.32.0107311813220.19357-100000@ziggy.localdomain.fake>

On Tue, 31 Jul 2001, Guido van Rossum wrote:
> Why not add this to cgi.py instead?

Because that would seem to defeat most of the point.

The point was to provide an instant, effortless improvement for all
of the Python CGI scripts out there.  If programmers have to manually
edit all of their CGI scripts to insert

    import sys, cgitb
    sys.excepthook = cgitb.excepthook

then it's just as annoying as inserting

    import sys
    sys.stderr = sys.stdout

I don't want people to have to edit every single script.

> I don't like new additions that are
> irrelevant for most apps (CGI is a tiny niche for Python IMO).

I think this is where our perceptions differ.  I think of CGI as
the application that totally "made" Perl, and as the quickest, easiest
way that many beginners get early payoff from a scripting language.
My impression is that it has been a big "hook" for bringing people to
Perl and Python -- it's the shortest path to building and deploying
something useful to a huge and unlimited audience.

Wouldn't you say there are more Python CGI programmers out there
than, say, Zope developers?  Or think of it like this: what fraction
of Web developers are CGI programmers, and what fraction of those
use Python?

Maybe i'm wrong?  I welcome more opinions from others -- how do you
see people coming to Python?  What's the first "real" thing they do
with Python that motivates them to try it?


-- ?!ng



From ping@lfw.org  Wed Aug  1 02:25:32 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Tue, 31 Jul 2001 18:25:32 -0700 (PDT)
Subject: [Python-Dev] cgitb.py for Python 2.2
In-Reply-To: <15206.36478.421953.437702@anthem.wooz.org>
Message-ID: <Pine.LNX.4.32.0107311823510.19357-100000@ziggy.localdomain.fake>

On Tue, 31 Jul 2001, Barry A. Warsaw wrote:
> I'm at best +0 on adding it to site.py too.  E.g. for performance
> reasons Mailman's cgi wrappers invoke Python with -S to avoid the
> expensive overhead of importing site.py for each cgi hit.

This is just one environment variable check -- doesn't touch the disk at
all.  Most of site.py's time is spent scanning paths and such, i believe.


-- ?!ng



From esr@thyrsus.com  Wed Aug  1 03:06:27 2001
From: esr@thyrsus.com (Eric S. Raymond)
Date: Tue, 31 Jul 2001 22:06:27 -0400
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: <Pine.LNX.4.32.0107311813220.19357-100000@ziggy.localdomain.fake>; from ping@lfw.org on Tue, Jul 31, 2001 at 06:23:39PM -0700
References: <200107310622.CAA11742@cj20424-a.reston1.va.home.com> <Pine.LNX.4.32.0107311813220.19357-100000@ziggy.localdomain.fake>
Message-ID: <20010731220627.G1708@thyrsus.com>

Ka-Ping Yee <ping@lfw.org>:
> I think this is where our perceptions differ.  I think of CGI as
> the application that totally "made" Perl, and as the quickest, easiest
> way that many beginners get early payoff from a scripting language.
> My impression is that it has been a big "hook" for bringing people to
> Perl and Python -- it's the shortest path to building and deploying
> something useful to a huge and unlimited audience.

I strongly agree with Ping on this.  My experience is that CGI is an
extremely important application category for Python, and one worth
significant effort to serve.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

Good intentions will always be pleaded for every assumption of
authority. It is hardly too strong to say that the Constitution was
made to guard the people against the dangers of good intentions. There
are men in all ages who mean to govern well, but they mean to
govern. They promise to be good masters, but they mean to be masters.
	-- Daniel Webster


From gnat@oreilly.com  Wed Aug  1 03:26:47 2001
From: gnat@oreilly.com (Nathan Torkington)
Date: Tue, 31 Jul 2001 20:26:47 -0600
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <3B668608.F68B5953@ActiveState.com>
References: <20010730051831.B1122@thyrsus.com>
 <20010731012432.G20676@xs4all.nl>
 <15206.8708.811000.468489@gargle.gargle.HOWL>
 <20010731115521.I20676@xs4all.nl>
 <3B668608.F68B5953@ActiveState.com>
Message-ID: <15207.26855.65000.718694@gargle.gargle.HOWL>

Paul Prescod writes:
> One of the things I picked up from the Perl conference is that Perl
> users *seem* (to me) to have a higher tolerance for code breakage than
> Python users. (and Python users have a higher tolerance than (let's say)
> Java users) Even if we put aside Perl 6, Perlers talk pretty glibly
> about ripping little used features out in Perl 5.8.0 and Perl 5.10 and
> so forth. 

(off-topic, but he started it!)

That's a really interesting comment.  In many ways it's probably true,
although make no mistake about it--we take it in the neck for every
language change we make!

The features we're talking about removing are either deprecated or
experimental.  With that said, we've probably been less rigorous than
we should be about segregating experimental features from blessed
ones.  I suspect that because experimental features are the only way
to do something, they still get used no matter how much we caution
that they may go away.

I've been eyeing your 'import from the future' system of getting
experimental features.  I'll be interested to see how this holds up.

Perl and Python believe in active maintenance and development with
many small incremental releases (unlike Java with occasional bursty
releases, and C++ and C which are to all intents and purposes static).
Changes to the language cause problems we share, and the other
mainstream languages don't provide us with any direction on how to
approach the problem.

We in the Perl camp have bandied around the idea of "use perl 5.004"
as a way to get all the behaviour of the 5.004 release.  That's a
noble idea, but forces us to keep a lot of crap around when one of the
goals of active development is to be able to shed obstructive missteps
in language evolution.

Nat



From pedroni@inf.ethz.ch  Wed Aug  1 03:36:48 2001
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Wed, 1 Aug 2001 04:36:48 +0200
Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12
References: <200107301959.VAA11733@core.inf.ethz.ch> <2m4rrt78pn.fsf@starship.python.net>
Message-ID: <000f01c11a32$f24241c0$525821c0@newmexico>

Hi.

[Michael Hudson]
> Samuele Pedroni <pedroni@inf.ethz.ch> writes:
>
> > ...
> > > > >
> > > > > Does codeop currently work in Jython?  The solution should continue
to
> > > > > work in Jython then.
> > > > We have our interface compatible version of codeop that works.
> > >
> > > Would implementing the new interfaces I sketched out for codeop.py be
> > > possible in Jython?  That's the bit I care about, not so much the
> > > interface to __builtin__.compile.
> > Yes, it's of possible.
>
> Good; hopefully we can get somewhere then.
>
> > > > > Does Jython support the same flag bit values as
> > > > > CPython?  If not, Paul Prescod's suggestion to use keyword arguments
> > > > > becomes very relevant.
> > > > we support a subset of the co_flags, CO_NESTED e.g. is there with the
same
> > > > value.
> > > >
> > > > But the embedding API is very different, my implementation of nested
> > > > scopes does not define any Py_CF... flags, we have an internal
CompilerFlags
> > > > object but is more similar to PyFutureFeatures ...
> > >
> > > Is this object exposed to Python code at all?
> > Not publicily, but in Jython the separating line is a bit different,
> > because public java classes are always accessible from jython,
> > even most of the internals. That does not mean and every use of that
> > is welcome and supported.
>
> Ah, of course.  I'd forgotten how cool Jython was in some ways.
>
> > >  One approach would be
> > > PyObject-izing PyFutureFlags and making *that* the fourth argument to
> > > compile...
> > >
> > > class Compiler:
> > >     def __init__(self):
> > >         self.ff = ff.new() # or whatever
> > >     def __call__(self, source, filename, start_symbol):
> > >         code = compile(source, filename, start_symbol, self.ff)
> > >         self.ff.merge(code.co_flags)
> > >         return code
> > I see, "internally" we already have a compiler_flags function
> > that do the same of:
> > >         code = compile(source, filename, start_symbol, self.ff)
> > >         self.ff.merge(code.co_flags)
> >
> > where self.ff is a CompuilerFlags object.
> >
> > I can re-arrange things for any interface,
>
> Well, I don't want to make more work for you - I imagine Guido's doing
> enough of that for two!
>
> > I was only trying to explain our approach and situation and a
> > possible way to avoid duplicating some internal code in Python.
>
> Can you point me to the code in CVS that implements this sort of
> thing?  I don't really know Java but I can probably muddle through to
> some extent.  We might as well have CPython copy Jython for once...
I don't know if this can help you but the relevant stuff is in:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/jython/jython/org/python/modules
/codeop.java

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/jython/jython/org/python/core/Co
mpilerFlags.java

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/jython/jython/org/python/core/Py
.java
see compile_flags

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/jython/jython/org/python/compile
r/Module.java
see compile and PyCode

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/jython/jython/org/python/compile
r/CodeCompiler.java

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/jython/jython/org/python/compile
r/Future.java

However implementing this approach in CPython is a bit more demanding because
we can import any java class from jython code, in CPython you cannot import a C
struct directly (for the obvious reasons)
you need an Extension Type :(.

regards, Samuele Pedroni.



From guido@zope.com  Wed Aug  1 03:51:30 2001
From: guido@zope.com (Guido van Rossum)
Date: Tue, 31 Jul 2001 22:51:30 -0400
Subject: [Python-Dev] Revised decimal type PEP
In-Reply-To: Your message of "Tue, 31 Jul 2001 16:55:26 PDT."
 <20010731235526.8E8ED99C82@waltz.rahul.net>
References: <20010731235526.8E8ED99C82@waltz.rahul.net>
Message-ID: <200108010251.WAA18015@cj20424-a.reston1.va.home.com>

> Guido van Rossum wrote:
> > 
> > int/long < decimal < rational < float < complex
> > \---------------------------/   \-------------/
> >             exact                  inexact
> 
> Note that in Cowlishaw's implementation of decimal numbers, decimals are
> *not* exact.  Truncation (rounding), overflow, and underflow errors can
> occur under addition, subtraction, and multiplication.  It's trivial to
> set them to be unbounded, but then Cowlishaw provides no mechanism for
> determining the truncation of division.

I know all that.  Cowlishaw's numbers are a replacement for float (and
for the real/imag parts of complex).  I was toying with a *different*
idea, more appropriate perhaps for accountants, that guarantees exact
results for decimal numbers, giving decimal results when possible,
rational otherwise.  This is based on my guess about Michael's target
audience.  Michael makes a big deal of not allowing "decimalness" to
be contaminated by mixing with floats.  Possible this could be
implemented using a subclass of your decimals that uses unbounded
precision but overrides division to produce a rational number if
needed.

But it's possible that Michael's audience would in fact be very happy
with your/Cowlishaw's decimal numbers.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From pedroni@inf.ethz.ch  Wed Aug  1 03:52:28 2001
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Wed, 1 Aug 2001 04:52:28 +0200
Subject: R: [Python-Dev] Parrot -- should life imitate satire?
References: <20010731041443.A26075@thyrsus.com> <Pine.LNX.4.32.0107311809160.19357-100000@ziggy.localdomain.fake> <20010731095941.E1708@thyrsus.com>
Message-ID: <001f01c11a35$01bc6fc0$525821c0@newmexico>

[Eric S. Raymond]
> No, but we want to be able to interoperate with Perl and have if possible
> have just one back end on which efforts to do things like native code
> compilation can be concentrated.

>From my experience of the relative pain of a dynamic language over a static
typed VM,
and from the fact that native (dynamic?) compilation of Python/Perl would
reduce
the amount of code that must/is perceived to have to  be written C, if the goal
is to work on the long run on native compilation, that's a *great* goal.

Sharing C extesion would be also a good thing, no more duplicated efforts
for writing wrapping code for important libraries.

Sharing Perl libraries I feel is somehow different, using a library needs often
to read the code,
what would be the sentiment if a large part of the code a python program calls
is written in Perl.

IMHO also the fact of not merging the type "ontologies" but carrying somehow
both around
is a bit scary.

regards, Samuele Pedroni.



From guido@zope.com  Wed Aug  1 03:56:06 2001
From: guido@zope.com (Guido van Rossum)
Date: Tue, 31 Jul 2001 22:56:06 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: Your message of "Tue, 31 Jul 2001 17:33:15 PDT."
 <3B674E4B.1B849A06@ActiveState.com>
References: <Your message of "Tue, 31 Jul 2001 04:14:43 EDT." <20010731041443.A26075@thyrsus.com> <20010730051831.B1122@thyrsus.com> <Pine.NXT.4.21.0107301836260.258-100000@localhost.virginia.edu> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1393.232974.785433@beluga.mojam.com> <20010731041443.A26075@thyrsus.com> <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48> <5.1.0.14.0.20010731200907.02747f58@24.8.96.48>
 <3B674E4B.1B849A06@ActiveState.com>
Message-ID: <200108010256.WAA18063@cj20424-a.reston1.va.home.com>

BTW, for a VM flexible enough to deal with Python and Perl, JavaScript
should be a piece of cake.  (JavaScript is much closer to Python than
to Perl in its semantic model, for the most part.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@zope.com  Wed Aug  1 04:00:17 2001
From: guido@zope.com (Guido van Rossum)
Date: Tue, 31 Jul 2001 23:00:17 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: Your message of "Tue, 31 Jul 2001 18:12:55 PDT."
 <Pine.LNX.4.32.0107311809160.19357-100000@ziggy.localdomain.fake>
References: <Pine.LNX.4.32.0107311809160.19357-100000@ziggy.localdomain.fake>
Message-ID: <200108010300.XAA18095@cj20424-a.reston1.va.home.com>

> On Tue, 31 Jul 2001, Eric S. Raymond wrote:
> > Things Python would bring to this party: our serious-cool GC, our
> > C extension/embedding system (*much* nicer than XS).  Things Perl would
> > bring: blazingly fast regexps, taint, flexitypes, references.
> 
> I don't really understand the motivation.  Do we want any of those things?
> 
> Are our current regexps not good enough?
> What are flexitypes, and why would we really want them?
> Don't we already have references?
> 
> Taint might be worth something, and i would see the regexp engine as
> incrementally better perhaps, but that doesn't seem enough to justify
> a massive re-engineering of both languages.

Python won't be reengineered.  If the Perl VM happens to be able to
retargetable for Python, maybe some synergy might happen (like
reusable C extensions).  But I don't see Python making any compromises
in its semantic model, and the new VM would only be used for something
like Python 3.0 (or more likely 4.0) if it really offers significant
advantages.  Dan's recent contributions make me a lot less optimistic.
Don't ask me to explain -- I don't have the time, sorry.  The other
participants in the thread are doing a good job.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@zope.com  Wed Aug  1 04:06:14 2001
From: guido@zope.com (Guido van Rossum)
Date: Tue, 31 Jul 2001 23:06:14 -0400
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: Your message of "Tue, 31 Jul 2001 18:23:39 PDT."
 <Pine.LNX.4.32.0107311813220.19357-100000@ziggy.localdomain.fake>
References: <Pine.LNX.4.32.0107311813220.19357-100000@ziggy.localdomain.fake>
Message-ID: <200108010306.XAA18133@cj20424-a.reston1.va.home.com>

> On Tue, 31 Jul 2001, Guido van Rossum wrote:
> > Why not add this to cgi.py instead?
> 
> Because that would seem to defeat most of the point.
> 
> The point was to provide an instant, effortless improvement for all
> of the Python CGI scripts out there.  If programmers have to manually
> edit all of their CGI scripts to insert
> 
>     import sys, cgitb
>     sys.excepthook = cgitb.excepthook
> 
> then it's just as annoying as inserting
> 
>     import sys
>     sys.stderr = sys.stdout
> 
> I don't want people to have to edit every single script.

You misunderstand.  You proposed a few lines that would automatically
do this in site.py, which is always imported.  I propose to add those
same lines to cgi.py, so that any code that imports cgi.py
*automatically* has he feature enabled.  I assume that all CGI scripts
have an import of cgi.  There are non-CGI scripts that import cgi, but
those will be protected by the check for an environment variable that
you propose.

> > I don't like new additions that are
> > irrelevant for most apps (CGI is a tiny niche for Python IMO).
> 
> I think this is where our perceptions differ.  I think of CGI as
> the application that totally "made" Perl, and as the quickest, easiest
> way that many beginners get early payoff from a scripting language.

Yeah.  But it's not doing that for Python IMO.  Most Python apps (even
those that do web stuff) are not CGI apps.

> My impression is that it has been a big "hook" for bringing people to
> Perl and Python -- it's the shortest path to building and deploying
> something useful to a huge and unlimited audience.
> 
> Wouldn't you say there are more Python CGI programmers out there
> than, say, Zope developers?  Or think of it like this: what fraction
> of Web developers are CGI programmers, and what fraction of those
> use Python?

I don't want Python to become a wannabe CGI language, and I don't want
to make choices that benefit CGI programmers to the detriment of
others (CGI is actually a pretty lame way of producing active web
content).  Python is a decent language for CGI, but Perl is the
established standard and then there's PHP which also has way more
users than Python.

> Maybe i'm wrong?  I welcome more opinions from others -- how do you
> see people coming to Python?  What's the first "real" thing they do
> with Python that motivates them to try it?

All sorts of stuff.  Using NumPy.  GUI apps.  Database apps.  Unix
scripting.  App steering.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dan@sidhe.org  Wed Aug  1 04:16:05 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Tue, 31 Jul 2001 23:16:05 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <Pine.LNX.4.32.0107311809160.19357-100000@ziggy.localdomain
 .fake>
References: <20010731041443.A26075@thyrsus.com>
Message-ID: <5.1.0.14.0.20010731231443.02298fd8@24.8.96.48>

At 06:12 PM 7/31/2001 -0700, Ka-Ping Yee wrote:
>On Tue, 31 Jul 2001, Eric S. Raymond wrote:
> > Things Python would bring to this party: our serious-cool GC, our
> > C extension/embedding system (*much* nicer than XS).  Things Perl would
> > bring: blazingly fast regexps, taint, flexitypes, references.
>
>I don't really understand the motivation.  Do we want any of those things?
>
>Taint might be worth something, and i would see the regexp engine as
>incrementally better perhaps, but that doesn't seem enough to justify
>a massive re-engineering of both languages.

Honestly, neither language should need any reengineering. If it does, then 
that means I've failed as a designer. The point isn't to redesign Python to 
run on the Parrot back-end--it's to design the Parrot back end such that 
Python runs just fine on it. Which doesn't mean you *can't* redesign, just 
that you shouldn't have to.

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From guido@zope.com  Wed Aug  1 04:21:58 2001
From: guido@zope.com (Guido van Rossum)
Date: Tue, 31 Jul 2001 23:21:58 -0400
Subject: [Python-Dev] numerical models and numpy
In-Reply-To: Your message of "Tue, 31 Jul 2001 17:19:45 PDT."
 <ADEOIFHFONCLEEPKCACCCEIMCLAA.paul@pfdubois.com>
References: <ADEOIFHFONCLEEPKCACCCEIMCLAA.paul@pfdubois.com>
Message-ID: <200108010321.XAA18226@cj20424-a.reston1.va.home.com>

> I'm a little worried that you all may be assuming that I'm keeping up with
> this flood of stuff on python-dev about the numerical model. I have tried
> but I just can't manage that much volume. My silence doesn't mean approval.
> 
> Whatever "native" types you decide on there will be considerable pressure on
> us to include corresponding types in Numerical Python, but I don't think
> that can happen. At the moment there seem to be few people with any time to
> spare to help me. I won't be able to do such work myself as my job just has
> no use for decimals, rationals, etc.
> 
> My community would like a native Python type corresponding to a C float (and
> a corresponding complex made out of two floats). Numpy already has those. I
> was considering just hacking the source file for the native Python type
> corresponding to a C double -- until I looked at it. What a complicated
> beast it is. I have room in my draft implementation of PEP 242 for the beast
> and I was going to do it in there.

I have read your PEP and see it as a way to provide control over very
different aspects of numbers.  I expect that most of it is motivated
out of a desire to control the range, size and accuracy of floating
point more precisely, *and* to have scalar types matching the NumPy
types.  I think we can give you what you want for float and complex,
because I have no great plans to change those (a decimal type might be
added, but we can't remove binary floats).  I'm nor sure if you really
need various width integer types once the native Python integer uses
unbounded precision (the int/long unification).

> To those of you thinking about this that haven't read PEP 242,
> please do so.  The Fortran committee thought hard about these issues
> and concluded that some numeric "types" are not really different
> types but different "kinds" of the same basic type.

Yeah, but Python is not Fortran, and programmer needs do differ.

I do like the terminological distinction, it allows us to talk with
more clarity about these aspects of our numbers!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dan@sidhe.org  Wed Aug  1 04:31:07 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Tue, 31 Jul 2001 23:31:07 -0400
Subject: R: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <001f01c11a35$01bc6fc0$525821c0@newmexico>
References: <20010731041443.A26075@thyrsus.com>
 <Pine.LNX.4.32.0107311809160.19357-100000@ziggy.localdomain.fake>
 <20010731095941.E1708@thyrsus.com>
Message-ID: <5.1.0.14.0.20010731231925.022a68e0@24.8.96.48>

At 04:52 AM 8/1/2001 +0200, Samuele Pedroni wrote:
>[Eric S. Raymond]
> > No, but we want to be able to interoperate with Perl and have if possible
> > have just one back end on which efforts to do things like native code
> > compilation can be concentrated.
>
> From my experience of the relative pain of a dynamic language over a static
>typed VM, and from the fact that native (dynamic?) compilation of 
>Python/Perl would reduce the amount of code that must/is perceived to have 
>to  be written C, if the goal is to work on the long run on native 
>compilation, that's a *great* goal.

This is one of the goals, yes. It's next on the list after getting a 
working interpreter. There will probably be a TIL version on the road to 
native compilation.

>IMHO also the fact of not merging the type "ontologies" but carrying somehow
>both around is a bit scary.

It's mildly scary, sure. On the other hand, it's not really any more scary 
than, say, writing code in COBOL, C++, Basic, or Fortran and calling them 
from C. Or vice versa. While that's a tad odd, you can pretty easily get 
used to it, and it works out well.

You also don't have to mix languages. That's certainly one of the 
advantages, but it's far from required. (This is assuming, of course, that 
there's a performance gain to be had in targeting the parrot back end--if 
there isn't, then it's reasonably pointless to do so)

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From dan@sidhe.org  Wed Aug  1 04:41:55 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Tue, 31 Jul 2001 23:41:55 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <20010731080128.B2757@ute.cnri.reston.va.us>
References: <15206.8511.147000.832644@gargle.gargle.HOWL>
 <20010730051831.B1122@thyrsus.com>
 <20010731012432.G20676@xs4all.nl>
 <20010730205657.A2298@ute.cnri.reston.va.us>
 <15206.8511.147000.832644@gargle.gargle.HOWL>
Message-ID: <5.1.0.14.0.20010731233118.022a6640@24.8.96.48>

At 08:01 AM 7/31/2001 -0400, Andrew Kuchling wrote:
>On Mon, Jul 30, 2001 at 08:08:47PM -0700, Nathan Torkington wrote:
> >Andrew Kuchling writes:
> >The core loop would just be the usual opcode dispatch loop ("call the
> >function for the current operation, which returns the next
> >operation").  The only difference is that some of the opcodes would be
> >specific to RE matches.  (I'm unclear on how much special logic RE
>
>The big difference I see between regex opcodes and language opcodes is
>that regexes need to backtrack and language ones don't.

Backtracking in regexes isn't all that much different from some of the 
things you do at the language level, and certainly it's a straightforward 
mapping of regexes onto a series of string & integer register manipulations 
with simple branch-and-test code, with perhaps some stack space thrown in 
for deeply nested regexes.

>Unless the
>idea is to compile a regex to actual VM code similar to that generated
>by Python/Perl code, but then wouldn't that sacrifice efficiency?

The idea is (currently, barring bad performance) to compile down to VM code 
similar to what Ptyhon or perl would generate. There's the potential to 
slow things down, which I'm rather nervous about. On the other hand, the 
sort of code that would get emitted for regexes maps very well onto the 
instruction sets of hardware CPUs, so you'd get a corresponding boost in 
speed once translated to machine code. (Though granted you'd need to be 
able to do the conversion on the fly, as the regex itself isn't guaranteed 
to be known at compile time)

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From guido@zope.com  Wed Aug  1 04:46:50 2001
From: guido@zope.com (Guido van Rossum)
Date: Tue, 31 Jul 2001 23:46:50 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: Your message of "Tue, 31 Jul 2001 23:41:55 EDT."
 <5.1.0.14.0.20010731233118.022a6640@24.8.96.48>
References: <15206.8511.147000.832644@gargle.gargle.HOWL> <20010730051831.B1122@thyrsus.com> <20010731012432.G20676@xs4all.nl> <20010730205657.A2298@ute.cnri.reston.va.us> <15206.8511.147000.832644@gargle.gargle.HOWL>
 <5.1.0.14.0.20010731233118.022a6640@24.8.96.48>
Message-ID: <200108010346.XAA18484@cj20424-a.reston1.va.home.com>

Dan,

Have you looked at the byte code used in SRE, Python's (mostly)
Perl-compatible regex package?  Worth a look!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dan@sidhe.org  Wed Aug  1 04:49:02 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Tue, 31 Jul 2001 23:49:02 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <200108010300.XAA18095@cj20424-a.reston1.va.home.com>
References: <Your message of "Tue, 31 Jul 2001 18:12:55 PDT." <Pine.LNX.4.32.0107311809160.19357-100000@ziggy.localdomain.fake>
 <Pine.LNX.4.32.0107311809160.19357-100000@ziggy.localdomain.fake>
Message-ID: <5.1.0.14.0.20010731234359.022a1298@24.8.96.48>

At 11:00 PM 7/31/2001 -0400, Guido van Rossum wrote:
>Dan's recent contributions make me a lot less optimistic.
>Don't ask me to explain -- I don't have the time, sorry.  The other
>participants in the thread are doing a good job.

I would appreciate it if, when you have a chance, you (or someone) would 
clue me in. I've apparently offended, and that was not, in any way, my 
intent. My apologies all around.

I think I've overstayed my welcome, so I'll bow out here before muddying 
things up any worse.

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From dan@sidhe.org  Wed Aug  1 04:52:01 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Tue, 31 Jul 2001 23:52:01 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <200108010346.XAA18484@cj20424-a.reston1.va.home.com>
References: <Your message of "Tue, 31 Jul 2001 23:41:55 EDT." <5.1.0.14.0.20010731233118.022a6640@24.8.96.48>
 <15206.8511.147000.832644@gargle.gargle.HOWL>
 <20010730051831.B1122@thyrsus.com>
 <20010731012432.G20676@xs4all.nl>
 <20010730205657.A2298@ute.cnri.reston.va.us>
 <15206.8511.147000.832644@gargle.gargle.HOWL>
 <5.1.0.14.0.20010731233118.022a6640@24.8.96.48>
Message-ID: <5.1.0.14.0.20010731234925.02298ae8@24.8.96.48>

At 11:46 PM 7/31/2001 -0400, Guido van Rossum wrote:
>Have you looked at the byte code used in SRE, Python's (mostly)
>Perl-compatible regex package?  Worth a look!

Nope, not yet, but I will. (You folks will end up making a Python 
programmer out of me yet, I expect... :)

This and the C extension interface are the two big Python things I've got 
on my "must peer deeply into" list. Are there other places I should be 
concentrating on? (I'm poking about with the interpreter loop and 
associated code, but that's the sort of thing an expert with 20 minutes 
free will get me much farther than a week or two with the source)

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From pedroni@inf.ethz.ch  Wed Aug  1 04:51:23 2001
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Wed, 1 Aug 2001 05:51:23 +0200
Subject: [Python-Dev] Parrot -- should life imitate satire?
References: <20010731041443.A26075@thyrsus.com><Pine.LNX.4.32.0107311809160.19357-100000@ziggy.localdomain.fake><20010731095941.E1708@thyrsus.com> <5.1.0.14.0.20010731231925.022a68e0@24.8.96.48>
Message-ID: <003c01c11a3d$3cb1d860$665821c0@newmexico>

[Dan Sugalski]
>
> >IMHO also the fact of not merging the type "ontologies" but carrying somehow
> >both around is a bit scary.
>
> It's mildly scary, sure. On the other hand, it's not really any more scary
> than, say, writing code in COBOL, C++, Basic, or Fortran and calling them
> from C. Or vice versa. While that's a tad odd, you can pretty easily get
> used to it, and it works out well.
>
> You also don't have to mix languages. That's certainly one of the
> advantages, but it's far from required. (This is assuming, of course, that
> there's a performance gain to be had in targeting the parrot back end--if
> there isn't, then it's reasonably pointless to do so)O
If  one have to call conversion function from Python/Perl code to deal with
the other ontology or deal with unnatural behavior avoiding mixing languages
will remain the best thing, also for the problem of  reading code in the other
language.

But for C extensions the situation is a bit different, they should target one
ontology (or not?)
and then using the extension from both languages will be not that natural.
And C extension in Python typically extend the ontology, introduce new types
...

regards, Samuele Pedroni.



From dan@sidhe.org  Wed Aug  1 04:43:45 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Tue, 31 Jul 2001 23:43:45 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <200107312312.TAA17483@cj20424-a.reston1.va.home.com>
References: <Your message of "Tue, 31 Jul 2001 13:31:01 MDT." <15207.1909.395000.123189@gargle.gargle.HOWL>
 <20010730051831.B1122@thyrsus.com>
 <Pine.NXT.4.21.0107301836260.258-100000@localhost.virginia.edu>
 <15206.51329.561652.565480@beluga.mojam.com>
 <200107311708.NAA16497@cj20424-a.reston1.va.home.com>
 <15206.61296.958360.72700@beluga.mojam.com>
 <200107311900.PAA17062@cj20424-a.reston1.va.home.com>
 <15207.1909.395000.123189@gargle.gargle.HOWL>
Message-ID: <5.1.0.14.0.20010731231748.0228f808@24.8.96.48>

At 07:12 PM 7/31/2001 -0400, Guido van Rossum wrote:
> > Guido van Rossum writes:
> > > Yeah, but the runtime could offer a choice of data types -- for Python
> > > code the constants table would contain Python ints and strings etc., for
> > > Perl code it would contain Perl string-number objects.  Maybe.
> >
> > A perl6 value have a vtable, essentially an array of function pointers
> > which comprises the standard operations on that value.  I talked to
> > Dan (the perl6 internals guy, dan@sidhe.org) about an impedence
> > mismatch between Perl and Python data types, and he pointed out that
> > you can have Perl values and Python values, each with their own
> > semantics, simply by having separate vtables (and thus separate
> > functions to implement the behaviour of those types).  Code can work
> > with either type because the type carries around (in its vtable) the
> > knowledge of how it should behave.
>
>The vtable looks a lot like Python's type object.  Is Perl's vtable an
>object in its own right?

Depends on your definition of object, I suppose. There is a mapping from 
the low-level bits the interpreter needs to interpreted-language level 
things--while the vtable won't actually be directly manipulatable at the 
HLL level, manipulating HLL constructs will alter the vtable, so it's 
effectively the same thing.

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From guido@zope.com  Wed Aug  1 05:03:58 2001
From: guido@zope.com (Guido van Rossum)
Date: Wed, 01 Aug 2001 00:03:58 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: Your message of "Tue, 31 Jul 2001 23:43:45 EDT."
 <5.1.0.14.0.20010731231748.0228f808@24.8.96.48>
References: <Your message of "Tue, 31 Jul 2001 13:31:01 MDT." <15207.1909.395000.123189@gargle.gargle.HOWL> <20010730051831.B1122@thyrsus.com> <Pine.NXT.4.21.0107301836260.258-100000@localhost.virginia.edu> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1909.395000.123189@gargle.gargle.HOWL>
 <5.1.0.14.0.20010731231748.0228f808@24.8.96.48>
Message-ID: <200108010403.AAA18645@cj20424-a.reston1.va.home.com>

> >The vtable looks a lot like Python's type object.  Is Perl's vtable an
> >object in its own right?
> 
> Depends on your definition of object, I suppose. There is a mapping from 
> the low-level bits the interpreter needs to interpreted-language level 
> things--while the vtable won't actually be directly manipulatable at the 
> HLL level, manipulating HLL constructs will alter the vtable, so it's 
> effectively the same thing.

In Python, the type pointer really does point to a full-fledged
object.  Which probably means something quite different than it means
in Perl. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From pedroni@inf.ethz.ch  Wed Aug  1 05:02:58 2001
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Wed, 1 Aug 2001 06:02:58 +0200
Subject: [Python-Dev] Parrot -- should life imitate satire?
References: <20010731041443.A26075@thyrsus.com><Pine.LNX.4.32.0107311809160.19357-100000@ziggy.localdomain.fake><20010731095941.E1708@thyrsus.com> <5.1.0.14.0.20010731231925.022a68e0@24.8.96.48> <003c01c11a3d$3cb1d860$665821c0@newmexico>
Message-ID: <006801c11a3e$dac4a720$665821c0@newmexico>

>
> But for C extensions the situation is a bit different, they should target one
> ontology (or not?)
> and then using the extension from both languages will be not that natural.
> And C extension in Python typically extend the ontology, introduce new types
> ...
One more thing, with Python 2.2 you will be able to subclass those type from
C Extensions through Python code.

> regards, Samuele Pedroni.




From michel@digicool.com  Wed Aug  1 05:10:56 2001
From: michel@digicool.com (Michel Pelletier)
Date: Tue, 31 Jul 2001 21:10:56 -0700
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
References: <Pine.LNX.4.32.0107311813220.19357-100000@ziggy.localdomain.fake>
Message-ID: <3B678150.B8462B7B@digicool.com>

Ka-Ping Yee wrote:
> 
> The point was to provide an instant, effortless improvement for all
> of the Python CGI scripts out there.  If programmers have to manually
> edit all of their CGI scripts to insert
> 
>     import sys, cgitb
>     sys.excepthook = cgitb.excepthook
> 
> then it's just as annoying as inserting
> 
>     import sys
>     sys.stderr = sys.stdout

But Zope would have to do:

  import sys
  _sys_excepthook = sys.excepthook
  from cgi import FieldStorage, escape
  sys.excepthook = _sys_excepthook

and I don't think we'd be the only one that would have to do that.  Zope
has the environment variable set when accessed from CGI, but its stderr
gets redirected to a log file for plain text human consumption. 
importing cgi would have an unintended side effect. I would prefer an
explicit interface than a side effect.  +1 on it being in the standard
lib, or course.

-Michel


From dan@sidhe.org  Wed Aug  1 05:16:43 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Wed, 01 Aug 2001 00:16:43 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <200108010403.AAA18645@cj20424-a.reston1.va.home.com>
References: <Your message of "Tue, 31 Jul 2001 23:43:45 EDT." <5.1.0.14.0.20010731231748.0228f808@24.8.96.48>
 <Your message of "Tue, 31 Jul 2001 13:31:01 MDT." <15207.1909.395000.123189@gargle.gargle.HOWL>
 <20010730051831.B1122@thyrsus.com>
 <Pine.NXT.4.21.0107301836260.258-100000@localhost.virginia.edu>
 <15206.51329.561652.565480@beluga.mojam.com>
 <200107311708.NAA16497@cj20424-a.reston1.va.home.com>
 <15206.61296.958360.72700@beluga.mojam.com>
 <200107311900.PAA17062@cj20424-a.reston1.va.home.com>
 <15207.1909.395000.123189@gargle.gargle.HOWL>
 <5.1.0.14.0.20010731231748.0228f808@24.8.96.48>
Message-ID: <5.1.0.14.0.20010801000522.0238d970@24.8.96.48>

At 12:03 AM 8/1/2001 -0400, Guido van Rossum wrote:
> > >The vtable looks a lot like Python's type object.  Is Perl's vtable an
> > >object in its own right?
> >
> > Depends on your definition of object, I suppose. There is a mapping from
> > the low-level bits the interpreter needs to interpreted-language level
> > things--while the vtable won't actually be directly manipulatable at the
> > HLL level, manipulating HLL constructs will alter the vtable, so it's
> > effectively the same thing.
>
>In Python, the type pointer really does point to a full-fledged
>object.  Which probably means something quite different than it means
>in Perl. :-)

Ah. There's going to be a level of indirection there with Parrot. That'll 
introduce some speed hits, but only when altering the behaviour of the 
objects. It's been my experience that altering behaviour happens rather 
less frequently than the behaviour itself happens, so it makes sense to 
optimize both ends (the interface presented to the HLL programmer and the 
interface presented to the interpreter engine) and let the middle layer be 
a bit slower.

In this case, the vtable interface doesn't cover general method calls 
(except to the extent that a variable will have a vtable entry to handle 
method calls, in case you want to override the inheritance behaviour on a 
per-object basis, or enable multimethod dispatch for some objects), which 
are another beast entirely. The vtable interface, at the moment, covers the 
operators you'd expect to be able to override--assignment, simple math, 
most string operations, conversion duties, and suchlike things. So this code:

    a = b + c

would call the add vtable method for b, and the assign vtable method for a, 
while this:

    a.foo()

would make a generic method call on a looking for the foo method, and the 
vtables generally wouldn't be involved other than handling the (possibly 
cached) lookup of foo.

The point is to put the commonly called things in the vtable in a way that 
you can avoid as much conditional code as possible, while less common 
things get dealt with in ways you'd generally expect. (Dynamic lookups with 
caching and suchlike things)

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From tim.one@home.com  Wed Aug  1 05:26:30 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 1 Aug 2001 00:26:30 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <3B667087.EBBE8938@ActiveState.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIELALCAA.tim.one@home.com>

[Paul Prescod]
> Also, the .NET CLR is standardized at ECMA so we could (at least in
> theory!) go to the meetings and try to influence version 2.

[Guido]
> Notice the addition "in theory".  In practice, this is BS.

[Paul Prescod]
> It depends on the rules and politics of each particular standards group.
> It is fundamentally a social activity. It also depends how much effort
> you are willing to put into promoting your cause.

And cash:  being an active member (== the only way to be an *effective*
member) of a stds committee is expensive and time-consuming.  In my 15-year
compiler-geek career, I worked for companies who put major money behind
language development, and wrote off several person-years per annum to doing
language committee work.  Somehow I don't picture your boss springing for
that -- which isn't a knock, since I can't picture the people up my chain of
command even letting me finish explaining the idea <wink>.

> ...
> Working within a standards body is a gamble.  It can pay off big or
> it can completely fail.

I don't know how ECMA works; indeed, I never heard of it before it agreed to
fast-track ECMAScript (nee JavaScript).  ANSI/ISO committees work by
consensus, and one intransigent member can paralyze the entire effort.  Lots
of power there!  The breadth and depth of compromises made to make progress
in the end also account for the "a camel is a horse designed by a committee"
outcomes.

Note:  I was an active member of ANSI X3J17 for a couple of years.  This was
tasked to come up with a high-level x-language model for expressing parallel
computation, plus bindings for a few languages.  In order to protect an
ancient implementation of Fortran, the IBM representative vetoed any idea
that required so much as a runtime *stack* -- or any other dynamic feature.
In the end, X3J17 did the honorable thing:  it voted to put itself out of
its misery <wink>.

had-enough-of-that-myself-ly y'rs  - tim



From dan@sidhe.org  Wed Aug  1 05:42:28 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Wed, 01 Aug 2001 00:42:28 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <003c01c11a3d$3cb1d860$665821c0@newmexico>
References: <20010731041443.A26075@thyrsus.com>
 <Pine.LNX.4.32.0107311809160.19357-100000@ziggy.localdomain.fake>
 <20010731095941.E1708@thyrsus.com>
 <5.1.0.14.0.20010731231925.022a68e0@24.8.96.48>
Message-ID: <5.1.0.14.0.20010801002628.0228b9a0@24.8.96.48>

At 05:51 AM 8/1/2001 +0200, Samuele Pedroni wrote:
>[Dan Sugalski]
> >
> > >IMHO also the fact of not merging the type "ontologies" but carrying 
> somehow
> > >both around is a bit scary.
> >
> > It's mildly scary, sure. On the other hand, it's not really any more scary
> > than, say, writing code in COBOL, C++, Basic, or Fortran and calling them
> > from C. Or vice versa. While that's a tad odd, you can pretty easily get
> > used to it, and it works out well.
> >
> > You also don't have to mix languages. That's certainly one of the
> > advantages, but it's far from required. (This is assuming, of course, that
> > there's a performance gain to be had in targeting the parrot back end--if
> > there isn't, then it's reasonably pointless to do so)O
>If  one have to call conversion function from Python/Perl code to deal with
>the other ontology or deal with unnatural behavior avoiding mixing languages
>will remain the best thing, also for the problem of  reading code in the other
>language.

I'm not entirely sure there'll be as much of a mismatch between the 
languages as you might think. One thing that'll help on the perl end is the 
introduction of optionally typed data--some scalars may be tagged as 
integer, or arrays as string only, so a lot of the pain of needing to 
handle conversion and impedance matching will already have been dealt with.

>But for C extensions the situation is a bit different, they should target one
>ontology (or not?)
>and then using the extension from both languages will be not that natural.
>And C extension in Python typically extend the ontology, introduce new types
>...

It's possible there'll be a bigger mismatch, but I'm not that sure here, 
either. (Though I'm stumbling around a bit here, as I've no experience 
writing Python extensions in C) Some classes of extension will be 
essentially language neutral--an array's an array's an array and, while 
some of the details are interesting (like whether it's typed or sized) 
those are, ultimately details. Important ones, to be sure, but far from an 
obstacle.

New types are a bit more of a challenge. Some things, like declaring 
methods or overloaded behaviour, are also reasonably small details. Other 
things, like things I'm not thinking of, are less so.

Don't forget I'm also looking at this from the bottom up, not the top down. 
Parrot's going to be a dynamic language friendly, mildly OO CPU. From that 
level, these (And I'd get fancy here, but I don't know how to do so without 
embarrassment in Python):

     $foo->bar($a, $b, $c)
     foo.bar(a, b, c)

are pretty much syntactically equivalent, and the fact that perl declares 
methods in packages and handles inheritance on a per-class basis via the 
@ISA array and Python does something else entirely syntactically is pretty 
straightforwardly abstractable, and isn't hugely important *for the 
execution engine* as long as that abstraction is sufficiently powerful. 
While Perl and Python aren't quite just ALGOL with funny hats, they're 
really not that different from each other from the level I'm working at. 
Certainly far less different than they are together from, say, LISP or 
Forth, or even .NET. (I expect we face similar performance issues with 
.NET, and the JVM, as neither seem particularly fond of dynamic languages. 
Though I do see that Jython gets pretty good performance from some of the 
JIT JVMs)

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From tim.one@home.com  Wed Aug  1 05:42:42 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 1 Aug 2001 00:42:42 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <200107311708.NAA16497@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAELCLCAA.tim.one@home.com>

[Guido]
> Actually, this may not be as big a deal as I thought before.  The PVM
> doesn't have a lot of knowledge about types built into its instruction
> set.  It knows a bit about classes, lists, dicts, but not e.g. about
> ints and strings.  The opcodes are mostly very abstract: BINARY_ADD etc.

I agree overall, but you picked an unfortunate example.  BINARY_DIVIDE is a
perfect example:

		case BINARY_DIVIDE:
			w = POP();
			v = POP();
			x = PyNumber_Divide(v, w);
			Py_DECREF(v);
			Py_DECREF(w);
			PUSH(x);
			if (x != NULL) continue;
			break;

(For Perl'ers, an operation in Python returns a PyObject*, and returns NULL
iff the operation wants to raise an exception; the "continue" if x != NULL
tells it to go straight back to the top of the eval loop (to fetch the next
opcode) if there is no exception; else control flows out of the switch stmt,
and masses of code figure out what to *do* with the exception.)

That code works unchanged for any pair of objects whatsoever that think they
know what to do with an infix "/", since all the intelligence is in
PyNumber_Divide().

But BINARY_ADD knows everything there is to know about Python ints, and
that's an important speed optimization:

		case BINARY_ADD:
			w = POP();
			v = POP();
			if (PyInt_Check(v) && PyInt_Check(w)) {
				/* INLINE: int + int */
				register long a, b, i;
				a = PyInt_AS_LONG(v);
				b = PyInt_AS_LONG(w);
				i = a + b;
				if ((i^a) < 0 && (i^b) < 0) {
					PyErr_SetString(PyExc_OverflowError,
							"integer addition");
					x = NULL;
				}
				else
					x = PyInt_FromLong(i);
			}
			else
				x = PyNumber_Add(v, w);
			Py_DECREF(v);
			Py_DECREF(w);
			PUSH(x);
			if (x != NULL) continue;
			break;

While we don't peek under the covers often in the eval loop, the places we
do were huge benefit/effort wins.



From tim.one@home.com  Wed Aug  1 05:47:35 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 1 Aug 2001 00:47:35 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <E15Rfqs-0000vf-00@darjeeling>
Message-ID: <LNBBLJKPBEHFEDALKOLCIELCLCAA.tim.one@home.com>

[Moshe Zadka]
> PUSH "1"
> PUSH "2"
> BINARY_ADD
>
> In Python that gives "12". In Perl that gives 3.
> Unless you suggest a PERL_BINARY_ADD and a PYTHON_BINARY_ADD, I
> don't see how you can around these things.

Perl needs distinct flavors of BINARY_ADD for its infix "+" and "."
operators.  Once you accept that, there's no real problem here (it simply
means that Python's and Perl's "+" operators would need to map to different
opcodes).



From tim.one@home.com  Wed Aug  1 06:23:59 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 1 Aug 2001 01:23:59 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <5.1.0.14.0.20010731200907.02747f58@24.8.96.48>
Message-ID: <LNBBLJKPBEHFEDALKOLCIELELCAA.tim.one@home.com>

[Paul Prescod]
> What is the downside of the global lock on the average single
> processor machine? I tend to think that the "default" threading model
> should allow simple and easy, everything-shared multi-threading on
> ordinary machines.  Having a multi-processor-friendly advanced mode is
> a great extension for the wizards.

[Dan Sugalski]
> If you hold the lock during an I/O operation, you'll lose time you
> could have otherwise used.  Getting and releasing a global lock
> frequently also costs performance you might otherwise have used in other
> places.  Mutex releases require memory coherency, which will force your
> CPU to flush any pending writes that might be hanging about, which will
> tend to drop it's efficiency, especially on heavily out-of-order
> machines like the Alpha.
>
> Also, that is a zillion and a half mutex aquisition and releases, most
> of which you probably have no need of.

Python doesn't actually suffer from either of these problems:  while there's
a pair of acquire/release-global-lock macros around potentially blocking I/O
calls in Python's runtime (ditto sleep(), etc), no mutex is actually
allocated before *somebody* calls PyEval_InitThreads:

void
PyEval_InitThreads(void)
{
	if (interpreter_lock)
		return;
	_PyThread_Started = 1;
	interpreter_lock = PyThread_allocate_lock();
	PyThread_acquire_lock(interpreter_lock, 1);
	main_thread = PyThread_get_thread_ident();
}

In most uses of Python, no thread other than the main thread ever gets
created, that routine never gets called, interpreter_lock remains NULL, and
all the global-lock acquire/release code reduces to a cheap test against
NULL.

However, Python calls the platform's thread-safe libraries regardless, and
*that* can be a huge speed hit.  A minor example is that system malloc() is
more expensive in Microsoft's thread-safe version of libc.  A monster
example is speed of line-at-a-time input:  we only recently discovered that
Python's getc()-in-a-loop was killing us on many platforms because the
platform threadsafe library implementation locked and unlocked the stream
for each character.  Worming around that brought our input speed much closer
to Perl's (up to 50x faster on Tru64 Unix).  It's still slower on most
boxes, though, because we're still threadsafe, but, last I looked, Perl's
line-at-a-time input tricks mucked with stdio structs directly without
benefit of exclusion (and are not threadsafe).

> ...
> (I do work on SMP machines as a rule, so I am a little biased against
> things that single-thread me when I don't need it--what's the point of
> 500% idle time?)

Greg Stein is the fellow to talk with about about "free threading" of
Python.  He had that at least mostly working several years ago, but it was a
major project, that patch is way out of date now, and Python is much more
elegant now.  Oops!  I didn't mean "elegant", I meant "bigger" <wink>.



From dan@sidhe.org  Wed Aug  1 07:20:10 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Wed, 01 Aug 2001 02:20:10 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIELELCAA.tim.one@home.com>
References: <5.1.0.14.0.20010731200907.02747f58@24.8.96.48>
Message-ID: <5.1.0.14.0.20010801020813.022a56b8@24.8.96.48>

At 01:23 AM 8/1/2001 -0400, Tim Peters wrote:
>[Paul Prescod]
> > What is the downside of the global lock on the average single
> > processor machine? I tend to think that the "default" threading model
> > should allow simple and easy, everything-shared multi-threading on
> > ordinary machines.  Having a multi-processor-friendly advanced mode is
> > a great extension for the wizards.
>
>[Dan Sugalski]
> > If you hold the lock during an I/O operation, you'll lose time you
> > could have otherwise used.  Getting and releasing a global lock
> > frequently also costs performance you might otherwise have used in other
> > places.  Mutex releases require memory coherency, which will force your
> > CPU to flush any pending writes that might be hanging about, which will
> > tend to drop it's efficiency, especially on heavily out-of-order
> > machines like the Alpha.
> >
> > Also, that is a zillion and a half mutex aquisition and releases, most
> > of which you probably have no need of.
>
>Python doesn't actually suffer from either of these problems:  while there's
>a pair of acquire/release-global-lock macros around potentially blocking I/O
>calls in Python's runtime (ditto sleep(), etc), no mutex is actually
>allocated before *somebody* calls PyEval_InitThreads:

Yeah, I figured you didn't initialize or aquire the mutex until it was 
actually needed. One of the nice side-benefits of the global opcode-aquired 
lock.

>However, Python calls the platform's thread-safe libraries regardless, and
>*that* can be a huge speed hit.  A minor example is that system malloc() is
>more expensive in Microsoft's thread-safe version of libc.

Everywhere else too, I'd bet. I've been considering thread-specific memory 
pools because of this. (Well, this is one reason, at least)

>A monster
>example is speed of line-at-a-time input:  we only recently discovered that
>Python's getc()-in-a-loop was killing us on many platforms because the
>platform threadsafe library implementation locked and unlocked the stream
>for each character.

Ouch, I'd bet that hurts. Has anyone timed the difference between making 
lots of getc calls and making a few larger reads and managing the buffers 
internally? I can see it going either way, and another data point would be 
useful to have.

>Worming around that brought our input speed much closer
>to Perl's (up to 50x faster on Tru64 Unix).

FWIW, at least on Tru64 and VMS, there are a number of faster thread calls 
if you don't mind bypassing some library error checking. (Which is OK if 
you're guaranteed to have correct parameters) I can dig them up if you 
like. A tweak of pthread_get_specific made a depressingly large performance 
boost for me on my VMS box. (I did say perl's threading models weren't that 
good... :) Not worth it if you don't make many calls, though.

>It's still slower on most
>boxes, though, because we're still threadsafe, but, last I looked, Perl's
>line-at-a-time input tricks mucked with stdio structs directly without
>benefit of exclusion (and are not threadsafe).

Yep, perl's pthread-based threading model doesn't guarantee threadsafe I/O 
on many platforms. The alternative threading model doesn't use that 
codepath. (It's currently primarily windows-based, and we don't do the 
buffer lookbehind stuff there)

> > ...
> > (I do work on SMP machines as a rule, so I am a little biased against
> > things that single-thread me when I don't need it--what's the point of
> > 500% idle time?)
>
>Greg Stein is the fellow to talk with about about "free threading" of
>Python.  He had that at least mostly working several years ago, but it was a
>major project, that patch is way out of date now, and Python is much more
>elegant now.  Oops!  I didn't mean "elegant", I meant "bigger" <wink>.

:) I'm as much (or more, which is generally an odd thing, but threads are 
almost inherently odd) interested in how things look at the programmer 
level and what guarantees are made as how things look under the hood. (It's 
all just a SMOP, right?)

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From tim.one@home.com  Wed Aug  1 06:03:18 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 1 Aug 2001 01:03:18 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <5.1.0.14.0.20010731234359.022a1298@24.8.96.48>
Message-ID: <LNBBLJKPBEHFEDALKOLCKELDLCAA.tim.one@home.com>

[Guido van Rossum]
> Dan's recent contributions make me a lot less optimistic.
> Don't ask me to explain -- I don't have the time, sorry.  The other
> participants in the thread are doing a good job.
>

[Dan Sugalski]
> I would appreciate it if, when you have a chance, you (or someone)
> would clue me in. I've apparently offended, and that was not, in any
> way, my intent. My apologies all around.

Ah.  My primary job in the Python Community is channeling Guido:  By "the
other participants are doing a good job", he doesn't mean you *aren't* doing
a good job.  Not at all!  He means that they're doing a good enough job of
expressing his concerns that he doesn't feel terrible about not having
enough time of his own to participate.  No offense taken!  You're doing a
good job too.

> I think I've overstayed my welcome,

Not at all -- it's a good discussion, and perfectly appropriate for
Python-Dev.  I wish I had more time for it too.

> so I'll bow out here before muddying things up any worse.

Please stay.  Just try to avoid mentioning Perl, lest you offend Guido again
<wink>.



From mal@lemburg.com  Wed Aug  1 08:48:10 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 01 Aug 2001 09:48:10 +0200
Subject: [Python-Dev] Revised decimal type PEP
References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073023115207.02466@fermi.eeel.nist.gov> <3B66DE8C.C9C62012@lemburg.com>
 <01073103271101.02004@fermi.eeel.nist.gov> <200107312326.TAA17540@cj20424-a.reston1.va.home.com>
Message-ID: <3B67B43A.D29AF194@lemburg.com>

Guido van Rossum wrote:
> 
> > Also, why do you consider a float to be a "larger" value type than decimal?
> > Do you mean that a float is less precise?
> 
> (Warning: I think the following is a sound model, but I'm still
> practicing how to explain it right.)
> 
> I have this ordering of the types in mind:
> 
> int/long < decimal < rational < float < complex
> \---------------------------/   \-------------/
>             exact                  inexact
> 
> This is different from the Scheme numeric "tower" -- I no longer agree
> with the Scheme model any more.
> 
> The ordering is only to determine what happens on mixed arithmetic:
> the result has the rightmost type in the diagram (or a type further on
> the right in some cases).

Interesting. Here's what I use in mxNumber:

                     mx.Number.Float
                            ^
                            |
            --------> Python float
           |                ^
           |                |
           |         mx.Number.Rational
           |                ^
           |                |
     Python long --> mx.Number.Integer
           ^                ^
           |                |
            --------  Python integer
 
> The ints are a subset of the decimal numbers, and the decimal numbers
> (in this view) are a subset of the rational numbers.  Ints and
> decimals aren't closed under division -- the result of division on
> these (in general) is a rational.  While the exact values of floats
> are a subset of the rationals, the inexactness property (which I give
> all floats) makes that each float stands for an infinite set of
> numbers *including* the exact value.  When a binary operation involves
> an exact and an inexact operand, the result is inexact.
> 
> Tim's "numeric context" contains a bunch of flags controlling detailed
> behavior of numeric operations.  It could specify that mixing exact
> and inexact numbers is illegal, and that would be Michael's pedantic
> mode.  It could also specify warnings.  (I would never call a mode
> that issues warnings "safe" :-)

Could you perhaps write this coercion scheme up as informational
PEP ? I think it would help a lot as reference to what Python should
do and serve well for numeric extension writers as basis for their
coercion decisions.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From mal@lemburg.com  Wed Aug  1 08:50:48 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 01 Aug 2001 09:50:48 +0200
Subject: [Python-Dev] Revised decimal type PEP
References: <20010801010750.E045299C80@waltz.rahul.net>
Message-ID: <3B67B4D8.2B319336@lemburg.com>

Aahz Maruch wrote:
> 
> Greg Ewing wrote:
> > aahz@rahul.net (Aahz Maruch):
> >>
> >> Truncation (rounding), overflow, and underflow errors can
> >> occur under addition, subtraction, and multiplication.  It's trivial to
> >> set them to be unbounded, but then Cowlishaw provides no mechanism for
> >> determining the truncation of division.
> >
> > If you allow for the representation of repeating parts in your
> > unbounded decimals, they could be closed under division. (I think --
> > does the division of one repeating decimal by another always lead to
> > a third repeating decimal?  Yes, it must, because every rational can
> > be expressed as a repeating decimal and vice versa, IIRC. Hmmm, that
> > means we'd just be implementing rationals another way...)
> 
> <shudder>  Tell ya what, *you* write the algorithm and I'll think about
> it sometime in the next five years.  ;-)

Decimals are just a different way of displaying Rationals and
depending on their preset precision show a different behaviour
in numeric operations.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From mal@lemburg.com  Wed Aug  1 08:55:58 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 01 Aug 2001 09:55:58 +0200
Subject: [Python-Dev] Revised decimal type PEP
References: <20010731235526.8E8ED99C82@waltz.rahul.net> <200108010251.WAA18015@cj20424-a.reston1.va.home.com>
Message-ID: <3B67B60E.201CA0C@lemburg.com>

Guido van Rossum wrote:
> 
> > Guido van Rossum wrote:
> > >
> > > int/long < decimal < rational < float < complex
> > > \---------------------------/   \-------------/
> > >             exact                  inexact
> >
> > Note that in Cowlishaw's implementation of decimal numbers, decimals are
> > *not* exact.  Truncation (rounding), overflow, and underflow errors can
> > occur under addition, subtraction, and multiplication.  It's trivial to
> > set them to be unbounded, but then Cowlishaw provides no mechanism for
> > determining the truncation of division.
> 
> I know all that.  Cowlishaw's numbers are a replacement for float (and
> for the real/imag parts of complex).  I was toying with a *different*
> idea, more appropriate perhaps for accountants, that guarantees exact
> results for decimal numbers, giving decimal results when possible,
> rational otherwise.  This is based on my guess about Michael's target
> audience.  Michael makes a big deal of not allowing "decimalness" to
> be contaminated by mixing with floats.  Possible this could be
> implemented using a subclass of your decimals that uses unbounded
> precision but overrides division to produce a rational number if
> needed.
> 
> But it's possible that Michael's audience would in fact be very happy
> with your/Cowlishaw's decimal numbers.

I believe that Rationals should form the basis for Decimal types
-- the different possible semantics can then be added either by 
subclassing Rationals or by providing some context or specialized
methods for different operations, e.g. for special rounding 
requirements like you have in financial applications.

Note that I implemented mxNumber for just this reason (the float
and long integer support is merely a side effect of GMP providing
it ;-).

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From mal@lemburg.com  Wed Aug  1 09:11:37 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 01 Aug 2001 10:11:37 +0200
Subject: [Python-Dev] Revised decimal type PEP
References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073023115207.02466@fermi.eeel.nist.gov> <3B66DE8C.C9C62012@lemburg.com> <01073103271101.02004@fermi.eeel.nist.gov>
Message-ID: <3B67B9B9.37FF7237@lemburg.com>

Michael McLay wrote:
> 
> On Tuesday 31 July 2001 12:36 pm, M.-A. Lemburg wrote:
> 
> > I'd suggest to follow the rules for the SQL definitions
> > of DECIMAL(,).
> 
> > Well, there are several options. I support that the IBM paper
> > on decimal types has good hints as to what the type should do.
> > Again, SQL is probably a good source for inspiration too, since
> > it deals with decimals a lot.
> 
> Ok, I know about the IBM paper.  is there online document on the SQL
> semantics that can be referenced in the PEP?

The spec is not available online since it is an ISO paper (you pay
per page), but I did find a review draft dated July 1992 for SQL-92:

http://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt

which has all the relevant information in text format (section 4.4.).
 
> > I see, the small 'b' still looks funny to me though. Wouldn't
> > 1.23f and 25i be more intuitive ?
> 
> I originally used 'f' for both the integer and float.    The use of 'b' was
> suggested by Guido. There were two reasons not to use 'i' for integers.  The
> first has to do with how the tokenizer works.  It doesn't distringuish
> between float and int when the token string is passed to parsenumber().  Both
> float and int are processed by the same function.  I could have got around
> this problem by having the switch statement in parsenumber recognize both 'i'
> and 'f', but there is another problem with using 'i'.  The 25i would be
> confusing for someone if they was trying to use an imaginary numbers If they
> accidentally typed 25i instead of 25j they would get an integer instead of an
> imaginary number.  The error might not be detected since 3.0 + 4i would
> evaluate properly.

Well, if we manage to have a registry of some sort for these
number literal modifiers, I think that the issues in which code
to choose are secondary.
 
> > > > I'd rather have this explicit in the sense that you define which
> > > > assumptions will be made and what issues arise (rounding, truncation,
> > > > loss of precision, etc.).
> > >
> > > Can you give an example of how this might be implemented.
> >
> > You would typically first coerce the types to the "larger"
> > type, e.g. float + decimal -> float + float -> float, so
> > you'd only have to document how the conversion is done and
> > which accuracy to expect.
> 
> I would be concerned about the float + decimal automatically generating a
> float.  Would it generate an error message if the pedantic flag was set?
> Would it generate a warning in safe mode?
> 
> Also, why do you consider a float to be a "larger" value type than decimal?
> Do you mean that a float is less precise?

See Guido's post on this. 

"Larger" usually means two things:
- more precise
- larger range

This does not necessarily make the decision any easier, though,
since the two terms don't discriminate numeric values too well,
e.g. how do complex and longs fit into the picture ? As a result,
this becomes a language design question.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From m@moshez.org  Wed Aug  1 09:37:47 2001
From: m@moshez.org (Moshe Zadka)
Date: Wed, 01 Aug 2001 11:37:47 +0300
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIELCLCAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCIELCLCAA.tim.one@home.com>
Message-ID: <E15RrVj-0002RV-00@darjeeling>

[Moshe Zadka]
> PUSH "1"
> PUSH "2"
> BINARY_ADD
>
> In Python that gives "12". In Perl that gives 3.
> Unless you suggest a PERL_BINARY_ADD and a PYTHON_BINARY_ADD, I
> don't see how you can around these things.

[Tim]
> Perl needs distinct flavors of BINARY_ADD for its infix "+" and "."
> operators.  Once you accept that, there's no real problem here (it simply
> means that Python's and Perl's "+" operators would need to map to different
> opcodes).

And if you call Python's + opcode PYTHON_BINARY_ADD and Perl's
PERL_BINARY_ADD, it's exactly what I said, isn't it?
Are you agreeing or disagreeing with me? ;-)
And more importantly, what do you think it means for the feasibility
of such a project?
-- 
gpg --keyserver keyserver.pgp.com --recv-keys 46D01BD6 54C4E1FE
Secure (inaccessible): 4BD1 7705 EEC0 260A 7F21  4817 C7FC A636 46D0 1BD6
Insecure (accessible): C5A5 A8FA CA39 AB03 10B8  F116 1713 1BCF 54C4 E1FE
Learn Python! http://www.ibiblio.org/obp/thinkCSpy


From thomas@xs4all.net  Wed Aug  1 09:45:09 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Wed, 1 Aug 2001 10:45:09 +0200
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <15207.26855.65000.718694@gargle.gargle.HOWL>
References: <20010730051831.B1122@thyrsus.com> <20010731012432.G20676@xs4all.nl> <15206.8708.811000.468489@gargle.gargle.HOWL> <20010731115521.I20676@xs4all.nl> <3B668608.F68B5953@ActiveState.com> <15207.26855.65000.718694@gargle.gargle.HOWL>
Message-ID: <20010801104509.D626@xs4all.nl>

On Tue, Jul 31, 2001 at 08:26:47PM -0600, Nathan Torkington wrote:

> I've been eyeing your 'import from the future' system of getting
> experimental features.  I'll be interested to see how this holds up.

It's really not that different form Perl's "use". They're both scanned for
by the parser/tokenizer, they're both used to change the semantics of an
operation, and they also double as a normal 'get and use a package'
operation (which is, in fact, the primary task of 'import' and 'use' both,
right ? :) The main difference is that changing the behaviour of the Python
interpreter is a lot less common than changing the behaviour of the Perl
one, and Guido wants to keep it that way. The purpously 'magic' spelling of
the future-import statement is part of that.

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From thomas@xs4all.net  Wed Aug  1 09:48:22 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Wed, 1 Aug 2001 10:48:22 +0200
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <5.1.0.14.0.20010731234925.02298ae8@24.8.96.48>
Message-ID: <20010801104822.E626@xs4all.nl>

On Tue, Jul 31, 2001 at 11:52:01PM -0400, Dan Sugalski wrote:
> At 11:46 PM 7/31/2001 -0400, Guido van Rossum wrote:
> >Have you looked at the byte code used in SRE, Python's (mostly)
> >Perl-compatible regex package?  Worth a look!

> Nope, not yet, but I will. (You folks will end up making a Python 
> programmer out of me yet, I expect... :)

Well, at least you know it's happening... most people don't realize until
it's too late :)

> This and the C extension interface are the two big Python things I've got 
> on my "must peer deeply into" list. Are there other places I should be 
> concentrating on? (I'm poking about with the interpreter loop and 
> associated code, but that's the sort of thing an expert with 20 minutes 
> free will get me much farther than a week or two with the source)

What do you say I write up a longish email explaining quickly how Python
objects work, and how the Python interpreter and extension code works with
them ? I think it'd help with getting us talking about the same thing when
we say 'interpreter' or 'type' :-) And the beauty of my being in a different
timezone is that noone else is posting right now, so I can take the time
without someone else doing it spread out over four posts in separate
subthreads :)

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From thomas@xs4all.net  Wed Aug  1 09:58:13 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Wed, 1 Aug 2001 10:58:13 +0200
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <E15RrVj-0002RV-00@darjeeling>
Message-ID: <20010801105813.F626@xs4all.nl>

On Wed, Aug 01, 2001 at 11:37:47AM +0300, Moshe Zadka wrote:
> [Moshe Zadka]
> > PUSH "1"
> > PUSH "2"
> > BINARY_ADD
> >
> > In Python that gives "12". In Perl that gives 3.
> > Unless you suggest a PERL_BINARY_ADD and a PYTHON_BINARY_ADD, I
> > don't see how you can around these things.
> 
> [Tim]
> > Perl needs distinct flavors of BINARY_ADD for its infix "+" and "."
> > operators.  Once you accept that, there's no real problem here (it simply
> > means that Python's and Perl's "+" operators would need to map to different
> > opcodes).

> And if you call Python's + opcode PYTHON_BINARY_ADD and Perl's
> PERL_BINARY_ADD, it's exactly what I said, isn't it?
> Are you agreeing or disagreeing with me? ;-)

He's disagreeing. It's not a PERL vs. PYTHON ADD at all; it's a "string
concatenation add" vs. "numerical add". Perl code using the string
concatenation operator (apparently, it's not going to be ".", which scares
me shitless: Perl6 gets a new string concat operator, but it isn't going to
be "+" ? If it is going to be '+', how does it flexitype ?) would use the
string-concat add opcode, and Perl code and Python code doing '+' would get
the normal BINARY_ADD. The string-to-int conversion in Perl's '+' would be
put into the 'Scalar' type.

Channeling-Tim--probably-wrong-<wink>-ly y'rs,
-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From m@moshez.org  Wed Aug  1 10:21:52 2001
From: m@moshez.org (Moshe Zadka)
Date: Wed, 01 Aug 2001 12:21:52 +0300
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <20010801105813.F626@xs4all.nl>
References: <20010801105813.F626@xs4all.nl>
Message-ID: <E15RsCO-0002c5-00@darjeeling>

On Wed, 1 Aug 2001 10:58:13 +0200, Thomas Wouters <thomas@xs4all.net> wrote:

> He's disagreeing. It's not a PERL vs. PYTHON ADD at all; it's a "string
> concatenation add" vs. "numerical add". Perl code using the string
> concatenation operator (apparently, it's not going to be ".", which scares
> me shitless: Perl6 gets a new string concat operator, but it isn't going to
> be "+" ? If it is going to be '+', how does it flexitype ?) would use the
> string-concat add opcode, and Perl code and Python code doing '+' would get
> the normal BINARY_ADD. The string-to-int conversion in Perl's '+' would be
> put into the 'Scalar' type.

I think I've lost you.
But if you're thinking about Perl compiled to something like

PUSH "1"
CONVERT_TO_INT
PUSH "2"
CONVERT_TO_INT
BINARY_ADD

Then it's only a matter of time before the Perl guys find out it's hella
slow and optimize it to:

PUSH "1"
PUSH "2"
CONVERT_TO_INT_TOP_2
BINARY_ADD

And then to

PUSH "1"
PUSH "2"
CONVERT_TO_INT_TOP_2_AND_BINARY_ADD

And then a simple

#define CONVERT_TO_INT_TOP_2_AND_BINARY_ADD PERL_ADD

Would finish the 2*PI circle to land exactly where we started with. ;-)

-- 
gpg --keyserver keyserver.pgp.com --recv-keys 46D01BD6 54C4E1FE
Secure (inaccessible): 4BD1 7705 EEC0 260A 7F21  4817 C7FC A636 46D0 1BD6
Insecure (accessible): C5A5 A8FA CA39 AB03 10B8  F116 1713 1BCF 54C4 E1FE
Learn Python! http://www.ibiblio.org/obp/thinkCSpy


From thomas@xs4all.net  Wed Aug  1 10:53:17 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Wed, 1 Aug 2001 11:53:17 +0200
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <E15RsCO-0002c5-00@darjeeling>
References: <20010801105813.F626@xs4all.nl> <E15RsCO-0002c5-00@darjeeling>
Message-ID: <20010801115317.F624@xs4all.nl>

On Wed, Aug 01, 2001 at 12:21:52PM +0300, Moshe Zadka wrote:
> On Wed, 1 Aug 2001 10:58:13 +0200, Thomas Wouters <thomas@xs4all.net> wrote:
> 
> > He's disagreeing. It's not a PERL vs. PYTHON ADD at all; it's a "string
> > concatenation add" vs. "numerical add". Perl code using the string
> > concatenation operator (apparently, it's not going to be ".", which scares
> > me shitless: Perl6 gets a new string concat operator, but it isn't going to
> > be "+" ? If it is going to be '+', how does it flexitype ?) would use the
> > string-concat add opcode, and Perl code and Python code doing '+' would get
> > the normal BINARY_ADD. The string-to-int conversion in Perl's '+' would be
> > put into the 'Scalar' type.

> I think I've lost you.


> But if you're thinking about Perl compiled to something like

> PUSH "1"
> CONVERT_TO_INT
> PUSH "2"
> CONVERT_TO_INT
> BINARY_ADD

No, I'm not. I was, but I'm not.

Okay, Moshe, lets try this again. Imagine a Python object, "Scalar", that
behaves like a Perl $calar. We're not going to talk about merging the Perl
and Python VM here, just about a Python object that behaves somewhat like
a Perl scalar, and (for the sake of the discussion) just the string<->int
part of the flexitype. 

The Scalar holds its value as a string, unless it's been stored as an int or
'modified' into an int. Pretend we have an operator "@" to explicitly
string-concatenate, exactly like Perl's ".". Now look at this code snippet:

   one = Scalar("1")
   two = Scalar("2")
   three_num = one + two
   three_s = one @ two

"one + two" would simply all BINARY_ADD, and the Scalar type's
tp_as_number->nb_add would convert its value internally to an int before
handling the operation. "one @ two" would call BINARY_CONCAT (or whatever),
which would call a different method to calculate the result. Effectively a
different operation.

But this isn't "perl add versus python add". "perl add" is just the same as
"python add" with a slightly different type, we just get a new opcode for
the "perl string concat" operator. 

Like I (and Eric) said before, there's a lot to be done, and there's bound
to be a lot of issues (like how to get *both* scalars to convert their
value to a number before arithmatic, above, without resulting to something
like PyObject_AsInt() for all binary arithmatic) but those are most likely
not insurmountable.

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From dan@sidhe.org  Wed Aug  1 12:21:47 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Wed, 01 Aug 2001 07:21:47 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <20010801104822.E626@xs4all.nl>
References: <5.1.0.14.0.20010731234925.02298ae8@24.8.96.48>
Message-ID: <5.1.0.14.0.20010801072126.022f2400@24.8.96.48>

At 10:48 AM 8/1/2001 +0200, Thomas Wouters wrote:
>On Tue, Jul 31, 2001 at 11:52:01PM -0400, Dan Sugalski wrote:
> > This and the C extension interface are the two big Python things I've got
> > on my "must peer deeply into" list. Are there other places I should be
> > concentrating on? (I'm poking about with the interpreter loop and
> > associated code, but that's the sort of thing an expert with 20 minutes
> > free will get me much farther than a week or two with the source)
>
>What do you say I write up a longish email explaining quickly how Python
>objects work, and how the Python interpreter and extension code works with
>them ? I think it'd help with getting us talking about the same thing when
>we say 'interpreter' or 'type' :-) And the beauty of my being in a different
>timezone is that noone else is posting right now, so I can take the time
>without someone else doing it spread out over four posts in separate
>subthreads :)

That'd be most helpful, thanks.

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From gmcm@hypernet.com  Wed Aug  1 14:11:36 2001
From: gmcm@hypernet.com (Gordon McMillan)
Date: Wed, 1 Aug 2001 09:11:36 -0400
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: <Pine.LNX.4.32.0107311813220.19357-100000@ziggy.localdomain.fake>
References: <200107310622.CAA11742@cj20424-a.reston1.va.home.com>
Message-ID: <3B67C7C8.13312.16DF23AD@localhost>

Ka-Ping Yee wrote:

[Guido]
> > Why not add this to cgi.py instead?
> 
> Because that would seem to defeat most of the point.

[snip]

> Wouldn't you say there are more Python CGI programmers out there
> than, say, Zope developers?  Or think of it like this: what
> fraction of Web developers are CGI programmers, and what fraction
> of those use Python?
> 
> Maybe i'm wrong?  I welcome more opinions from others -- how do
> you see people coming to Python?  What's the first "real" thing
> they do with Python that motivates them to try it?

Web stuff is certainly a major doorway. Not sure about cgi.py 
itself.

Web stuff accounts for (very, very roughly) 20% of what I do. I 
have never used cgi.py. In fact, the only web related module in 
the std lib I use regularly is urllib (mostly for the quote stuff). 
So neither site.py nor cgi.py would do me any good. (In fact, 
cgitb.py wouldn't either, since my source is not on the server 
and sys.std* have nothing to do with requests/responses).

Judging by the list at
 http://www.paul.boddie.net/Python/web_modules.html
I don't think I'm all that unusual in this regard.

(BTW, line 133 should be
filename = tempfile.mktemp('.html')
).

- Gordon


From gmcm@hypernet.com  Wed Aug  1 14:11:36 2001
From: gmcm@hypernet.com (Gordon McMillan)
Date: Wed, 1 Aug 2001 09:11:36 -0400
Subject: [Python-Dev] Revised decimal type PEP
In-Reply-To: <200108010251.WAA18015@cj20424-a.reston1.va.home.com>
References: Your message of "Tue, 31 Jul 2001 16:55:26 PDT."             <20010731235526.8E8ED99C82@waltz.rahul.net>
Message-ID: <3B67C7C8.5814.16DF229F@localhost>

[Guido]

> ....  I was toying with a
> *different* idea, more appropriate perhaps for accountants, that
> guarantees exact results for decimal numbers, giving decimal
> results when possible, rational otherwise.  

My experience with accountants (~5 yrs in another life) is that 
they're perfectly happy with fixed point decimal. They may 
make you do things to the 1/100 of a penny & then round in 
funny ways that favor their bosses, but they want every 
calculation to come out the same as on their calculator.

- Gordon


From guido@zope.com  Wed Aug  1 14:22:22 2001
From: guido@zope.com (Guido van Rossum)
Date: Wed, 01 Aug 2001 09:22:22 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: Your message of "Wed, 01 Aug 2001 00:42:42 EDT."
 <LNBBLJKPBEHFEDALKOLCAELCLCAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCAELCLCAA.tim.one@home.com>
Message-ID: <200108011322.JAA19671@cj20424-a.reston1.va.home.com>

[Tim]
> But BINARY_ADD knows everything there is to know about Python ints, and
> that's an important speed optimization:

Which makes it a perfect example IMO: the abstract *semantics* of the
opcode are as neutral as can be, and the *implementation* can grease
the path for a common case.  A complete redesign can still do that.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido@zope.com  Wed Aug  1 14:25:46 2001
From: guido@zope.com (Guido van Rossum)
Date: Wed, 01 Aug 2001 09:25:46 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: Your message of "Wed, 01 Aug 2001 00:16:43 EDT."
 <5.1.0.14.0.20010801000522.0238d970@24.8.96.48>
References: <Your message of "Tue, 31 Jul 2001 23:43:45 EDT." <5.1.0.14.0.20010731231748.0228f808@24.8.96.48> <Your message of "Tue, 31 Jul 2001 13:31:01 MDT." <15207.1909.395000.123189@gargle.gargle.HOWL> <20010730051831.B1122@thyrsus.com> <Pine.NXT.4.21.0107301836260.258-100000@localhost.virginia.edu> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1909.395000.123189@gargle.gargle.HOWL> <5.1.0.14.0.20010731231748.0228f808@24.8.96.48>
 <5.1.0.14.0.20010801000522.0238d970@24.8.96.48>
Message-ID: <200108011325.JAA19689@cj20424-a.reston1.va.home.com>

> The vtable interface, at the moment, covers the 
> operators you'd expect to be able to override--assignment, simple math, 
> most string operations, conversion duties, and suchlike things. So this code:
> 
>     a = b + c
> 
> would call the add vtable method for b, and the assign vtable method for a, 

But in Python, this is not an operation on a at all!  It's an
operation on the namespace containing a, and a cannot override it.
I'm sure your VM design can accommodate this, but it points out the
fundamental difference between the languages in their ideas of what a
"variable" is.  Python has at least two types of namespaces in this
context: some namespaces (like module globals) are dictionaries,
others (like function locals) have mapped the variable names to an
array of object references.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@zope.com  Wed Aug  1 14:33:37 2001
From: guido@zope.com (Guido van Rossum)
Date: Wed, 01 Aug 2001 09:33:37 -0400
Subject: [Python-Dev] Revised decimal type PEP
In-Reply-To: Your message of "Wed, 01 Aug 2001 09:50:48 +0200."
 <3B67B4D8.2B319336@lemburg.com>
References: <20010801010750.E045299C80@waltz.rahul.net>
 <3B67B4D8.2B319336@lemburg.com>
Message-ID: <200108011333.JAA19785@cj20424-a.reston1.va.home.com>

> Decimals are just a different way of displaying Rationals and
> depending on their preset precision show a different behaviour
> in numeric operations.

Not necessarily.  If decimals are used for (in-memory) representation,
they can't represent all rationals.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From thomas@xs4all.net  Wed Aug  1 14:36:07 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Wed, 1 Aug 2001 15:36:07 +0200
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <5.1.0.14.0.20010731234925.02298ae8@24.8.96.48>
Message-ID: <20010801153607.G626@xs4all.nl>

On Tue, Jul 31, 2001 at 11:52:01PM -0400, Dan Sugalski wrote:

> [SRE] and the C extension interface are the two big Python things I've got
> on my "must peer deeply into" list.

I can't say much about SRE, but the C extension interface should be on
anyone's "must peer deeply into" list, except that there isn't a real need
to peer deeply into it :) Lets peer into it. I'm going to discuss the
pre-2.2 state of affairs, mostly because I know jack shit about the
2.2-type-dichotomy-healing :) In case you aren't aware, we are currently
into the 2.2 alphas, where this story is slightly different because you can
now subclass types. If Guido or anyone else involved in the descr-branch
wants to tune in and correct the explanation for Python 2.2, that's fine by
me.

As you might have heard, everything in Python is an object. This is true
even for the C side of things. A Python object, any Python object, is
essentially a struct like this:

stuct PyExample {
    int ob_refcnt;                     /* reference count */
    struct _typeobject *ob_type;       /* type of this object */

    /* type-specific data goes here */

}

The common header is made available as a #define, so that we can add extra
info in severe-debug mode (though this breaks binary compatibility, so it
should not be done in production builds.)

For instance, the struct to hold a (normal, unbounded) Python integer is

typedef struct {
    PyObject_HEAD
    long ob_ival;
} PyIntObject;

The struct _typeobject pointer is a pointer to a Type object, which also
acts as a vtable. It contains all the information about the type, including
how to operate on it. The typeobject is itself also an object! Its type is
the PyType type, which is its own type. The typeobject is actually what you
get back when you do 'type(object)' in Python code. All this is also
explained in Include/object.h, by the way.

The type object holds things like the type name, how to print it, howmuch
memory should be allocated, a couple of flags to implement binary
compatibility, and a whole boatload of function pointers. The function
pointers for operations are spread out over the type struct and several
separate structs, to which the type struct points: tp_as_number,
tp_as_sequence, and tp_as_mapping. Each of these can be NULL if the type
doesn't implement the operations.

Some of the operations that are defined on the type struct itself are
get-attribute, set-attribute, comparison, get-hashvalue, convert-to-string
and call-as-function.

The PyNumberMethods struct (tp_as_number) contains arithmatic operations
(such as adding, mulitplying, shifting, XORing, and in-place variations of
each for use with augmented assignment), truth-value-testing, and converting
to various other integer types.

The PySequenceMethods struct (tp_as_sequence) contains sequence operations
such as membership-test, item- and slice-retrieval and -assignment, and
sequence concatenation and repetition. The latter two are somewhat of an odd
duck, since they implement the same Python operation ("+" and "*") as the
'add' and 'multiply' methods from the PyNumberMethods struct. They're mainly
so the Python runtime can properly handle "number * list" as well as "list *
number" without attempting to convert list to an integer.

Lastly there's PyMappingMethods. I'm not at all sure why this is a separate
sturct, as the three operations it contains (length inquiry, item-retrieval
and item-assignment) are also covered by the PySequenceMethods, but I
suspect some kind of hysterical raisin crept in here :)

To see this construct in action, take a look at Objects/intobject.c, in the
Python source tree (doesn't really matter which version.) It defines a bunch
of functions to do the arithmatic, fills a PyNumberMethods struct with the
appropriate pointers (leaving unsupported ones NULL), and fills a new
PyTypeObject with the right info for the "int" type.

To contrast, the PyFunction type is nearly empty. It defines getattr and
setattr, and a 'repr' function (convert to string). You can't compare it (it
will always compare false to anything but itself), add it, index it, or get
its length, but you can get attributes from it. And the average function has
some interesting attributes: you can, from Python code and C code alike,
easily get at things like the bytecode it consists of, the number of
arguments it expects, the names of those arguments, etc. Each of those is,
of course, a Python object in its own right :) Function objects are defined
in Objects/funcobject.c, though the actual execution of function objects is
done elsewhere.

An 'instance' object is a slightly different case in this story (more so in
the post-2.2 story, but I'm not sure howmuch more.) An instance object is a
Python class, defined in Python code by the user/programmer. It can choose
to implement any of the above operations by defining magically named methods
on the class. For instance, to implement the 'getitem' protocol, it defines
a function __getitem__, and to implement addition, __add__. The Type object
for instance objects acts as a proxy for the Python class, though this does
represent some problems: instance objects need to be treated in some areas
of the interpreter (but this is almost certain to have been changed in 2.2.)

But back to generic Python objects. Obviously, when writing C code, you
don't want to manipulate Python objects by directly accessing all those
layers of C structs all the time. So each builtin type defines useful C
functions and macros to help. For instance, to create a new list, you call
PyList_New(size). To test whether something is a list, PyList_Check(object).
To index a list, PyList_GetItem(list, index), or a macro version of that,
PyList_GET_ITEM. The macro version doesn't do type or boundary checks, so it
should only be used when you already did those checks. Likewise, there is
PyInt_AsLong (convert a PyInt to a C 'long') and PyInt_AS_LONG.

But even that isn't that helpful in a dynamically typed language, since you
have to know an object's type in order to call the type-specific functions.
So we also have two more abstraction layers. The most abstract one is the
Object layer. Basically, regardless of the type of object, you can always
manipulate it using PyObject_*() functions. Some PyObject_ functions have a
direct relation to Python code, others are only useful to C code. There are
PyObject_ functions such as PyObject_IsTrue, PyObject_Compare,
PyObject_GetAttr, PyObject_CallObject, etcetera. All these work regardless
of the type. If the type (or instance) does not implement the operation, an
exception is raised (see below.)

The other layer is the slightly less abstract layer that differentiates in
the same way as the PyNumberMethods, PySequenceMethods, PyMappingMethods
structs. To add two objects, you call PyNumber_Add(o1, o2). To explicitly
concatenate two objects, you call PySequence_Concat(o1, o2). All of these
work for any type that defines the behaviour, including instance types. Note
that calling one of these functions can end up executing arbitrary Python
code, so they shouldn't be called in any time-critical situation :-)

As a last part of the API, there is the reference counting. As I showed a
couple of pages back, all objects contain a reference count. Operations that
copy a reference should increment the reference count, using Py_INCREF, and
deleting a reference should be done using Py_DECREF. The latter also does
the deallocation and cleanup if the DECREF causes the refcount to drop to 0.
All API functions that return a reference define whether they return a new
reference, which the caller has to DECREF when throwing it away, or a
borrowed reference, which shouldn't be DECREF'ed, or kept around without
INCREFing it.

Exceptions, by the way, are set by setting a global (or thread-local, in a
threaded interpreter) variable to contain the Python exception object, and
then returning an error flag (either NULL, -1 or 0, depending on the
function) to the caller. All code should check all return values and always
return an error value if a called function does so, unless it's prepared to
handle the exception itself. To compare errors with a particular error
'class', there is PyErr_ExceptionMatches (which also handles 'subclasses' of
exception types.) If the error value returned by a particular function is
also a real possible value (such as the value returned by PyNumber_AsLong),
there is PyErr_Occured() to see if it was a real exception, or a legitimate
-1 value.

So that's it for the API. All Python objects can be manipulated, created and
destroyed this way. Most extension types provide their functionality either
in builtin-operations (in the type object) or by providing methods that can
be called (using PyObject_GetAttr and PyObject_CallObject), and only rarely
need to export an explicit API of their own. (And as you probably know,
exporting an API from a shared library to another shared library is...
slightly tricky :)


Now, the Python bytecode-interpreter itself is very small. All it does is
interpret bytecode, and then call out to the API to make it do the actual
work. It has some special knowledge about some types, for optimzation, and
it does all the real exception handling and namespace handling (name lookup
and such), but the actual manipulation of objects is left to the API. Even
so, it has enough to do. In the case of exceptions, it has to search up the
function for a 'catching' block (a try/except or try/finally statement), or
break out of the function to let the error percolate to the caller.

Namespaces are a complete story in their own right. Python has three
distinct namespaces: the 'local' namespace inside a function (or 'code
block'), the 'global' or 'module-level' namespace at the toplevel of a file,
and the 'builtin' namespace which holds builtin functions and names. The
search order is always local->global->builtin, though for code that is
executed at the module level (not inside a function), local is the same as
global.

In Python, variables are defined by assignment. As a consequence, any
variable that is assigned to in a code block, is local to that block. Any
variable that isn't assigned to but is just referenced thus has to be
defined in the global namespace. The compiler keeps track of this, and
generates 'LOAD_FAST' (for loading from the local namespace, inside a
function), 'LOAD_GLOBAL' (for loading from the global or builtin namespace,
skipping the local namespace), and 'LOAD_NAME' (for occurances where it's
truly not known where a variable comes from, and both local and global
namespaces have to be searched.) The latter is necessary because Python has
a few operations that can modify a local namespace at runtime (such as 'from
module import *', and 'exec'.)

Since Python 2.1, Python has 'nested scopes', which makes code blocks
defined inside other code blocks slightly special: the 'parent' scope is
also searched for any variable that isn't local to the nested scope. Most of
this, but not all, is done at compiletime, IIRC. (Jeremy can correct me if
I'm wrong :)

I tried to make the distinction between a 'function' and a 'code block',
since the VM really deals with the second, not the first. I suspect that the
current Python VM's mechanics to deal with nested scopes could be
generalized so that Perl's lexical closures could fit as well (I assume that
won't change in perl6 :). Then again, Perl already knows where a variable
comes from, so the VM will probably want to put that to good use... The main
reason Python does the name-searching at runtime is that a global namespace
can be altered from outside the currently-executing codeblock (threads, or
other modules, can do 'module.range = my_fake_range', and suddenly the
range() function does something completely different throughout the entire
module.)

A good overview of how the VM uses the API can be obtained by looking at
Python/ceval.c, and looking for "switch (opcode)". You'll end up at the top
of the main bytecode switch. Most of the opcode names are pretty
descriptive, but if you want a better overview, take a look at the 'dis'
module's documentation. (www.python.org is still down, but you can use one
of the mirrors:

http://python.mirrors.netnumina.com/doc/current/lib/module-dis.html
http://python.mirrors.netnumina.com/doc/current/lib/bytecodes.html

And an easy way to see how a Python operation is actually implemented is
using the 'dis' module interactively:

>>> import dis
>>> def spam(x):
...     print "Hello Perl World!"
>>> dis.dis(spam)
          0 SET_LINENO               1

          3 SET_LINENO               2
          6 SETUP_LOOP              44 (to 53)

    >>    9 SET_LINENO               2
         12 LOAD_FAST                0 (x)
         15 JUMP_IF_FALSE           33 (to 51)
         18 POP_TOP             

         19 SET_LINENO               3
         22 LOAD_CONST               1 ('Hello ')
         25 LOAD_CONST               2 ('Perl ')
         28 BINARY_ADD          
         29 LOAD_CONST               3 ('World!')
         32 BINARY_ADD          
         33 PRINT_ITEM          
         34 PRINT_NEWLINE       

         35 SET_LINENO               4
         38 LOAD_FAST                0 (x)
         41 LOAD_CONST               4 (1)
         44 INPLACE_SUBTRACT    
         45 STORE_FAST               0 (x)
         48 JUMP_ABSOLUTE            9
    >>   51 POP_TOP             
         52 POP_BLOCK           
    >>   53 LOAD_CONST               0 (None)
         56 RETURN_VALUE        


Keep in mind that this is stack-based, and all operations manipulate the
stack. (for instance, LOAD_FAST pushes the variable onto the stack, and
BINARY_ADD pops two values, adds them, and pushes the result.)

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From guido@zope.com  Wed Aug  1 14:48:30 2001
From: guido@zope.com (Guido van Rossum)
Date: Wed, 01 Aug 2001 09:48:30 -0400
Subject: [Python-Dev] Revised decimal type PEP
In-Reply-To: Your message of "Wed, 01 Aug 2001 09:48:10 +0200."
 <3B67B43A.D29AF194@lemburg.com>
References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073023115207.02466@fermi.eeel.nist.gov> <3B66DE8C.C9C62012@lemburg.com> <01073103271101.02004@fermi.eeel.nist.gov> <200107312326.TAA17540@cj20424-a.reston1.va.home.com>
 <3B67B43A.D29AF194@lemburg.com>
Message-ID: <200108011348.JAA19995@cj20424-a.reston1.va.home.com>

> Could you perhaps write this coercion scheme up as informational
> PEP ? I think it would help a lot as reference to what Python should
> do and serve well for numeric extension writers as basis for their
> coercion decisions.

I'm not sure which coercion scheme you are referring to.  I'm also
overwhelmed with other stuff so I think I'll pass, but I'll gladly
help steer a draft by someone else in the right direction.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mclay@nist.gov  Wed Aug  1 02:50:57 2001
From: mclay@nist.gov (Michael McLay)
Date: Tue, 31 Jul 2001 21:50:57 -0400
Subject: [Python-Dev] Revised decimal type PEP
In-Reply-To: <200107312326.TAA17540@cj20424-a.reston1.va.home.com>
References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073103271101.02004@fermi.eeel.nist.gov> <200107312326.TAA17540@cj20424-a.reston1.va.home.com>
Message-ID: <01073121505701.02964@fermi.eeel.nist.gov>

On Tuesday 31 July 2001 07:26 pm, Guido van Rossum wrote:
> > Also, why do you consider a float to be a "larger" value type than
> > decimal? Do you mean that a float is less precise?
>
> (Warning: I think the following is a sound model, but I'm still
> practicing how to explain it right.)
>
>
> I have this ordering of the types in mind:
>
> int/long < decimal < rational < float < complex
> \---------------------------/   \-------------/
>             exact                  inexact
>
> This is different from the Scheme numeric "tower" -- I no longer agree
> with the Scheme model any more.
>
> The ordering is only to determine what happens on mixed arithmetic:
> the result has the rightmost type in the diagram (or a type further on
> the right in some cases).
>
> The ints are a subset of the decimal numbers, and the decimal numbers
> (in this view) are a subset of the rational numbers.  Ints and
> decimals aren't closed under division -- the result of division on
> these (in general) is a rational.  While the exact values of floats
> are a subset of the rationals, the inexactness property (which I give
> all floats) makes that each float stands for an infinite set of
> numbers *including* the exact value.  When a binary operation involves
> an exact and an inexact operand, the result is inexact.

Hmm, am I understanding you explaination?

Here is a rational expression:

   9/4 * 4/3 =3D 3

With floats this ends up being close, but with rounding errors.

>>> 2.25*1.333333
2.9999992500000001
>>>

If this is expressed as a product of 2.25b * 1.333333d the result would b=
e an=20
inexact value.  A binary number would be returned, instead of the decimal=
=20
number 3.


> Tim's "numeric context" contains a bunch of flags controlling detailed
> behavior of numeric operations.  It could specify that mixing exact
> and inexact numbers is illegal, and that would be Michael's pedantic
> mode.  It could also specify warnings.  (I would never call a mode
> that issues warnings "safe" :-)

Where is Tim's "numeric context" located? =20


M.-A. Lemburg"  suggested looking at the SQL specification for Decimal=20
datatypes.  A decimal type is also defined as a type in XML Schema.  Sinc=
e=20
this is an XML datatype there isn't a definition for how these numbers ar=
e=20
created.

  NOTE: All =B7minimally conforming=B7 processors =B7must=B7 support deci=
mal numbers
  with a minimum of 18 decimal digits (i.e., with a =B7totalDigits=B7 of =
18).
  However, =B7minimally conforming=B7 processors =B7may=B7 set an applica=
tion-defined
  limit on the maximum number of decimal digits they are prepared to supp=
ort,
  in which case that application-defined maximum number =B7must=B7 be cle=
arly
  documented.
	- http://www.w3.org/TR/xmlschema-2/#decimal



From guido@zope.com  Wed Aug  1 15:55:02 2001
From: guido@zope.com (Guido van Rossum)
Date: Wed, 01 Aug 2001 10:55:02 -0400
Subject: [Python-Dev] Revised decimal type PEP
In-Reply-To: Your message of "Tue, 31 Jul 2001 21:50:57 EDT."
 <01073121505701.02964@fermi.eeel.nist.gov>
References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073103271101.02004@fermi.eeel.nist.gov> <200107312326.TAA17540@cj20424-a.reston1.va.home.com>
 <01073121505701.02964@fermi.eeel.nist.gov>
Message-ID: <200108011455.KAA20439@cj20424-a.reston1.va.home.com>

> Hmm, am I understanding you explaination?
> 
> Here is a rational expression:
> 
>    9/4 * 4/3 = 3
> 
> With floats this ends up being close, but with rounding errors.
> 
> >>> 2.25*1.333333
> 2.9999992500000001
> >>>
> 
> If this is expressed as a product of 2.25b * 1.333333d the result would be an 
> inexact value.  A binary number would be returned, instead of the decimal 
> number 3.

Correct.

> Where is Tim's "numeric context" located?  

In his mind. :-)

I believe it is typically global per thread, but that's up to the
langage binding.  A Java binding for Cowlishaw's decimals apparently
requires passing in a context as a third argument on each operation.

> M.-A. Lemburg"  suggested looking at the SQL specification for Decimal 
> datatypes.  A decimal type is also defined as a type in XML Schema.  Since 
> this is an XML datatype there isn't a definition for how these numbers are 
> created.

Do these say anything about semantics under numeric operations?  That
would seem to be outside the realm of XML and possibly even outside
SQL.  So I'm not sure how these help.

>   NOTE: All ·minimally conforming· processors ·must· support decimal numbers
>   with a minimum of 18 decimal digits (i.e., with a ·totalDigits· of 18).
>   However, ·minimally conforming· processors ·may· set an application-defined
>   limit on the maximum number of decimal digits they are prepared to support,
>   in which case that application-defined maximum number ·must· be clearly
>   documented.
> 	- http://www.w3.org/TR/xmlschema-2/#decimal

I followed the URL and found only external representation issues,
nothing that can help us.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@zope.com  Wed Aug  1 16:16:41 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Wed, 1 Aug 2001 11:16:41 -0400 (EDT)
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <200108011325.JAA19689@cj20424-a.reston1.va.home.com>
References: <Your message of "Tue, 31 Jul 2001 23:43:45 EDT." <5.1.0.14.0.20010731231748.0228f808@24.8.96.48>
 <Your message of "Tue, 31 Jul 2001 13:31:01 MDT." <15207.1909.395000.123189@gargle.gargle.HOWL>
 <20010730051831.B1122@thyrsus.com>
 <Pine.NXT.4.21.0107301836260.258-100000@localhost.virginia.edu>
 <15206.51329.561652.565480@beluga.mojam.com>
 <200107311708.NAA16497@cj20424-a.reston1.va.home.com>
 <15206.61296.958360.72700@beluga.mojam.com>
 <200107311900.PAA17062@cj20424-a.reston1.va.home.com>
 <15207.1909.395000.123189@gargle.gargle.HOWL>
 <5.1.0.14.0.20010731231748.0228f808@24.8.96.48>
 <5.1.0.14.0.20010801000522.0238d970@24.8.96.48>
 <200108011325.JAA19689@cj20424-a.reston1.va.home.com>
Message-ID: <15208.7513.169067.995475@slothrop.digicool.com>

>>>>> "GvR" == Guido van Rossum <guido@zope.com> writes:

  GvR> But in Python, this is not an operation on a at all!  It's an
  GvR> operation on the namespace containing a, and a cannot override
  GvR> it.  I'm sure your VM design can accommodate this, but it
  GvR> points out the fundamental difference between the languages in
  GvR> their ideas of what a "variable" is.  Python has at least two
  GvR> types of namespaces in this context: some namespaces (like
  GvR> module globals) are dictionaries, others (like function locals)
  GvR> have mapped the variable names to an array of object
  GvR> references.

In the future, I expect module namespaces will be implemented more
like function namespaces.  That's the "low-hanging fruit" optimization
idea we've kicked around for a while.  I hope to work on it for 2.3.

This discussion of variables also raises a question for the Perl guys:
Is there a Perl language reference ala the Python reference manual?

Jeremy




From dan@sidhe.org  Wed Aug  1 16:22:11 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Wed, 01 Aug 2001 11:22:11 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <15208.7513.169067.995475@slothrop.digicool.com>
References: <200108011325.JAA19689@cj20424-a.reston1.va.home.com>
 <Your message of "Tue, 31 Jul 2001 23:43:45 EDT." <5.1.0.14.0.20010731231748.0228f808@24.8.96.48>
 <Your message of "Tue, 31 Jul 2001 13:31:01 MDT." <15207.1909.395000.123189@gargle.gargle.HOWL>
 <20010730051831.B1122@thyrsus.com>
 <Pine.NXT.4.21.0107301836260.258-100000@localhost.virginia.edu>
 <15206.51329.561652.565480@beluga.mojam.com>
 <200107311708.NAA16497@cj20424-a.reston1.va.home.com>
 <15206.61296.958360.72700@beluga.mojam.com>
 <200107311900.PAA17062@cj20424-a.reston1.va.home.com>
 <15207.1909.395000.123189@gargle.gargle.HOWL>
 <5.1.0.14.0.20010731231748.0228f808@24.8.96.48>
 <5.1.0.14.0.20010801000522.0238d970@24.8.96.48>
 <200108011325.JAA19689@cj20424-a.reston1.va.home.com>
Message-ID: <5.1.0.14.0.20010801112115.0240c7a8@24.8.96.48>

At 11:16 AM 8/1/2001 -0400, Jeremy Hylton wrote:
>This discussion of variables also raises a question for the Perl guys:
>Is there a Perl language reference ala the Python reference manual?

Not for perl 6, nope. (Larry's working on that) The Camel and the 
implementation are pretty much what we've got for perl 5.

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From mal@lemburg.com  Wed Aug  1 17:54:35 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 01 Aug 2001 18:54:35 +0200
Subject: [Python-Dev] Revised decimal type PEP
References: <20010801010750.E045299C80@waltz.rahul.net>
 <3B67B4D8.2B319336@lemburg.com> <200108011333.JAA19785@cj20424-a.reston1.va.home.com>
Message-ID: <3B68344B.9BFBB666@lemburg.com>

Guido van Rossum wrote:
> 
> > Decimals are just a different way of displaying Rationals and
> > depending on their preset precision show a different behaviour
> > in numeric operations.
> 
> Not necessarily.  If decimals are used for (in-memory) representation,
> they can't represent all rationals.

I know, that's what I wanted to say with "depending on their preset
precision"; if you have a decimal with precision 2 then 1/3 will 
come out as "0.33" -- my point was that by implementing a Rational
type instead of a decimal type we can have all semantics of decimals
by simply subclassing the Rational type. 

This setup will be more flexible than choosing one of the many 
different variants of "how to add two decimal numbers" 
(e.g. calculator style, with hidden extra digits, with truncation, 
mathematical rounding, (one of the various) financial rounding 
modes, etc.).

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/



From m@moshez.org  Wed Aug  1 18:01:32 2001
From: m@moshez.org (Moshe Zadka)
Date: Wed, 01 Aug 2001 20:01:32 +0300
Subject: [Python-Dev] Revised decimal type PEP
In-Reply-To: <200108011333.JAA19785@cj20424-a.reston1.va.home.com>
References: <200108011333.JAA19785@cj20424-a.reston1.va.home.com>, <20010801010750.E045299C80@waltz.rahul.net>
 <3B67B4D8.2B319336@lemburg.com>
Message-ID: <E15RzNE-0003au-00@darjeeling>

[Marc]
> Decimals are just a different way of displaying Rationals and
> depending on their preset precision show a different behaviour
> in numeric operations.

[Guido]
> Not necessarily.  If decimals are used for (in-memory) representation,
> they can't represent all rationals.

Different definitions of decimals? Marc was thinking repeating decimals,
if I understood him correctly.
-- 
gpg --keyserver keyserver.pgp.com --recv-keys 46D01BD6 54C4E1FE
Secure (inaccessible): 4BD1 7705 EEC0 260A 7F21  4817 C7FC A636 46D0 1BD6
Insecure (accessible): C5A5 A8FA CA39 AB03 10B8  F116 1713 1BCF 54C4 E1FE
Learn Python! http://www.ibiblio.org/obp/thinkCSpy


From fdrake@acm.org  Wed Aug  1 18:23:20 2001
From: fdrake@acm.org (Fred L. Drake)
Date: Wed,  1 Aug 2001 13:23:20 -0400 (EDT)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20010801172320.BE5A72879B@cj42289-a.reston1.va.home.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

Incremental updates, including some useful improvements in the docs for
the re module.



From DavidA@ActiveState.com  Wed Aug  1 18:32:22 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Wed, 01 Aug 2001 10:32:22 -0700
Subject: [Python-Dev] Parrot -- should life imitate satire?
References: <200108011325.JAA19689@cj20424-a.reston1.va.home.com>
 <Your message of "Tue, 31 Jul 2001 23:43:45 EDT." <5.1.0.14.0.20010731231748.0228f808@24.8.96.48>
 <Your message of "Tue, 31 Jul 2001 13:31:01 MDT." <15207.1909.395000.123189@gargle.gargle.HOWL>
 <20010730051831.B1122@thyrsus.com>
 <Pine.NXT.4.21.0107301836260.258-100000@localhost.virginia.edu>
 <15206.51329.561652.565480@beluga.mojam.com>
 <200107311708.NAA16497@cj20424-a.reston1.va.home.com>
 <15206.61296.958360.72700@beluga.mojam.com>
 <200107311900.PAA17062@cj20424-a.reston1.va.home.com>
 <15207.1909.395000.123189@gargle.gargle.HOWL>
 <5.1.0.14.0.20010731231748.0228f808@24.8.96.48>
 <5.1.0.14.0.20010801000522.0238d970@24.8.96.48>
 <200108011325.JAA19689@cj20424-a.reston1.va.home.com> <5.1.0.14.0.20010801112115.0240c7a8@24.8.96.48>
Message-ID: <3B683D26.5ECEFAF2@ActiveState.com>

Dan Sugalski wrote:
> 
> At 11:16 AM 8/1/2001 -0400, Jeremy Hylton wrote:
> >This discussion of variables also raises a question for the Perl guys:
> >Is there a Perl language reference ala the Python reference manual?
> 
> Not for perl 6, nope. (Larry's working on that) The Camel and the
> implementation are pretty much what we've got for perl 5.

For Perl5, don't forget perlguts and perlguts illustrated, which may be
useful to Pythonistas:

http://aspn.activestate.com/ASPN/Products/ActivePerl/lib/Pod/perlguts.html
http://gisle.aas.no/perl/illguts/

--david (don't blame me!) ascher


From Samuele Pedroni <pedroni@inf.ethz.ch>  Wed Aug  1 20:03:33 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Wed, 1 Aug 2001 21:03:33 +0200 (MET DST)
Subject: [Python-Dev] Parrot -- should life imitate satire?
Message-ID: <200108011903.VAA06122@core.inf.ethz.ch>

Hi.

> 
> The point is to put the commonly called things in the vtable in a way that 
> you can avoid as much conditional code as possible, while less common 
> things get dealt with in ways you'd generally expect. (Dynamic lookups with 
> caching and suchlike things)

If I'm right, you're designing a object-based VM. I don't how
typical is OO programming in Perl, but in Python that plays a central role,
if your long run goal is to compile to native code you should have
a "hard-wired" concept of classes and them like, because an operation
like getting the class of an instance should be as direct and fast as possible
if you want to use any of the "nice" optimization for a VM for a OO dynamic
language:
  inline polymorphic caches
  customization
  or if/when possible: direct inlining.
  etc.
  
If any of the high-level OO system can not be used, you have to choose
a lower level one and map these to it, 
a vtable bytecode mapping model is too underspecified.

regards, Samuele Pedroni.
  



From mclay@nist.gov  Wed Aug  1 09:26:58 2001
From: mclay@nist.gov (Michael McLay)
Date: Wed, 1 Aug 2001 04:26:58 -0400
Subject: [Python-Dev] Revised decimal type PEP
In-Reply-To: <200108011455.KAA20439@cj20424-a.reston1.va.home.com>
References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073121505701.02964@fermi.eeel.nist.gov> <200108011455.KAA20439@cj20424-a.reston1.va.home.com>
Message-ID: <01080104265804.02964@fermi.eeel.nist.gov>

On Wednesday 01 August 2001 10:55 am, Guido van Rossum wrote:
>
> I believe it is typically global per thread, but that's up to the
> langage binding.  A Java binding for Cowlishaw's decimals apparently
> requires passing in a context as a third argument on each operation.

I'm trying to understand how a decimal number context would work.  Is the=
=20
context a variable and/or flag that defines the rounding rules and precis=
ion=20
of a number when it is used in a calculation?  How is it associated with =
a=20
number or a calculation?  The "global per thread" description seems to=20
associate the context with threads.  Can the context be altered inside th=
e=20
thread?  Is it possible to change the context at different levels in a=20
stackframe?=20

I would assume there is a default context will be used until the context =
is=20
changed.  If this is the case I would expect a default context would be=20
defined at startup.

Would it make sense to have a simple decimal type with no features that c=
an=20
be modified (a fixed context)? This simple type could be extended by deri=
ving=20
a new numerical type from the base decimal type.  This base decimal type=20
would be targeted at the newbie user.  It would have no surprises.  It wo=
uld=20
have a default precision of 18 and the rules for rounding would emulate t=
he=20
typical hand held calculator. Accountants who need special rounding rules=
=20
would use a derived type that allowed the default rules to be overridden.

It would be possible to round numbers of the simple based type, but it wo=
uld=20
be an explicit step to remove insignificant digits.  An accounting decima=
l=20
type might automatically round calculations to the smallest denomination.=
 =20
For instance, an accounting context might have automatically managed the=20
final rounding in the following calculation:

p>>> quantity =3D 6
>>> tax =3D .06
>>> price =3D 2.99
>>> total =3D price * quantity * (1 + tax)
>>> total
19.0164
>>> round(total,2)
19.02
>>>

> > M.-A. Lemburg"  suggested looking at the SQL specification for Decima=
l
> > datatypes.  A decimal type is also defined as a type in XML Schema.=20
> > Since this is an XML datatype there isn't a definition for how these
> > numbers are created.
>
> Do these say anything about semantics under numeric operations?  That
> would seem to be outside the realm of XML and possibly even outside
> SQL.  So I'm not sure how these help.

You are correct that it doesn't deal with numeric operations.  It does de=
fine=20
a minimum precision requirement.  I am only referencing it here because i=
t is=20
another instance where having a decimal type in Python would be useful an=
d=20
because they have set a minimum requirement.  Setting this minmum as a=20
default behavior would probably make newbies comfortable with the languag=
e. =20

> >   NOTE: All =B7minimally conforming=B7 processors =B7must=B7 support =
decimal
> > numbers with a minimum of 18 decimal digits (i.e., with a =B7totalDig=
its=B7
> > of 18). However, =B7minimally conforming=B7 processors =B7may=B7 set =
an
> > application-defined limit on the maximum number of decimal digits the=
y
> > are prepared to support, in which case that application-defined maxim=
um
> > number =B7must=B7 be clearly documented.
> > 	- http://www.w3.org/TR/xmlschema-2/#decimal


From guido@zope.com  Wed Aug  1 22:44:37 2001
From: guido@zope.com (Guido van Rossum)
Date: Wed, 01 Aug 2001 17:44:37 -0400
Subject: [Python-Dev] Revised decimal type PEP
In-Reply-To: Your message of "Wed, 01 Aug 2001 04:26:58 EDT."
 <01080104265804.02964@fermi.eeel.nist.gov>
References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073121505701.02964@fermi.eeel.nist.gov> <200108011455.KAA20439@cj20424-a.reston1.va.home.com>
 <01080104265804.02964@fermi.eeel.nist.gov>
Message-ID: <200108012144.RAA27076@cj20424-a.reston1.va.home.com>

> I'm trying to understand how a decimal number context would work.  Is the 
> context a variable and/or flag that defines the rounding rules and precision 
> of a number when it is used in a calculation?

Yes -- I believe advanced HP calculators have such functionality.  So
does IEEE 754 for binary floating point.

> How is it associated with a 
> number or a calculation?

As I wrote, this depends on the language binding.  I believe it's
associated with a calculation, not with a number.

> The "global per thread" description seems to 
> associate the context with threads.  Can the context be altered inside the 
> thread?  Is it possible to change the context at different levels in a 
> stackframe? 

Again, this would depend on the language.  I believe typically you can
change it but it's not stacked (you'd have to do that yourself).

> I would assume there is a default context will be used until the context is 
> changed.  If this is the case I would expect a default context would be 
> defined at startup.

Me too.

> Would it make sense to have a simple decimal type with no features that can 
> be modified (a fixed context)?

That's like providing IEEE 754 floating point without the controls.
That's what C does, but at times it's painful.  For MOST users this
would be fine, but for advanced use you need the control, and claiming
"IEEE 754 std" is unfair without the controls.

> This simple type could be extended by deriving 
> a new numerical type from the base decimal type.  This base decimal type 
> would be targeted at the newbie user.  It would have no surprises.

It's hard to avoid surprises of the kind (1/3)*3 != 1.  My calculator
gives 0.99999999, but it's still a surprise.  On the other hand for
someone who thinks they know how a calculator does it, returning 1
would be the surprise!

What kind of surprises do you specifically want to avoid?

> It would 
> have a default precision of 18 and the rules for rounding would emulate the 
> typical hand held calculator. Accountants who need special rounding rules 
> would use a derived type that allowed the default rules to be overridden.
> 
> It would be possible to round numbers of the simple based type, but it would 
> be an explicit step to remove insignificant digits.  An accounting decimal 
> type might automatically round calculations to the smallest denomination.  
> For instance, an accounting context might have automatically managed the 
> final rounding in the following calculation:
> 
> p>>> quantity = 6
> >>> tax = .06
> >>> price = 2.99
> >>> total = price * quantity * (1 + tax)
> >>> total
> 19.0164
> >>> round(total,2)
> 19.02
> >>>

Looks good to me.  This would be a nice goal to strive for.

> > > M.-A. Lemburg" suggested looking at the SQL specification for
> > > Decimal datatypes.  A decimal type is also defined as a type in
> > > XML Schema.  Since this is an XML datatype there isn't a
> > > definition for how these numbers are created.
> >
> > Do these say anything about semantics under numeric operations?
> > That would seem to be outside the realm of XML and possibly even
> > outside SQL.  So I'm not sure how these help.
> 
> You are correct that it doesn't deal with numeric operations.  It
> does define a minimum precision requirement.  I am only referencing
> it here because it is another instance where having a decimal type
> in Python would be useful and because they have set a minimum
> requirement.  Setting this minmum as a default behavior would
> probably make newbies comfortable with the language.

Good point.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@zope.com  Wed Aug  1 22:49:19 2001
From: guido@zope.com (Guido van Rossum)
Date: Wed, 01 Aug 2001 17:49:19 -0400
Subject: [Python-Dev] Revised decimal type PEP
In-Reply-To: Your message of "Wed, 01 Aug 2001 17:44:37 EDT."
 <200108012144.RAA27076@cj20424-a.reston1.va.home.com>
References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073121505701.02964@fermi.eeel.nist.gov> <200108011455.KAA20439@cj20424-a.reston1.va.home.com> <01080104265804.02964@fermi.eeel.nist.gov>
 <200108012144.RAA27076@cj20424-a.reston1.va.home.com>
Message-ID: <200108012149.RAA27144@cj20424-a.reston1.va.home.com>

> > I'm trying to understand how a decimal number context would work.
> > Is the context a variable and/or flag that defines the rounding
> > rules and precision of a number when it is used in a calculation?
> 
> Yes -- I believe advanced HP calculators have such functionality.  So
> does IEEE 754 for binary floating point.

I forgot: also which exceptional conditions raise exceptions, and a
bunch of (resettable?) state flags that tell you whether the last
calculation did certain things like lose precision or overflow etc.

I know nothing about this stuff, but Tim talks about it a lot and I
seem to learn by osmosis. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From greg@cosc.canterbury.ac.nz  Thu Aug  2 00:09:12 2001
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 02 Aug 2001 11:09:12 +1200 (NZST)
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <5.1.0.14.0.20010801000522.0238d970@24.8.96.48>
Message-ID: <200108012309.LAA00931@s454.cosc.canterbury.ac.nz>

Dan Sugalski <dan@sidhe.org>:

> In this case, the vtable interface doesn't cover general method
> calls ... The vtable interface, at the moment, covers the 
> operators you'd expect to be able to override--assignment, simple math, 
> most string operations, conversion duties, and suchlike things.

It sounds like a vtable is very similar to a Python TypeObject,
the main difference being that TypeObjects are first-class
objects.

TypeObjects are immutable, though - you can't use Python code to
change one of it's method slots. Whereas, if I understand you
correctly, you can use Perl code to change a method in a vtable?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From tim.one@home.com  Thu Aug  2 05:28:47 2001
From: tim.one@home.com (Tim Peters)
Date: Thu, 2 Aug 2001 00:28:47 -0400
Subject: [Python-Dev] descr-branch is dead; long live the trunk
Message-ID: <LNBBLJKPBEHFEDALKOLCEEPFLCAA.tim.one@home.com>

I completed merging descr-branch back into the trunk.  Checkins should no
longer occur on descr-branch.

The trunk was tagged before and after the merge:

before tag:  date2001-08-01
after  tag:  after-descr-branch-merge

After a few quiet months have passed, I intend to get rid of the assorted
dateYYYY-MM-DD tags added for repeated merging of the trunk back into the
branch.

One problem:  test_weakref.py fails now.  It didn't fail in the trunk before
the merge, or in descr-branch.  All the weakref files (_weakref.c,
weakref.py, test_weakref.py) are the same in both trunk and branch, so
there's no obvious accounting for this.  Most of test_weakref works fine:

C:\Code\python\dist\src\PCbuild>python ../lib/test/test_weakref.py
test_basic_callback (__main__.ReferencesTestCase) ... FAIL
test_basic_proxy (__main__.ReferencesTestCase) ... ok
test_basic_ref (__main__.ReferencesTestCase) ... ok
test_callable_proxy (__main__.ReferencesTestCase) ... ok
test_getweakrefcount (__main__.ReferencesTestCase) ... ok
test_getweakrefs (__main__.ReferencesTestCase) ... ok
test_multiple_callbacks (__main__.ReferencesTestCase) ... ok
Make sure all references are invalidated before callbacks ... ok
test_proxy_ref (__main__.ReferencesTestCase) ... ok
test_proxy_reuse (__main__.ReferencesTestCase) ... ok
test_ref_reuse (__main__.ReferencesTestCase) ... ok

======================================================================
FAIL: test_basic_callback (__main__.ReferencesTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_weakref.py", line 52, in test_basic_callback
    self.check_basic_callback(create_unbound_method)
  File "../lib/test/test_weakref.py", line 114, in check_basic_callback
    "callback did not properly set 'cbcalled'")
  File "C:\CODE\PYTHON\DIST\SRC\lib\unittest.py", line 249, in failUnless
    if not expr: raise self.failureException, msg
AssertionError: callback did not properly set 'cbcalled'
----------------------------------------------------------------------
Ran 11 tests in 0.000s

FAILED (failures=1)
Traceback (most recent call last):
  File "../lib/test/test_weakref.py", line 410, in ?
    run_unittest(ReferencesTestCase)
  File "../lib/test\test_support.py", line 146, in run_unittest
    raise TestFailed(str(err[1]))
test_support.TestFailed: callback did not properly set 'cbcalled'

C:\Code\python\dist\src\PCbuild>

Fred, would you take a look at that and see whether anything punches you in
the eye?  CVS made the merge back into the trunk much harder than it should
have been, and I don't to want hold things up for what's *probably* a
shallow problem I'm just too fried now to pick up on.

gamblingly y'rs  - tim



From tim.one@home.com  Thu Aug  2 05:46:51 2001
From: tim.one@home.com (Tim Peters)
Date: Thu, 2 Aug 2001 00:46:51 -0400
Subject: [Python-Dev] descr-branch is dead; long live the trunk
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEPFLCAA.tim.one@home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEPGLCAA.tim.one@home.com>

[Tim said]
> test_weakref.py fails now.

Never mind -- I flubbed resolving one of "the conflicts" CVS insisted
existed in classobject.c.  Fixed now.  All tests pass (but 24 are skipped on
Windows).

Still, keep your eyes open!  Lots of code just changed.



From tim.one@home.com  Thu Aug  2 06:16:55 2001
From: tim.one@home.com (Tim Peters)
Date: Thu, 2 Aug 2001 01:16:55 -0400
Subject: [Python-Dev] PyOS_snprintf() / PyOS_vsnprintf()
In-Reply-To: <3B66E96C.FBAB8A62@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEPILCAA.tim.one@home.com>

[MAL]
> How about always enabling our version in the alpha cycle and then
> reverting back to the native one in the betas ?

If we have to ship our own code for it anyway, why ever revert to the native
one?  Historically, all that gives us is boundless opportunities to catalog
and #ifdef our way around gratuitous discrepancies among platform C
libraries.

since-we-switched-to-our-own-getopt-everywhere-we-no-longer-get-
    getopt-bug-reports-anywhere-ly y'rs  - tim



From paulp@ActiveState.com  Thu Aug  2 17:51:02 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Thu, 02 Aug 2001 09:51:02 -0700
Subject: [Python-Dev] Parrot -- should life imitate satire?
References: <20010730051831.B1122@thyrsus.com>
 <20010731012432.G20676@xs4all.nl>
 <15206.8708.811000.468489@gargle.gargle.HOWL>
 <20010731115521.I20676@xs4all.nl>
 <3B668608.F68B5953@ActiveState.com> <15207.26855.65000.718694@gargle.gargle.HOWL>
Message-ID: <3B6984F6.1AF3853F@ActiveState.com>

Nathan Torkington wrote:
> 
> ...
> 
> I've been eyeing your 'import from the future' system of getting
> experimental features.  I'll be interested to see how this holds up.

Actually, Python doesn't usually have experimental features. "import
from future" is about introducing new features that will break code
"early". On the one hand, some people want the feature as soon as
possible. On the other, people are afraid of the code breakage (e.g.
adding a keyword in Python breaks code) and need time to adjust. The
future construct is the compromise.

Python has no real mechanism for experimental features other than
documentation, patches and so forth. I can only imagine one or two
experimental features in the history of the language, though.

>...
> We in the Perl camp have bandied around the idea of "use perl 5.004"
> as a way to get all the behaviour of the 5.004 release.  That's a
> noble idea, but forces us to keep a lot of crap around when one of the
> goals of active development is to be able to shed obstructive missteps
> in language evolution.

Yes, Guido has rejected that idea for Python as too much effort. Imagine
the regression testing problems alone!
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From mal@lemburg.com  Thu Aug  2 10:58:26 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 02 Aug 2001 11:58:26 +0200
Subject: [Python-Dev] PyOS_snprintf() / PyOS_vsnprintf()
References: <LNBBLJKPBEHFEDALKOLCIEPILCAA.tim.one@home.com>
Message-ID: <3B692442.E0C14931@lemburg.com>

Tim Peters wrote:
> 
> [MAL]
> > How about always enabling our version in the alpha cycle and then
> > reverting back to the native one in the betas ?
> 
> If we have to ship our own code for it anyway, why ever revert to the native
> one?  Historically, all that gives us is boundless opportunities to catalog
> and #ifdef our way around gratuitous discrepancies among platform C
> libraries.
> 
> since-we-switched-to-our-own-getopt-everywhere-we-no-longer-get-
>     getopt-bug-reports-anywhere-ly y'rs  - tim

Well, the emulation is not as robust and fast as the native 
implementation is, plus it cannot recover from a buffer overrun;
such an overrun is likely to cause a core dump due to sprintf()
writing into the heap -- still better than allowing a cracker to 
hack your system by exploiting a stack overrun, but not as good as a
true snprintf() implementation will do.

If we do get complaints about snprintf() failures on systems which
do have a native API, then we can still switch to the emulation
for all platforms.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/




From barry@zope.com  Thu Aug  2 13:20:08 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 2 Aug 2001 08:20:08 -0400
Subject: [Python-Dev] post descr-branch merge results
Message-ID: <15209.17784.192118.671324@anthem.wooz.org>

Everything looks pretty good from here (Linux, RH6.1-ish).  143 tests
passed, and the usual 17 skipped.

gcc spit out a bunch of warnings in socketmodule.c, all of which look
to be openssl related.  It's certainly possible that I have an old
openssl package laying around.  Other than that, it looks good.

Thanks Tim!

-Barry

gcc -g -O3 -Wall -Wstrict-prototypes -fPIC -DUSE_SSL=1 -I/usr/local/ssl/include -I. -I/home/barry/projects/python/./Include -I/usr/local/include -IInclude/ -c /home/barry/projects/python/Modules/socketmodule.c -o build/temp.linux-i686-2.2/socketmodule.o
In file included from /home/barry/projects/python/Modules/socketmodule.c:208:
/usr/local/ssl/include/openssl/rsa.h:219: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/rsa.h:220: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/rsa.h:221: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/rsa.h:223: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/rsa.h:224: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/rsa.h:227: warning: function declaration isn't a prototype
In file included from /usr/local/ssl/include/openssl/objects.h:960,
                 from /usr/local/ssl/include/openssl/evp.h:138,
                 from /usr/local/ssl/include/openssl/x509.h:67,
                 from /home/barry/projects/python/Modules/socketmodule.c:210:
/usr/local/ssl/include/openssl/asn1.h:377: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/asn1.h:378: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/asn1.h:379: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/asn1.h:380: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/asn1.h:753: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/asn1.h:755: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/asn1.h:806: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/asn1.h:806: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/asn1.h:809: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/asn1.h:809: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/asn1.h:810: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/asn1.h:817: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/asn1.h:817: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/asn1.h:818: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/asn1.h:854: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/asn1.h:856: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/asn1.h:858: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/asn1.h:859: warning: function declaration isn't a prototype
In file included from /usr/local/ssl/include/openssl/x509.h:67,
                 from /home/barry/projects/python/Modules/socketmodule.c:210:
/usr/local/ssl/include/openssl/evp.h:267: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/evp.h:268: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/evp.h:269: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/evp.h:271: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/evp.h:272: warning: function declaration isn't a prototype
In file included from /home/barry/projects/python/Modules/socketmodule.c:210:
/usr/local/ssl/include/openssl/x509.h:122: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/x509.h:123: warning: function declaration isn't a prototype
In file included from /usr/local/ssl/include/openssl/x509_vfy.h:69,
                 from /usr/local/ssl/include/openssl/x509.h:520,
                 from /home/barry/projects/python/Modules/socketmodule.c:210:
/usr/local/ssl/include/openssl/lhash.h:90: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/lhash.h:91: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/lhash.h:123: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/lhash.h:123: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/lhash.h:128: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/lhash.h:129: warning: function declaration isn't a prototype
In file included from /home/barry/projects/python/Modules/socketmodule.c:210:
/usr/local/ssl/include/openssl/x509.h:967: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/x509.h:970: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/x509.h:973: warning: function declaration isn't a prototype
In file included from /home/barry/projects/python/Modules/socketmodule.c:211:
/usr/local/ssl/include/openssl/pem.h:491: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/pem.h:493: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/pem.h:505: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/pem.h:507: warning: function declaration isn't a prototype
In file included from /usr/local/ssl/include/openssl/ssl.h:63,
                 from /home/barry/projects/python/Modules/socketmodule.c:212:
/usr/local/ssl/include/openssl/comp.h:15: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/comp.h:16: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/comp.h:17: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/comp.h:18: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/comp.h:19: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/comp.h:20: warning: function declaration isn't a prototype
In file included from /home/barry/projects/python/Modules/socketmodule.c:212:
/usr/local/ssl/include/openssl/ssl.h:224: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/ssl.h:225: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/ssl.h:226: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/ssl.h:442: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/ssl.h:445: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/ssl.h:467: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/ssl.h:574: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/ssl.h:653: warning: function declaration isn't a prototype
In file included from /home/barry/projects/python/Modules/socketmodule.c:212:
/usr/local/ssl/include/openssl/ssl.h:1032: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/ssl.h:1068: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/ssl.h:1070: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/ssl.h:1143: warning: function declaration isn't a prototype
/usr/local/ssl/include/openssl/ssl.h:1144: warning: function declaration isn't a prototype


From dan@sidhe.org  Thu Aug  2 15:02:42 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Thu, 02 Aug 2001 10:02:42 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <200108011903.VAA06122@core.inf.ethz.ch>
Message-ID: <5.1.0.14.0.20010802095102.021f9e68@24.8.96.48>

At 09:03 PM 8/1/2001 +0200, Samuele Pedroni wrote:
>Hi.
>
> >
> > The point is to put the commonly called things in the vtable in a way that
> > you can avoid as much conditional code as possible, while less common
> > things get dealt with in ways you'd generally expect. (Dynamic lookups 
> with
> > caching and suchlike things)
>
>If I'm right, you're designing a object-based VM.

More or less, yep, with a semi-generic two-arg dispatch for the binary 
methods. The vtable associated with each variable has a fixed set of 
required functions--add with 2nd arg an int, add with second arg a bigint, 
add with second arg a generic variable, for example. Basically all the 
things one would commonly do on data (add, subtract, multiply, divide, 
modulus, some string ops, get integer/real/string representation, copy, 
destroy, etc) have entries, with a catchall "generic method call" to 
handle, well, generic method calls.

It's all designed with high-speed dispatch and minimal conditional branch 
requirements in mind, as well as encapsulating all the "what do I do on 
data" functions. Basically the opcodes generally handle control flow, 
register/stack ops, and VM management, while actual operations on variables 
is left to the vtable methods attached to the variables.

I expect things to get mildly incestuous for speed reasons, but I'm OK with 
that. :)

>I don't how
>typical is OO programming in Perl, but in Python that plays a central role,
>if your long run goal is to compile to native code you should have
>a "hard-wired" concept of classes and them like,

Yep. There's a fast "get name of object's class" and "get pointer to 
object's class stash/variable table/methods/subs/<insert generic class 
terminology here>".

>  because an operation
>like getting the class of an instance should be as direct and fast as possible
>if you want to use any of the "nice" optimization for a VM for a OO dynamic
>language:
>   inline polymorphic caches

Yup. I'm leaning towards a class based cache for inherited methods and 
suchlike things, but I'm not sure that's sufficient if we're going to have 
objects whose inheritance tree is handled on a per-object basis.

>   customization

Details? I'm not sure what you're talking about here.

>   or if/when possible: direct inlining.

Yep, though the potential to mess with a class' methods at runtime tends to 
shoot this one down. (Perl's got similar issues with this, plus a few extra 
real nasty optimizer killers) I'm pondering a "check_if_changed" branch 
opcode that'll check a method/function/sub's definition to see if it's 
changed since the code was generated, and do the inline stuff if it hasn't. 
Though that still limits what you can do with code motion and other 
optimizations.

>If any of the high-level OO system can not be used, you have to choose
>a lower level one and map these to it,
>a vtable bytecode mapping model is too underspecified.

Oh, sure, saying "vtable bytecode mapping model" is an awful lot like 
saying "procedural language"--it's informative without actually being 
useful... :)

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From skip@pobox.com (Skip Montanaro)  Thu Aug  2 15:07:21 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Thu, 2 Aug 2001 09:07:21 -0500
Subject: [Python-Dev] test results
Message-ID: <15209.24217.765601.582838@beluga.mojam.com>

On my Mandrake 8.0 laptop I ran make test after grabbing the latest stuff
from cvs and wound up with

    145 tests OK.
    1 test failed: test_linuxaudiodev
    14 tests skipped: test_al test_cd test_cl test_dl test_gl
        test_imgfile test_largefile test_nis test_ntpath test_socketserver
        test_sunaudiodev test_unicode_file test_winreg test_winsound

The linuxaudiodev test never seems to work for me (though I do hear an
agonizing wail (is that supposed to be Homer Simpson?) from my laptop's
speakers when it's run).  The result is always:

    test test_linuxaudiodev crashed -- linuxaudiodev.error: (16, 'Device or
        resource busy') 

The socketserver test works when run manually.

Skip


From guido@zope.com  Thu Aug  2 15:20:44 2001
From: guido@zope.com (Guido van Rossum)
Date: Thu, 02 Aug 2001 10:20:44 -0400
Subject: [Python-Dev] testing
Message-ID: <200108021420.KAA32192@cj20424-a.reston1.va.home.com>

I have noticed PSU activity on this channel.  This is a test message
to check wh


From Samuele Pedroni <pedroni@inf.ethz.ch>  Thu Aug  2 15:32:34 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Thu, 2 Aug 2001 16:32:34 +0200 (MET DST)
Subject: [Python-Dev] Parrot -- should life imitate satire?
Message-ID: <200108021432.QAA28270@core.inf.ethz.ch>

Hi.

> > >
> > > The point is to put the commonly called things in the vtable in a way that
> > > you can avoid as much conditional code as possible, while less common
> > > things get dealt with in ways you'd generally expect. (Dynamic lookups 
> > with
> > > caching and suchlike things)
> >
> >If I'm right, you're designing a object-based VM.
> 
> More or less, yep, with a semi-generic two-arg dispatch for the binary 
> methods. The vtable associated with each variable has a fixed set of 
> required functions--add with 2nd arg an int, add with second arg a bigint, 
> add with second arg a generic variable, for example. Basically all the 
> things one would commonly do on data (add, subtract, multiply, divide, 
> modulus, some string ops, get integer/real/string representation, copy, 
> destroy, etc) have entries, with a catchall "generic method call" to 
> handle, well, generic method calls.
A question: when you say variable you mean variable (perl sense of that)
or object. It has already been pointed out but it's really confusing
from the point of view of python terminology. Will perl6 have only
variables which contain objects, truly references to objects like Python
or ...?
I should repeat that your explanation that assigment is somehow performed
calling a method on the "variable" is quiet a strange notion in general,
I can imagine having a slot called on assigment that eventually does a copy
or return just the object but assigment as an operation on the lvalue
is something very peculiar. I know that perl5 assignment is an operator
returning an lvalue, is this related?


> It's all designed with high-speed dispatch and minimal conditional branch 
> requirements in mind, as well as encapsulating all the "what do I do on 
> data" functions. Basically the opcodes generally handle control flow, 
> register/stack ops, and VM management, while actual operations on variables 
> is left to the vtable methods attached to the variables.
> 
> I expect things to get mildly incestuous for speed reasons, but I'm OK with 
> that. :)
> 
> >I don't how
> >typical is OO programming in Perl, but in Python that plays a central role,
> >if your long run goal is to compile to native code you should have
> >a "hard-wired" concept of classes and them like,
> 
> Yep. There's a fast "get name of object's class" and "get pointer to 
> object's class stash/variable table/methods/subs/<insert generic class 
> terminology here>".
> 
> >  because an operation
> >like getting the class of an instance should be as direct and fast as 
possible
> >if you want to use any of the "nice" optimization for a VM for a OO dynamic
> >language:
> >   inline polymorphic caches
> 
> Yup. I'm leaning towards a class based cache for inherited methods and 
> suchlike things, but I'm not sure that's sufficient if we're going to have 
> objects whose inheritance tree is handled on a per-object basis.
Make sense for the interpreted version, but for speed call-site
caches is far more promising when native compiling, but also more complicated. I 
imagine you already know e.g. the Self project literature.

> >   customization
> 
> Details? I'm not sure what you're talking about here.
Compiling different versions of the same method for example wrt to 
the receiver type in single dispatch case. See below

> >   or if/when possible: direct inlining.
> 
> Yep, though the potential to mess with a class' methods at runtime tends to 
> shoot this one down. (Perl's got similar issues with this, plus a few extra 
> real nasty optimizer killers) I'm pondering a "check_if_changed" branch 
> opcode that'll check a method/function/sub's definition to see if it's 
> changed since the code was generated, and do the inline stuff if it hasn't. 
> Though that still limits what you can do with code motion and other 
> optimizations.
> 
> >If any of the high-level OO system can not be used, you have to choose
> >a lower level one and map these to it,
> >a vtable bytecode mapping model is too underspecified.
> 
> Oh, sure, saying "vtable bytecode mapping model" is an awful lot like 
> saying "procedural language"--it's informative without actually being 
> useful... :)

I imagine you already know: especially when compiling to native code,
it is all a matter of optimizing for the common case, and be prepared
to be quiet slow otherwise, especially when dealing with dynamic changes,
both in Perl and Python there is no clear distiction from normal ops and
dynamic changes to methods... but in any case they are rare compared to the
rest. Threading adds complexity to the problem.

For the rest is just speed vs. memory, customization is a clear example
for that.

regards, Samuele Pedroni.



From nas@python.ca  Thu Aug  2 19:42:13 2001
From: nas@python.ca (Neil Schemenauer)
Date: Thu, 2 Aug 2001 11:42:13 -0700
Subject: [Python-Dev] test results
In-Reply-To: <15209.24217.765601.582838@beluga.mojam.com>; from skip@pobox.com on Thu, Aug 02, 2001 at 09:07:21AM -0500
References: <15209.24217.765601.582838@beluga.mojam.com>
Message-ID: <20010802114213.B17147@glacier.fnational.com>

Skip Montanaro wrote:
>     test test_linuxaudiodev crashed -- linuxaudiodev.error: (16, 'Device or
>         resource busy') 

Something has /dev/dsp or /dev/audio open.  Are you playing MP3s or
running something like esd?

  Neil


From gward@python.net  Thu Aug  2 15:15:00 2001
From: gward@python.net (Greg Ward)
Date: Thu, 2 Aug 2001 10:15:00 -0400
Subject: [Python-Dev] Good news about ExtensionClass and Python 2.2a1
In-Reply-To: <20010731135457.A15139@mems-exchange.org>; from nascheme@mems-exchange.org on Tue, Jul 31, 2001 at 01:54:57PM -0400
References: <20010731135457.A15139@mems-exchange.org>
Message-ID: <20010802101500.A4616@gerg.ca>

On 31 July 2001, Neil Schemenauer said:
> After a few tweaks to ExtensionClass and a few small fixes to some of
> our introspection code I'm happy to say that Python 2.2a1 passes our
> unit test suite.  This is significant since there are about 45000 lines
> of code (counted by "wc -l") tested by 3569 test cases.  Since we use
> ZODB ExtensionClasses are quite widely used.  Merging descr_branch into
> HEAD sounds like a good idea to me.  Well done Guido.

I hate to be a wet blanket, Neil, but we (the MEMS Exchange virtual fab
project) really have more like 18000 lines of formally tested code, 6700
lines of test code, and 10000 lines of UI code (which isn't formally
tested).  It's still a pretty big pile, though, and the fact that it can
be beaten into submission to work with 2.2 is great news.

(LoC figures are courtesy of Dinu Gherman's pycount, which distinguishes
code, docs, comments, and blank lines.  I'm just reporting the "code"
figure.)

> I'm going to spend a bit of time trying to rewrite the ZODB Persistent
> class as a type.

Excellent!

        Greg
-- 
Greg Ward - Unix geek                                   gward@python.net
http://starship.python.net/~gward/
Drive defensively -- buy a tank.


From skip@pobox.com (Skip Montanaro)  Thu Aug  2 20:29:12 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Thu, 2 Aug 2001 14:29:12 -0500
Subject: [Python-Dev] test results
In-Reply-To: <20010802114213.B17147@glacier.fnational.com>
References: <15209.24217.765601.582838@beluga.mojam.com>
 <20010802114213.B17147@glacier.fnational.com>
Message-ID: <15209.43528.631408.210878@beluga.mojam.com>

    >> test test_linuxaudiodev crashed -- linuxaudiodev.error: (16, 'Device
    >> or resource busy')

    Neil> Something has /dev/dsp or /dev/audio open.  Are you playing MP3s
    Neil> or running something like esd?

Nope, not that I'm aware of.  I poked around ps auxww output but didn't find
anything that looked like an audio server.  /dev/dsp and /dev/audio* are
owned by user "skip", group "audio".  My usual login is "skip" and I am in
group "audio".  The test succeeds outside of X but no sound is produced.

Skip



From fdrake@acm.org  Thu Aug  2 20:26:24 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Aug 2001 15:26:24 -0400 (EDT)
Subject: [Python-Dev] test results
In-Reply-To: <20010802114213.B17147@glacier.fnational.com>
References: <15209.24217.765601.582838@beluga.mojam.com>
 <20010802114213.B17147@glacier.fnational.com>
Message-ID: <15209.43360.641499.705190@cj42289-a.reston1.va.home.com>

Neil Schemenauer writes:
 > Something has /dev/dsp or /dev/audio open.  Are you playing MP3s or
 > running something like esd?

  This is exactly the problem I was having several months ago.  I
"solved" it by changing the permissions on the audio device:

cj42289-a(.../python/linux); ls -lF /dev/dsp
crw-rw----    1 root     audio     14,   3 Sep 27  2000 /dev/dsp
cj42289-a(.../python/linux); ./python ../Lib/test/regrtest.py test_linuxaudiodevtest_linuxaudiodev
test test_linuxaudiodev skipped --  (13, 'Permission denied', '/dev/dsp')
1 test skipped: test_linuxaudiodev

  As a side effect, the machine no longer makes that horrid screeching
sound.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From fdrake@acm.org  Thu Aug  2 20:29:13 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Aug 2001 15:29:13 -0400 (EDT)
Subject: [Python-Dev] Good news about ExtensionClass and Python 2.2a1
In-Reply-To: <20010802101500.A4616@gerg.ca>
References: <20010731135457.A15139@mems-exchange.org>
 <20010802101500.A4616@gerg.ca>
Message-ID: <15209.43529.943039.929906@cj42289-a.reston1.va.home.com>

Greg Ward writes:
 > (LoC figures are courtesy of Dinu Gherman's pycount, which distinguishes
 > code, docs, comments, and blank lines.  I'm just reporting the "code"

  You don't test your documentation?  Shame on you!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From cgw@alum.mit.edu  Thu Aug  2 20:48:20 2001
From: cgw@alum.mit.edu (Charles G Waldman)
Date: Thu, 2 Aug 2001 14:48:20 -0500
Subject: [Python-Dev] linuxaudiodev
In-Reply-To: <E15SOIH-0003w1-00@mail.python.org>
References: <E15SOIH-0003w1-00@mail.python.org>
Message-ID: <15209.44676.292017.955013@nyx.dyndns.org>

Skip Montanaro write:

 > The linuxaudiodev test never seems to work for me (though I do hear an
 > agonizing wail (is that supposed to be Homer Simpson?) from my laptop's
 > speakers when it's run).  The result is always:
 > 
 >     test test_linuxaudiodev crashed -- linuxaudiodev.error: (16, 'Device or
 >         resource busy') 
 > 
 > The socketserver test works when run manually.

Neil Schemenauer replied:

 > Something has /dev/dsp or /dev/audio open.  Are you playing MP3s or
 > running something like esd?

If another process was holding /dev/whatever open, it's hard to
understand how the agonizing wail was produced.

The code in linuxaudiodev does its initializations in an order which
is explicitly proscribed by the OSS programming guide.

I have a SWIG-based linux audio module that I use in my code, not
wanting anything to do with the (to me) highly suspect linuxaudiodev
module.  Should I dust off this code and contribute it?

Of course, if we really want to provide high-quality audio support on
Linux, we have to account for a number of different options - at
least, OSS and ALSA...  this is sort of a can-of-worms.  But I do
believe that the current linuxaudiodev module should be deprecated.





From guido@zope.com  Thu Aug  2 20:48:39 2001
From: guido@zope.com (Guido van Rossum)
Date: Thu, 02 Aug 2001 15:48:39 -0400
Subject: [Python-Dev] post descr-branch merge results
In-Reply-To: Your message of "Thu, 02 Aug 2001 08:20:08 EDT."
 <15209.17784.192118.671324@anthem.wooz.org>
References: <15209.17784.192118.671324@anthem.wooz.org>
Message-ID: <200108021948.f72JmeU11410@odiug.digicool.com>

> gcc spit out a bunch of warnings in socketmodule.c, all of which look
> to be openssl related.  It's certainly possible that I have an old
> openssl package laying around.

It does that to me too.  I think our gcc option requiring function
prototypes is too picky for the openssl header files.

Is there a way to teach setup.py to turn off this option when openssl
is detected?  It's already a separate case in setup.py.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From cgw@alum.mit.edu  Thu Aug  2 20:49:51 2001
From: cgw@alum.mit.edu (Charles G Waldman)
Date: Thu, 2 Aug 2001 14:49:51 -0500
Subject: [Python-Dev] linuxaudiodev, ps
Message-ID: <15209.44767.10607.967674@nyx.dyndns.org>

Skip - did you try "fuser /dev/dsp" ?



From guido@zope.com  Thu Aug  2 20:52:32 2001
From: guido@zope.com (Guido van Rossum)
Date: Thu, 02 Aug 2001 15:52:32 -0400
Subject: [Python-Dev] test results
In-Reply-To: Your message of "Thu, 02 Aug 2001 11:42:13 PDT."
 <20010802114213.B17147@glacier.fnational.com>
References: <15209.24217.765601.582838@beluga.mojam.com>
 <20010802114213.B17147@glacier.fnational.com>
Message-ID: <200108021952.f72JqW411441@odiug.digicool.com>

> Skip Montanaro wrote:
> >     test test_linuxaudiodev crashed -- linuxaudiodev.error: (16, 'Device or
> >         resource busy') 
> 
> Something has /dev/dsp or /dev/audio open.  Are you playing MP3s or
> running something like esd?

Fred experiences this too with Mandrake on his Dell laptop, and before
someone hacked the extension module I experienced it on a Dell desktop
with Red Hat too -- no MP# players.  The wail that Skip hears is the
first second of "Nobody expects the Spanish inquisition".  I think
that different audio hardware has different Linux drivers that act
differently, and the extension module doesn't know how to properly
wait until enough buffer space is available with some drivers.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh@python.net  Thu Aug  2 20:55:40 2001
From: mwh@python.net (Michael Hudson)
Date: 02 Aug 2001 15:55:40 -0400
Subject: [Python-Dev] test results
In-Reply-To: Skip Montanaro's message of "Thu, 2 Aug 2001 09:07:21 -0500"
References: <15209.24217.765601.582838@beluga.mojam.com>
Message-ID: <2mvgk6jkxv.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

> The linuxaudiodev test never seems to work for me (though I do hear
> an agonizing wail (is that supposed to be Homer Simpson?)

Cardinal Biggles, I think.

> from my laptop's speakers when it's run).

Sorry I can't be of more use.

Cheers,
M.

-- 
  While preceding your entrance with a grenade is a good tactic in
  Quake, it can lead to problems if attempted at work.    -- C Hacking
               -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html


From guido@zope.com  Thu Aug  2 22:13:28 2001
From: guido@zope.com (Guido van Rossum)
Date: Thu, 02 Aug 2001 17:13:28 -0400
Subject: [Python-Dev] linuxaudiodev
In-Reply-To: Your message of "Thu, 02 Aug 2001 14:48:20 CDT."
 <15209.44676.292017.955013@nyx.dyndns.org>
References: <E15SOIH-0003w1-00@mail.python.org>
 <15209.44676.292017.955013@nyx.dyndns.org>
Message-ID: <200108022113.f72LDSi11865@odiug.digicool.com>

> The code in linuxaudiodev does its initializations in an order which
> is explicitly proscribed by the OSS programming guide.
> 
> I have a SWIG-based linux audio module that I use in my code, not
> wanting anything to do with the (to me) highly suspect linuxaudiodev
> module.  Should I dust off this code and contribute it?
> 
> Of course, if we really want to provide high-quality audio support on
> Linux, we have to account for a number of different options - at
> least, OSS and ALSA...  this is sort of a can-of-worms.  But I do
> believe that the current linuxaudiodev module should be deprecated.

Sure.  Do you want to take the lead here?  Nobody at PythonLabs has
any interest in this, and without someone putting a stake into the
ground, there's no point in even talking about it.  It sounds you know
what you're talking about, and you have CVS permission, so if you want
to do it, go ahead.

(For b/w compatibility we may have to keep the linuxaudio extension
around for a while, in a deprecated state.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Thu Aug  2 22:18:07 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Aug 2001 17:18:07 -0400 (EDT)
Subject: [Python-Dev] linuxaudiodev
In-Reply-To: <200108022113.f72LDSi11865@odiug.digicool.com>
References: <E15SOIH-0003w1-00@mail.python.org>
 <15209.44676.292017.955013@nyx.dyndns.org>
 <200108022113.f72LDSi11865@odiug.digicool.com>
Message-ID: <15209.50063.245244.530635@cj42289-a.reston1.va.home.com>

Guido van Rossum writes:
 > Sure.  Do you want to take the lead here?  Nobody at PythonLabs has
 > any interest in this, and without someone putting a stake into the
 > ground, there's no point in even talking about it.  It sounds you know
 > what you're talking about, and you have CVS permission, so if you want
 > to do it, go ahead.
 > 
 > (For b/w compatibility we may have to keep the linuxaudio extension
 > around for a while, in a deprecated state.)

  Would it make sense to include a Python implementation that wraps
aronud whatever low-level modules are needed?  Or to use the same API
and just replace the implementation?
  I don't know enough about the potential replacements, but it would
be nice to maintain the availability of the API; it's been available
for several versions now.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From dan@sidhe.org  Thu Aug  2 22:54:46 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Thu, 02 Aug 2001 17:54:46 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <200108021432.QAA28270@core.inf.ethz.ch>
Message-ID: <5.1.0.14.0.20010802165055.0221dd00@24.8.96.48>

At 04:32 PM 8/2/2001 +0200, Samuele Pedroni wrote:
>Hi.
>
> > > >
> > > > The point is to put the commonly called things in the vtable in a 
> way that
> > > > you can avoid as much conditional code as possible, while less common
> > > > things get dealt with in ways you'd generally expect. (Dynamic lookups
> > > with
> > > > caching and suchlike things)
> > >
> > >If I'm right, you're designing a object-based VM.
> >
> > More or less, yep, with a semi-generic two-arg dispatch for the binary
> > methods. The vtable associated with each variable has a fixed set of
> > required functions--add with 2nd arg an int, add with second arg a bigint,
> > add with second arg a generic variable, for example. Basically all the
> > things one would commonly do on data (add, subtract, multiply, divide,
> > modulus, some string ops, get integer/real/string representation, copy,
> > destroy, etc) have entries, with a catchall "generic method call" to
> > handle, well, generic method calls.
>A question: when you say variable you mean variable (perl sense of that)
>or object. It has already been pointed out but it's really confusing
>from the point of view of python terminology. Will perl6 have only
>variables which contain objects, truly references to objects like Python
>or ...?

Well, it sort of looks like:

   name--->variable--->value

The name is a symbol table entry that points to the variable. The variable 
is a data structure that has the vtable info, some pointer data, some 
flags, and general whatnots. (The docs refer to this as a PMC, or Parrot 
Magic Cookie, FWIW. Which isn't much) The value is the actual contents.

We also have to track properties on both a variable and a value basis. For 
example:

    declare c, b
      declare a is const = 0 but true
      b = a
      c = \a
    dereference(c) = 1

In this case, the variable represented by a has the property 'const', and a 
value of 0 which has the property 'true'. The assignment to b copies the 
value, and the properties on the value (true in this case), but not the 
properties on the variable, so b isn't const. c is a reference to the 
variable a represents, so the attempt to assign to the dereference of c 
will fail, since the variable c points to has the const property. We can't 
depend on the properties to be attached to the *name* a, since (in my crude 
attempt to use python-style blocking) the name a has gone out of scope by 
the time we dereference c, and so would any properties attached to the name 
as such.

I think the variable here would correspond to a Python object, but I'll 
need to go digging into Python semantics more to be sure. Currently there's 
not a lot planned around the actual symbol table entry, since it's not 
required for variable existence, and there's not much I can think of to 
attach to it anyway. (References mean we can't use it for much, really)

My bet, and this is just an off-hand thing, is that the parser would spit 
out bytecode tying the symbol table entry and the PMC structure more 
closely than it might for other languages. At least do a lot more symbolic 
fetch/store ops, if need be, which we might not.

>I should repeat that your explanation that assigment is somehow performed
>calling a method on the "variable" is quiet a strange notion in general,
>I can imagine having a slot called on assigment that eventually does a copy
>or return just the object but assigment as an operation on the lvalue
>is something very peculiar. I know that perl5 assignment is an operator
>returning an lvalue, is this related?

I don't think so, but I'm betting we're talking terminologically 
cross-wise. Assume, for example, the following pseudoish code:

    declare kitchen isa Thermostat
    kitchen = 20

In this case the kitchen variable is active, and assigning to it changes 
the temperature the thermostat in the kitchen is set to.

The magic properties aren't attached to the name per se, at least in 
perl--it's just a handy way to get the PMC that underlies the thing. I can 
get an indirect handle on the data that the name 'kitchen' refers to and 
update it that way, or if I have that handle the 'kitchen' name can go 
completely out of scope and the only way to refer to the object 
representing (and allowing control over) my kitchen temperature is via the 
handle I've got on what's now an anonymous object. (Or, if I'm feeling 
really wacky, have two or more names that look like separate things but 
actually all point to the same PMC structure)

I think it's the potential existence of references (or pointers, or 
whatever you want to call them) that necessitate the three-level split. It 
sounds like the things I'm calling 'name' and 'variable' are much more 
tightly bound in Python.

There are several reasons to call a PMC assignment operator, rather than 
just unconditionally slam a new PMC into the name slot:

1) In the kitchen example above, assigning 20 to kitchen doesn't suddenly 
make kitchen an integer variable--it needs to stay a Thermostat, and just 
do something because we assigned to it. (And I understand this might not be 
the way Python would do things, in which case Python variable vtables might 
just have an "overwrite myself with the incoming info" function, rather 
than allow it to be overridden)

2) Since we can't really assign properties like constant-ness to a name (in 
which case we could get a reference to a variable and bypass the properties 
on the name by referring to it by reference) that needs to be enforced in 
other ways. In this case with a custom assignment vtable function which 
pitches a fit if you try to alter the variable.

3) Some other thing or other I apparently can't think of. It's a really 
good reason, though.

> > >  because an operation
> > >like getting the class of an instance should be as direct and fast as
>possible
> > >if you want to use any of the "nice" optimization for a VM for a OO 
> dynamic
> > >language:
> > >   inline polymorphic caches
> >
> > Yup. I'm leaning towards a class based cache for inherited methods and
> > suchlike things, but I'm not sure that's sufficient if we're going to have
> > objects whose inheritance tree is handled on a per-object basis.
>Make sense for the interpreted version, but for speed call-site
>caches is far more promising when native compiling, but also more 
>complicated. I
>imagine you already know e.g. the Self project literature.

Self I've not dug into. Despite all the "I haven't see that" that keeps 
popping up in my conversation, I *have* done a lot of research. (The pile 
of digested books at one point outweighed my kids. They've grown, but the 
new research pile's about to grow too...)

> > >   customization
> >
> > Details? I'm not sure what you're talking about here.
>Compiling different versions of the same method for example wrt to
>the receiver type in single dispatch case. See below

Ah, aggressive inlining of a sort. I think. Yep, on the list of things to 
do with the optimizer. (Not on the first version, probably, but certainly 
later)

> > >   or if/when possible: direct inlining.
> >
> > Yep, though the potential to mess with a class' methods at runtime 
> tends to
> > shoot this one down. (Perl's got similar issues with this, plus a few 
> extra
> > real nasty optimizer killers) I'm pondering a "check_if_changed" branch
> > opcode that'll check a method/function/sub's definition to see if it's
> > changed since the code was generated, and do the inline stuff if it 
> hasn't.
> > Though that still limits what you can do with code motion and other
> > optimizations.
> >
> > >If any of the high-level OO system can not be used, you have to choose
> > >a lower level one and map these to it,
> > >a vtable bytecode mapping model is too underspecified.
> >
> > Oh, sure, saying "vtable bytecode mapping model" is an awful lot like
> > saying "procedural language"--it's informative without actually being
> > useful... :)
>
>I imagine you already know: especially when compiling to native code,
>it is all a matter of optimizing for the common case, and be prepared
>to be quiet slow otherwise, especially when dealing with dynamic changes,
>both in Perl and Python there is no clear distiction from normal ops and
>dynamic changes to methods... but in any case they are rare compared to the
>rest.

Yup. I am profoundly tempted, and I think it will go in, to have a "stable 
code" switch that can be given to the optimizer which guarantees that, once 
the compilation phase is done and we finally get to run for real, code will 
not change. And then apply all the advanced optimization techniques at that 
point.

>Threading adds complexity to the problem.

Nah, not too badly. A little synchronization issue, but that's reasonably 
trivial.

>For the rest is just speed vs. memory, customization is a clear example
>for that.

Sure. The one nice thing that both perl and python have going for them is 
the ultimate presence of all the source (or a high-enough level 
representation of it to feed to the optimizer) at run time. We can tell 
with more surety what side-effects various methods and functions in 
external code libraries will have, and we can get more aggressive (maybe) 
because of it. Unlike, say, C, where passing a pointer to a routine outside 
what the compiler can see will hobble the optimizer a whole lot in many 
cases...

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From dan@sidhe.org  Thu Aug  2 23:01:42 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Thu, 02 Aug 2001 18:01:42 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <20010801153607.G626@xs4all.nl>
References: <5.1.0.14.0.20010731234925.02298ae8@24.8.96.48>
Message-ID: <5.1.0.14.0.20010802180025.0232e130@24.8.96.48>

At 03:36 PM 8/1/2001 +0200, Thomas Wouters wrote:
> > [SRE] and the C extension interface are the two big Python things I've got
> > on my "must peer deeply into" list.
>
>I can't say much about SRE, but the C extension interface should be on
>anyone's "must peer deeply into" list, except that there isn't a real need
>to peer deeply into it :) Lets peer into it. I'm going to discuss the
>pre-2.2 state of affairs, mostly because I know jack shit about the
>2.2-type-dichotomy-healing :)

Ah, I see. (and thanks heaps) Well, I just rejigged my thinking a bit on 
what we need to export, then. My take's been both more agressively 
black-box about things, and much less OO. Too much C on the brain, I expect.

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From skip@pobox.com (Skip Montanaro)  Thu Aug  2 23:06:40 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Thu, 2 Aug 2001 17:06:40 -0500
Subject: [Python-Dev] Re: linuxaudiodev, ps
In-Reply-To: <15209.44767.10607.967674@nyx.dyndns.org>
References: <15209.44767.10607.967674@nyx.dyndns.org>
Message-ID: <15209.52976.938845.241434@beluga.mojam.com>

    Charles> Skip - did you try "fuser /dev/dsp" ?

Not until you suggested it.  Didn't even know it existed.  It displayed
nothing for either /dev/dsp or /dev/audio0:

    % fuser -a -v /dev/audio0 /dev/dsp

                         USER        PID ACCESS COMMAND
    /dev/audio0         
    No process references; use -v for the complete list
    /dev/dsp            
    No process references; use -v for the complete list

Skip



From guido@zope.com  Thu Aug  2 23:04:45 2001
From: guido@zope.com (Guido van Rossum)
Date: Thu, 02 Aug 2001 18:04:45 -0400
Subject: [Python-Dev] linuxaudiodev
In-Reply-To: Your message of "Thu, 02 Aug 2001 17:18:07 EDT."
 <15209.50063.245244.530635@cj42289-a.reston1.va.home.com>
References: <E15SOIH-0003w1-00@mail.python.org> <15209.44676.292017.955013@nyx.dyndns.org> <200108022113.f72LDSi11865@odiug.digicool.com>
 <15209.50063.245244.530635@cj42289-a.reston1.va.home.com>
Message-ID: <200108022204.SAA32634@cj20424-a.reston1.va.home.com>

>   I don't know enough about the potential replacements, but it would
> be nice to maintain the availability of the API; it's been available
> for several versions now.

Yup.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com (Skip Montanaro)  Thu Aug  2 23:11:23 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Thu, 2 Aug 2001 17:11:23 -0500
Subject: [Python-Dev] test results
In-Reply-To: <200108021952.f72JqW411441@odiug.digicool.com>
References: <15209.24217.765601.582838@beluga.mojam.com>
 <20010802114213.B17147@glacier.fnational.com>
 <200108021952.f72JqW411441@odiug.digicool.com>
Message-ID: <15209.53259.443303.220952@beluga.mojam.com>

    Guido> Fred experiences this too with Mandrake on his Dell laptop, and
    Guido> before someone hacked the extension module I experienced it on a
    Guido> Dell desktop with Red Hat too -- no MP# players.  

Nice to know I'm not alone.  I also have Mandrake 8.0 on a Dell laptop.

Since it's something the wizards know about, I won't pursue it further.

Skip



From skip@pobox.com (Skip Montanaro)  Thu Aug  2 23:17:31 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Thu, 2 Aug 2001 17:17:31 -0500
Subject: [Python-Dev] linuxaudiodev
In-Reply-To: <15209.44676.292017.955013@nyx.dyndns.org>
References: <E15SOIH-0003w1-00@mail.python.org>
 <15209.44676.292017.955013@nyx.dyndns.org>
Message-ID: <15209.53627.333118.300560@beluga.mojam.com>

    Charles> I have a SWIG-based linux audio module that I use in my code,
    Charles> not wanting anything to do with the (to me) highly suspect
    Charles> linuxaudiodev module.  Should I dust off this code and
    Charles> contribute it?

    Charles> Of course, if we really want to provide high-quality audio
    Charles> support on Linux, we have to account for a number of different
    Charles> options - at least, OSS and ALSA...  this is sort of a
    Charles> can-of-worms.  But I do believe that the current linuxaudiodev
    Charles> module should be deprecated.

Charles, is your SWIG-based thing still Linux-specific?  (I sort of suspect
it is.)  While having to support multiple options will be a pain, I think
they will each tend to be more platform-neutral than the current approach.

Skip


From akuchlin@mems-exchange.org  Thu Aug  2 23:16:37 2001
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Thu, 02 Aug 2001 18:16:37 -0400
Subject: [Python-Dev] Outdated Misc/cheatsheet
Message-ID: <E15SQlh-0007hk-00@ute.cnri.reston.va.us>

Misc/cheatsheet is way out of date (the 'access' keyword?).  I'd like
to replace it with the text version of Simon Brunning and Richard
Gruet's Python cheatsheet.   Issues:

1) Simon, do you grant permission to do that?  Should I ask Richard,
too?

2) The new file is significantly larger: 100K instead of just 22K.
   Is that OK?

3) The reference will need updating for 2.1 and 2.2 changes, but 
   slightly obsolete is better than waaay obsolete.  Simon, do you 
   want help updating it for 2.1/2.2?

--amk



From nas@python.ca  Thu Aug  2 23:35:26 2001
From: nas@python.ca (Neil Schemenauer)
Date: Thu, 2 Aug 2001 15:35:26 -0700
Subject: [Python-Dev] Good news about ExtensionClass and Python 2.2a1
In-Reply-To: <20010802101500.A4616@gerg.ca>; from gward@python.net on Thu, Aug 02, 2001 at 10:15:00AM -0400
References: <20010731135457.A15139@mems-exchange.org> <20010802101500.A4616@gerg.ca>
Message-ID: <20010802153526.A18271@glacier.fnational.com>

Greg Ward wrote:
> Neil:
> > I'm going to spend a bit of time trying to rewrite the ZODB Persistent
> > class as a type.
> 
> Excellent!

Unfortunately it's looks to be too much work for me to tackle right now.
cPersistent is subclass by lots of other ZODB extension classes.  My
idea of doing think piecemeal is not going to work.

  Neil


From barry@zope.com  Thu Aug  2 23:37:08 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 2 Aug 2001 18:37:08 -0400
Subject: [Python-Dev] linuxaudiodev
References: <E15SOIH-0003w1-00@mail.python.org>
 <15209.44676.292017.955013@nyx.dyndns.org>
 <200108022113.f72LDSi11865@odiug.digicool.com>
Message-ID: <15209.54804.552103.106351@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@zope.com> writes:

    GvR> Sure.  Do you want to take the lead here?  Nobody at
    GvR> PythonLabs has any interest in this, and without someone
    GvR> putting a stake into the ground, there's no point in even
    GvR> talking about it.  It sounds you know what you're talking
    GvR> about, and you have CVS permission, so if you want to do it,
    GvR> go ahead.

I have interest, but no time.  Although I'm sure Charles can do
everything necessary on his own, if he needs help from inside
Pythonlabs I'll volunteer.

-Barry


From guido@zope.com  Fri Aug  3 00:43:35 2001
From: guido@zope.com (Guido van Rossum)
Date: Thu, 02 Aug 2001 19:43:35 -0400
Subject: [Python-Dev] Good news about ExtensionClass and Python 2.2a1
In-Reply-To: Your message of "Thu, 02 Aug 2001 15:35:26 PDT."
 <20010802153526.A18271@glacier.fnational.com>
References: <20010731135457.A15139@mems-exchange.org> <20010802101500.A4616@gerg.ca>
 <20010802153526.A18271@glacier.fnational.com>
Message-ID: <200108022343.TAA00415@cj20424-a.reston1.va.home.com>

> > Neil:
> > > I'm going to spend a bit of time trying to rewrite the ZODB Persistent
> > > class as a type.
> 
> Unfortunately it's looks to be too much work for me to tackle right now.
> cPersistent is subclass by lots of other ZODB extension classes.  My
> idea of doing think piecemeal is not going to work.

Don't worry about it.  Eventually this will become a Zope
Corp. project.  (If you do get to this, I'd like to hear about it of
course. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)



From esr@thyrsus.com  Fri Aug  3 03:36:32 2001
From: esr@thyrsus.com (Eric S. Raymond)
Date: Thu, 2 Aug 2001 22:36:32 -0400
Subject: [Python-Dev] Proposed new module for the Python library
Message-ID: <20010802223632.A31724@thyrsus.com>

There's a kind of boilerplate code I've written way too often.  The most
recent time, I got fed up and wrote a candidate library module so I'd
never have to do it again.

Here's the header comment:

---------------------------------------------------------------------------
ccframe -- framework code for building compiler-like programs.

There is a common `compiler-like' pattern in Unix scripts which is useful
for translation utilities of all sorts.  A program following this pattern
behaves as a filter when no argument files are specified on the command
line, but otherwise transforms each file individually into a corresponding
output file.

This module provides framework and glue code to make such programs easy
to write.  You supply a function to massage the file data; depending
on which entry point you use, it can take input and output file pointers,
or it can take a string consisting of the entire file's data and return
a replacement, or it can take in succession strings consisting of each
of the file's lines and return a translated line for each.

Argument files are transformed in left to right order on the command line.
A filename consisting of a dash is interpreted as a directive to read from
standard input (this can be useful in pipelines).

Replacement of each file is atomic and doesn't occur until the
translation of that file has completed.  Any tempfiles are removed
automatically on any exception thrown by the translation function,
and the exception is then passed upwards.

The entry points return 0 on success, 1 to signal a failed file open, and
2 to signal a failed tempfile open or rename.  Error messages are emitted
to stderr.
---------------------------------------------------------------------------

Design comments?  Critiques?  Code on request.

I'm already considering throwing exceptions on open and rename errors
instead of complaining to stderr and returning an error status.  That
would be more Pythonic, though slightly less convenient in the
commonest cases.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

The two pillars of `political correctness' are, 
  a) willful ignorance, and
  b) a steadfast refusal to face the truth
	-- George MacDonald Fraser


From paulp@ActiveState.com  Fri Aug  3 04:13:26 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Thu, 02 Aug 2001 20:13:26 -0700
Subject: [Python-Dev] Proposed new module for the Python library
References: <20010802223632.A31724@thyrsus.com>
Message-ID: <3B6A16D6.1DB6C964@ActiveState.com>

Sounds "import fileinput-ish" to me. Are there fixes to fileinput that
will make it applicable to your problems?
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From esr@thyrsus.com  Fri Aug  3 04:30:45 2001
From: esr@thyrsus.com (Eric S. Raymond)
Date: Thu, 2 Aug 2001 23:30:45 -0400
Subject: [Python-Dev] Proposed new module for the Python library
In-Reply-To: <3B6A16D6.1DB6C964@ActiveState.com>; from paulp@ActiveState.com on Thu, Aug 02, 2001 at 08:13:26PM -0700
References: <20010802223632.A31724@thyrsus.com> <3B6A16D6.1DB6C964@ActiveState.com>
Message-ID: <20010802233045.A32400@thyrsus.com>

Paul Prescod <paulp@ActiveState.com>:
> Sounds "import fileinput-ish" to me. Are there fixes to fileinput that
> will make it applicable to your problems?

No.  The existing code in the fileinput module is intended to treat a
sequence of files as one continuous input source.  My module is
designed to iterate over files, transforming each one to a
corresponsing named output file. 

They're both input frameworks, but otherwise quite different.  The
fileinput model is appropriate for filters like cat(1) and tr(1).  The
ccframe model, on the other hand, is appropriate for compilers and 
file-format converters.
--
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

Our society won't be truly free until "None of the Above" is always an option.


From simon@netthink.co.uk  Thu Aug  2 23:01:09 2001
From: simon@netthink.co.uk (Simon Cozens)
Date: Thu, 2 Aug 2001 15:01:09 -0700
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: <20010731220627.G1708@thyrsus.com>
Message-ID: <20010802150109.B2033@netthink.co.uk>

On Tue, Jul 31, 2001 at 10:06:27PM -0400, Eric S. Raymond wrote:
> I strongly agree with Ping on this.  My experience is that CGI is an
> extremely important application category for Python, and one worth
> significant effort to serve.

I also strongly agree. CGI's one application that draws a lot of people
into programming, and many of them for the first time - you get a lot of
web designers and the like wanting to go a bit further, and it'd be
nice to get them into the good kind of snake (Python) instead of the
bad kind of snake. (ASP)

Damn, I'm a traitor to my own language.

Simon


From simon@netthink.co.uk  Thu Aug  2 18:33:15 2001
From: simon@netthink.co.uk (Simon Cozens)
Date: Thu, 2 Aug 2001 10:33:15 -0700
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <15206.8918.603000.448728@gargle.gargle.HOWL>
Message-ID: <20010802103315.I1667@netthink.co.uk>

On Mon, Jul 30, 2001 at 08:15:34PM -0700, Nathan Torkington wrote:
> The people designing and implementing perl6 have already agreed on a
> "do it clean, then make it faster" approach.  We can all see the
> problems with the current Perl internals, and have no desire to repeat
> the mistakes of the past.

May I just say, as one of those people, yea amen to that.

Simon


From simon@netthink.co.uk  Thu Aug  2 18:22:48 2001
From: simon@netthink.co.uk (Simon Cozens)
Date: Thu, 2 Aug 2001 10:22:48 -0700
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <3B683D26.5ECEFAF2@ActiveState.com>
Message-ID: <20010802102248.F1667@netthink.co.uk>

On Wed, Aug 01, 2001 at 10:32:22AM -0700, David Ascher wrote:
> For Perl5, don't forget perlguts and perlguts illustrated, which may be
> useful to Pythonistas:
> 
> http://aspn.activestate.com/ASPN/Products/ActivePerl/lib/Pod/perlguts.html
> http://gisle.aas.no/perl/illguts/

Let's kick this in again because people might have missed it:

    http://www.netthink.co.uk/downloads/

I spent a lot of time working on that, so use it. :)


From simon@netthink.co.uk  Thu Aug  2 18:20:25 2001
From: simon@netthink.co.uk (Simon Cozens)
Date: Thu, 2 Aug 2001 10:20:25 -0700
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <20010731225445.B626@xs4all.nl>
Message-ID: <20010802102025.E1667@netthink.co.uk>

On Tue, Jul 31, 2001 at 10:54:45PM +0200, Thomas Wouters wrote:
> Python objects all have vtables too (though they're structs, not arrays...
> I'm not sure why you'd use arrays

He means structs. :)

Simon


From simon@netthink.co.uk  Thu Aug  2 17:54:09 2001
From: simon@netthink.co.uk (Simon Cozens)
Date: Thu, 2 Aug 2001 09:54:09 -0700
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <20010801153607.G626@xs4all.nl>
Message-ID: <20010802095409.A1667@netthink.co.uk>

On Wed, Aug 01, 2001 at 03:36:07PM +0200, Thomas Wouters wrote:
> A Python object, any Python object, is essentially a struct like this:
> 
> stuct PyExample {
>     int ob_refcnt;                     /* reference count */
>     struct _typeobject *ob_type;       /* type of this object */
> 
>     /* type-specific data goes here */
> 
> }

This looks rather like a Perl SV (scalar value):
struct STRUCT_SV {      /* struct sv { */
    void*   sv_any;     /* pointer to something */
    U32     sv_refcnt;  /* how many references to us */
    U32     sv_flags;   /* what we are */
};

> For instance, the struct to hold a (normal, unbounded) Python integer is
> 
> typedef struct {
>     PyObject_HEAD
>     long ob_ival;
> } PyIntObject;

We use double indirection so that we can switch between types very quickly;
the sv_any pointer points to another structure or an integer/float.

> The struct _typeobject pointer is a pointer to a Type object, which also
> acts as a vtable.

That's sounding more like how we plan to do things for Perl 6.

> >>> def spam(x):
> ...     print "Hello Perl World!"
> >>> dis.dis(spam)
>           0 SET_LINENO               1
[...]

% ./perl -Dts -e 'print "Hello Python people!\n"'
    =>
(-e:1)  null
    =>
(-e:1)  const(PV("Hello Python people!\12"\0))
    =>  PV("Hello Python people!\12"\0)
(-e:1)  stringify

    [ That was constant folding, BTW. - SC]

EXECUTING...

    =>
(-e:0)  enter
    =>
(-e:0)  nextstate
    =>
(-e:1)  pushmark
    =>  *
(-e:1)  const(PV("Hello Python people!\12"\0))
    =>  *  PV("Hello Python people!\12"\0)
(-e:1)  print
Hello Python people!
    =>  SV_YES
(-e:1)  leave

perl -MO=Bytecode,-S -e 'print "Hello Python people!\n"'
will give you a complete disassembly of the bytecode, but
that's the version that can be serialised to disk, so it's
a lot more verbose than it need be - it sets up all the
opcodes, their flags, their sequence numbers, their connections
to other nodes in the opcode tree and so on. 

If you really want to get stuck in, read the internals tutorial
at http://www.netthink.co.uk/downloads/

Simon


From simon@netthink.co.uk  Thu Aug  2 18:06:06 2001
From: simon@netthink.co.uk (Simon Cozens)
Date: Thu, 2 Aug 2001 10:06:06 -0700
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <200108011903.VAA06122@core.inf.ethz.ch>
Message-ID: <20010802100606.B1667@netthink.co.uk>

On Wed, Aug 01, 2001 at 09:03:33PM +0200, Samuele Pedroni wrote:
> If I'm right, you're designing a object-based VM. I don't how
> typical is OO programming in Perl, but in Python that plays a central role,

In Perl, the OO-ness is central at the *implementation* level,
regardless of how OO the end user's code is. For instance, 
"$a = $b + $c" is essentially calling "a.store(b.get_integer() +
c.get_integer())" deep down, but the end user doesn't have to
know or care about that.

Simon


From simon@netthink.co.uk  Thu Aug  2 18:24:14 2001
From: simon@netthink.co.uk (Simon Cozens)
Date: Thu, 2 Aug 2001 10:24:14 -0700
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <20010731223259.A626@xs4all.nl>
Message-ID: <20010802102414.G1667@netthink.co.uk>

On Tue, Jul 31, 2001 at 10:32:59PM +0200, Thomas Wouters wrote:
> PUSH "1"
> PUSH "2"
> BINARY_ADD
> 
> for it's 
> 
> "1" . "2"

*cough*. Hell no! Addition and string concatenation would be
separate opcodes. (I think.)

Simon


From simon@netthink.co.uk  Thu Aug  2 22:54:18 2001
From: simon@netthink.co.uk (Simon Cozens)
Date: Thu, 2 Aug 2001 14:54:18 -0700
Subject: [Python-Dev] Python API version & optional features
In-Reply-To: <200107301427.f6UERW802779@odiug.digicool.com>
Message-ID: <20010802145418.A2033@netthink.co.uk>

On Mon, Jul 30, 2001 at 10:27:32AM -0400, Guido van Rossum wrote:
> I'm trying to convince myself that the magic number patch is okay, and
> here's what I come up with.  If someone builds a Python with a
> non-standard Unicode width and accidentally uses a directory full of
> extensions built for the standard Unicode width on his platform, he
> deserves a warning.

Shouldn't Python automatically Do The Right Thing in that case? That
would mean wrapping the UCS4 calls in a conversion layer, which isn't
hard. I mean, it's just a question of adding or taking away zeroes, right? :)

Simon


From simon@netthink.co.uk  Thu Aug  2 18:30:06 2001
From: simon@netthink.co.uk (Simon Cozens)
Date: Thu, 2 Aug 2001 10:30:06 -0700
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAELCLCAA.tim.one@home.com>
Message-ID: <20010802103006.H1667@netthink.co.uk>

On Wed, Aug 01, 2001 at 12:42:42AM -0400, Tim Peters wrote:
> 		case BINARY_DIVIDE:

case? Wowsers. Hey, Ruby does that too. We use function pointers,
FWIW.

Oh, and here's our divide, for comparison:

PP(pp_divide)
{
    dSP; dATARGET; tryAMAGICbin(div,opASSIGN);
    {
      dPOPPOPnnrl;
      NV value;
      if (right == 0.0)
        DIE(aTHX_ "Illegal division by zero");
      value = left / right;
      PUSHn( value );
      RETURN;
    }
}

See, this is why we need a new interpreter. :)

In slightly more pseudo code:

PP(pp_divide)
{
    get_the_stack_pointer;
    find_somewhere_to_put_result;
    check_if_overloaded;
    {
      NV right = SvNV(POP);
      NV left  = SvNV(POP);
      NV value;
      if (right == 0.0)
        DIE(aTHX_ "Illegal division by zero");
      value = left / right;
      PUSHn( value );
      RETURN;
    }
}


From greg@cosc.canterbury.ac.nz  Fri Aug  3 05:14:07 2001
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri, 03 Aug 2001 16:14:07 +1200 (NZST)
Subject: [Python-Dev] Proposed new module for the Python library
In-Reply-To: <20010802233045.A32400@thyrsus.com>
Message-ID: <200108030414.QAA01131@s454.cosc.canterbury.ac.nz>

"Eric S. Raymond" <esr@thyrsus.com>

> The existing code in the fileinput module is intended to treat a
> sequence of files as one continuous input source.  My module is
> designed to iterate over files, transforming each one to a
> corresponsing named output file. 

Still, they sound similar enough that the functionality
should perhaps be merged into one module, with options
for getting "concatenated" or "separate" input files.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From sdm7g@Virginia.EDU  Fri Aug  3 05:20:42 2001
From: sdm7g@Virginia.EDU (Steven D. Majewski)
Date: Fri, 3 Aug 2001 00:20:42 -0400 (EDT)
Subject: [Python-Dev] Proposed new module for the Python library
In-Reply-To: <20010802233045.A32400@thyrsus.com>
Message-ID: <Pine.NXT.4.21.0108022344310.835-100000@localhost.virginia.edu>


On Thu, 2 Aug 2001, Eric S. Raymond wrote:

> No.  The existing code in the fileinput module is intended to treat a
> sequence of files as one continuous input source.  My module is
> designed to iterate over files, transforming each one to a
> corresponsing named output file. 

 I did a FileUtil module for MacPython -- perhaps it was more 
complicated because it did all of the GUI also -- progress dialog
with info lines and a  cancel button, message boxes on errors, etc. 
You could drop a folder full of files on it and it would iterate 
thru them. 

 But I found it difficult to generalize sufficiently -- there was 
not only different file processing methods to be defined, there 
were different input/output filename transformation rules, different
message templates, different file selection rules, etc. Still -- it was
useful, but I think I kept fiddling with it almost every time I used it. 

 I think I started with a function that was passed a process callback
 function. 

 It ended up a class that could be extended for each particular 
   file processing job. 

 But I think if I did it again now, I'ld use some sort of generator. 
 There was some discussion of this: that the os.path.walk way of
 iterating with callbacks is really inside-out. 
 
 Nested generators might be a handier building block for this sort
  of job -- kind of the python equivalent of a unix pipeline command. 
 
-- Steve 



From esr@thyrsus.com  Fri Aug  3 06:45:29 2001
From: esr@thyrsus.com (Eric S. Raymond)
Date: Fri, 3 Aug 2001 01:45:29 -0400
Subject: [Python-Dev] Proposed new module for the Python library
In-Reply-To: <200108030414.QAA01131@s454.cosc.canterbury.ac.nz>; from greg@cosc.canterbury.ac.nz on Fri, Aug 03, 2001 at 04:14:07PM +1200
References: <20010802233045.A32400@thyrsus.com> <200108030414.QAA01131@s454.cosc.canterbury.ac.nz>
Message-ID: <20010803014528.A1743@thyrsus.com>

Greg Ewing <greg@cosc.canterbury.ac.nz>:
> > The existing code in the fileinput module is intended to treat a
> > sequence of files as one continuous input source.  My module is
> > designed to iterate over files, transforming each one to a
> > corresponsing named output file. 
> 
> Still, they sound similar enough that the functionality
> should perhaps be merged into one module, with options
> for getting "concatenated" or "separate" input files.

I'd be open to that -- it's why I said "existing" code :-).
It doesn't matter to me whether my functions live in the
fileinput module or in a sepate ccframe module.

I wrote the documentation page this morning.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

Faith may be defined briefly as an illogical belief in the occurrence
of the improbable...A man full of faith is simply one who has lost (or
never had) the capacity for clear and realistic thought. He is not a
mere ass: he is actually ill.	-- H. L. Mencken 


From thomas@xs4all.net  Fri Aug  3 07:11:44 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Fri, 3 Aug 2001 08:11:44 +0200
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <20010802102414.G1667@netthink.co.uk>
References: <20010802102414.G1667@netthink.co.uk>
Message-ID: <20010803081144.J626@xs4all.nl>

On Thu, Aug 02, 2001 at 10:24:14AM -0700, Simon Cozens wrote:
> On Tue, Jul 31, 2001 at 10:32:59PM +0200, Thomas Wouters wrote:
> > PUSH "1"
> > PUSH "2"
> > BINARY_ADD
> > 
> > for it's 
> > 
> > "1" . "2"

> *cough*. Hell no! Addition and string concatenation would be
> separate opcodes. (I think.)

Note that I later in the same thread change my mind and say that
string-concatenation should be a separate opcode :)

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From thomas@xs4all.net  Fri Aug  3 07:21:37 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Fri, 3 Aug 2001 08:21:37 +0200
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: <Pine.LNX.4.32.0107311813220.19357-100000@ziggy.localdomain.fake>
Message-ID: <20010803082137.K626@xs4all.nl>

On Tue, Jul 31, 2001 at 06:23:39PM -0700, Ka-Ping Yee wrote:
> On Tue, 31 Jul 2001, Guido van Rossum wrote:
> > Why not add this to cgi.py instead?

> Because that would seem to defeat most of the point.

> The point was to provide an instant, effortless improvement for all
> of the Python CGI scripts out there.  If programmers have to manually
> edit all of their CGI scripts to insert

> I don't want people to have to edit every single script.

I'm sorry, Ping, but I have to disagree with you there... You might not want
people to edit scripts, but I'm not sure howmany people want the
functionality of their CGI script(s) significantly changed without editing
their scripts. I know I wouldn't, even though nothing would break; I
wouldn't feel comfortable displaying the innards of some of the CGI scripts
to whoever manages to feed it faulty data. I would strongly suggest making
people add a single line, along the lines of

cgi.enable_debug()

or

cgi.install_reporter()

and doing the exception-hook replacement in there.

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From martin@loewis.home.cs.tu-berlin.de  Fri Aug  3 07:24:02 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Fri, 3 Aug 2001 08:24:02 +0200
Subject: [Python-Dev] Python API version & optional features
Message-ID: <200108030624.f736O2B00890@mira.informatik.hu-berlin.de>

> Shouldn't Python automatically Do The Right Thing in that case? That
> would mean wrapping the UCS4 calls in a conversion layer, which
> isn't hard. I mean, it's just a question of adding or taking away
> zeroes, right? :)

Given the smiley at the end of your message, I don't know whether you
meant your point seriously or not.

I believe it is impossible for Python to do what you seem to think as
the "Right Thing". E.g. modules that use PyUnicode_AS_UNICODE will
behave differently when sizeof(Py_UNICODE) changes. Since this is a
macro, you cannot change their behavior at run-time; they have
assumption on the size of Py_UNICODE compiled-in.

Regards,
Martin



From SBrunning@trisystems.co.uk  Fri Aug  3 08:53:39 2001
From: SBrunning@trisystems.co.uk (Simon Brunning)
Date: Fri, 3 Aug 2001 08:53:39 +0100
Subject: [Python-Dev] RE: Outdated Misc/cheatsheet
Message-ID: <31575A892FF6D1118F5800600846864D78BF67@intrepid>

> From:	Andrew Kuchling [SMTP:akuchlin@mems-exchange.org]
> Misc/cheatsheet is way out of date (the 'access' keyword?).  I'd like
> to replace it with the text version of Simon Brunning and Richard
> Gruet's Python cheatsheet.   Issues:
 
Andy,
Sounds like a good idea to me.

> 1) Simon, do you grant permission to do that?  Should I ask Richard,
> too?
 
Yes and yes. I've copied Richard in on this.
 
> 2) The new file is significantly larger: 100K instead of just 22K.
>    Is that OK?
> 
> 3) The reference will need updating for 2.1 and 2.2 changes, but 
>    slightly obsolete is better than waaay obsolete.  Simon, do you 
>    want help updating it for 2.1/2.2?
 
Thanks, Andy, but I'm on the Python 2.1 version - it's no more than a week
away. Honest.

I'll also do a 2.2 version as soon as I know what's going to be in it!
Iterators and generators seem to be in for sure, and after reading your
'What's New in Python 2.2' document I understand what they do, so they
aren't a problem. Ditto the other changes mentioned in your document.

But I'm not sure about the Type/Class unification stuff - I don't know if
it's going in, and I don't understand it, either! So here, I could use some
help.

Cheers,
Simon Brunning
TriSystems Ltd.
sbrunning@trisystems.co.uk





-----------------------------------------------------------------------
The information in this email is confidential and may be legally privileged.
It is intended solely for the addressee. Access to this email by anyone else
is unauthorised. If you are not the intended recipient, any disclosure,
copying, distribution, or any action taken or omitted to be taken in
reliance on it, is prohibited and may be unlawful. TriSystems Ltd. cannot
accept liability for statements made which are clearly the senders own.


From mal@lemburg.com  Fri Aug  3 10:13:56 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Aug 2001 11:13:56 +0200
Subject: [Python-Dev] Python API version & optional features
References: <200108030624.f736O2B00890@mira.informatik.hu-berlin.de>
Message-ID: <3B6A6B54.3BA008C9@lemburg.com>

"Martin v. Loewis" wrote:
> 
> > Shouldn't Python automatically Do The Right Thing in that case? That
> > would mean wrapping the UCS4 calls in a conversion layer, which
> > isn't hard. I mean, it's just a question of adding or taking away
> > zeroes, right? :)
> 
> Given the smiley at the end of your message, I don't know whether you
> meant your point seriously or not.
> 
> I believe it is impossible for Python to do what you seem to think as
> the "Right Thing". E.g. modules that use PyUnicode_AS_UNICODE will
> behave differently when sizeof(Py_UNICODE) changes. Since this is a
> macro, you cannot change their behavior at run-time; they have
> assumption on the size of Py_UNICODE compiled-in.

I've already checked in a patch which applies API name mangling
to the Unicode APIs. The linker will then generate error
messages which Python reports as ImportError in case there
should be a mismatch in the sizeof(Py_UNICODE).

The only thing which we require is that a Unicode aware
extension use at least one of the mangled APIs -- shouldn't
be much of a requirement, though :-)

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From jack@oratrix.nl  Fri Aug  3 11:44:51 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Fri, 03 Aug 2001 12:44:51 +0200
Subject: [Python-Dev] test_quopri is iso-latin-1 centric
Message-ID: <20010803104451.88356303181@snelboot.oratrix.nl>

I think test_quopri is too latin-1 centric. Alternatively, I'm mistaken, 
please enlighten me (is quopri supposed to convert to/from latin1, in stead of 
to/from local convention, as I expect?).

What happens is the following. The test_quopri.py source contains lots of 
special characters, for instance the spanish-upsidedown-exclamation (to 
mention the first one). It converts this with quopri and expects to see =A1.

This all works fine if your Python source is interpreted in latin-1, but it 
fails for other encoding.

For instance, on my Mac, Python source is in MacRoman encoding. CVS knows all 
about this, so it happily converts the latin-1-upsidedown-exclam to a 
macroman-upsidedown-exclam, and if I look at the source code I see the same 
glyph as I see on Unix.

However, the MacRoman encoding for this character is =D1, so test_quopri fails.

I'm surprised the test doesn't fail on Windows as well, or do Windows 
pythonistas generally work with source in latin1?
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | ++++ see http://www.xs4all.nl/~tank/ ++++




From loewis@informatik.hu-berlin.de  Fri Aug  3 12:43:01 2001
From: loewis@informatik.hu-berlin.de (Martin von Loewis)
Date: Fri, 3 Aug 2001 13:43:01 +0200
Subject: [Python-Dev] test_quopri is iso-latin-1 centric
Message-ID: <200108031143.f73Bh1708106@artagnan.informatik.hu-berlin.de>

> I think test_quopri is too latin-1 centric.

Strictly speaking, there is nothing latin-1 centric in test_quopri.py
whatsoever.

> For instance, on my Mac, Python source is in MacRoman encoding. CVS
> knows all about this, so it happily converts the
> latin-1-upsidedown-exclam to a macroman-upsidedown-exclam, and if I
> look at the source code I see the same glyph as I see on Unix.

This is the problem. Python source code is not in Latin-1; bytes
inside strings and comments are "as-is". So the CVS "binary" mode
would come closer as to how python files should be treated, although
you'd still would want to convert line-endings.

> I'm surprised the test doesn't fail on Windows as well, or do
> Windows pythonistas generally work with source in latin1?

Most of these people probably use code page 1252, which is identical
to latin-1 except in the range 0x80 to 0x9f.

For test_quopri.py, the best thing would be to replace the characters
outside range(128) to \x escaped ones, to avoid the problem with Mac
CVS (which really is the problem here - if you unpack Python from the
source distribution, the test should pass fine on your system).

Regards,
Martin


From just@letterror.com  Fri Aug  3 12:49:12 2001
From: just@letterror.com (Just van Rossum)
Date: Fri,  3 Aug 2001 13:49:12 +0200
Subject: [Python-Dev] test_quopri is iso-latin-1 centric
In-Reply-To: <200108031143.f73Bh1708106@artagnan.informatik.hu-berlin.de>
Message-ID: <20010803134915-r01010700-a5cd514b-0910-010c@10.0.0.9>

Martin von Loewis wrote:

> For test_quopri.py, the best thing would be to replace the characters
> outside range(128) to \x escaped ones, to avoid the problem with Mac
> CVS (which really is the problem here - if you unpack Python from the
> source distribution, the test should pass fine on your system).

It doesn't matter what the problem is: the Python sources that are supposed to
be portable should be 7-bit clean.

Just


From guido@zope.com  Fri Aug  3 13:33:40 2001
From: guido@zope.com (Guido van Rossum)
Date: Fri, 03 Aug 2001 08:33:40 -0400
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: Your message of "Thu, 02 Aug 2001 15:01:09 PDT."
 <20010802150109.B2033@netthink.co.uk>
References: <20010802150109.B2033@netthink.co.uk>
Message-ID: <200108031233.IAA01557@cj20424-a.reston1.va.home.com>

> I also strongly agree. CGI's one application that draws a lot of people
> into programming, and many of them for the first time - you get a lot of
> web designers and the like wanting to go a bit further, and it'd be
> nice to get them into the good kind of snake (Python) instead of the
> bad kind of snake. (ASP)

OK.  CGI is important.  But Ping's tracebackhook still shouldn't be
the default.  I'm willing to entertain a new command line option for
it though, so that CGI scripts can use

   #!/usr/bin/env python -C

as their first line.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jack@oratrix.nl  Fri Aug  3 13:38:57 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Fri, 03 Aug 2001 14:38:57 +0200
Subject: [Python-Dev] test_quopri is iso-latin-1 centric
In-Reply-To: Message by Martin von Loewis <loewis@informatik.hu-berlin.de> ,
 Fri, 3 Aug 2001 13:43:01 +0200 , <200108031143.f73Bh1708106@artagnan.informatik.hu-berlin.de>
Message-ID: <20010803123857.99202303181@snelboot.oratrix.nl>

> This is the problem. Python source code is not in Latin-1; bytes
> inside strings and comments are "as-is". So the CVS "binary" mode
> would come closer as to how python files should be treated, although
> you'd still would want to convert line-endings.

Is this an "official" policy? As such I disagree with it. Either all Python 
source should be 7-bit clean or the source should be converted to the local 
convention, even for strings and comments. But if it is an official policy I'd 
like to hear it (who wants to channel Guido here?), because there's heaps of 
MacPython stuff that'll have to be converted back (I did the forward 
conversion a couple of months ago).

As a point-in-case for local convention, you wouldn't want to be know as 
Martin von Lupsidedownquestionmarkwis by the chinese MacPython community or 
the Polish PythonWin users, would you?
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 




From guido@zope.com  Fri Aug  3 13:47:43 2001
From: guido@zope.com (Guido van Rossum)
Date: Fri, 03 Aug 2001 08:47:43 -0400
Subject: [Python-Dev] test_quopri is iso-latin-1 centric
In-Reply-To: Your message of "Fri, 03 Aug 2001 13:43:01 +0200."
 <200108031143.f73Bh1708106@artagnan.informatik.hu-berlin.de>
References: <200108031143.f73Bh1708106@artagnan.informatik.hu-berlin.de>
Message-ID: <200108031247.IAA01719@cj20424-a.reston1.va.home.com>

> > I think test_quopri is too latin-1 centric.
> 
> Strictly speaking, there is nothing latin-1 centric in test_quopri.py
> whatsoever.
> 
> > For instance, on my Mac, Python source is in MacRoman encoding. CVS
> > knows all about this, so it happily converts the
> > latin-1-upsidedown-exclam to a macroman-upsidedown-exclam, and if I
> > look at the source code I see the same glyph as I see on Unix.
> 
> This is the problem. Python source code is not in Latin-1; bytes
> inside strings and comments are "as-is". So the CVS "binary" mode
> would come closer as to how python files should be treated, although
> you'd still would want to convert line-endings.
> 
> > I'm surprised the test doesn't fail on Windows as well, or do
> > Windows pythonistas generally work with source in latin1?
> 
> Most of these people probably use code page 1252, which is identical
> to latin-1 except in the range 0x80 to 0x9f.
> 
> For test_quopri.py, the best thing would be to replace the characters
> outside range(128) to \x escaped ones, to avoid the problem with Mac
> CVS (which really is the problem here - if you unpack Python from the
> source distribution, the test should pass fine on your system).

Well, exactly because Python source code is not in Latin-1, the
test_quopri.py file is wrong IMO, because it uses non-ASCII
characters.  So please go ahead and change it into hex constants...

(This doesn't occur on Windows because, I'm assuming, the Windows CVS
client isn't as "smart" as the Mac CVS client.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From akuchlin@mems-exchange.org  Fri Aug  3 13:48:23 2001
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Fri, 3 Aug 2001 08:48:23 -0400
Subject: [Python-Dev] RE: Outdated Misc/cheatsheet
In-Reply-To: <31575A892FF6D1118F5800600846864D78BF67@intrepid>; from SBrunning@trisystems.co.uk on Fri, Aug 03, 2001 at 08:53:39AM +0100
References: <31575A892FF6D1118F5800600846864D78BF67@intrepid>
Message-ID: <20010803084823.A30133@ute.cnri.reston.va.us>

On Fri, Aug 03, 2001 at 08:53:39AM +0100, Simon Brunning wrote:
>Thanks, Andy, but I'm on the Python 2.1 version - it's no more than a week
>away. Honest.

Great!  I'll wait for it, then.

>But I'm not sure about the Type/Class unification stuff - I don't know if
>it's going in, and I don't understand it, either! So here, I could use some
>help.

Guido wrote up a description that's at
http://www.mirror.ac.uk/sites/ftp.python.org/pub/www.python.org/2.2/descrintro.html
.  I do plan to write a section on it, now that it seems closer to
inevitable, but I don't think it affects beginner or
intermediate-level programmers much.  Mostly the changes will affect
wizards who want to write ExtensionClass-like things.

And thanks!

--amk


From guido@zope.com  Fri Aug  3 13:49:11 2001
From: guido@zope.com (Guido van Rossum)
Date: Fri, 03 Aug 2001 08:49:11 -0400
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: Your message of "Fri, 03 Aug 2001 08:33:40 EDT."
 <200108031233.IAA01557@cj20424-a.reston1.va.home.com>
References: <20010802150109.B2033@netthink.co.uk>
 <200108031233.IAA01557@cj20424-a.reston1.va.home.com>
Message-ID: <200108031249.IAA01751@cj20424-a.reston1.va.home.com>

> I'm willing to entertain a new command line option for
> it though, so that CGI scripts can use
> 
>    #!/usr/bin/env python -C
> 
> as their first line.

But it's fine to require making a call to cgi.whatever() too, and
that's certainly a lot less work!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@zope.com  Fri Aug  3 13:50:35 2001
From: guido@zope.com (Guido van Rossum)
Date: Fri, 03 Aug 2001 08:50:35 -0400
Subject: [Python-Dev] test_quopri is iso-latin-1 centric
In-Reply-To: Your message of "Fri, 03 Aug 2001 14:38:57 +0200."
 <20010803123857.99202303181@snelboot.oratrix.nl>
References: <20010803123857.99202303181@snelboot.oratrix.nl>
Message-ID: <200108031250.IAA01771@cj20424-a.reston1.va.home.com>

> > This is the problem. Python source code is not in Latin-1; bytes
> > inside strings and comments are "as-is". So the CVS "binary" mode
> > would come closer as to how python files should be treated, although
> > you'd still would want to convert line-endings.
> 
> Is this an "official" policy? As such I disagree with it. Either all
> Python source should be 7-bit clean or the source should be
> converted to the local convention, even for strings and
> comments. But if it is an official policy I'd like to hear it (who
> wants to channel Guido here?), because there's heaps of MacPython
> stuff that'll have to be converted back (I did the forward
> conversion a couple of months ago).

Until we've picked a way to indicate the source encoding, it should
definitely be 7-bit clean.

> As a point-in-case for local convention, you wouldn't want to be
> know as Martin von Lupsidedownquestionmarkwis by the chinese
> MacPython community or the Polish PythonWin users, would you?

ROFL :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@zope.com  Fri Aug  3 13:54:17 2001
From: guido@zope.com (Guido van Rossum)
Date: Fri, 03 Aug 2001 08:54:17 -0400
Subject: [Python-Dev] WebDAV in python 2.2?
In-Reply-To: Your message of "Thu, 02 Aug 2001 22:18:44 EDT."
 <mailman.996828301.19910.clpa-moderators@python.org>
References: <mailman.996828301.19910.clpa-moderators@python.org>
Message-ID: <200108031254.IAA01818@cj20424-a.reston1.va.home.com>

> GvR also wondered if Python 2.2 should support DAV ("That's another
> open protocol that Python could easily support out of the box.") and
> /F agreed, but Greg Stein, the DAV expert and champion on python-dev,
> remained silent.

Greg, I now know where you were.  Since you're back, any comments?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gward@python.net  Fri Aug  3 02:49:44 2001
From: gward@python.net (Greg Ward)
Date: Thu, 2 Aug 2001 21:49:44 -0400
Subject: [Python-Dev] test results
In-Reply-To: <15209.24217.765601.582838@beluga.mojam.com>; from skip@pobox.com on Thu, Aug 02, 2001 at 09:07:21AM -0500
References: <15209.24217.765601.582838@beluga.mojam.com>
Message-ID: <20010802214944.A8810@gerg.ca>

On 02 August 2001, Skip Montanaro said:
> The linuxaudiodev test never seems to work for me (though I do hear an
> agonizing wail (is that supposed to be Homer Simpson?) from my laptop's
> speakers when it's run).  The result is always:

I am pleased-as-punch to report that test_linuxaudiodev *finally* works
for me, both in 2.2a1 and the CVS as of this afternoon (after Tim's big
merge).

Also, the rest of the test suite seems to have passed just fine for me
with the latest CVS; this is a Progeny Debian 1.0 system (pretty close
to Debian 2.2 stable).  ("Seems to have passed" because there's some
aborted work from a year or so ago where I was trying to fix the lexer's
handling of line termination.  Gave up in despair -- brain too small --
but I still have non-working tokenizer tests sitting in my CVS working
dir.  Sigh.)

        Greg
-- 
Greg Ward - programmer-at-large                         gward@python.net
http://starship.python.net/~gward/
Always look on the bright side of life.


From nas@python.ca  Fri Aug  3 14:53:16 2001
From: nas@python.ca (Neil Schemenauer)
Date: Fri, 3 Aug 2001 06:53:16 -0700
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <20010802103006.H1667@netthink.co.uk>; from simon@netthink.co.uk on Thu, Aug 02, 2001 at 10:30:06AM -0700
References: <LNBBLJKPBEHFEDALKOLCAELCLCAA.tim.one@home.com> <20010802103006.H1667@netthink.co.uk>
Message-ID: <20010803065316.A19638@glacier.fnational.com>

Simon Cozens wrote:
> On Wed, Aug 01, 2001 at 12:42:42AM -0400, Tim Peters wrote:
> > 		case BINARY_DIVIDE:
> 
> case? Wowsers. Hey, Ruby does that too. We use function pointers,
> FWIW.

People (including me) have tried using threaded code
(http://www.complang.tuwien.ac.at/forth/threaded-code.html) and have not
measured any significant advantage.  I suspect the opcodes to much
longer to evaluate than doing the dispatch.

> Oh, and here's our divide, for comparison:
> 
> PP(pp_divide)
> {
>     dSP; dATARGET; tryAMAGICbin(div,opASSIGN);
>     {
>       dPOPPOPnnrl;
>       NV value;
>       if (right == 0.0)
>         DIE(aTHX_ "Illegal division by zero");
>       value = left / right;
>       PUSHn( value );
>       RETURN;
>     }
> }

Does every Perl opcode do a function call?

  Neil


From nas@python.ca  Fri Aug  3 14:57:36 2001
From: nas@python.ca (Neil Schemenauer)
Date: Fri, 3 Aug 2001 06:57:36 -0700
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: <20010803082137.K626@xs4all.nl>; from thomas@xs4all.net on Fri, Aug 03, 2001 at 08:21:37AM +0200
References: <Pine.LNX.4.32.0107311813220.19357-100000@ziggy.localdomain.fake> <20010803082137.K626@xs4all.nl>
Message-ID: <20010803065736.B19638@glacier.fnational.com>

Thomas Wouters wrote:
> I'm sorry, Ping, but I have to disagree with you there... You might not want
> people to edit scripts, but I'm not sure howmany people want the
> functionality of their CGI script(s) significantly changed without editing
> their scripts. I know I wouldn't [...]

I wouldn't either.  Doing an import and/or a function is not too hard.
I'm -1 on magically enabling Ping's traceback thing (cool as it is).

  Neil


From thomas@xs4all.net  Fri Aug  3 15:17:04 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Fri, 3 Aug 2001 16:17:04 +0200
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <20010802103006.H1667@netthink.co.uk>
References: <20010802103006.H1667@netthink.co.uk>
Message-ID: <20010803161704.M626@xs4all.nl>

On Thu, Aug 02, 2001 at 10:30:06AM -0700, Simon Cozens wrote:
> On Wed, Aug 01, 2001 at 12:42:42AM -0400, Tim Peters wrote:
> > 		case BINARY_DIVIDE:

> case? Wowsers. Hey, Ruby does that too. We use function pointers,
> FWIW.

Well, we use *both* :) But function-pointers alone aren't enough, are they ?
Or are the bytecodes themselves the function pointers ? *shudder*. I guess
you do opcodes as indices into an array-of-function-pointers ? There was
talk about doing that for Python too, but I believe it ended up being not
significantly faster. Some people have seen bigger speedups just
reorganizing the case's in the Big Switch, but those same changes slowed
things down on other platforms.

> Oh, and here's our divide, for comparison:

> In slightly more pseudo code:

> PP(pp_divide)
> {
>     get_the_stack_pointer;
>     find_somewhere_to_put_result;
>     check_if_overloaded;
>     {
>       NV right = SvNV(POP);
>       NV left  = SvNV(POP);
>       NV value;
>       if (right == 0.0)
>         DIE(aTHX_ "Illegal division by zero");
>       value = left / right;
>       PUSHn( value );
>       RETURN;
>     }
> }

Right... Almost the same, except that the check_if_overloaded is a bigger
check for Python, and more often true (I suspect.) And, as was said before,
assignment isn't a value-filling operation, so instead of 'value = left /
right' we have 'PyObject *value = new_object(left / right)' (pseudocode,
again) and just push value onto the stack (to be stored into a variable by a
subsequent opcode, or just kept in the stack for the running expression.)

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From just@letterror.com  Fri Aug  3 15:44:47 2001
From: just@letterror.com (Just van Rossum)
Date: Fri,  3 Aug 2001 16:44:47 +0200
Subject: [Python-Dev] (Idea) subclassable string: path object?
Message-ID: <20010803164453-r01010700-1c882179-0910-010c@10.0.0.11>

Every once in a while I wished for an path object to manipulate file system
paths. Things like
   os.path.join(a, b, c, os.path.splitext(os.path.basename(p))[0] + ".ext")
quickly get frustrating (so of course I never write them like that ;-).

I thought of implementing a path object several times, but always stopped when I
realized (for the Nth time ;-) that you'd then have to do something like
   file = open(p.tostring())
whenever you want to *use* your pat. That doesn't help at all.

But: since strings are now subclassable (there are, aren't they?) this should no
longer be a problem!

Would it be a worthwile project to design and implement a path object for the
standard library?

Just


From skip@pobox.com (Skip Montanaro)  Fri Aug  3 16:00:26 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Fri, 3 Aug 2001 10:00:26 -0500
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: <20010803082137.K626@xs4all.nl>
References: <Pine.LNX.4.32.0107311813220.19357-100000@ziggy.localdomain.fake>
 <20010803082137.K626@xs4all.nl>
Message-ID: <15210.48266.673906.863831@beluga.mojam.com>

    >> I don't want people to have to edit every single script.

    Thomas> I'm sorry, Ping, but I have to disagree with you there... You
    Thomas> might not want people to edit scripts, but I'm not sure howmany
    Thomas> people want the functionality of their CGI script(s)
    Thomas> significantly changed without editing their scripts.

I have to agree.  Perhaps just as significant, what would happen to a CGI
script that already attempts to do something to direct error output to a
file or mail message the webmaster will see?

Most of my users wouldn't know a traceback (simpler or otherwise) from a
hole in the ground.  I think I have perhaps two users who would understand
that I was trying to create a more readable report.

Skip


From jack@oratrix.nl  Fri Aug  3 16:00:09 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Fri, 03 Aug 2001 17:00:09 +0200
Subject: [Python-Dev] (Idea) subclassable string: path object?
In-Reply-To: Message by Just van Rossum <just@letterror.com> ,
 Fri, 3 Aug 2001 16:44:47 +0200 , <20010803164453-r01010700-1c882179-0910-010c@10.0.0.11>
Message-ID: <20010803150014.BF9D7162E14@oratrix.oratrix.nl>

One problem I see is that what you'd really like is to overload + and
split and such on path objects. But this creates a problem if you then
pass this path object to something that expects old-fashioned strings:
if it wants to manipulate that path it will use string operations,
which suddenly have different semantics...

Recently, Just van Rossum <just@letterror.com> said:
> Every once in a while I wished for an path object to manipulate file system
> paths. Things like
>    os.path.join(a, b, c, os.path.splitext(os.path.basename(p))[0] + ".ext")
> quickly get frustrating (so of course I never write them like that ;-).
> 
> I thought of implementing a path object several times, but always stopped whe
> n I
> realized (for the Nth time ;-) that you'd then have to do something like
>    file = open(p.tostring())
> whenever you want to *use* your pat. That doesn't help at all.
> 
> But: since strings are now subclassable (there are, aren't they?) this should
>  no
> longer be a problem!
> 
> Would it be a worthwile project to design and implement a path object for the
> standard library?
> 
> Just
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> 
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 


From just@letterror.com  Fri Aug  3 16:11:09 2001
From: just@letterror.com (Just van Rossum)
Date: Fri,  3 Aug 2001 17:11:09 +0200
Subject: [Python-Dev] (Idea) subclassable string: path object?
In-Reply-To: <20010803150014.BF9D7162E14@oratrix.oratrix.nl>
Message-ID: <20010803171113-r01010700-2e14948d-0910-010c@10.0.0.11>

Jack Jansen wrote:

> One problem I see is that what you'd really like is to overload + and
> split and such on path objects.

Yup. Although it's unclear to me what would be the best behavior for +, because
it could mean several things:

1) p = Path(a_directory) + filename  # os.path.join(a_directory, filename)
2) p = Path(a_file) + ".ext"  # a_file + ".ext"

1) is very useful, but then 2) wouldn't work... So it's not all straightforward.

> But this creates a problem if you then
> pass this path object to something that expects old-fashioned strings:
> if it wants to manipulate that path it will use string operations,
> which suddenly have different semantics...

Yes, that's important to consider, too.

On the other hand: the os.path.* functions could be made aware of the path
object, similar to string.py, except it should fall back to the existing code
when a string gets passed.

Just


From Samuele Pedroni <pedroni@inf.ethz.ch>  Fri Aug  3 16:30:48 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Fri, 3 Aug 2001 17:30:48 +0200 (MET DST)
Subject: [Python-Dev] Parrot -- should life imitate satire?
Message-ID: <200108031530.RAA14515@core.inf.ethz.ch>

[Dan Sugalski]
> >A question: when you say variable you mean variable (perl sense of that)
> >or object. It has already been pointed out but it's really confusing
> >from the point of view of python terminology. Will perl6 have only
> >variables which contain objects, truly references to objects like Python
> >or ...?
> 
> Well, it sort of looks like:
> 
>    name--->variable--->value
> 
Ok, I got your explanation. I see that perl6 will still have a rather 
convoluted model and quiet different from Python...

> I think the variable here would correspond to a Python object
Possible but not in a natural way.


> I think it's the potential existence of references (or pointers, or 
> whatever you want to call them) that necessitate the three-level split.
Makes perfectly sense.


> sounds like the things I'm calling 'name' and 'variable' are much more 
> tightly bound in Python.
No,

a name/variable is just slot in a namespace/ symbol table,

that contains a pointer to a value/object which contains data and also 
points to the type-specific vtable.

Quiet different.

Do your values come with a vtable that then is "copied" in the variable?
Your vtable are for things like scalar, vs array vs. hash.
Do they possibly change when the value change (e.g. string to integer
 for a scalar var)?


> the way Python would do things, in which case Python variable vtables might 
> just have an "overwrite myself with the incoming info" function, rather 
> than allow it to be overridden)
No, unless you have a generic Python PMC which mirrors/ or uses a vtable
that goes together with the value (in the sense of your scheme).

Otherwise I think python assigment would be more some flavor
of reference (perl sense) assignment.

If you map Python built-in data object to PMC then assignment
is a just a matter of namespace manipulation.

For the momement I don't really see a natural way of mapping python
semantics over your three-level scheme, maybe is just the lack of details.
Especially I see a lot of problem if you want a mapping that can also
allow interoperability beetween Python and Perl code.

> Yup. I am profoundly tempted, and I think it will go in, to have a "stable 
> code" switch that can be given to the optimizer which guarantees that, once 
> the compilation phase is done and we finally get to run for real, code will 
> not change. And then apply all the advanced optimization techniques at that 
> point.
> 
I don't really get this. In python the kind of dynamic changes you can
do are reloading code, changing the base classes of a class, adding/substituting
methods in a class or instance. When an application really need one of this
techniques with your philosophy it will be treated as 2nd-class citizen.
What I was referring were de-optimization techniques (yes they are a mess
to deal with for the compiler writer).
Reading about native compilation of dynamic languages I have been conviced
that in general performance is a matter of implemention and it better
to keep it transparent to the user, having constructs that should
be avoided not for design reasons but because the compiler uses
some tricks and not others is a bad thing, for a dynamic language IMHO.
And in any case you still have the problem with polymorphic calls
and to deal with them - in case you want to speed them up - you still
have to make asssumptions at runtime and possibly take them back.

> 
> >For the rest is just speed vs. memory, customization is a clear example
> >for that.
> 
> Sure. The one nice thing that both perl and python have going for them is 
> the ultimate presence of all the source (or a high-enough level 
> representation of it to feed to the optimizer) at run time. We can tell 
> with more surety what side-effects various methods and functions in 
> external code libraries will have, and we can get more aggressive (maybe) 
> because of it.
That is probably not directly true for polimorphic call sites. But of
course you can type-profile the program at runtime.

> Unlike, say, C, where passing a pointer to a routine outside 
> what the compiler can see will hobble the optimizer a whole lot in many 
> cases...
> 

regards, Samuele Pedroni.



From cgw@alum.mit.edu  Fri Aug  3 17:13:40 2001
From: cgw@alum.mit.edu (Charles G Waldman)
Date: Fri, 3 Aug 2001 11:13:40 -0500
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <E15ShO3-0000fz-00@mail.python.org>
References: <E15ShO3-0000fz-00@mail.python.org>
Message-ID: <15210.52660.887708.675013@transamoeba.dyndns.org>

On Fri, 3 Aug 2001 at 16:17:04 +0200, Thomas Wouters wrote:

 > Some people have seen bigger speedups just
 > reorganizing the case's in the Big Switch, but those same changes slowed
 > things down on other platforms.

Clearly, part of Python's ./configure should try all different
permutations of the 107 "case" statements in the Big Switch, and 
find the arrangement that maximizes Pystones on a per-platform
basis.

just-kidding-ly y'rs,
		-C



From barry@zope.com  Fri Aug  3 17:39:50 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 3 Aug 2001 12:39:50 -0400
Subject: [Python-Dev] cgitb.py for Python 2.2
References: <Pine.LNX.4.32.0107301905210.4535-100000@ziggy.localdomain.fake>
 <15206.8227.652539.471067@anthem.wooz.org>
 <20010731075258.A2757@ute.cnri.reston.va.us>
Message-ID: <15210.54230.552520.602993@anthem.wooz.org>

>>>>> "AK" == Andrew Kuchling <akuchlin@mems-exchange.org> writes:

    AK> +0 from me, though I also think it would be better in cgi.py
    AK> and not in site.py.  It would also be useful if it could mail
    AK> tracebacks and return a non-committal but secure error message
    AK> to the browser; I'll contribute that as a patch if cgitb.py
    AK> goes in.  (Or should that be cgi/tb.py?  Hmm...)

Whatever happened to PEP 222? :)


From barry@zope.com  Fri Aug  3 17:46:15 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 3 Aug 2001 12:46:15 -0400
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
References: <Pine.LNX.4.32.0107311813220.19357-100000@ziggy.localdomain.fake>
 <20010803082137.K626@xs4all.nl>
 <15210.48266.673906.863831@beluga.mojam.com>
Message-ID: <15210.54615.364193.422123@anthem.wooz.org>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    SM> I have to agree.  Perhaps just as significant, what would
    SM> happen to a CGI script that already attempts to do something
    SM> to direct error output to a file or mail message the webmaster
    SM> will see?

That's my major concern about enabling by default too.  I'd now have
to have a way to /disable/ it explicitly, but of course, I'd have to
wrap that in a try/except in case I'm using an older cgi.py.

    SM> Most of my users wouldn't know a traceback (simpler or
    SM> otherwise) from a hole in the ground.  I think I have perhaps
    SM> two users who would understand that I was trying to create a
    SM> more readable report.

I agree.  cgitb.py is without a doubt Very Cool and should definitely
be in the std library.  But it's of use only to a handful of people,
primarily the developers of cgi module-based scripts.

+1 on additing it to std library
+1 on turning it on only with an explicit action (although I could
    live with enabling via side-effect on import of cgitb.py, count me
    -0 for that).

-Barry


From guido@zope.com  Fri Aug  3 17:50:23 2001
From: guido@zope.com (Guido van Rossum)
Date: Fri, 03 Aug 2001 12:50:23 -0400
Subject: [Python-Dev] (Idea) subclassable string: path object?
In-Reply-To: Your message of "Fri, 03 Aug 2001 16:44:47 +0200."
 <20010803164453-r01010700-1c882179-0910-010c@10.0.0.11>
References: <20010803164453-r01010700-1c882179-0910-010c@10.0.0.11>
Message-ID: <200108031650.MAA03181@cj20424-a.reston1.va.home.com>

> Every once in a while I wished for an path object to manipulate file
> system paths. Things like
>    os.path.join(a, b, c, os.path.splitext(os.path.basename(p))[0] + ".ext")
> quickly get frustrating (so of course I never write them like that ;-).
> 
> I thought of implementing a path object several times, but always
> stopped when I realized (for the Nth time ;-) that you'd then have
> to do something like
>    file = open(p.tostring())
> whenever you want to *use* your pat. That doesn't help at all.

Two better solutions:

(1) If the Path class defines a __str__() method, you can write open(p).

(2) You could add an open method to the Path class, and write p.open().

> But: since strings are now subclassable (there are, aren't they?)
> this should no longer be a problem!
> 
> Would it be a worthwile project to design and implement a path
> object for the standard library?

I'm lukewarm on adding this to the std library.  Java has such a thing
and I never warmed up to it.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@zope.com  Fri Aug  3 17:52:03 2001
From: guido@zope.com (Guido van Rossum)
Date: Fri, 03 Aug 2001 12:52:03 -0400
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: Your message of "Fri, 03 Aug 2001 12:46:15 EDT."
 <15210.54615.364193.422123@anthem.wooz.org>
References: <Pine.LNX.4.32.0107311813220.19357-100000@ziggy.localdomain.fake> <20010803082137.K626@xs4all.nl> <15210.48266.673906.863831@beluga.mojam.com>
 <15210.54615.364193.422123@anthem.wooz.org>
Message-ID: <200108031652.MAA03215@cj20424-a.reston1.va.home.com>

> +1 on additing it to std library
> +1 on turning it on only with an explicit action (although I could

Agreed.

>     live with enabling via side-effect on import of cgitb.py, count me
>     -0 for that).

-1 on enabling it with "import cgitb".  Modules with import side
effects are evil (and trickier to introspect, as Ping should know :-).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@zope.com  Fri Aug  3 18:11:36 2001
From: guido@zope.com (Guido van Rossum)
Date: Fri, 03 Aug 2001 13:11:36 -0400
Subject: [Python-Dev] www.python.org is back!
Message-ID: <200108031711.NAA03374@cj20424-a.reston1.va.home.com>

The Python sites http://www.python.org and ftp://ftp.python.org are
back up, thanks to the energetic help of Thomas Wouters and XS4ALL
(the dutch ISP that's his employer).

We're still fetching some missing files from a mirror, so please be
patient if your favorite download isn't there yet.  We're also still
working on restoring the FAQ wizard and the 2.0 MoinMoin and
restarting the Ultraseek engine (search.python.org) -- I hope this
will be completed later today.

Thanks again to XS4ALL (http://www.xs4all.nl/) and Thomas!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From just@letterror.com  Fri Aug  3 18:21:53 2001
From: just@letterror.com (Just van Rossum)
Date: Fri,  3 Aug 2001 19:21:53 +0200
Subject: [Python-Dev] (Idea) subclassable string: path object?
In-Reply-To: <200108031650.MAA03181@cj20424-a.reston1.va.home.com>
Message-ID: <20010803192155-r01010700-eaaed0b3-0910-010c@10.0.0.11>

Guido van Rossum wrote:

> Two better solutions:
> 
> (1) If the Path class defines a __str__() method, you can write open(p).

Ah, I had no idea that works these days: excellent, much nicer!

> (2) You could add an open method to the Path class, and write p.open().

Mwah. I quite like (1). Maybe I'll just write a Path class and use it for a
while, then I can be more sure of it's usefulness (or unusefulness...).

I think it'll be a lot like string methods, in the sense that if all you know is
the string module string methods seems unneccesary and overkill, but once you
start using them you notice how much better it is. I see a Path object as an
attempt to OO-ify os.path. I'll report back if it works out well.

Just


From tim.one@home.com  Fri Aug  3 18:38:33 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 3 Aug 2001 13:38:33 -0400
Subject: [Python-Dev] www.python.org is back!
In-Reply-To: <200108031711.NAA03374@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEELLDAA.tim.one@home.com>

> The Python sites http://www.python.org and ftp://ftp.python.org are
> back up, thanks to the energetic help of Thomas Wouters and XS4ALL
> (the dutch ISP that's his employer).

Not only up, but my response time browsing python.org is dramatically faster
going to the Netherlands than to CNRI down the street.  Downloads are much
zippier too.  Sweet.

wondering-why-we-had-to-twist-thomas's-arm-so-hard<wink>-ly y'rs  - tim



From DavidA@ActiveState.com  Fri Aug  3 19:27:27 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Fri, 03 Aug 2001 11:27:27 -0700
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
References: <20010802150109.B2033@netthink.co.uk>
 <200108031233.IAA01557@cj20424-a.reston1.va.home.com> <200108031249.IAA01751@cj20424-a.reston1.va.home.com>
Message-ID: <3B6AED0F.AC515C76@ActiveState.com>

Guido van Rossum wrote:
> 
> > I'm willing to entertain a new command line option for
> > it though, so that CGI scripts can use
> >
> >    #!/usr/bin/env python -C
> >
> > as their first line.
> 
> But it's fine to require making a call to cgi.whatever() too, and
> that's certainly a lot less work!

Furthermore, I believe that it's not always easy to get web servers to
respect command line options set in shebang lines, so my guess is that
doing cgi.whatever() is going to work for more people (e.g. people who
have CGI access but not webserver config access).

--david


From sdm7g@Virginia.EDU  Fri Aug  3 19:19:24 2001
From: sdm7g@Virginia.EDU (Steven D. Majewski)
Date: Fri, 3 Aug 2001 14:19:24 -0400 (EDT)
Subject: [Python-Dev] www.python.org is back!
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEELLDAA.tim.one@home.com>
Message-ID: <Pine.NXT.4.21.0108031415320.334-100000@localhost.virginia.edu>


On Fri, 3 Aug 2001, Tim Peters wrote:

> > The Python sites http://www.python.org and ftp://ftp.python.org are
> > back up, thanks to the energetic help of Thomas Wouters and XS4ALL
> > (the dutch ISP that's his employer).
> 
> Not only up, but my response time browsing python.org is dramatically faster
> going to the Netherlands than to CNRI down the street.  Downloads are much
> zippier too.  Sweet.

Is the Netherlands spinward or anti-spinward from your location with 
respect to the earth's rotation?  I think you are 'down spin' . 
How about from CNRI ?  That might explain it:  less friction on the
electrons flowing spinward. 

Waiting for my Nobel Prize! 

-- Steve 



From fdrake@acm.org  Fri Aug  3 19:18:01 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 3 Aug 2001 14:18:01 -0400 (EDT)
Subject: [Python-Dev] www.python.org is back!
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEELLDAA.tim.one@home.com>
References: <200108031711.NAA03374@cj20424-a.reston1.va.home.com>
 <LNBBLJKPBEHFEDALKOLCCEELLDAA.tim.one@home.com>
Message-ID: <15210.60121.118788.294847@cj42289-a.reston1.va.home.com>

Tim Peters writes:
 > wondering-why-we-had-to-twist-thomas's-arm-so-hard<wink>-ly y'rs  - tim

heck-just-wondering-why-we-had-to-drive-so-many-nails-in-CNRI's-
  coffin-ly y'rs


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From gmcm@hypernet.com  Fri Aug  3 20:07:51 2001
From: gmcm@hypernet.com (Gordon McMillan)
Date: Fri, 3 Aug 2001 15:07:51 -0400
Subject: [Python-Dev] www.python.org is back!
In-Reply-To: <Pine.NXT.4.21.0108031415320.334-100000@localhost.virginia.edu>
References: <LNBBLJKPBEHFEDALKOLCCEELLDAA.tim.one@home.com>
Message-ID: <3B6ABE47.2090.22720523@localhost>

Steven D. Majewski wrote:
> 
> On Fri, 3 Aug 2001, Tim Peters wrote:
> 
> > > The Python sites http://www.python.org and
> > > ftp://ftp.python.org are back up, thanks to the energetic
> > > help of Thomas Wouters and XS4ALL (the dutch ISP that's his
> > > employer).
> > 
> > Not only up, but my response time browsing python.org is
> > dramatically faster going to the Netherlands than to CNRI down
> > the street.  Downloads are much zippier too.  Sweet.
> 
> Is the Netherlands spinward or anti-spinward from your location
> with respect to the earth's rotation?  I think you are 'down
> spin' . How about from CNRI ?  That might explain it:  less
> friction on the electrons flowing spinward. 
> 
> Waiting for my Nobel Prize! 

That's spinward from you, so unless you go pick it up, you'll 
never get it.

- Gordon


From gstein@lyra.org  Fri Aug  3 20:12:54 2001
From: gstein@lyra.org (Greg Stein)
Date: Fri, 3 Aug 2001 12:12:54 -0700
Subject: [Python-Dev] Re: WebDAV in python 2.2?
In-Reply-To: <200108031254.IAA01818@cj20424-a.reston1.va.home.com>; from guido@zope.com on Fri, Aug 03, 2001 at 08:54:17AM -0400
References: <mailman.996828301.19910.clpa-moderators@python.org> <200108031254.IAA01818@cj20424-a.reston1.va.home.com>
Message-ID: <20010803121254.B29583@lyra.org>

On Fri, Aug 03, 2001 at 08:54:17AM -0400, Guido van Rossum wrote:
> > GvR also wondered if Python 2.2 should support DAV ("That's another
> > open protocol that Python could easily support out of the box.") and
> > /F agreed, but Greg Stein, the DAV expert and champion on python-dev,
> > remained silent.
> 
> Greg, I now know where you were.  Since you're back, any comments?

I saw that thread while I was deleting over 200,000 duplicate messages from
my inbox :-( ... I just haven't got to that part of my "real" inbox yet.

My intent was to write a short PEP because there would be a couple modules
to add to the lib:

1) httpauth -- subclasses httplib.HTTPConnection to add authenticated
   connections. Initially using the Basic auth method, but it could also be
   expanded to Digest (and others)

2) davlib -- my existing module. needs a couple items of work: allow
   alternate/fallback XML parsers (it uses Expat + qp_xml right now), and
   use the authenticated HTTP connections.

That is at least a start on what needs to happen. We can flesh it out with a
PEP or by checking in the modules.
(Moshe and I talked about httpauth a bit at the last Python Conference; I
 know he is interested in working on something like that)

I'm not sure how we normally write tests for network-based modules, but it
shouldn't be too hard to have them run against test.webdav.org.

Doc should be pretty straight-forward.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From tim.one@home.com  Fri Aug  3 21:32:45 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 3 Aug 2001 16:32:45 -0400
Subject: [Python-Dev] test_quopri is iso-latin-1 centric
In-Reply-To: <20010803104451.88356303181@snelboot.oratrix.nl>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEFJLDAA.tim.one@home.com>

IIRC, I checked in test_quopri.py without noticing this (although I did
notice that it relied on significant trailing whitespace!), so I'll fix it
as penance.

Note that the C std says the result of writing a text-mode file is *defined*
only if the characters written are in range(32, 127) (note:  chr(127) isn't
suppoted) plus tab and newline.  Also that space characters before a newline
may disappear, and that the last line written must terminate with a newline.
Jack's bad experience is one example of why the std says all this stuff.

iow-c-text-mode-is-useless<wink>-ly y'rs  - tim



From tim.one@home.com  Fri Aug  3 21:51:43 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 3 Aug 2001 16:51:43 -0400
Subject: [Python-Dev] test_quopri is iso-latin-1 centric
In-Reply-To: <20010803123857.99202303181@snelboot.oratrix.nl>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEFLLDAA.tim.one@home.com>

[Martin von Lupsidedownquestionmarkwis]
> This is the problem. Python source code is not in Latin-1; bytes;
> inside strings and comments are "as-is".

The latter is a platform-dependent accident -- the ref man says Python
source is 7-bit ASCII, and that was the intent.  That C's ctype functions
later grew locale-dependent behavior accounts for why Python did too, but
even according to the C standard you can't count on transporting C text-mode
files across platforms unless they avoid that stuff.

> So the CVS "binary" mode would come closer as to how python files
> should be treated, although you'd still would want to convert line-
> endings.

[Jack]
> Is this an "official" policy?

My policy is that Python source in the *distribution* adheres to the
restrictions of the C std for C text-mode files, until such time as Python
changes to read and write them in C binary mode, or stops using C stdio
altogether.  And that's as official as I am <wink>.

> As such I disagree with it. Either all Python source should be 7-bit
> clean or the source should be converted to the local convention, even
> for strings and comments. But if it is an official policy I'd
> like to hear it (who wants to channel Guido here?), because
> there's heaps of MacPython stuff that'll have to be converted back
> (I did the forward conversion a couple of months ago).

Sorry, I don't know what "forward" and "back" mean to you.



From sdm7g@Virginia.EDU  Fri Aug  3 22:21:29 2001
From: sdm7g@Virginia.EDU (Steven D. Majewski)
Date: Fri, 3 Aug 2001 17:21:29 -0400 (EDT)
Subject: [Python-Dev] RFC: generator combinator syntax
In-Reply-To: <Pine.NXT.4.21.0108031643320.334-100000@localhost.virginia.edu>
Message-ID: <Pine.NXT.4.21.0108031702000.334-100000@localhost.virginia.edu>

FYI: In the thread on comp.lang.python with the subject:

"Nested generators is the python equivalent of unix pipe cmds."

we're hashing out some ideas for functions for "the standard 
generator library" and syntax for operators to combine iterators.

This thread started after thinking about my comments on Eric's
request-for-comments on a file processing framework. 

I'm not actually in favor of following unix pipeline syntax so
closely as to use the __or__ operator -- I think it's confusing
python syntax -- but by having   'iterator | function' 
return a modified iterator, you can build iterator pipeline
chain commands. 

For example:

	Files('.') | isGif 

Files( '.' ) is a generator that recursively enumerates all the files 
	     starting from the current (unix) working directory. 

isGif = lambda s : s.lower().find( '.gif' ) >= 0 

and the operator returns the iterator that returns all of the 
 ".gif" files below the current working directory. 


I'm actually proposing "&" to be the operator to do the above.
I think "|", despite the unix convention, would be better for
alternation ( i.e. the generator that yields one from column A,
one from column (generator) B, etc. ) 


-- Steve 



From paulp@ActiveState.com  Sat Aug  4 00:07:51 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Fri, 03 Aug 2001 16:07:51 -0700
Subject: [Python-Dev] RFC: generator combinator syntax
References: <Pine.NXT.4.21.0108031702000.334-100000@localhost.virginia.edu>
Message-ID: <3B6B2EC7.93F61652@ActiveState.com>

"Steven D. Majewski" wrote:
> 
> 
>         Files('.') | isGif
> 
> Files( '.' ) is a generator that recursively enumerates all the files
>              starting from the current (unix) working directory.
> 
> isGif = lambda s : s.lower().find( '.gif' ) >= 0

I propose:

[yield file for file in Files('.') 
       if file.lower.find(.'gif')>=0]

By reusing the yield keyword we can get rid of the need for the new "|"
or "&" syntax.

Any kind of combination that can be done with list comprehensions would
be possible with generator comprehensions.
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From sdm7g@Virginia.EDU  Sat Aug  4 00:35:04 2001
From: sdm7g@Virginia.EDU (Steven D. Majewski)
Date: Fri, 3 Aug 2001 19:35:04 -0400 (EDT)
Subject: [Python-Dev] RFC: generator combinator syntax
In-Reply-To: <3B6B2EC7.93F61652@ActiveState.com>
Message-ID: <Pine.NXT.4.21.0108031914170.334-100000@localhost.virginia.edu>


On Fri, 3 Aug 2001, Paul Prescod wrote:

> I propose:
> 
> [yield file for file in Files('.') 
>        if file.lower.find(.'gif')>=0]
> 
> By reusing the yield keyword we can get rid of the need for the new "|"
> or "&" syntax.
> 
> Any kind of combination that can be done with list comprehensions would
> be possible with generator comprehensions.

I assume you intend that "[yield file for..."  expression to evaluate
to a generator, not a list. 

I like it. 

I'm not sure that I want to get rid of the operator syntax. 
I'm still getting used to it. 

Either works a lot better than using filter: I was complaining 
in an earlier thread about having to explicitly ask for the
.next attrib. It just took a while to figure out how they are
meant to be used. 

Generators & Iterators really seem to open up functional programming
in Python. 

I'm trying to put together some more basic file processing functionals: 
for example, fileExt(ext) instead of isGif:

def fileExt(ext):
        return lambda s: os.path.splitext( s )[-1].lower() == ext.lower() 

Then isGif becomes:

isGif = fileExt( '.gif' )

and others like fileSize, owner, etc.

def fileSize( test ):
	return lambda s: test( os.stat(s)[stat.ST_SIZE] ) 


and folding the Genops class into the File function, I can 
do things like: 


for file in Files() & isGif & fileSize( lambda s : s > 1024 ): 
	process( file )


-- Steve 



From simon@netthink.co.uk  Sat Aug  4 01:54:04 2001
From: simon@netthink.co.uk (Simon Cozens)
Date: Fri, 3 Aug 2001 17:54:04 -0700
Subject: [Python-Dev] Python API version & optional features
In-Reply-To: <3B6A6B54.3BA008C9@lemburg.com>
Message-ID: <20010803175404.C13102@netthink.co.uk>

On Fri, Aug 03, 2001 at 11:13:56AM +0200, M.-A. Lemburg wrote:
> > Given the smiley at the end of your message, I don't know whether you
> > meant your point seriously or not.
> 
> The only thing which we require is that a Unicode aware
> extension use at least one of the mangled APIs -- shouldn't
> be much of a requirement, though :-)

My point was that given a Py_UCS2_Foo it should be easy to automatically
produce a Py_UCS4_Foo (or vice versa) so that people who are using the "wrong"
mangled API can still get what they expect.

But I'm probably misunderstanding something.

Simon


From simon@netthink.co.uk  Sat Aug  4 01:56:02 2001
From: simon@netthink.co.uk (Simon Cozens)
Date: Fri, 3 Aug 2001 17:56:02 -0700
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: <200108031233.IAA01557@cj20424-a.reston1.va.home.com>
Message-ID: <20010803175602.D13102@netthink.co.uk>

On Fri, Aug 03, 2001 at 08:33:40AM -0400, Guido van Rossum wrote:
> But Ping's tracebackhook still shouldn't be the default.

Yep, understood.

> I'm willing to entertain a new command line option for
> it though, so that CGI scripts can use
>    #!/usr/bin/env python -C
> as their first line.

How about making it the default iff certain CGI-oid enviroment
variables are set? Or would that be way too magical?

Simon


From barry@zope.com  Sat Aug  4 01:24:41 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 3 Aug 2001 20:24:41 -0400
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
References: <200108031233.IAA01557@cj20424-a.reston1.va.home.com>
 <20010803175602.D13102@netthink.co.uk>
Message-ID: <15211.16585.852075.358121@anthem.wooz.org>

>>>>> "SC" == Simon Cozens <simon@netthink.co.uk> writes:

    >> I'm willing to entertain a new command line option for it
    >> though, so that CGI scripts can use #!/usr/bin/env python -C as
    >> their first line.

    SC> How about making it the default iff certain CGI-oid enviroment
    SC> variables are set? Or would that be way too magical?

I'm concerned about magically enabling it, because then my existing
cgi scripts have to be modified to turn it off or they will/might
break.  And I'll have to conditionalize the code to turn them off in
case my users have an older version of Python.

explicit-is-better-than-implicit-ly y'rs,
-Barry


From aahz@rahul.net  Sat Aug  4 02:38:46 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Fri, 3 Aug 2001 18:38:46 -0700 (PDT)
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: <15211.16585.852075.358121@anthem.wooz.org> from "Barry A. Warsaw" at Aug 03, 2001 08:24:41 PM
Message-ID: <20010804013846.8B64B99C97@waltz.rahul.net>

Barry A. Warsaw wrote:
> 
>     SC> How about making it the default iff certain CGI-oid enviroment
>     SC> variables are set? Or would that be way too magical?
> 
> I'm concerned about magically enabling it, because then my existing
> cgi scripts have to be modified to turn it off or they will/might
> break.  And I'll have to conditionalize the code to turn them off in
> case my users have an older version of Python.
> 
> explicit-is-better-than-implicit-ly y'rs,
> -Barry

OTOH, looking for a CGITB variable might work.  On the gripping hand,
anyone who wants to work it that way can do so with two lines of code.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From fdrake@acm.org  Sat Aug  4 03:03:38 2001
From: fdrake@acm.org (Fred L. Drake)
Date: Fri,  3 Aug 2001 22:03:38 -0400 (EDT)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20010804020338.338B82879B@cj42289-a.reston1.va.home.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

A variety of updates, including the first installment for the distutils
API docs (in "Distributing Python Modules"), and some contributed
material on embedding Python (in "Extending and Embedding the Python
Interpreter").

There were many other small improvements and bug fixes.



From gstein@lyra.org  Sat Aug  4 04:22:27 2001
From: gstein@lyra.org (Greg Stein)
Date: Fri, 3 Aug 2001 20:22:27 -0700
Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?]
Message-ID: <20010803202227.A31091@lyra.org>

[ misaddressed... ]

----- Forwarded message from Guido van Rossum <guido@zope.com> -----

From: Guido van Rossum <guido@zope.com>
Subject: Re: [Python-Dev] Re: WebDAV in python 2.2?
To: Greg Stein <gstein@lyra.org>
Date: Fri, 03 Aug 2001 15:18:37 -0400

> My intent was to write a short PEP because there would be a couple modules
> to add to the lib:
> 
> 1) httpauth -- subclasses httplib.HTTPConnection to add authenticated
>    connections. Initially using the Basic auth method, but it could also be
>    expanded to Digest (and others)
> 
> 2) davlib -- my existing module. needs a couple items of work: allow
>    alternate/fallback XML parsers (it uses Expat + qp_xml right now), and
>    use the authenticated HTTP connections.
> 
> That is at least a start on what needs to happen. We can flesh it out with a
> PEP or by checking in the modules.
> (Moshe and I talked about httpauth a bit at the last Python Conference; I
>  know he is interested in working on something like that)

Good!

> I'm not sure how we normally write tests for network-based modules, but it
> shouldn't be too hard to have them run against test.webdav.org.

Can someone help Greg with his question about testing network modules?
Generally the test suite doesn't assume internet access; network code
is "tested" using a second thread and localhost.

--Guido van Rossum (home page: http://www.python.org/~guido/)

----- End forwarded message -----

-- 
Greg Stein, http://www.lyra.org/


From tim.one@home.com  Sat Aug  4 06:33:03 2001
From: tim.one@home.com (Tim Peters)
Date: Sat, 4 Aug 2001 01:33:03 -0400
Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler
Message-ID: <LNBBLJKPBEHFEDALKOLCCEGNLDAA.tim.one@home.com>

Some Socket Wizard should worry about the attached.  What's life like on
64-bit Linux (etc)?  Perhaps more autoconf crap could fudge this.

once-again-windows-shows-the-way<wink>-ly y'rs  - tim


Background info from an earlier Python-Help msg:

> I work on a Solaris 8.0 (64 bit) system. Python is compiled with
> Sun CC (Forte 6.1) compiler with threading support.  A small program
> telnetting to a computer often dumps core.

-----Original Message-----
From: python-help-admin@python.org
Sent: Friday, August 03, 2001 5:06 AM
To: help@python.org
Subject: Re: [Python-Help] Python threads with suncc (Forte 6.1)
compiler

...

I've received some answers that this code running fine in the same
environment.  A collegaue of mine compiled Python 2.0 as I wrote
yesterday, and later he debugged this error.  Purify showed us stack
read and write outside the stack in Modules/selectmodule.c in
function:


Modules/selectmodule.c
.
.
static PyObject *
select_select(PyObject *self, PyObject *args)
{
#ifdef MS_WINDOWS
	/* This would be an awful lot of stack space on Windows! */
	pylist *rfd2obj, *wfd2obj, *efd2obj;
#else
	pylist rfd2obj[FD_SETSIZE + 3];
	pylist wfd2obj[FD_SETSIZE + 3];
	pylist efd2obj[FD_SETSIZE + 3];
#endif
.
.
.
}

In our environment FD_SETSIZE is 65536 as defined in sys/select.h (see
below). The allocated stack space in select_select is
3*sizeof(rfd2obj)*(FD_SETSIZE+3). It is more than 3MB. My collegaue told
me that the difference between the addresses of the same variable in two
seperate threads is about 2MB. Lets suppose char *p1 = (char *)rfd2obj
in thread N and char *p2 = (char *)rfd2obj in thread N + 1, abs(p1-p2)
is about 2MB. The stack is overwritten between the seperate threads. He
could solve the problem, as solved on Windows with allocating these
variables on the heap. Do you have any ideas, how can it happen?

/usr/include/sys/select.h:
.
.
#ifndef	FD_SETSIZE
#ifdef _LP64
#define	FD_SETSIZE	65536
#else
#define	FD_SETSIZE	1024
#endif	/* _LP64 */
.
.

...

_______________________________________________
Python-Help maillist  -  Python-Help@python.org
http://mail.python.org/mailman/listinfo/python-help



From guido@zope.com  Sat Aug  4 07:29:08 2001
From: guido@zope.com (Guido van Rossum)
Date: Sat, 04 Aug 2001 02:29:08 -0400
Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler
In-Reply-To: Your message of "Sat, 04 Aug 2001 01:33:03 EDT."
 <LNBBLJKPBEHFEDALKOLCCEGNLDAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCCEGNLDAA.tim.one@home.com>
Message-ID: <200108040629.CAA10668@cj20424-a.reston1.va.home.com>

> Some Socket Wizard should worry about the attached.  What's life like on
> 64-bit Linux (etc)?

Dunno, but I doubt 64-bit Linux would be so dumb as to declare
FD_SETSIZE to be 2**16.

> Perhaps more autoconf crap could fudge this.

I think we can avoid this.

> once-again-windows-shows-the-way<wink>-ly y'rs  - tim

Yes, in a very simple way: instead of testing for MS_WINDOWS, we could
test for a preposterous value of FD_SETSIZE.  For example, change all
three occurrences of

    #ifdef MS_WINDOWS

into

    #if FD_SETSIZE > 1024

Looking at the code, a better solution may be to always allocate this
thing on the heap, and to let list2set allocate it.  The actual number
of items *used* is the length of the list of file descriptors + 1 (a
sentinel).  Even if FD_SETSIZE is not preposterous, the array size
allocated is the theoretical maximum, which is always ways larger than
the size needed.  (I also fail to understand why 3 extra elements are
allocated; the algorithm seems to need only one extra.)

Yet another approach would (shrudder :) be to forget about arrays and
use a Python dict, since that's the usage: the data structure stores
the mapping between file descriptors (unique small ints) and objects
in the argument lists (either Python ints or objects with a fileno()
method).

Volunteers?  Not me! :)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From ping@lfw.org  Sat Aug  4 07:45:05 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Fri, 3 Aug 2001 23:45:05 -0700 (PDT)
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: <15211.16585.852075.358121@anthem.wooz.org>
Message-ID: <Pine.LNX.4.32.0108032324550.579-100000@ziggy.localdomain.fake>

Hi all,

I'm glad this is being discussed, and i understand that some of you
are concerned about changing default behaviour, but, uh, it kind of
irks me that the major concerns aren't that well based in truth.

Thomas Wouters wrote:
> You might not want people to edit scripts, but I'm not sure howmany
> people want the functionality of their CGI script(s) significantly
> changed without editing their scripts.

Skip Montanaro wrote:
> I have to agree.  Perhaps just as significant, what would happen to a CGI
> script that already attempts to do something to direct error output to a
> file or mail message the webmaster will see?

Nothing!  It would still work.  No functionality would change at all!

That's the whole point.  This feature only kicks in at the point
where a program has ALREADY FAILED.  By definition, there is nothing
the program could have done at that point.  It's about to die.

Barry Warsaw wrote:
> I'm concerned about magically enabling it, because then my existing
> cgi scripts have to be modified to turn it off or they will/might
> break.

They *couldn't* break.  I mean, they could only "break" in the case
where they're already broken.

There seems to be this general nervousness about making a change,
but -- well, i don't really see what you all are going on about.
It seems to me that if you think about it for a bit, you'll see
why the change couldn't break anything.

If anything is there (e.g. some sort of CGI-imitation environment)
running the script and catching exceptions, then there's no issue.
sys.excepthook is only a last resort.

Setting os.environ['GATEWAY_INTERFACE'] after the program starts
doesn't magically install the feature -- it's only enabled if the
GATEWAY_INTERFACE variable is present when Python starts, which
can only be the case if the caller of the Python binary wants the
script to behave like a CGI script, and cgitb meets that contract.

Skip Montanaro wrote:
> Most of my users wouldn't know a traceback (simpler or otherwise) from a
> hole in the ground.  I think I have perhaps two users who would understand
> that I was trying to create a more readable report.

It is certainly a legitimate concern that you may want to keep
your tracebacks secret.

However, this seems to be just as good an argument that ordinary
Python programs shouldn't print tracebacks on stderr.  Oh, no --
what if a user sees them?

Well, the user saw it because something went wrong.  Of course
you don't want a typical user to worry about tracebacks.  If your
program was correct, that would never happen.  When you do get one,
something truly exceptional has occurred.  And yet we do display
tracebacks on stderr nevertheless, because

    You Never Know An Error Is About To Happen Before It Happens.

And when it does happen, gosh! am i ever grateful i got that traceback.


-- ?!ng



From guido@zope.com  Sat Aug  4 08:32:04 2001
From: guido@zope.com (Guido van Rossum)
Date: Sat, 04 Aug 2001 03:32:04 -0400
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: Your message of "Fri, 03 Aug 2001 23:45:05 PDT."
 <Pine.LNX.4.32.0108032324550.579-100000@ziggy.localdomain.fake>
References: <Pine.LNX.4.32.0108032324550.579-100000@ziggy.localdomain.fake>
Message-ID: <200108040732.DAA10879@cj20424-a.reston1.va.home.com>

Give it up, Ping.  cgitb writes the tb to stdout, not to stderr, and
that's enough of a difference in behavior to require explicit
enabling.  It's not worth your energy to try and convince us.

Let's have the cgitb module as an optional feature.  Let's update
various documentation to suggest this boilerplate at the top of CGI
modules:

  import cgi
  import cgitb; cgitb.enable()

Regarding the secrecy of tracebacks: normal tracebacks are seen by a
user who has logged in to the system and who can look at the source
code anyway.  Web tracebacks can be seen as invitations to hackerz
without any source access to look for weaknesses in the script.
That's a very different thing from a security point of view!
(Security by obscurity, maybe, but nevertheless better than hanging
out your dirty laundry to dry in public.  Enough hackerz know Python.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From m@moshez.org  Sat Aug  4 08:55:44 2001
From: m@moshez.org (Moshe Zadka)
Date: Sat, 04 Aug 2001 10:55:44 +0300
Subject: [Python-Dev] Re: WebDAV in python 2.2?
In-Reply-To: <20010803121254.B29583@lyra.org>
References: <20010803121254.B29583@lyra.org>, <mailman.996828301.19910.clpa-moderators@python.org> <200108031254.IAA01818@cj20424-a.reston1.va.home.com>
Message-ID: <E15SwHg-0004Bs-00@darjeeling>

On Fri, 3 Aug 2001 12:12:54 -0700, Greg Stein <gstein@lyra.org> wrote:

> That is at least a start on what needs to happen. We can flesh it out with a
> PEP or by checking in the modules.
> (Moshe and I talked about httpauth a bit at the last Python Conference; I
>  know he is interested in working on something like that)

Let me just clarify what we talked about: today, the HTTPConnection already
know how to respond to code 100. urllib2 has a very nifty mechanism of handling
all kinds of errors. I have a feeling the responsiblity split is not correct,
and I want to think about some kind of better responsibility split.

I'll try and write something up soonish.
-- 
Moshe Zadka - http://moshez.geek
(if you're not cool, http://moshez.org is still working)


From tim.one@home.com  Sat Aug  4 10:01:07 2001
From: tim.one@home.com (Tim Peters)
Date: Sat, 4 Aug 2001 05:01:07 -0400
Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler
In-Reply-To: <200108040629.CAA10668@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEHCLDAA.tim.one@home.com>

[Guido]
> Dunno, but I doubt 64-bit Linux would be so dumb as to declare
> FD_SETSIZE to be 2**16.

Quick search on google shows it's "the normal case" on that 64-bit Solaris,
nothing special under Tru64, and probably allowed on some others.

> ...
> instead of testing for MS_WINDOWS, we could test for a preposterous
> value of FD_SETSIZE.  For example, change all three occurrences of
>
>     #ifdef MS_WINDOWS
>
> into
>
>     #if FD_SETSIZE > 1024

Sure.

> Looking at the code, a better solution may be to always allocate this
> thing on the heap, and to let list2set allocate it.  The actual number
> of items *used* is the length of the list of file descriptors + 1 (a
> sentinel).  Even if FD_SETSIZE is not preposterous, the array size
> allocated is the theoretical maximum, which is always ways larger than
> the size needed.

I assume the current code is just trying to save 3 malloc/free pairs in the
usual case.  Since I'm not a select wizard, I don't know whether the speed
hit would matter (OTOH, if there are few descriptors it's cheap regardless,
and if there are many it's expensive regardless).

> (I also fail to understand why 3 extra elements are allocated; the
> algorithm seems to need only one extra.)

I've stared uncomprehendingly at that too.  Anyone?  It discouraged me from
becoming a select wizard a couple of years ago <wink>.

> Yet another approach would (shrudder :) be to forget about arrays and
> use a Python dict, since that's the usage: the data structure stores
> the mapping between file descriptors (unique small ints) and objects
> in the argument lists (either Python ints or objects with a fileno()
> method).
>
> Volunteers?  Not me! :)

Ditto.  Someone who stopped at the first suggestion could be finished
already, though <wink>.

BTW, do we actually need *any* auxiliary data structure here?  Looks to me
like all these giant pylist thingies could be tossed in favor of making
shallow copies of the input ifdlist etc list arguments.  The giant pylists
don't *appear* to accomplish anything except cache the result of a
PyObject_AsFileDescriptor() call per object.  But if that's not an expensive
call (doesn't look expensive to me), it's not worth all this trouble no
matter how spelled.



From gward@python.net  Sat Aug  4 16:06:53 2001
From: gward@python.net (Greg Ward)
Date: Sat, 4 Aug 2001 11:06:53 -0400
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: <200108031233.IAA01557@cj20424-a.reston1.va.home.com>; from guido@zope.com on Fri, Aug 03, 2001 at 08:33:40AM -0400
References: <20010802150109.B2033@netthink.co.uk> <200108031233.IAA01557@cj20424-a.reston1.va.home.com>
Message-ID: <20010804110653.A910@gerg.ca>

On 03 August 2001, Guido van Rossum said:
> it though, so that CGI scripts can use
> 
>    #!/usr/bin/env python -C
> 
> as their first line.

Completely off-topic: the "/usr/bin/env" hack doesn't work with
command-line arguments under Linux:

$ cat t.py 
#!/usr/bin/env python -i

print "hello"

$ ./t.py  
/usr/bin/env: python -i: No such file or directory

Grumble.  Andrew and I tried to track this down once, and found
suspicious-looking code in the kernel (ie., I don't think it's the fault
of GNU env).

        Greg
-- 
Greg Ward - Python bigot                                gward@python.net
http://starship.python.net/~gward/
"What do you mean -- a European or an African swallow?"


From pedroni@inf.ethz.ch  Sat Aug  4 16:09:37 2001
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Sat, 4 Aug 2001 17:09:37 +0200
Subject: [Python-Dev] Zope porting to jython
References: <20010802103006.H1667@netthink.co.uk> <20010803161704.M626@xs4all.nl>
Message-ID: <004c01c11cf7$7b7eec60$b75821c0@newmexico>

Hi,

from some recent messages on the jython lists it seems that different
people/groups are starting to work
on recreating some of the necessary software components in order to port Zope
or a create a
similar compatible framework on top of Java/Jython.

Here's a link to latest related thread:

http://aspn.activestate.com/ASPN/Mail/Message/Jython-dev/712337

Jeremy has already chimed in,

my personal position is to help them as all other jython users, fixing
incompatibilities with CPython and
answering to questions. Nothing special. I can imagine that one of the big
problem could be restricted execution (if Zope uses that), admitted that any C
code functionality can be rewritten in Java.

If the PythonLabs have an "official" position/opinion on this and in particular
on the last quite speculative
message (from Joseph Grace) in the thread, they may post something. I can say
something about technical aspects,
but I don't feel that I should channel/or invent PythonLabs opinions.

regards, Samuele Pedroni.



From gward@python.net  Sat Aug  4 16:18:47 2001
From: gward@python.net (Greg Ward)
Date: Sat, 4 Aug 2001 11:18:47 -0400
Subject: [Python-Dev] Proposed new module for the Python library
In-Reply-To: <20010802223632.A31724@thyrsus.com>; from esr@thyrsus.com on Thu, Aug 02, 2001 at 10:36:32PM -0400
References: <20010802223632.A31724@thyrsus.com>
Message-ID: <20010804111847.B910@gerg.ca>

On 02 August 2001, Eric S. Raymond said:
> There is a common `compiler-like' pattern in Unix scripts which is useful
> for translation utilities of all sorts.  A program following this pattern
> behaves as a filter when no argument files are specified on the command
> line, but otherwise transforms each file individually into a corresponding
> output file.

Sounds like a useful tool to have around.  I'm not sure the name
'ccframe' conjures up images of the above paragraph, but I'm not sure
what name *does*.  This is the hardest sort of software to name!

> Design comments?  Critiques?  Code on request.
> 
> I'm already considering throwing exceptions on open and rename errors
> instead of complaining to stderr and returning an error status.  That
> would be more Pythonic, though slightly less convenient in the
> commonest cases.

How to handle failure to open an input file depends on circumstances:
  * sometimes, it is utter calamity, and raising an exception is the
    right thing to do

  * sometimes, it's a user-level error, and
      try:
          file = open(filename)
      except IOError, err:
          sys.exit("%s: %s" % (filename, err.strerror))

    is the right thing to do

  * sometimes, it's just something the user ought to know about, and
    the "except" clause above can be turned into
          sys.stderr.write("warning: %s: %s" % (filename, err.strerror))
          continue   # next file, please

  * conceivably, it's something you can just ignore

IOW, I think this should be an option for users of the framework.  The
default should be to throw an exception -- kaBOOM!  For me, the middle
two options would be most commonly used, but modules should not
sys.exit() or sys.stderr.write() unless they are explicitly told to do
so.

        Greg
-- 
Greg Ward - nerd                                        gward@python.net
http://starship.python.net/~gward/
My opinions may have changed, but not the fact that I am right.


From fdrake@acm.org  Sat Aug  4 17:03:06 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sat, 4 Aug 2001 12:03:06 -0400 (EDT)
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: <20010804110653.A910@gerg.ca>
References: <20010802150109.B2033@netthink.co.uk>
 <200108031233.IAA01557@cj20424-a.reston1.va.home.com>
 <20010804110653.A910@gerg.ca>
Message-ID: <15212.7354.720723.721035@cj42289-a.reston1.va.home.com>

Greg Ward writes:
 > Grumble.  Andrew and I tried to track this down once, and found
 > suspicious-looking code in the kernel (ie., I don't think it's the fault
 > of GNU env).

  I had always understood that the sh-bang line could only provide a
single argument to begin with.  I don't know what you found in the
kernel sources, but I'd expect it to be implementing that policy.  So
/usr/bin/env gets one parameter from the sh-bang line, "python", and
then another parameter added by the kernel: the name of the script.
  Sounds about right to me.  Whether we like the constraint that there
be only one arg from the sh-bang line doesn't really matter; lots of
systems enforce this constraint, so to be portable we need to live
with it.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From itojun@iijlab.net  Sat Aug  4 14:21:57 2001
From: itojun@iijlab.net (Jun-ichiro itojun Hagino)
Date: Sat, 04 Aug 2001 22:21:57 +0900
Subject: [Python-Dev] segmentation fault in tuple parsing
Message-ID: <20010804132157.6F3F67BC@starfruit.itojun.org>

	it looks that there's some issue with tuple passing
	(PyArg_ParseTuple).  the last argument of vgetargs1(), which is 0,
	is dereferenced in convertsimple().  sorry if this is due to
	my mistake in socketmodule IPv6 support.

itojun


(gdb) run
Starting program: /usr/home/itojun/work/python/python/dist/src/python 
Python 2.2a1 (#12, Aug  4 2001, 10:17:23) 
[GCC egcs-2.91.66 19990314 (egcs-1.1.2 release)] on netbsd1
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.getnameinfo(('::1', 80, 0, 0), 0)

Program received signal SIGSEGV, Segmentation fault.
convertsimple (arg=0x80e43ac, p_format=0xbfbfcda4, p_va=0xbfbfcf80, 
    msgbuf=0xbfbfce68 "\af\027H ŽÇ\021\b\b+\030H") at Python/getargs.c:501
501                             *p = ival;
(gdb) bt
#0  convertsimple (arg=0x80e43ac, p_format=0xbfbfcda4, p_va=0xbfbfcf80, 
    msgbuf=0xbfbfce68 "\af\027H ŽÇ\021\b\b+\030H") at Python/getargs.c:501
#1  0x806ab6c in convertitem (arg=0x80e43ac, p_format=0xbfbfcf74, 
    p_va=0xbfbfcf80, levels=0xbfbfcde8, 
    msgbuf=0xbfbfce68 "\af\027H ŽÇ\021\b\b+\030H") at Python/getargs.c:351
#2  0x8068a72 in vgetargs1 (args=0x813d18c, format=0x481980b0 "i", 
    p_va=0xbfbfcf80, compat=0) at Python/getargs.c:197
#3  0x80686fd in PyArg_ParseTuple (args=0x813d18c, format=0x481980ac "si|ii")
    at Python/getargs.c:52
#4  0x48194319 in PySocket_getnameinfo (self=0x0, args=0x811c3cc)
    at /usr/home/itojun/work/python/python/dist/src/Modules/socketmodule.c:2430
#5  0x8059f35 in call_cfunction (func=0x81871cc, arg=0x811c3cc, kw=0x0)
    at Python/ceval.c:3072
#6  0x805744d in eval_frame (f=0x80e7c0c) at Python/ceval.c:1912
#7  0x8058461 in PyEval_EvalCodeEx (co=0x80fbdc0, globals=0x80f2d0c, 
    locals=0x80f2d0c, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0, 
    defcount=0, closure=0x0) at Python/ceval.c:2517
#8  0x8059ea3 in PyEval_EvalCode (co=0x80fbdc0, globals=0x80f2d0c, 
    locals=0x80f2d0c) at Python/ceval.c:490
#9  0x8073f48 in run_node (n=0x80f5c00, filename=0x80b4975 "<stdin>", 
    globals=0x80f2d0c, locals=0x80f2d0c, flags=0xbfbfd65c)
    at Python/pythonrun.c:1058
#10 0x8072a0e in PyRun_InteractiveOneFlags (fp=0x80dafd0, 
    filename=0x80b4975 "<stdin>", flags=0xbfbfd65c) at Python/pythonrun.c:578
#11 0x8073997 in PyRun_InteractiveLoopFlags (fp=0x80dafd0, 
    filename=0x80b4975 "<stdin>", flags=0xbfbfd65c) at Python/pythonrun.c:514
#12 0x8073802 in PyRun_AnyFileExFlags (fp=0x80dafd0, 
    filename=0x80b4975 "<stdin>", closeit=0, flags=0xbfbfd65c)
    at Python/pythonrun.c:477
#13 0x8051da3 in Py_Main (argc=1, argv=0xbfbfd6b8) at Modules/main.c:332
#14 0x8051742 in main (argc=1, argv=0xbfbfd6b8) at Modules/python.c:10
#15 0x8051579 in ___start ()
(gdb) frame 1
#1  0x806ab6c in convertitem (arg=0x80e43ac, p_format=0xbfbfcf74, 
    p_va=0xbfbfcf80, levels=0xbfbfcde8, 
    msgbuf=0xbfbfce68 "\af\027H ŽÇ\021\b\b+\030H") at Python/getargs.c:351
351                     msg = convertsimple(arg, &format, p_va, msgbuf);
(gdb) print p_va
$1 = (va_list *) 0xbfbfcf80
(gdb) frame 2
#2  0x8068a72 in vgetargs1 (args=0x813d18c, format=0x481980b0 "i", 
    p_va=0xbfbfcf80, compat=0) at Python/getargs.c:197
197                     msg = convertitem(PyTuple_GET_ITEM(args, i), &format, p_va,
(gdb) list
192             }
193             
194             for (i = 0; i < len; i++) {
195                     if (*format == '|')
196                             format++;
197                     msg = convertitem(PyTuple_GET_ITEM(args, i), &format, p_va,
198                                       levels, msgbuf);
199                     if (msg) {
200                             seterror(i+1, msg, levels, fname, message);
201                             return 0;
(gdb) print format
$2 = 0x481980b0 "i"
(gdb) frame 3
#3  0x80686fd in PyArg_ParseTuple (args=0x813d18c, format=0x481980ac "si|ii")
    at Python/getargs.c:52
52              retval = vgetargs1(args, format, &va, 0);
(gdb) list
47      {
48              int retval;
49              va_list va;
50              
51              va_start(va, format);
52              retval = vgetargs1(args, format, &va, 0);
53              va_end(va);
54              return retval;
55      }
56      


From itojun@iijlab.net  Sat Aug  4 17:20:05 2001
From: itojun@iijlab.net (Jun-ichiro itojun Hagino)
Date: Sun, 05 Aug 2001 01:20:05 +0900
Subject: [Python-Dev] IPv6 docs
Message-ID: <20010804162005.E1D3B7BB@starfruit.itojun.org>

	Here's a diff for adding documentation for IPv6 funcs in socket module.
	I have no TeX on my laptop these days, so not sure if it compiles clean.
	also if someone could go through my broken english, it would be highly
	appreciated.

itojun


Index: Doc/lib/libftplib.tex
===================================================================
RCS file: /cvsroot/python/python/dist/src/Doc/lib/libftplib.tex,v
retrieving revision 1.32
diff -u -r1.32 libftplib.tex
--- Doc/lib/libftplib.tex	2001/07/06 19:28:48	1.32
+++ Doc/lib/libftplib.tex	2001/08/04 16:18:22
@@ -191,9 +191,9 @@
 
 \begin{methoddesc}{transfercmd}{cmd\optional{, rest}}
 Initiate a transfer over the data connection.  If the transfer is
-active, send a \samp{PORT} command and the transfer command specified
+active, send a \samp{EPRT} or  \samp{PORT} command and the transfer command specified
 by \var{cmd}, and accept the connection.  If the server is passive,
-send a \samp{PASV} command, connect to it, and start the transfer
+send a \samp{EPSV} or \samp{PASV} command, connect to it, and start the transfer
 command.  Either way, return the socket for the connection.
 
 If optional \var{rest} is given, a \samp{REST} command is
Index: Doc/lib/libsocket.tex
===================================================================
RCS file: /cvsroot/python/python/dist/src/Doc/lib/libsocket.tex,v
retrieving revision 1.51
diff -u -r1.51 libsocket.tex
--- Doc/lib/libsocket.tex	2001/01/10 19:34:52	1.51
+++ Doc/lib/libsocket.tex	2001/08/04 16:18:24
@@ -19,6 +19,8 @@
 of information on the details of socket semantics.  For \UNIX, refer
 to the manual pages; for Windows, see the WinSock (or Winsock 2)
 specification.
+For IPv6-ready APIs, readers may want to refer to RFC2553 titled
+\cite{Basic Socket Interface Extensions for IPv6}.
 
 The Python interface is a straightforward transliteration of the
 \UNIX{} system call and library interface for sockets to Python's
@@ -30,20 +32,43 @@
 receive operations is automatic, and buffer length is implicit on send
 operations.
 
-Socket addresses are represented as a single string for the
-\constant{AF_UNIX} address family and as a pair
-\code{(\var{host}, \var{port})} for the \constant{AF_INET} address
+Socket addresses are represented as follows.
+A single string is used for the
+\constant{AF_UNIX} address family.
+A pair
+\code{(\var{host}, \var{port})} is used for the \constant{AF_INET} address
 family, where \var{host} is a string representing
 either a hostname in Internet domain notation like
-\code{'daring.cwi.nl'} or an IP address like \code{'100.50.200.5'},
-and \var{port} is an integral port number.  Other address families are
+\code{'daring.cwi.nl'} or an IPv4 address like \code{'100.50.200.5'},
+and \var{port} is an integral port number.
+For \constant{AF_INET6} address family, a four-tuple \code{(\var{host},
+\var{port}, \var{flowinfo}, \var{scopeid})} is used, where
+\var{flowinfo} and \var{scopeid} represents \code{sin6_flowinfo}
+and \code{sin6_scope_id} member in \constant{struct sockaddr_in6}
+in C.
+For \module{socket} module methods,
+\var{flowinfo} and \var{scopeid} can be omitted just for backward compatibility.
+Note, however, omission of \var{scopeid} can cause problems in manipulating
+scoped IPv6 addresses.
+Other address families are
 currently not supported.  The address format required by a particular
 socket object is automatically selected based on the address family
 specified when the socket object was created.
 
-For IP addresses, two special forms are accepted instead of a host
+For IPv4 addresses, two special forms are accepted instead of a host
 address: the empty string represents \constant{INADDR_ANY}, and the string
 \code{'<broadcast>'} represents \constant{INADDR_BROADCAST}.
+The behavior is not available for IPv6 for backward compatibility,
+therefore, you may want to avoid these if you intend to support IPv6 with
+your Python programs.
+
+If you use a hostname in \var{host} portion of IPv4/v6 socket address,
+the program may present a nondeterministic behavior,
+as Python uses the first address returned from the DNS resolution.
+The socket address will be resolved differently into an actual
+IPv4/v6 address, depending on the results from DNS resolution and/or
+the host configuration.
+For deterministic behavior use a numeric address in \var{host} portion.
 
 All errors raise exceptions.  The normal exceptions for invalid
 argument types and out-of-memory conditions can be raised; errors
@@ -57,7 +82,7 @@
 
 
 \begin{excdesc}{error}
-This exception is raised for socket- or address-related errors.
+This exception is raised for socket-related errors.
 The accompanying value is either a string telling what went wrong or a
 pair \code{(\var{errno}, \var{string})}
 representing an error returned by a system
@@ -66,8 +91,30 @@
 names for the error codes defined by the underlying operating system.
 \end{excdesc}
 
+\begin{excdesc}{herror}
+This exception is raised for address-related errors, for functions that uses
+\var{h_errno} in C API.
+It includes \function{gethostbyname} and \function{gethostbyaddr}.
+The accompanying value is either a string telling what went wrong or a
+pair \code{(\var{h_errno}, \var{string})}
+representing an error returned by a library call.
+\var{string} represents the description of \var{h_errno}, as returned
+by \cfunction{hstrerror} C API.
+\end{excdesc}
+
+\begin{excdesc}{gaierror}
+This exception is raised for address-related errors, for \function{getaddrinfo}
+and \function{getnameinfo}.
+The accompanying value is either a string telling what went wrong or a
+pair \code{(\var{error}, \var{string})}
+representing an error returned by a library call.
+\var{string} represents the description of \var{error}, as returned
+by \cfunction{gai_strerror} C API.
+\end{excdesc}
+
 \begin{datadesc}{AF_UNIX}
 \dataline{AF_INET}
+\dataline{AF_INET6}
 These constants represent the address (and protocol) families,
 used for the first argument to \function{socket()}.  If the
 \constant{AF_UNIX} constant is not defined then this protocol is
@@ -93,6 +140,10 @@
 \dataline{IPPORT_*}
 \dataline{INADDR_*}
 \dataline{IP_*}
+\dataline{IPV6_*}
+\dataline{EAI_*}
+\dataline{AI_*}
+\dataline{NI_*}
 Many constants of these forms, documented in the \UNIX{} documentation on
 sockets and/or the IP protocol, are also defined in the socket module.
 They are generally used in arguments to the \method{setsockopt()} and
@@ -101,6 +152,30 @@
 for a few symbols, default values are provided.
 \end{datadesc}
 
+\begin{funcdesc}{getaddrinfo}{host, port\optional{, family, socktype, proto, flags}}
+Resolves \var{host}/\var{port} argument,
+into a sequence of 5-tuples that contain all the necessary argument for
+the sockets manipulation.
+\var{host} is a domain name,
+an string representation of IPv4/v6 address or \code{None}.
+\var{port} is a string service name (like \code{http}),
+a numeric port number or \code{None}.
+The rest of the arguments are optional and must be numeric if specified.
+For \var{host} and \var{port}, by passing either an empty string or \code{None},
+you can pass \code{NULL} to the C API.
+The \function{getaddrinfo()} function returns a list of the following 5-tuples:
+\code{(\var{family}, \var{socktype}, \var{proto}, \var{canonname}, \var{sockaddr})}.
+\var{family}, \var{socktype}, \var{proto} are all integer and are meant to
+be passed to the \function{socket()} function.
+\var{canonname} is a string representing the canonical name of the \var{host}.
+It can be a numeric IPv4/v6 address when \code{AI_CANONNAME} is specified
+against numeric \var{host}.
+\var{sockaddr} is a tuple describing a socket address, as described above.
+See \code{Lib/httplib.py} and other library files
+for a typical usage of the function.
+\versionadded{X.0}
+\end{funcdesc}
+
 \begin{funcdesc}{getfqdn}{\optional{name}}
 Return a fully qualified domain name for \var{name}.
 If \var{name} is omitted or empty, it is interpreted as the local
@@ -113,26 +188,33 @@
 \end{funcdesc}
 
 \begin{funcdesc}{gethostbyname}{hostname}
-Translate a host name to IP address format.  The IP address is
+Translate a host name to IPv4 address format.  The IPv4 address is
 returned as a string, e.g.,  \code{'100.50.200.5'}.  If the host name
-is an IP address itself it is returned unchanged.  See
+is an IPv4 address itself it is returned unchanged.  See
 \function{gethostbyname_ex()} for a more complete interface.
+\function{gethostbyname()} does not support IPv6 name resolution, and
+\function{getaddrinfo()} should be used instead for IPv4/v6 dual stack support.
 \end{funcdesc}
 
 \begin{funcdesc}{gethostbyname_ex}{hostname}
-Translate a host name to IP address format, extended interface.
+Translate a host name to IPv4 address format, extended interface.
 Return a triple \code{(hostname, aliaslist, ipaddrlist)} where
 \code{hostname} is the primary host name responding to the given
 \var{ip_address}, \code{aliaslist} is a (possibly empty) list of
 alternative host names for the same address, and \code{ipaddrlist} is
-a list of IP addresses for the same interface on the same
+a list of IPv4 addresses for the same interface on the same
 host (often but not always a single address).
+\function{gethostbyname_ex()} does not support IPv6 name resolution, and
+\function{getaddrinfo()} should be used instead for IPv4/v6 dual stack support.
 \end{funcdesc}
 
 \begin{funcdesc}{gethostname}{}
 Return a string containing the hostname of the machine where 
-the Python interpreter is currently executing.  If you want to know the
-current machine's IP address, use \code{gethostbyname(gethostname())}.
+the Python interpreter is currently executing.
+If you want to know the current machine's IP address, you may want to use
+\code{gethostbyname(gethostname())}.
+This operation assumes that there is a valid address-to-host mapping for
+the host, and the assumption does not always hold.
 Note: \function{gethostname()} doesn't always return the fully qualified
 domain name; use \code{gethostbyaddr(gethostname())}
 (see below).
@@ -143,10 +225,19 @@
 \var{ipaddrlist})} where \var{hostname} is the primary host name
 responding to the given \var{ip_address}, \var{aliaslist} is a
 (possibly empty) list of alternative host names for the same address,
-and \var{ipaddrlist} is a list of IP addresses for the same interface
+and \var{ipaddrlist} is a list of IPv4/v6 addresses for the same interface
 on the same host (most likely containing only a single address).
 To find the fully qualified domain name, use the function
 \function{getfqdn()}.
+\function{gethostbyaddr} supports both IPv4 and IPv6.
+\end{funcdesc}
+
+\begin{funcdesc}{getnameinfo}{sockaddr, flags}
+Translate a socket address \var{sockaddr} into a 2-tuple
+\code{(\var{host}, \var{port})}.
+Depending on the settings of \var{flags}, the result can contain
+a fully-qualified domain name or numeric address representation in \var{host}.
+Similarly, \var{port} can contain a string port name or a numeric port number.
 \end{funcdesc}
 
 \begin{funcdesc}{getprotobyname}{protocolname}
@@ -166,7 +257,7 @@
 
 \begin{funcdesc}{socket}{family, type\optional{, proto}}
 Create a new socket using the given address family, socket type and
-protocol number.  The address family should be \constant{AF_INET} or
+protocol number.  The address family should be \constant{AF_INET}, \constant{AF_INET6} or
 \constant{AF_UNIX}.  The socket type should be \constant{SOCK_STREAM},
 \constant{SOCK_DGRAM} or perhaps one of the other \samp{SOCK_} constants.
 The protocol number is usually zero and may be omitted in that case.
@@ -209,7 +300,7 @@
 \end{funcdesc}
 
 \begin{funcdesc}{inet_aton}{ip_string}
-Convert an IP address from dotted-quad string format
+Convert an IPv4 address from dotted-quad string format
 (e.g.\ '123.45.67.89') to 32-bit packed binary format, as a string four
 characters in length.
 
@@ -217,14 +308,17 @@
 and needs objects of type \ctype{struct in_addr}, which is the C type
 for the 32-bit packed binary this function returns.
 
-If the IP address string passed to this function is invalid,
+If the IPv4 address string passed to this function is invalid,
 \exception{socket.error} will be raised. Note that exactly what is
 valid depends on the underlying C implementation of
 \cfunction{inet_aton()}.
+
+\function{inet_aton} does not support IPv6, and
+\function{getnameinfo()} should be used instead for IPv4/v6 dual stack support.
 \end{funcdesc}
 
 \begin{funcdesc}{inet_ntoa}{packed_ip}
-Convert a 32-bit packed IP address (a string four characters in
+Convert a 32-bit packed IPv4 address (a string four characters in
 length) to its standard dotted-quad string representation
 (e.g. '123.45.67.89').
 
@@ -234,6 +328,9 @@
 
 If the string passed to this function is not exactly 4 bytes in
 length, \exception{socket.error} will be raised.
+
+\function{inet_ntoa} does not support IPv6, and
+\function{getnameinfo()} should be used instead for IPv4/v6 dual stack support.
 \end{funcdesc}
 
 \begin{datadesc}{SocketType}
@@ -306,14 +403,14 @@
 
 \begin{methoddesc}[socket]{getpeername}{}
 Return the remote address to which the socket is connected.  This is
-useful to find out the port number of a remote IP socket, for instance.
+useful to find out the port number of a remote IPv4/v6 socket, for instance.
 (The format of the address returned depends on the address family ---
 see above.)  On some systems this function is not supported.
 \end{methoddesc}
 
 \begin{methoddesc}[socket]{getsockname}{}
 Return the socket's own address.  This is useful to find out the port
-number of an IP socket, for instance.
+number of an IPv4/v6 socket, for instance.
 (The format of the address returned depends on the address family ---
 see above.)
 \end{methoddesc}
@@ -413,7 +510,7 @@
 
 \subsection{Example \label{socket-example}}
 
-Here are two minimal example programs using the TCP/IP protocol:\ a
+Here are four minimal example programs using the TCP/IP protocol:\ a
 server that echoes all data that it receives back (servicing only one
 client), and a client using it.  Note that a server must perform the
 sequence \function{socket()}, \method{bind()}, \method{listen()},
@@ -424,6 +521,8 @@
 socket it is listening on but on the new socket returned by
 \method{accept()}.
 
+The first two examples support IPv4 only.
+
 \begin{verbatim}
 # Echo server program
 import socket
@@ -450,6 +549,82 @@
 PORT = 50007              # The same port as used by the server
 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
 s.connect((HOST, PORT))
+s.send('Hello, world')
+data = s.recv(1024)
+s.close()
+print 'Received', `data`
+\end{verbatim}
+
+The next two examples are identical to the above two, but support both
+IPv4 and IPv6.
+The server side will listen to the first address family available
+(it should listen to both instead).
+On most of IPv6-ready systems, IPv6 will take precedence
+and the server may not accept IPv4 traffic.
+The client side will try to connect to the all addresses returned as a result
+of the name resolution, and sends traffic to the first one connected
+successfully.
+
+\begin{verbatim}
+# Echo server program
+import socket
+import sys
+
+HOST = ''                 # Symbolic name meaning the local host
+PORT = 50007              # Arbitrary non-privileged port
+s = None
+for res in socket.getaddrinfo('', PORT, socket.AF_UNSPEC, socket.SOCK_STREAM, 0, socket.AI_PASSIVE):
+    af, socktype, proto, canonname, sa = res
+    try:
+	s = socket.socket(af, socktype, proto)
+    except socket.error, msg:
+	s = None
+	continue
+    try:
+	s.bind(sa)
+	s.listen(1)
+    except socket.error, msg:
+	s.close()
+	s = None
+	continue
+    break
+if s is None:
+    print 'could not open socket'
+    sys.exit(1)
+conn, addr = s.accept()
+print 'Connected by', addr
+while 1:
+    data = conn.recv(1024)
+    if not data: break
+    conn.send(data)
+conn.close()
+\end{verbatim}
+
+\begin{verbatim}
+# Echo client program
+import socket
+import sys
+
+HOST = 'daring.cwi.nl'    # The remote host
+PORT = 50007              # The same port as used by the server
+s = None
+for res in socket.getaddrinfo(HOST, PORT, socket.AF_UNSPEC, socket.SOCK_STREAM):
+    af, socktype, proto, canonname, sa = res
+    try:
+	s = socket.socket(af, socktype, proto)
+    except socket.error, msg:
+	s = None
+	continue
+    try:
+	s.connect(sa)
+    except socket.error, msg:
+	s.close()
+	s = None
+	continue
+    break
+if s is None:
+    print 'could not open socket'
+    sys.exit(1)
 s.send('Hello, world')
 data = s.recv(1024)
 s.close()


From itojun@iijlab.net  Sat Aug  4 17:20:38 2001
From: itojun@iijlab.net (Jun-ichiro itojun Hagino)
Date: Sun, 05 Aug 2001 01:20:38 +0900
Subject: [Python-Dev] socketmodule fix for IPv6 (mostly additional symbols)
Message-ID: <20010804162038.144F47BB@starfruit.itojun.org>

Index: Modules/socketmodule.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Modules/socketmodule.c,v
retrieving revision 1.160
diff -u -r1.160 socketmodule.c
--- Modules/socketmodule.c	2001/08/03 10:02:29	1.160
+++ Modules/socketmodule.c	2001/08/04 16:18:28
@@ -2433,6 +2433,7 @@
 	PyOS_snprintf(pbuf, sizeof(pbuf), "%d", port);
 	memset(&hints, 0, sizeof(hints));
 	hints.ai_family = PF_UNSPEC;
+	hints.ai_socktype = SOCK_DGRAM;	/* make numeric port happy */
 	error = getaddrinfo(hostp, pbuf, &hints, &res);
 	if (error) {
 		PyGAI_Err(error);
@@ -3135,6 +3136,9 @@
 #else
 	insint(d, "IPPROTO_IP", 0);
 #endif
+#ifdef	IPPROTO_HOPOPTS
+	insint(d, "IPPROTO_HOPOPTS", IPPROTO_HOPOPTS);
+#endif
 #ifdef	IPPROTO_ICMP
 	insint(d, "IPPROTO_ICMP", IPPROTO_ICMP);
 #else
@@ -3146,6 +3150,12 @@
 #ifdef	IPPROTO_GGP
 	insint(d, "IPPROTO_GGP", IPPROTO_GGP);
 #endif
+#ifdef	IPPROTO_IPV4
+	insint(d, "IPPROTO_IPV4", IPPROTO_IPV4);
+#endif
+#ifdef	IPPROTO_IPIP
+	insint(d, "IPPROTO_IPIP", IPPROTO_IPIP);
+#endif
 #ifdef	IPPROTO_TCP
 	insint(d, "IPPROTO_TCP", IPPROTO_TCP);
 #else
@@ -3174,12 +3184,54 @@
 #ifdef	IPPROTO_TP
 	insint(d, "IPPROTO_TP", IPPROTO_TP);
 #endif
+#ifdef	IPPROTO_IPV6
+	insint(d, "IPPROTO_IPV6", IPPROTO_IPV6);
+#endif
+#ifdef	IPPROTO_ROUTING
+	insint(d, "IPPROTO_ROUTING", IPPROTO_ROUTING);
+#endif
+#ifdef	IPPROTO_FRAGMENT
+	insint(d, "IPPROTO_FRAGMENT", IPPROTO_FRAGMENT);
+#endif
+#ifdef	IPPROTO_RSVP
+	insint(d, "IPPROTO_RSVP", IPPROTO_RSVP);
+#endif
+#ifdef	IPPROTO_GRE
+	insint(d, "IPPROTO_GRE", IPPROTO_GRE);
+#endif
+#ifdef	IPPROTO_ESP
+	insint(d, "IPPROTO_ESP", IPPROTO_ESP);
+#endif
+#ifdef	IPPROTO_AH
+	insint(d, "IPPROTO_AH", IPPROTO_AH);
+#endif
+#ifdef	IPPROTO_MOBILE
+	insint(d, "IPPROTO_MOBILE", IPPROTO_MOBILE);
+#endif
+#ifdef	IPPROTO_ICMPV6
+	insint(d, "IPPROTO_ICMPV6", IPPROTO_ICMPV6);
+#endif
+#ifdef	IPPROTO_NONE
+	insint(d, "IPPROTO_NONE", IPPROTO_NONE);
+#endif
+#ifdef	IPPROTO_DSTOPTS
+	insint(d, "IPPROTO_DSTOPTS", IPPROTO_DSTOPTS);
+#endif
 #ifdef	IPPROTO_XTP
 	insint(d, "IPPROTO_XTP", IPPROTO_XTP);
 #endif
 #ifdef	IPPROTO_EON
 	insint(d, "IPPROTO_EON", IPPROTO_EON);
 #endif
+#ifdef	IPPROTO_PIM
+	insint(d, "IPPROTO_PIM", IPPROTO_PIM);
+#endif
+#ifdef	IPPROTO_IPCOMP
+	insint(d, "IPPROTO_IPCOMP", IPPROTO_IPCOMP);
+#endif
+#ifdef	IPPROTO_VRRP
+	insint(d, "IPPROTO_VRRP", IPPROTO_VRRP);
+#endif
 #ifdef	IPPROTO_BIP
 	insint(d, "IPPROTO_BIP", IPPROTO_BIP);
 #endif
@@ -3242,7 +3294,7 @@
 	insint(d, "INADDR_NONE", 0xffffffff);
 #endif
 
-	/* IP [gs]etsockopt options */
+	/* IPv4 [gs]etsockopt options */
 #ifdef	IP_OPTIONS
 	insint(d, "IP_OPTIONS", IP_OPTIONS);
 #endif
@@ -3290,6 +3342,26 @@
 #endif
 #ifdef	IP_MAX_MEMBERSHIPS
 	insint(d, "IP_MAX_MEMBERSHIPS", IP_MAX_MEMBERSHIPS);
+#endif
+
+	/* IPv6 [gs]etsockopt options, defined in RFC2553 */
+#ifdef	IPV6_JOIN_GROUP
+	insint(d, "IPV6_JOIN_GROUP", IPV6_JOIN_GROUP);
+#endif
+#ifdef	IPV6_LEAVE_GROUP
+	insint(d, "IPV6_LEAVE_GROUP", IPV6_LEAVE_GROUP);
+#endif
+#ifdef	IPV6_MULTICAST_HOPS
+	insint(d, "IPV6_MULTICAST_HOPS", IPV6_MULTICAST_HOPS);
+#endif
+#ifdef	IPV6_MULTICAST_IF
+	insint(d, "IPV6_MULTICAST_IF", IPV6_MULTICAST_IF);
+#endif
+#ifdef	IPV6_MULTICAST_LOOP
+	insint(d, "IPV6_MULTICAST_LOOP", IPV6_MULTICAST_LOOP);
+#endif
+#ifdef	IPV6_UNICAST_HOPS
+	insint(d, "IPV6_UNICAST_HOPS", IPV6_UNICAST_HOPS);
 #endif
 
 	/* TCP options */


From cgw@alum.mit.edu  Sat Aug  4 18:02:04 2001
From: cgw@alum.mit.edu (Charles G Waldman)
Date: Sat, 4 Aug 2001 12:02:04 -0500
Subject: [Python-Dev] [offtopic] /usr/bin/env vs linux
In-Reply-To: <E15T3Dd-0005cV-00@mail.python.org>
References: <E15T3Dd-0005cV-00@mail.python.org>
Message-ID: <15212.10892.890406.103990@nyx.dyndns.org>

Greg Ward wrote:
 > 
 > Completely off-topic: the "/usr/bin/env" hack doesn't work with
 > command-line arguments under Linux:
 > 
 > Grumble.  Andrew and I tried to track this down once, and found
 > suspicious-looking code in the kernel (ie., I don't think it's the fault
 > of GNU env).

I've noticed this as well.  Do you have any pointers to the suspicious
kernel code?  Did you bring this up on the linux kernel development
list?


From guido@zope.com  Sat Aug  4 18:44:51 2001
From: guido@zope.com (Guido van Rossum)
Date: Sat, 04 Aug 2001 13:44:51 -0400
Subject: [Python-Dev] [offtopic] /usr/bin/env vs linux
In-Reply-To: Your message of "Sat, 04 Aug 2001 12:02:04 CDT."
 <15212.10892.890406.103990@nyx.dyndns.org>
References: <E15T3Dd-0005cV-00@mail.python.org>
 <15212.10892.890406.103990@nyx.dyndns.org>
Message-ID: <200108041744.NAA11809@cj20424-a.reston1.va.home.com>

> Greg Ward wrote:
>  > 
>  > Completely off-topic: the "/usr/bin/env" hack doesn't work with
>  > command-line arguments under Linux:
>  > 
>  > Grumble.  Andrew and I tried to track this down once, and found
>  > suspicious-looking code in the kernel (ie., I don't think it's the fault
>  > of GNU env).
> 
> I've noticed this as well.  Do you have any pointers to the suspicious
> kernel code?  Did you bring this up on the linux kernel development
> list?

This is age-old.  The kernel only passes on one argument (and it has
to be of limited length).  In "/usr/bin/env python", "python" is that
one argument.  I guess there's a really good reason (like resource
constraints) for this or it would've been fixed ages ago, wouldn't it?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Sat Aug  4 19:07:21 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sat, 4 Aug 2001 14:07:21 -0400 (EDT)
Subject: [Python-Dev] IPv6 docs
In-Reply-To: <20010804162005.E1D3B7BB@starfruit.itojun.org>
References: <20010804162005.E1D3B7BB@starfruit.itojun.org>
Message-ID: <15212.14809.33733.24175@cj42289-a.reston1.va.home.com>

Jun-ichiro itojun Hagino writes:
 > 	Here's a diff for adding documentation for IPv6 funcs in
 > 	socket module.  I have no TeX on my laptop these days, so not
 > 	sure if it compiles clean.  also if someone could go through
 > 	my broken english, it would be highly appreciated.

  If you could post this as a separate patch on SourceForge and label
it as documentation, I should be able to get to it this coming week.
(If it just sits in my inbox, it may take a lot longer. ;)
  Thanks!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From m@moshez.org  Sat Aug  4 19:06:51 2001
From: m@moshez.org (Moshe Zadka)
Date: Sat, 04 Aug 2001 21:06:51 +0300
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: <20010804110653.A910@gerg.ca>
References: <20010804110653.A910@gerg.ca>, <20010802150109.B2033@netthink.co.uk> <200108031233.IAA01557@cj20424-a.reston1.va.home.com>
Message-ID: <E15T5p5-0006uS-00@darjeeling>

On Sat, 4 Aug 2001, Greg Ward <gward@python.net> wrote:

> Completely off-topic: the "/usr/bin/env" hack doesn't work with
> command-line arguments under Linux:
> 
> $ cat t.py 
> #!/usr/bin/env python -i
> 
> print "hello"
> 
> $ ./t.py  
> /usr/bin/env: python -i: No such file or directory

That's what POSIX mandates, IIRC. Blame UNIX for using such an ugly
hack for shebang lines. It's probably better to use #!/usr/bin/python
and use binfmt_misc under linux with the correct echo > something/in/proc
magic to alias it to the correct interpreter.
-- 
Moshe Zadka - http://moshez.geek
(if you're not cool, http://moshez.org is still working)


From guido@zope.com  Sat Aug  4 19:30:09 2001
From: guido@zope.com (Guido van Rossum)
Date: Sat, 04 Aug 2001 14:30:09 -0400
Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler
In-Reply-To: Your message of "Sat, 04 Aug 2001 05:01:07 EDT."
 <LNBBLJKPBEHFEDALKOLCAEHCLDAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCAEHCLDAA.tim.one@home.com>
Message-ID: <200108041830.OAA12163@cj20424-a.reston1.va.home.com>

> BTW, do we actually need *any* auxiliary data structure here?  Looks
> to me like all these giant pylist thingies could be tossed in favor
> of making shallow copies of the input ifdlist etc list arguments.
> The giant pylists don't *appear* to accomplish anything except cache
> the result of a PyObject_AsFileDescriptor() call per object.  But if
> that's not an expensive call (doesn't look expensive to me), it's
> not worth all this trouble no matter how spelled.

Yes, I think we do need this.  PyObject_AsFileDescriptor() calls the
fileno() method which could be a Python call.

Maybe we could use a more sophisticated construct where we use a short
array allocated on the stack for lists of fewer than 100 (say) files
but use malloc for larger ones?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From ping@lfw.org  Sat Aug  4 19:37:52 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Sat, 4 Aug 2001 11:37:52 -0700 (PDT)
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: <200108010306.XAA18133@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.LNX.4.32.0108041129350.579-100000@ziggy.localdomain.fake>

On Tue, 31 Jul 2001, Guido van Rossum wrote:
> You misunderstand.  You proposed a few lines that would automatically
> do this in site.py, which is always imported.  I propose to add those
> same lines to cgi.py, so that any code that imports cgi.py
> *automatically* has he feature enabled.  I assume that all CGI scripts
> have an import of cgi.

Sorry that i misunderstood.  This sounds reasonable to me (though it
seems your opinions may have changed since you wrote this).

> I don't want Python to become a wannabe CGI language [...] Python
> is a decent language for CGI, but Perl is the established standard
> and then there's PHP which also has way more users than Python.

This attitude surprised me a little and sounded somewhat sad.  "Perl
is the established standard"?  Aw.  I say if people want a better tool,
they should use it.  The fact that Perl has more users than Python
doesn't make us give up on Python for system administration or text
processing or database access, even though Perl can do all these
things.  The fact is Python's quite good. :)

> I don't want to make choices that benefit CGI programmers to the
> detriment of others (CGI is actually a pretty lame way of producing
> active web content).

I guess i have a hard time seeing the "detriment" part.  What great
detriment to others is being caused here, by a feature that only
affects CGI to begin with?


-- ?!ng



From martin@loewis.home.cs.tu-berlin.de  Sat Aug  4 21:52:12 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sat, 4 Aug 2001 22:52:12 +0200
Subject: [Python-Dev] [offtopic] /usr/bin/env vs linux
Message-ID: <200108042052.f74KqCn02631@mira.informatik.hu-berlin.de>

> This is age-old.  The kernel only passes on one argument (and it has
> to be of limited length).  In "/usr/bin/env python", "python" is
> that one argument.  I guess there's a really good reason (like
> resource constraints) for this or it would've been fixed ages ago,
> wouldn't it?

It's in fs/binfmt_script.c, which I wrote about five years ago. The
specific code is around

	/*
	 * OK, we've parsed out the interpreter name and
	 * (optional) argument.
	 * Splice in (1) the interpreter's name for argv[0]
	 *           (2) (optional) argument to interpreter
	 *           (3) filename of shell script (replace argv[0])
	 *
	 * This is done in reverse order, because of how the
	 * user environment and arguments are stored.
	 */

AFAICS, there is no good reason to limit this to a single argument,
except for laziness. I copied this fragment from the earlier #!
implementation by tytso, and did not question it.

OTOH, the size constraint comes from BINPRM_BUF_SIZE, which is
128. The kernel only offers this much data initially to a binary
format module for format detection, and the binfmt_script driver
doesn't attempt to read any further data from disk (which it could, if
it wanted to).

It seems that Single Unix does not mandate any behaviour in this
respect; #! is still but a common extension. On Solaris, #! is
documented in exec(2) as

#! pathname [arg ]

so I guess Linux was just following SunOS once more here.

Regards,
Martin


From ping@lfw.org  Sat Aug  4 21:53:24 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Sat, 4 Aug 2001 13:53:24 -0700 (PDT)
Subject: [Python-Dev] Re: cgitb.py for Python 2.2
In-Reply-To: <200108040732.DAA10879@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.LNX.4.32.0108041236340.579-100000@ziggy.localdomain.fake>

On Sat, 4 Aug 2001, Guido van Rossum wrote:
> Give it up, Ping.  cgitb writes the tb to stdout, not to stderr, and
> that's enough of a difference in behavior to require explicit
> enabling.  It's not worth your energy to try and convince us.

Okay.  I defer to your sense of judgement.  (My last message was
composed before i read your message quoted above.)


-- ?!ng



From ping@lfw.org  Sat Aug  4 22:11:38 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Sat, 4 Aug 2001 14:11:38 -0700 (PDT)
Subject: [Python-Dev] Proposed new module for the Python library
In-Reply-To: <20010804111847.B910@gerg.ca>
Message-ID: <Pine.LNX.4.32.0108041410130.579-100000@ziggy.localdomain.fake>

On 02 August 2001, Eric S. Raymond said:
> There is a common `compiler-like' pattern in Unix scripts which is useful
> for translation utilities of all sorts.

On Sat, 4 Aug 2001, Greg Ward wrote:
> Sounds like a useful tool to have around.  I'm not sure the name
> 'ccframe' conjures up images of the above paragraph, but I'm not sure
> what name *does*.  This is the hardest sort of software to name!

I too find the name fairly cryptic.  "fileargs" perhaps?

But it would be better to think about merging this and fileinput.


-- ?!ng



From martin@loewis.home.cs.tu-berlin.de  Sat Aug  4 23:28:32 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 5 Aug 2001 00:28:32 +0200
Subject: [Python-Dev] IPv6 docs
Message-ID: <200108042228.f74MSWo10569@mira.informatik.hu-berlin.de>

> If you could post this as a separate patch on SourceForge and label
> it as documentation, I should be able to get to it this coming week.

I've applied itojun's patch, with slight modifications.

Itojun, I have a few questions on this documentation, though:

You write that socket.herror and socket.gaierror may both take either
a string or a pair of error number/message. Is this correct, or was
that a cut-n-paste error from socket.error?

You also write that socket.gethostbyname may raise socket.herror. Is
this true? It seems that you'll always get socket.gaierror instead.
I've changed it to say that gethostbyname_ex may raise herrors.

Anyway, thanks for the patch.

Regards,
Martin


From martin@loewis.home.cs.tu-berlin.de  Sat Aug  4 23:40:01 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 5 Aug 2001 00:40:01 +0200
Subject: [Python-Dev] socketmodule fix for IPv6 (mostly additional symbols)
Message-ID: <200108042240.f74Me1i10657@mira.informatik.hu-berlin.de>

Hi itojun,

Thanks for your patch; I just applied it. Please don't send patches to
python-dev, though. If you cannot use SourceForge for some reason,
please send them to me instead. Note that SF really is the preferred
channel for these things.

Regards,
Martin


From gstein@lyra.org  Sun Aug  5 01:35:21 2001
From: gstein@lyra.org (Greg Stein)
Date: Sat, 4 Aug 2001 17:35:21 -0700
Subject: [Python-Dev] Re: WebDAV in python 2.2?
In-Reply-To: <E15SwHg-0004Bs-00@darjeeling>; from m@moshez.org on Sat, Aug 04, 2001 at 10:55:44AM +0300
References: <20010803121254.B29583@lyra.org>, <mailman.996828301.19910.clpa-moderators@python.org> <200108031254.IAA01818@cj20424-a.reston1.va.home.com> <20010803121254.B29583@lyra.org> <E15SwHg-0004Bs-00@darjeeling>
Message-ID: <20010804173521.B31732@lyra.org>

On Sat, Aug 04, 2001 at 10:55:44AM +0300, Moshe Zadka wrote:
> On Fri, 3 Aug 2001 12:12:54 -0700, Greg Stein <gstein@lyra.org> wrote:
> 
> > That is at least a start on what needs to happen. We can flesh it out with a
> > PEP or by checking in the modules.
> > (Moshe and I talked about httpauth a bit at the last Python Conference; I
> >  know he is interested in working on something like that)
> 
> Let me just clarify what we talked about: today, the HTTPConnection already
> know how to respond to code 100.

Euh... no, it doesn't. It *should*, but it doesn't today. To do it right, a
client should be able to install a callback for when the 100-Continue is
sent by the server.

[ in fact, httplib's HTTPConnection and HTTPResponse should one day be
  reworked further to support a callback style model, passing the response
  object; that model allows for HTTP pipelining ]

> urllib2 has a very nifty mechanism of handling
> all kinds of errors. I have a feeling the responsiblity split is not correct,
> and I want to think about some kind of better responsibility split.

As I mentioned above, I'll be writing up a PEP discussing this stuff (since
it is non-trivial, and I'm guessing there is more than just a couple
people's input). Suggestions can go in there.

I realized that it shouldn't be "httpauth" but something like "httpext"
(extended stuff). We also want proxy support in there. Note that proxy
support is connection-based. Authentication is per-request, but we can
attach "defaults" for a connection.

urllib(2) would simply use the richer HTTPConnection class, shedding some of
its work. urllib is about handling a specific URL (of many varieties, not
just HTTP), while the connection is about a connection to a server for
multiple requests. We can bulk up the connection (IMO, via subclassing), and
keep the rich facilities of urllib's "single" request basis. (yes, urllib
caches and has richer error handling and stuff, but that is beside the point
of beefing up the connection for *all* HTTP clients)

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From gstein@lyra.org  Sun Aug  5 01:38:56 2001
From: gstein@lyra.org (Greg Stein)
Date: Sat, 4 Aug 2001 17:38:56 -0700
Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?]
In-Reply-To: <20010803202227.A31091@lyra.org>; from gstein@lyra.org on Fri, Aug 03, 2001 at 08:22:27PM -0700
References: <20010803202227.A31091@lyra.org>
Message-ID: <20010804173856.C31732@lyra.org>

> From: Guido van Rossum <guido@zope.com>
> Subject: Re: [Python-Dev] Re: WebDAV in python 2.2?
> To: Greg Stein <gstein@lyra.org>
> Date: Fri, 03 Aug 2001 15:18:37 -0400
>...
> > I'm not sure how we normally write tests for network-based modules, but it
> > shouldn't be too hard to have them run against test.webdav.org.
> 
> Can someone help Greg with his question about testing network modules?
> Generally the test suite doesn't assume internet access; network code
> is "tested" using a second thread and localhost.

Quite doable, but we don't really have a DAV server that davlib.py could hit
via localhost. I'm sure somebody could jury-rig a simple responder, though.
Avoiding the bailing-wire-and-glue bit was the reason for suggesting
test.webdav.org.

Hmm. Some kind of subclass of BaseHTTPServer or something could probably
just send back canned responses.

Note that even httplib.py doesn't attempt a localhost connection for
testing.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From guido@zope.com  Sun Aug  5 03:19:57 2001
From: guido@zope.com (Guido van Rossum)
Date: Sat, 04 Aug 2001 22:19:57 -0400
Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?]
In-Reply-To: Your message of "Sat, 04 Aug 2001 17:38:56 PDT."
 <20010804173856.C31732@lyra.org>
References: <20010803202227.A31091@lyra.org>
 <20010804173856.C31732@lyra.org>
Message-ID: <200108050219.WAA12942@cj20424-a.reston1.va.home.com>

> Quite doable, but we don't really have a DAV server that davlib.py could hit
> via localhost. I'm sure somebody could jury-rig a simple responder, though.
> Avoiding the bailing-wire-and-glue bit was the reason for suggesting
> test.webdav.org.

But we don't control how test.webdav.org will respond two (or ten)
years from now.  Plus there are firewalls, you don't want your modem
to start dialing, etc., etc.

> Hmm. Some kind of subclass of BaseHTTPServer or something could probably
> just send back canned responses.

Exactly.  That's how SocketServer tests itself these days.

> Note that even httplib.py doesn't attempt a localhost connection for
> testing.

The test in the test suite doesn't use the network at all.  The test
in the module is not used, exactly for this reason: it relies on
accessibility of external resources.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@home.com  Sun Aug  5 04:17:50 2001
From: tim.one@home.com (Tim Peters)
Date: Sat, 4 Aug 2001 23:17:50 -0400
Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler
In-Reply-To: <200108041830.OAA12163@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEINLDAA.tim.one@home.com>

[Guido, on reworking select()]
> Yes, I think we do need this.  PyObject_AsFileDescriptor() calls the
> fileno() method which could be a Python call.

Ya, but for me to believe that *matters*, I have to believe people worried
about high-performance select are passing objects with Python-implemented
fileno methods to begin with.  That's quite a stretch, isn't it?  They have
to pay for fileno() calls regardless, the question is only whether they pay
it once or twice per object.

> Maybe we could use a more sophisticated construct where we use a short
> array allocated on the stack for lists of fewer than 100 (say) files
> but use malloc for larger ones?

Another idea:  screw it <wink>.  We have to allocate three lists anyway to
hold the results.  Allocate them at the start instead of the end, and then
they can do double-duty, holding the cached fileno() results across the
select() call.

Fleshing it out a little,

def list2set(inputlist, fdset):
    "Return (list of file descriptors, biggest fd seen); set fdset slots."
    n = len(inputlist)
    result = [None] * n
    if _MSC_VER:
        biggest = 0
    else:
        biggest = -1
     for i in range(n):
        obj = inputlist[i]
        fd = PyObject_AsFileDescriptor(obj)
        if fd == -1:
            return NULL, -1
        result[i] = obj, fd
        FD_SET(fd, fdset)
        if not _MSC_VER:
            biggest = max(biggest, fd)
    return result, biggest

# iolist is the result of an earlier list2set() call.  It gets shrunk
# to hold just the objects corresponding to the fds in fdset.

def set2list(iolist, fdset):
    lastindex = 0
    for obj, fd in iolist:
        if FD_ISSET(fd, fdset):
            iolist[lastindex] = obj
            lastindex += 1
    if lastindex < len(iolist):
        del iolist[lastindex:]

So it still does the perversely <wink> optimized caching, but no stack space
required, no free() calls, only the 3 malloc() calls that are needed anyway,
and up to 3 realloc() calls to shrink the output lists.  It does create a
bunch of 2-tuples, but they're on a free list and come cheap.



From itojun@iijlab.net  Sun Aug  5 05:57:26 2001
From: itojun@iijlab.net (Jun-ichiro itojun Hagino)
Date: Sun, 05 Aug 2001 13:57:26 +0900
Subject: [Python-Dev] socketmodule fix for IPv6 (mostly additional symbols)
In-Reply-To: martin's message of Sun, 05 Aug 2001 00:40:01 +0200.
 <200108042240.f74Me1i10657@mira.informatik.hu-berlin.de>
Message-ID: <20010805045727.E78007BB@starfruit.itojun.org>

>Thanks for your patch; I just applied it. Please don't send patches to
>python-dev, though. If you cannot use SourceForge for some reason,
>please send them to me instead. Note that SF really is the preferred
>channel for these things.

	i see, next time i'll do so.  thanks!

itojun


From itojun@iijlab.net  Sun Aug  5 05:58:31 2001
From: itojun@iijlab.net (Jun-ichiro itojun Hagino)
Date: Sun, 05 Aug 2001 13:58:31 +0900
Subject: [Python-Dev] IPv6 docs
In-Reply-To: martin's message of Sun, 05 Aug 2001 00:28:32 +0200.
 <200108042228.f74MSWo10569@mira.informatik.hu-berlin.de>
Message-ID: <20010805045831.9A7F87BB@starfruit.itojun.org>

>You write that socket.herror and socket.gaierror may both take either
>a string or a pair of error number/message. Is this correct, or was
>that a cut-n-paste error from socket.error?

	yes, it is a cut-n-paste error.  herror and gaierror is always a
	2-tuple.

>You also write that socket.gethostbyname may raise socket.herror. Is
>this true? It seems that you'll always get socket.gaierror instead.
>I've changed it to say that gethostbyname_ex may raise herrors.

	with C API gethostbyname() should return h_error.  Python code
	internaly uses getnameinfo/getaddrinfo, so it returns socket.gaierror.
	you right.

itojun


From m@moshez.org  Sun Aug  5 06:07:19 2001
From: m@moshez.org (Moshe Zadka)
Date: Sun, 05 Aug 2001 08:07:19 +0300
Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?]
In-Reply-To: <20010804173856.C31732@lyra.org>
References: <20010804173856.C31732@lyra.org>, <20010803202227.A31091@lyra.org>
Message-ID: <E15TG8F-0000yq-00@darjeeling>

On Sat, 4 Aug 2001 17:38:56 -0700, Greg Stein <gstein@lyra.org> wrote:
 
> Quite doable, but we don't really have a DAV server that davlib.py could hit
> via localhost.

http://webdav.de has one.

> Note that even httplib.py doesn't attempt a localhost connection for
> testing.

But it should ;-)
-- 
Moshe Zadka - http://moshez.geek
(if you're not cool, http://moshez.org is still working)


From rnd@onego.ru  Sun Aug  5 08:54:14 2001
From: rnd@onego.ru (Roman Suzi)
Date: Sun, 5 Aug 2001 11:54:14 +0400 (MSD)
Subject: [Python-Dev] '*' instead of 'yield'?
Message-ID: <Pine.LNX.4.30.0108051150520.989-100000@rnd.onego.ru>

Hello!

Maybe it will be nicier to have unary * instead of 'yield'
for generators?

Example I already posted to c.l.p:

from __future__ import generators
import calendar, string
def calt(year, month):
    month_cal = calendar.monthcalendar(year, month)
    * """<B>%s.%s</B>""" % (month, year)
    * """<TABLE BORDER=1>"""
    for week in month_cal:
      * """\n<TR>"""
      for day in week:
        * """<TD>%s</TD>""" % (day or "&nbsp;")
      * """</TR>"""
    * """</TABLE>"""

print string.join(calt(2001, 8), "")

(This will also enable yields from expressions)

Sincerely yours, Roman Suzi
-- 
_/ Russia _/ Karelia _/ Petrozavodsk _/ rnd@onego.ru _/
_/ Sunday, August 05, 2001 _/ Powered by Linux RedHat 6.2 _/
_/ "By all means, let's not confuse ourselves with the facts!" _/




From fdrake@acm.org  Sun Aug  5 16:24:03 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sun, 5 Aug 2001 11:24:03 -0400 (EDT)
Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?]
In-Reply-To: <E15TG8F-0000yq-00@darjeeling>
References: <20010804173856.C31732@lyra.org>
 <20010803202227.A31091@lyra.org>
 <E15TG8F-0000yq-00@darjeeling>
Message-ID: <15213.25875.726178.186594@cj42289-a.reston1.va.home.com>

Moshe Zadka writes:
 > http://webdav.de has one.

  The pages there are inconsistent in their claims of status and
version number, so it's not clear what the actual features are.  I
suspect that only the front page is up to date.
  Attempting to actually download the software causes an
authentication dialog to pop up, so something is probably screwed up
as well.
  The pages look slick, though.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From mwh@python.net  Sun Aug  5 19:11:18 2001
From: mwh@python.net (Michael Hudson)
Date: 05 Aug 2001 14:11:18 -0400
Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler
In-Reply-To: "Tim Peters"'s message of "Sat, 4 Aug 2001 23:17:50 -0400"
References: <LNBBLJKPBEHFEDALKOLCMEINLDAA.tim.one@home.com>
Message-ID: <2mhevm4bsp.fsf@starship.python.net>

"Tim Peters" <tim.one@home.com> writes:

> [Guido, on reworking select()]
> > Yes, I think we do need this.  PyObject_AsFileDescriptor() calls the
> > fileno() method which could be a Python call.
> 
> Ya, but for me to believe that *matters*, I have to believe people worried
> about high-performance select are passing objects with Python-implemented
> fileno methods to begin with.  That's quite a stretch, isn't it?  They have
> to pay for fileno() calls regardless, the question is only whether they pay
> it once or twice per object.

Posted admittedly without checking any facts:

(1) Doesn't the Python fileno() method issue leave you open to
ridiculous fileno() methods that mutate the list passed in to
select()?

(2) AIUI, if you really care about performance you should be using
poll() anyway (tho' obviously this depends on your platform having
poll() in the first place, and I don't have the faintest idea how
widespread it is).

Cheers,
M.

-- 
  The ultimate laziness is not using Perl.  That saves you so much
  work you wouldn't believe it if you had never tried it.
                                        -- Erik Naggum, comp.lang.lisp


From tim.one@home.com  Sun Aug  5 23:50:06 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 5 Aug 2001 18:50:06 -0400
Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler
In-Reply-To: <2mhevm4bsp.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEKOLDAA.tim.one@home.com>

[Michael Hudson]
> ...
> (1) Doesn't the Python fileno() method issue leave you open to
> ridiculous fileno() methods that mutate the list passed in to
> select()?

Yes, but that's par for the course, and selectmodule is suitably paranoid
about it (unfortunately, this is *so* common in the implementation that it's
rarely mentioned in comments -- so there's a real danger of "simplifying"
seemingly overly-elaborate code into creating a hole).  list2set() is
unusual in letting PyList_GetItem catch it, though:

	int len = PyList_Size(list);
	...
	for (i = 0; i < len; i++)  {
		SOCKET v;

		/* any intervening fileno() calls could decr this refcnt */
		if (!(o = PyList_GetItem(list, i)))
                    return -1;
		Py_INCREF(o);

It's more common to see

	for (i = 0; i < PyList_Size(list); i++)  {

i.e. to recompute the list length on each trip, and so silently ignore
length-changing mutations.

> (2) AIUI,

New one on me!

at-least-as-i-understand-it<wink>-ly y'rs  - tim



From gstein@lyra.org  Sun Aug  5 23:58:17 2001
From: gstein@lyra.org (Greg Stein)
Date: Sun, 5 Aug 2001 15:58:17 -0700
Subject: [Python-Dev] '*' instead of 'yield'?
In-Reply-To: <Pine.LNX.4.30.0108051150520.989-100000@rnd.onego.ru>; from rnd@onego.ru on Sun, Aug 05, 2001 at 11:54:14AM +0400
References: <Pine.LNX.4.30.0108051150520.989-100000@rnd.onego.ru>
Message-ID: <20010805155817.B32658@lyra.org>

On Sun, Aug 05, 2001 at 11:54:14AM +0400, Roman Suzi wrote:
> Hello!
> 
> Maybe it will be nicier to have unary * instead of 'yield'
> for generators?

This is Python, not Perl. Why use an obscure character instead of a keyword?
It doesn't make sense to me...

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From gstein@lyra.org  Mon Aug  6 00:00:11 2001
From: gstein@lyra.org (Greg Stein)
Date: Sun, 5 Aug 2001 16:00:11 -0700
Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?]
In-Reply-To: <15213.25875.726178.186594@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Sun, Aug 05, 2001 at 11:24:03AM -0400
References: <20010804173856.C31732@lyra.org> <20010803202227.A31091@lyra.org> <E15TG8F-0000yq-00@darjeeling> <15213.25875.726178.186594@cj42289-a.reston1.va.home.com>
Message-ID: <20010805160011.C32658@lyra.org>

That is why I mentioned test.webdav.org :-)

But no matter... we can have a loopback-based test.

Cheers,
-g

On Sun, Aug 05, 2001 at 11:24:03AM -0400, Fred L. Drake, Jr. wrote:
> 
> Moshe Zadka writes:
>  > http://webdav.de has one.
> 
>   The pages there are inconsistent in their claims of status and
> version number, so it's not clear what the actual features are.  I
> suspect that only the front page is up to date.
>   Attempting to actually download the software causes an
> authentication dialog to pop up, so something is probably screwed up
> as well.
>   The pages look slick, though.  ;-)
> 
> 
>   -Fred
> 
> -- 
> Fred L. Drake, Jr.  <fdrake at acm.org>
> PythonLabs at Zope Corporation
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev

-- 
Greg Stein, http://www.lyra.org/


From tim.one@home.com  Mon Aug  6 00:12:40 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 5 Aug 2001 19:12:40 -0400
Subject: [Python-Dev] '*' instead of 'yield'?
In-Reply-To: <20010805155817.B32658@lyra.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCKELALDAA.tim.one@home.com>

[Roman Suzi]
> Maybe it will be nicier to have unary * instead of 'yield'
> for generators?

[Greg Stein]
> This is Python, not Perl. Why use an obscure character instead of
> a keyword?
> It doesn't make sense to me...

I doubt it would be warmly received in Perl-land, either.

Roman, this will never fly unless you can talk Guido into also allowing "**"
as a synonym for "return".  I wish you luck <wink>.



From thomas@xs4all.net  Mon Aug  6 00:12:28 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Mon, 6 Aug 2001 01:12:28 +0200
Subject: [Python-Dev] '*' instead of 'yield'?
In-Reply-To: <Pine.LNX.4.30.0108051150520.989-100000@rnd.onego.ru>
References: <Pine.LNX.4.30.0108051150520.989-100000@rnd.onego.ru>
Message-ID: <20010806011228.P626@xs4all.nl>

On Sun, Aug 05, 2001 at 11:54:14AM +0400, Roman Suzi wrote:

> Maybe it will be nicier to have unary * instead of 'yield'
> for generators?

Maybe not. Very certainly not for me, in any case :)

> Example I already posted to c.l.p:

> from __future__ import generators
> import calendar, string
> def calt(year, month):
>     month_cal = calendar.monthcalendar(year, month)
>     * """<B>%s.%s</B>""" % (month, year)
>     * """<TABLE BORDER=1>"""
>     for week in month_cal:
>       * """\n<TR>"""
>       for day in week:
>         * """<TD>%s</TD>""" % (day or "&nbsp;")
>       * """</TR>"""
>     * """</TABLE>"""

This looks so ugly and unreadable to me, it's just not funny. It makes '*'
look like some kind of comment character.

> (This will also enable yields from expressions)

There is no reason for 'yield' not to be an expression. "not" and "and" and
"lambda" and such are all expressions too, remember. But making 'yield' a
statement is a very good choice, in my eyes. What should

   ((yield a) ** (yield b) and (yield c)) ^ (yield e) / yield f

yield ? 'yield' is too similar to 'return' for it not to be a statement.

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From guido@zope.com  Mon Aug  6 01:30:36 2001
From: guido@zope.com (Guido van Rossum)
Date: Sun, 05 Aug 2001 20:30:36 -0400
Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler
In-Reply-To: Your message of "05 Aug 2001 14:11:18 EDT."
 <2mhevm4bsp.fsf@starship.python.net>
References: <LNBBLJKPBEHFEDALKOLCMEINLDAA.tim.one@home.com>
 <2mhevm4bsp.fsf@starship.python.net>
Message-ID: <200108060030.UAA19421@cj20424-a.reston1.va.home.com>

> "Tim Peters" <tim.one@home.com> writes:
> 
> > [Guido, on reworking select()]
> > > Yes, I think we do need this.  PyObject_AsFileDescriptor() calls the
> > > fileno() method which could be a Python call.
> > 
> > Ya, but for me to believe that *matters*, I have to believe people worried
> > about high-performance select are passing objects with Python-implemented
> > fileno methods to begin with.  That's quite a stretch, isn't it?  They have
> > to pay for fileno() calls regardless, the question is only whether they pay
> > it once or twice per object.

[And Tim posted an interesting alternative implementation]

That looks like a good way to avoid the double fileno() calls.
Drawback: the implementation you sketched seems to create a twople per
file descriptor.  That's not quite a malloc (twoples are cached), but
still.

If someone feels like implementing this, please go ahead!

[MH]
> Posted admittedly without checking any facts:
> 
> (1) Doesn't the Python fileno() method issue leave you open to
> ridiculous fileno() methods that mutate the list passed in to
> select()?

Yes, but the select code is immune to that -- it may get an
exception but it correctly propagates those.

> (2) AIUI, if you really care about performance you should be using
> poll() anyway (tho' obviously this depends on your platform having
> poll() in the first place, and I don't have the faintest idea how
> widespread it is).

Very widespread.  I forget -- what's the reason for poll()'s claim to
better performance?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@zope.com  Mon Aug  6 01:38:17 2001
From: guido@zope.com (Guido van Rossum)
Date: Sun, 05 Aug 2001 20:38:17 -0400
Subject: [Python-Dev] '*' instead of 'yield'?
In-Reply-To: Your message of "Mon, 06 Aug 2001 01:12:28 +0200."
 <20010806011228.P626@xs4all.nl>
References: <Pine.LNX.4.30.0108051150520.989-100000@rnd.onego.ru>
 <20010806011228.P626@xs4all.nl>
Message-ID: <200108060038.UAA21789@cj20424-a.reston1.va.home.com>

Of all the syntactic changes proposed on the newsgroup, by my estimate
less than 0.1% ever makes it into the language.  There's no need to
worry that this will change, "print >>" notwithstanding. :)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@home.com  Mon Aug  6 01:46:44 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 5 Aug 2001 20:46:44 -0400
Subject: [Python-Dev] '*' instead of 'yield'?
In-Reply-To: <20010806011228.P626@xs4all.nl>
Message-ID: <LNBBLJKPBEHFEDALKOLCMELGLDAA.tim.one@home.com>

[Thomas Wouters]
> ...
> There is no reason for 'yield' not to be an expression. "not" and
> "and" and "lambda" and such are all expressions too, remember. But
> making 'yield' a statement is a very good choice, in my eyes. What
> should
>
>    ((yield a) ** (yield b) and (yield c)) ^ (yield e) / yield f
>
> yield ? 'yield' is too similar to 'return' for it not to be a statement.

If we had been able to think of an "obviously correct" way for yield to
return a value, we would have -- but we didn't, so we didn't <wink>.

The temptation is that it's very handy for a coroutine transfer to "act
like" a function call from a transferor's POV:  this gives a way for *its*
resumer to communicate info back to it.  I first saw this in Icon, and it
was a huge and obvious improvement over Simula 67, wherein coroutines could
communicate only via stuffing values in shared globals.  Since PEP 255's
"Simple Generators" are only semi-coroutines, the need isn't acute, but
could still be handy at times.

Anyway, Neil implemented *lots* of ideas that didn't survive:  we all agreed
to slice out features left and right until the Simple in "Simple Generators"
earned its name.  They're still fancier than Icon's and CLU's generators
(because explicitly resumable even in the absence of magical control
context) all the same.

I'm curious to see how people use the "extra" power in practice, and can't
predict which kinds of extensions would be truly useful.  Until real life
makes that clear(er), I'm inclinded to resist any change; in the meantime,
we haven't precluded turning yield into an operator someday, neither
extending its syntax as a statement.

right-now-it's-the-simplest-thing-that-could-possibly-work-ly y'rs  - tim



From tim.one@home.com  Mon Aug  6 01:55:16 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 5 Aug 2001 20:55:16 -0400
Subject: [Python-Dev] '*' instead of 'yield'?
In-Reply-To: <200108060038.UAA21789@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOELHLDAA.tim.one@home.com>

[Guido]
> Of all the syntactic changes proposed on the newsgroup, by my estimate
> less than 0.1% ever makes it into the language.

Heh:  that shows how little you still read the newsgroup <0.9 wink>.  Note
that PEP 255 recorded all of these variants suggested for "yield":

       return 3 and continue
       return and continue 3
       return generating 3
       continue return 3
       return >> , 3
       from generator return 3
       return >> 3
       return << 3
       >> 3
       << 3

I suppose I should add

       * 3

to the list now.

if-you-accepted-0.1%-of-syntax-suggestions-it-would-have-stopped-
    looking-like-python-in-'94-ly y'rs  - tim



From skip@pobox.com (Skip Montanaro)  Mon Aug  6 05:11:35 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Sun, 5 Aug 2001 23:11:35 -0500
Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?]
In-Reply-To: <20010805160011.C32658@lyra.org>
References: <20010804173856.C31732@lyra.org>
 <20010803202227.A31091@lyra.org>
 <E15TG8F-0000yq-00@darjeeling>
 <15213.25875.726178.186594@cj42289-a.reston1.va.home.com>
 <20010805160011.C32658@lyra.org>
Message-ID: <15214.6391.208655.549066@beluga.mojam.com>

    Greg> That is why I mentioned test.webdav.org :-)
    Greg> But no matter... we can have a loopback-based test.

Tests are skipped because imports fail.  Seems reasonable to me that some
tests might be skipped if the network isn't up.  It shouldn't be difficult
to test if test.webdav.org is reachable and skip the davlib test if it's
not.

Skip


From guido@zope.com  Mon Aug  6 05:16:49 2001
From: guido@zope.com (Guido van Rossum)
Date: Mon, 06 Aug 2001 00:16:49 -0400
Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?]
In-Reply-To: Your message of "Sun, 05 Aug 2001 23:11:35 CDT."
 <15214.6391.208655.549066@beluga.mojam.com>
References: <20010804173856.C31732@lyra.org> <20010803202227.A31091@lyra.org> <E15TG8F-0000yq-00@darjeeling> <15213.25875.726178.186594@cj42289-a.reston1.va.home.com> <20010805160011.C32658@lyra.org>
 <15214.6391.208655.549066@beluga.mojam.com>
Message-ID: <200108060416.AAA22597@cj20424-a.reston1.va.home.com>

> Tests are skipped because imports fail.  Seems reasonable to me that some
> tests might be skipped if the network isn't up.  It shouldn't be difficult
> to test if test.webdav.org is reachable and skip the davlib test if it's
> not.

Sure, but only if we absolutely cannot test that module without
reaching out over the net.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com (Skip Montanaro)  Mon Aug  6 05:43:10 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Sun, 5 Aug 2001 23:43:10 -0500
Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?]
In-Reply-To: <200108060416.AAA22597@cj20424-a.reston1.va.home.com>
References: <20010804173856.C31732@lyra.org>
 <20010803202227.A31091@lyra.org>
 <E15TG8F-0000yq-00@darjeeling>
 <15213.25875.726178.186594@cj42289-a.reston1.va.home.com>
 <20010805160011.C32658@lyra.org>
 <15214.6391.208655.549066@beluga.mojam.com>
 <200108060416.AAA22597@cj20424-a.reston1.va.home.com>
Message-ID: <15214.8287.895.41269@beluga.mojam.com>

>>>>> "Guido" == Guido van Rossum <guido@zope.com> writes:

    >> Tests are skipped because imports fail.  Seems reasonable to me that
    >> some tests might be skipped if the network isn't up.  It shouldn't be
    >> difficult to test if test.webdav.org is reachable and skip the davlib
    >> test if it's not.

    Guido> Sure, but only if we absolutely cannot test that module without
    Guido> reaching out over the net.

I thought that was the whole idea of this thread.  There are apparently a
couple fully functional dav servers out there.  In the short-term, I think
it would be better to write a test that relies on them than having to write
a server - no matter how bare bones it is - just to perform some tests of a
client-side library.

who's-going-to-test-the-test-server?-ly y'rs,

Skip



From tim.one@home.com  Mon Aug  6 06:17:15 2001
From: tim.one@home.com (Tim Peters)
Date: Mon, 6 Aug 2001 01:17:15 -0400
Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler
In-Reply-To: <200108060030.UAA19421@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCELPLDAA.tim.one@home.com>

[Guido, on Tim's select() thought-algorithm]
> That looks like a good way to avoid the double fileno() calls.
> Drawback: the implementation you sketched seems to create a twople per
> file descriptor.  That's not quite a malloc (twoples are cached), but
> still.
>
> If someone feels like implementing this, please go ahead!

Yes, it does create twoples (as I said in my original msg).

That can also be avoided:  allocate result lists initially each the same
size as its corresponding input list.  Store just fds in the slots.  The
twoples are now implicit, in parallel lists (reusing the input lists as-is
to continue storing the objects).

The sole reason this can't work now is that a fileno() call may reach back
in and mutate one of the input lists -- that's the only thing creating
explicit twoples "off to the side" is guarding against, and the only real
reason I can see for why the current code is allocating giant pylist
thingies and storing "redundant" pointers to the objects in them.

So play the same trick list.sort plays:  redirect the input lists' type
pointers to the immutable list object for the duration of select().  It's
really no feature that select() can return ill-defined results if a fileno()
method mutates the list!  I'd rather the language raise an exception if
anyone *tries* to mutate an input list for the duration -- it's never a
reasonable thing for a fileno() method to do.

Then again, I was always the immutable-list trick's biggest fan <wink>.



From mwh@python.net  Mon Aug  6 10:18:33 2001
From: mwh@python.net (Michael Hudson)
Date: 06 Aug 2001 05:18:33 -0400
Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler
In-Reply-To: Guido van Rossum's message of "Sun, 05 Aug 2001 20:30:36 -0400"
References: <LNBBLJKPBEHFEDALKOLCMEINLDAA.tim.one@home.com> <2mhevm4bsp.fsf@starship.python.net> <200108060030.UAA19421@cj20424-a.reston1.va.home.com>
Message-ID: <2m4rrlwnpy.fsf@starship.python.net>

Guido van Rossum <guido@zope.com> writes:

> > (2) AIUI, if you really care about performance you should be using
> > poll() anyway (tho' obviously this depends on your platform having
> > poll() in the first place, and I don't have the faintest idea how
> > widespread it is).
> 
> Very widespread.  I forget -- what's the reason for poll()'s claim to
> better performance?

Appropriately enough, it avoids the need to much about with fd_sets
for each syscall, which makes a difference when yo are dealing with
huge numbers of sockets (again, I think).

Cheers,
M.

-- 
  I have long since given up dealing with people who hold idiotic
  opinions as if they had arrived at them through thinking about
  them.                                 -- Erik Naggum, comp.lang.lisp


From rnd@onego.ru  Mon Aug  6 10:21:42 2001
From: rnd@onego.ru (Roman Suzi)
Date: Mon, 6 Aug 2001 13:21:42 +0400 (MSD)
Subject: [Python-Dev] '*' instead of 'yield'?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOELHLDAA.tim.one@home.com>
Message-ID: <Pine.LNX.4.30.0108061311400.15766-100000@rnd.onego.ru>

On Sun, 5 Aug 2001, Tim Peters wrote:

>[Guido]
>> Of all the syntactic changes proposed on the newsgroup, by my estimate
>> less than 0.1% ever makes it into the language.
>
>Heh:  that shows how little you still read the newsgroup <0.9 wink>.  Note
>that PEP 255 recorded all of these variants suggested for "yield":
>
>       return 3 and continue
>       return and continue 3
>       return generating 3
>       continue return 3
>       return >> , 3
>       from generator return 3
>       return >> 3
>       return << 3
>       >> 3
>       << 3
>
>I suppose I should add
>
>       * 3

'*' looks right to be yield: like a hole after something got thru paper
;-)

I do not remember, but probably I also proposed:

        < 3

>to the list now.

Or, yes, and next step is to use '?' in place of 'print' ;-)

from __future__ import parrot

# must allow for such things:

= myfun(x, y, z):
  ? "how are you?"
  S = 0
  4 i <- x..y,z:
    S += i
  ** S

= mygen(x):
  4 k <- 1..x:
    * k

? myfun(1, 3, 5)
? mygen(20)

Another approach is lobby Python reserved words into Unicode.
(Like APL did, IIRC). Then they could be represented by nice
pictograms.

I think, yhis thread is not constructive any more, as 90%
of voices were AGAINST.

Sincerely yours, Roman Suzi
-- 
_/ Russia _/ Karelia _/ Petrozavodsk _/ rnd@onego.ru _/
_/ Monday, August 06, 2001 _/ Powered by Linux RedHat 6.2 _/
_/ "A pessimist is never disappointed." _/



From m@moshez.org  Mon Aug  6 13:47:17 2001
From: m@moshez.org (Moshe Zadka)
Date: Mon, 06 Aug 2001 15:47:17 +0300
Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?]
In-Reply-To: <15214.6391.208655.549066@beluga.mojam.com>
References: <15214.6391.208655.549066@beluga.mojam.com>, <20010804173856.C31732@lyra.org>
 <20010803202227.A31091@lyra.org>
 <E15TG8F-0000yq-00@darjeeling>
 <15213.25875.726178.186594@cj42289-a.reston1.va.home.com>
 <20010805160011.C32658@lyra.org>
Message-ID: <E15Tjmv-000601-00@darjeeling>

On Sun, 5 Aug 2001 23:11:35 -0500, Skip Montanaro <skip@pobox.com> wrote:

>     Greg> That is why I mentioned test.webdav.org :-)
>     Greg> But no matter... we can have a loopback-based test.
> 
> Tests are skipped because imports fail.  Seems reasonable to me that some
> tests might be skipped if the network isn't up.  It shouldn't be difficult
> to test if test.webdav.org is reachable and skip the davlib test if it's
> not.

The test for reachability might automatically dial-up for diald or its
windows equivalent.
-- 
Moshe Zadka - http://moshez.geek
(if you're not cool, http://moshez.org is still working)


From akuchlin@mems-exchange.org  Mon Aug  6 18:40:46 2001
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Mon, 06 Aug 2001 13:40:46 -0400
Subject: [Python-Dev] Cleaning up Misc/ directory
Message-ID: <E15ToMw-0004K4-00@ute.cnri.reston.va.us>

There are a few files in the Misc/ directory that can probably be
discarded:

BLURB, BLURB.*, and HYPE : several duplicated descriptions of Python.
                           I volunteer to edit them down to one BLURB file.
editline-fix : is the problem and fix described still relevant?
find_recursionlimit.py : unnecessary now that sys.getrecursionlimit()
                         exists.
fixfuncptrs.sh : seems to add casts to the type structure in a module
RFD : the RFD for comp.lang.python.  Can disappear now that the group
      has been created.

--amk


From skip@pobox.com (Skip Montanaro)  Mon Aug  6 19:28:52 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Mon, 6 Aug 2001 13:28:52 -0500
Subject: [Python-Dev] Cleaning up Misc/ directory
In-Reply-To: <E15ToMw-0004K4-00@ute.cnri.reston.va.us>
References: <E15ToMw-0004K4-00@ute.cnri.reston.va.us>
Message-ID: <15214.57828.64490.178664@beluga.mojam.com>

    amk> editline-fix : is the problem and fix described still relevant?

I sort of doubt it.  I'd delete it.  Editline has changed in the last six
years.  I haven't linked Python with it in nearly that long.  There are
other command line completion libraries available as well.  I just stumbled
upon tecla:

    http://www.astro.caltech.edu/~mcs/tecla/index.html

which seems to be actively maintained.  Anybody tried it?  It has an
X11-style license and is a little over half the size of libreadline.  Unlike
editline, it appears it doesn't attempt to conform to readline's API in the
areas the two libraries overlap.  Doesn't look like it supports history
files either.

Skip


From nas@python.ca  Mon Aug  6 19:39:06 2001
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 6 Aug 2001 11:39:06 -0700
Subject: [Python-Dev] Cleaning up Misc/ directory
In-Reply-To: <E15ToMw-0004K4-00@ute.cnri.reston.va.us>; from akuchlin@mems-exchange.org on Mon, Aug 06, 2001 at 01:40:46PM -0400
References: <E15ToMw-0004K4-00@ute.cnri.reston.va.us>
Message-ID: <20010806113906.A29653@glacier.fnational.com>

Andrew Kuchling wrote:
> find_recursionlimit.py : unnecessary now that sys.getrecursionlimit()
>                          exists.

Those two don't do the same thing.  sys.getrecursionlimit() returns the
current limit.  find_recursionlimit.py tries to find out how high you
can set it until you run out of stack space.

  Neil


From guido@zope.com  Mon Aug  6 19:40:17 2001
From: guido@zope.com (Guido van Rossum)
Date: Mon, 06 Aug 2001 14:40:17 -0400
Subject: [Python-Dev] Cleaning up Misc/ directory
In-Reply-To: Your message of "Mon, 06 Aug 2001 13:40:46 EDT."
 <E15ToMw-0004K4-00@ute.cnri.reston.va.us>
References: <E15ToMw-0004K4-00@ute.cnri.reston.va.us>
Message-ID: <200108061840.f76IeHO24574@odiug.digicool.com>

> There are a few files in the Misc/ directory that can probably be
> discarded:
> 
> BLURB, BLURB.*, and HYPE : several duplicated descriptions of Python.
>                            I volunteer to edit them down to one BLURB file.

I would just delete them -- we have plenty of such material on the
website.

> editline-fix : is the problem and fix described still relevant?

See Skip's response.  Toss it.

> find_recursionlimit.py : unnecessary now that sys.getrecursionlimit()
>                          exists.

Toss it.

> fixfuncptrs.sh : seems to add casts to the type structure in a module

Yeah, drop it.

> RFD : the RFD for comp.lang.python.  Can disappear now that the group
>       has been created.

Hm, *that* one is cute as a historic document.

You can also lose cheatsheet (a *very* old summary, there are much
better ones around), comparisons (we have comparisons on the website
now), faq2html.py (no longer used or relevant), and renumber.py (ditto
-- we now have the FAQ wizard so we no longer maintain the FAQ in text
form).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From akuchlin@mems-exchange.org  Mon Aug  6 20:32:13 2001
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Mon, 6 Aug 2001 15:32:13 -0400
Subject: [Python-Dev] Cleaning up Misc/ directory
In-Reply-To: <200108061840.f76IeHO24574@odiug.digicool.com>; from guido@zope.com on Mon, Aug 06, 2001 at 02:40:17PM -0400
References: <E15ToMw-0004K4-00@ute.cnri.reston.va.us> <200108061840.f76IeHO24574@odiug.digicool.com>
Message-ID: <20010806153212.G15095@ute.cnri.reston.va.us>

On Mon, Aug 06, 2001 at 02:40:17PM -0400, Guido van Rossum wrote:
>You can also lose cheatsheet ...

All deleted, except for find_recursionlimit and RFD, and cheatsheet
which has been updated to 2.0 (as previously discussed).

--amk




From klm@zope.com  Mon Aug  6 23:04:00 2001
From: klm@zope.com (Ken Manheimer)
Date: Mon, 6 Aug 2001 18:04:00 -0400 (EDT)
Subject: [Python-Dev] '*' instead of 'yield'?
In-Reply-To: <Pine.LNX.4.30.0108061311400.15766-100000@rnd.onego.ru>
Message-ID: <Pine.LNX.4.21.0108061749560.890-100000@serenade.digicool.com>

Roman, that was a sensible way to end this particular thread - forgive me
for gratuitously extending it a tiny bit, but i just can't resist...

On Mon, 6 Aug 2001, Roman Suzi wrote:

> [...]
> Or, yes, and next step is to use '?' in place of 'print' ;-)
> 
> from __future__ import parrot
> 
> # must allow for such things:
> 
> = myfun(x, y, z):
>   ? "how are you?"
>   S = 0
>   4 i <- x..y,z:
>     S += i
>   ** S
> 
> = mygen(x):
>   4 k <- 1..x:
>     * k
> 
> ? myfun(1, 3, 5)
> ? mygen(20)

I don't feel this is an, um, balanced example if you leave alone the only
non-expression in python that *is* signalled soley by punctuation, so:

  from __future__ import parrot

  comment must allow for such things: endcomment

  = myfun(x, y, z):
    ? "how are you?"
    S = 0
    4 i <- x..y,z:
      S += i
    ** S
  ...

:)

and-voluntary-block-delimeter-close-would-be-"comment end endcomment"'ly y'rs,

Ken Manheimer
klm@digicool.com



From gward@python.net  Tue Aug  7 01:51:42 2001
From: gward@python.net (Greg Ward)
Date: Mon, 6 Aug 2001 20:51:42 -0400
Subject: [Python-Dev] Interesting article: "Python Is Middleware"
Message-ID: <20010806205142.A4265@gerg.ca>

This was linked to from last week's Linux Weekly News, so you may have
already seen it.  If not, it's an interesting read:

  http://www.tundraware.com/Technology/Python-Is-Middleware/Python-Is-Middleware.html

Note in particular the last paragraph:

"""
  A Little Warning

  I worry about one thing and one thing only in the Python world. It is
  something I have witnessed in every new technology I've ever
  seen. Python is dangerously close to becoming a victim of Feeping
  Creaturism - not so much In Fact, but rather in this community's
  mindset of forever wanting to fiddle one more feature into the
  language. This will Kill Python commercially if it happens.  Too many
  variations on the core language theme make deployment and management
  of real systems too expensive. I, for one, would like to see a date
  picked for a permanent moratorium on the language proper, after which,
  only bug fixes and new modules could be added. After that date,
  language changes would have to be part of some new language ("Grail"?) 
  which would owe no allegiance to Python at all.
"""

        Greg
-- 
Greg Ward - programmer-at-large                         gward@python.net
http://starship.python.net/~gward/
"Question authority!"  "Oh yeah?  Says who?"


From guido@zope.com  Tue Aug  7 03:10:51 2001
From: guido@zope.com (Guido van Rossum)
Date: Mon, 06 Aug 2001 22:10:51 -0400
Subject: [Python-Dev] Interesting article: "Python Is Middleware"
In-Reply-To: Your message of "Mon, 06 Aug 2001 20:51:42 EDT."
 <20010806205142.A4265@gerg.ca>
References: <20010806205142.A4265@gerg.ca>
Message-ID: <200108070210.WAA24345@cj20424-a.reston1.va.home.com>

> This was linked to from last week's Linux Weekly News, so you may have
> already seen it.  If not, it's an interesting read:
> 
>   http://www.tundraware.com/Technology/Python-Is-Middleware/Python-Is-Middleware.html

Indeed.

> Note in particular the last paragraph:
[snip]

He's worrying too much, and under the influence of the recent division
flame war.  Understandable, but that causes me to weigh his remark
*less* than when it had come out of the blue.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From nas@python.ca  Tue Aug  7 03:43:53 2001
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 6 Aug 2001 19:43:53 -0700
Subject: [Python-Dev] Intel's Open Runtime Platform (ORP)
Message-ID: <20010806194353.A30524@glacier.fnational.com>

I just saw this while updating my Debian machine:

    Package: orp
    Description: Java VM and JIT from Intel Research Lab.
     ORP stands for Open Runtime Platform, which is an Intel(TM) Open
     Source research platform for investigating dynamic compilation and
     memory management technologies. The basic ORP system incorporates a
     fast code generating JIT (Just-In-Time) as well as an optimizing
     JIT. It also includes several GC (Garbage Collection) algorithms,
     ranging from a simple mark-sweep algorithm to an advanced train
     algorithm.

Its available at http://intel.com/research/mrl/orp/ and is released
under a BSD-like license.

  Neil


From tim.one@home.com  Tue Aug  7 05:57:03 2001
From: tim.one@home.com (Tim Peters)
Date: Tue, 7 Aug 2001 00:57:03 -0400
Subject: [Python-Dev] Interesting article: "Python Is Middleware"
In-Reply-To: <200108070210.WAA24345@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEONLDAA.tim.one@home.com>

[From Tim Daneliuk's article]
> I worry about one thing and one thing only in the Python world.  It
> is something I have witnessed in every new technology I've ever seen.
> Python is dangerously close to becoming a victim of Feeping Creaturism -
> not so much In Fact, but rather in this community's mindset of forever
> wanting to fiddle one more feature into the language.

With the exception of Unicode, I'm hard pressed to think of any major change
in Python that wasn't discussed for years and years, most back in '92
already (incl. augmented assignments, nested scopes, liberalizing the
iteration protocol, the type/class split, unifying ints and longs, and even
generators).

I've noted before that we're fast running out of this kind of thing (meaning
big features discussed for a long time).  Perhaps a formal notion of
interface, some flavor of optional static typing, and major tweaks to the
numeric model, will still occur, but that's "about it" for semantic features
that have been discussed forever.  What else is on the horizon?  Apart from
those, even the Py-in-the-Sky PEPs are either both minor and 100% backward
compatible ("Docstring Format", "Add a Built-in Set Type", "String
Interpolation"), or likely dead on arrival (even if nobody is rude enough to
say so <wink>).

> This will Kill Python commercially if it happens.

I suppose we should take the hard lesson from Visual Basic there <wink>.

> Too many variations on the core language theme make deployment and
> management of real systems too expensive.

Unfortunately for practical guidance, how many is "too many" can't be judged
without first determining how expensive is "too expensive" -- or, I suspect,
vice versa too.  Commercial interests in my eyes are defined by tangible
commitment of resource to achieve what they want; if they hired someone
full-time to ensure Python compatibility across releases, I bet they could
get it.

jaw-flapping-won't-solve-it-ly y'rs  - tim



From thomas@xs4all.net  Tue Aug  7 09:10:51 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Tue, 7 Aug 2001 10:10:51 +0200
Subject: [Python-Dev] Interesting article: "Python Is Middleware"
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEONLDAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCCEONLDAA.tim.one@home.com>
Message-ID: <20010807101051.T626@xs4all.nl>

On Tue, Aug 07, 2001 at 12:57:03AM -0400, Tim Peters wrote:

> I've noted before that we're fast running out of this kind of thing (meaning
> big features discussed for a long time).  Perhaps a formal notion of
> interface, some flavor of optional static typing, and major tweaks to the
> numeric model, will still occur, but that's "about it" for semantic features
> that have been discussed forever.

Let's not forget 'free threading', which is another one of those 'generally
benificial' features that will break code.

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From PyChecker <pychecker@metaslash.com>  Tue Aug  7 13:20:58 2001
From: PyChecker <pychecker@metaslash.com> (Neal Norwitz)
Date: Tue, 07 Aug 2001 08:20:58 -0400
Subject: [Python-Dev] ANN: PyChecker v0.8
Message-ID: <3B6FDD2A.85FB5B12@metaslash.com>

A new version of PyChecker is available for your hacking pleasure.

        PyChecker is a tool for finding common bugs in python source code.
        It finds problems that are typically caught by a compiler for less
        dynamic languages, like C and C++.

Comments, criticisms, new ideas, and other feedback is welcome.

Change Log from 0.7.5 to 0.8:
  * Add check for accessing list as list[1,2], should be slice 1:2
  * Add -J/maxargs to warn when using too many arguments
  * Add -K/maxlocals to warn when using too many local variables
  * Add -D/intdivide to warn when using integer division
  * Add -O/objattrs to warn when using object.attribute that doesn't exist
  * Add -M/reimportself to warn when a module imports itself
  * Add -E/unusednames to provide a list of unused names to ignore
        (default is: [ '_', 'empty', 'unused' ])
  * Major refactoring of warn.py -> utils, msgs, Warning, CodeChecks
  * Fix spurious warnings for No class attribute for dynamic classes
  * Fix spurious warnings when using objects from blacklisted modules
        (Instantiating object with arguments, but no constructor, etc)
  * Fix spurious warnings when using lambdas
  * Fix spurious warning (No global Y) when using: from X import Y
        (problem in python 1.5/1.6 only)

In case you missed 0.7.5, here's the change Log from 0.7 to 0.7.5:
  * Suppress warnings on a per module/function/class/method basis
        with new suppressions = {} in .pycheckrc
  * Suppress warnings by setting __pychecker__ in source code
  * Change long argument behaviour --arg sets arg to true,
        --no-arg sets arg to false (also works for warning suppression)
  * Add -U/--reuseattr check if function/class/method names are reused
  * Add -T/--argsused check for unused method/function arguments
  * Add -G/--selfused ignore if self is unused in a method (requires --argsused)
  * Add -q/--stdlib to disable warnings from the standard library
  * Add -o/--override warning when a method has different signature
        than one being overridden in a base class
  * Add -F/--rcfile to generate a .pycheckrc file
  * Fix checking files in standard library before local file
  * Fix spurious warning when using from X import Y and imports in methods
        (Module (m) re-imported)
  * Fix spurious warning when doing:  from X import Y, Z
  * Fix spurious warning when deriving from Exception() and
        instantiating object with multiple arguments
  * Fix method argument checks when calling base class methods
  * Fix error msg to base constructors (msg count was wrong)
  * Fix access to builtin module attributes (e.g., sys.exc_value)
        generating 'No attribute warnings'
  * Fix tests (forgot to add : after line number in expected results)


PyChecker is available on Source Forge:
    Web page:           http://pychecker.sourceforge.net/
    Project page:       http://sourceforge.net/projects/pychecker/

Neal
--
pychecker@metaslash.com


From guido@zope.com  Tue Aug  7 13:37:59 2001
From: guido@zope.com (Guido van Rossum)
Date: Tue, 07 Aug 2001 08:37:59 -0400
Subject: [Python-Dev] PyChecker v0.8
Message-ID: <200108071237.IAA27282@cj20424-a.reston1.va.home.com>

I just saw another announcement of PyChecker.  It's definitely growing
to become a very useful (if chatty :-) tool!  How about we attempt to
fix at least all the real bugs it detects in the standard library,
such as the typos in urllib2.py ...  Please help!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From arigo@ulb.ac.be  Tue Aug  7 16:15:44 2001
From: arigo@ulb.ac.be (Armin Rigo)
Date: Tue, 7 Aug 2001 17:15:44 +0200 (CEST)
Subject: [Python-Dev] PEP 252: class methods
Message-ID: <Pine.LNX.4.33L2.0108071712210.21615-100000@magma.unil.ch>

Hello,

PEP 252: as described, class methods have no equivalent in C++ or Java; I
would just like to point out that the Pascal dialect "Delphi" of Borland
has this feature (called "virtual class methods"), which I personnally
used a lot in one of my applications.


Armin



From jack@oratrix.nl  Tue Aug  7 16:34:04 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Tue, 07 Aug 2001 17:34:04 +0200
Subject: [Python-Dev] HAVE_HYPOT still needed?
Message-ID: <20010807153404.8D33D303181@snelboot.oratrix.nl>

As I was removing outdated #ifdefs my eye fell on HAVE_HYPOT, and I have this 
very vague recollection that some old 68K Macs were the only system where 
hypot() wasn't in the C library. Does anyone know this for sure? If so we 
could remove the whole hypot() stuff...
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | ++++ see http://www.xs4all.nl/~tank/ ++++




From jeremy@zope.com  Tue Aug  7 16:56:10 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Tue, 7 Aug 2001 11:56:10 -0400 (EDT)
Subject: [Python-Dev] Zope porting to jython
In-Reply-To: <004c01c11cf7$7b7eec60$b75821c0@newmexico>
References: <20010802103006.H1667@netthink.co.uk>
 <20010803161704.M626@xs4all.nl>
 <004c01c11cf7$7b7eec60$b75821c0@newmexico>
Message-ID: <15216.3994.244904.425445@slothrop.digicool.com>

>>>>> "SP" == Samuele Pedroni <pedroni@inf.ethz.ch> writes:

  SP> my personal position is to help them as all other jython users,
  SP> fixing incompatibilities with CPython and answering to
  SP> questions. Nothing special. I can imagine that one of the big
  SP> problem could be restricted execution (if Zope uses that),
  SP> admitted that any C code functionality can be rewritten in Java.

Zope doesn't use restricted execution.  In Zope 2.4, the
RestrictedPython project uses the compiler package to generate safe
bytecode from Python source code.  I presume something similar could
be achieved using jythonc, but don't know how hard that would be.

  SP> If the PythonLabs have an "official" position/opinion on this
  SP> and in particular on the last quite speculative message (from
  SP> Joseph Grace) in the thread, they may post something. I can say
  SP> something about technical aspects, but I don't feel that I
  SP> should channel/or invent PythonLabs opinions.

I don't think there's an official PythonLabs opinion on this in
general.  If we have one its this:

The project(s) -- including the just announced phabric
effort -- are worthwhile.  We are happy to give advice and make
changes to help the projects where we can.  I expect the same holds
true for the rest of Zope Corp.

If there are genuine CPython/Jython incompatibilities that hinder the
project, there's no reason not to fix them.  The project may also
serve as a vehicle to further explore what metaobject protocols should
be exported by Jython.

Jeremy



From tim@zope.com  Tue Aug  7 17:07:34 2001
From: tim@zope.com (Tim Peters)
Date: Tue, 7 Aug 2001 12:07:34 -0400
Subject: [Python-Dev] HAVE_HYPOT still needed?
In-Reply-To: <20010807153404.8D33D303181@snelboot.oratrix.nl>
Message-ID: <BIEJKCLHCIOIHAGOKOLHCEIICEAA.tim@zope.com>

[Jack Jansen]
> As I was removing outdated #ifdefs my eye fell on HAVE_HYPOT, and
> I have this very vague recollection that some old 68K Macs were the
> only system where hypot() wasn't in the C library.

The only comments about this in the codebase and in NEWS suggest that's so.

> Does anyone know this for sure?

Perhaps God -- the rest of us run on a finite number of machines <wink>.

> If so we could remove the whole hypot() stuff...

+1, including nuking Python/hypot.c.  hypot() is a std C library function,
so any platform without it is plain busted; and an alpha release is the only
time we dare test that hypot<wink>hesis.



From tim@zope.com  Tue Aug  7 17:50:12 2001
From: tim@zope.com (Tim Peters)
Date: Tue, 7 Aug 2001 12:50:12 -0400
Subject: [Python-Dev] HAVE_HYPOT still needed?
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHCEIICEAA.tim@zope.com>
Message-ID: <BIEJKCLHCIOIHAGOKOLHMEILCEAA.tim@zope.com>

> If so we could remove the whole hypot() stuff...

Maybe not!  I don't have a copy of the C89 std here, and web searches turn
up conflicting claims.  My favorite C89 web site:

    http://www-ccs.ucsd.edu/c/

does not list hypot among the C89 functions.  It's definitely required in
C99.  To play it safe, I expect we need to keep our own hypot after all.

Would still be nice to simplify this Mac-ish part of pyport.h:

#ifndef HAVE_HYPOT
extern double hypot(double, double);
#ifdef MWERKS_BEFORE_PRO4
#define hypot we_dont_want_faulty_hypot_decl
#endif
#endif

#ifndef HAVE_HYPOT
#ifdef __MWERKS__
#undef hypot
#endif
#endif

While we're at it, shame Barry into cleaning this up <wink>:

#ifdef __NeXT__
#ifdef __sparc__
/*
 * This works around a bug in the NS/Sparc 3.3 pre-release
 * limits.h header file.
 * 10-Feb-1995 bwarsaw@cnri.reston.va.us
 */
#undef LONG_MIN
#define LONG_MIN (-LONG_MAX-1)
#endif
#endif



From paulp@ActiveState.com  Tue Aug  7 18:33:21 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Tue, 07 Aug 2001 10:33:21 -0700
Subject: [Python-Dev] Interesting article: "Python Is Middleware"
References: <LNBBLJKPBEHFEDALKOLCCEONLDAA.tim.one@home.com> <20010807101051.T626@xs4all.nl>
Message-ID: <3B702661.DB18CD36@ActiveState.com>

Thomas Wouters wrote:
> 
> On Tue, Aug 07, 2001 at 12:57:03AM -0400, Tim Peters wrote:
> 
> > I've noted before that we're fast running out of this kind of thing (meaning
> > big features discussed for a long time).  Perhaps a formal notion of
> > interface, some flavor of optional static typing, and major tweaks to the
> > numeric model, will still occur, but that's "about it" for semantic features
> > that have been discussed forever.
> 
> Let's not forget 'free threading', which is another one of those 'generally
> benificial' features that will break code.

Is that true? I thought that you would have to ASK to use free threading
in a particular program.

I'm discussing threading models with our internal Perl guys and I'm
curious about one thing: what prevents a person from running two
non-communicating Python interpreters in two threads? What kind of data
is shared between interpeters?
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From guido@python.org  Tue Aug  7 18:46:00 2001
From: guido@python.org (Guido van Rossum)
Date: Tue, 07 Aug 2001 13:46:00 -0400
Subject: [Python-Dev] Free threading
In-Reply-To: Your message of "Tue, 07 Aug 2001 10:33:21 PDT."
 <3B702661.DB18CD36@ActiveState.com>
References: <LNBBLJKPBEHFEDALKOLCCEONLDAA.tim.one@home.com> <20010807101051.T626@xs4all.nl>
 <3B702661.DB18CD36@ActiveState.com>
Message-ID: <200108071746.f77Hk0I31236@odiug.digicool.com>

[Thomas]
> > Let's not forget 'free threading', which is another one of those 'generally
> > benificial' features that will break code.

[Paul]
> Is that true? I thought that you would have to ASK to use free threading
> in a particular program.

Dunno -- in any case I'm not even sure how free threading would break
code (as long as it shouldn't be allowed to cause core dumps, basic
data structures ought to be threadsafe under free threading).

> I'm discussing threading models with our internal Perl guys and I'm
> curious about one thing: what prevents a person from running two
> non-communicating Python interpreters in two threads? What kind of data
> is shared between interpeters?

Nothing stops you from doing that -- the intention is that it's
possible.  Immutable objects (like strings and ints) and all built-in
type objects (including extension types) are shared between all
interpreters.  Interned strings are shared.  Exceptions are shared
(there may actually be a bug related to initializing exceptions that I
still have to track down).  Extension modules that have global state
may cause additional confusion.  Operating system resources like file
descriptors are shared.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From paulp@ActiveState.com  Tue Aug  7 21:17:02 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Tue, 07 Aug 2001 13:17:02 -0700
Subject: [Python-Dev] Free threading
References: <LNBBLJKPBEHFEDALKOLCCEONLDAA.tim.one@home.com> <20010807101051.T626@xs4all.nl>
 <3B702661.DB18CD36@ActiveState.com> <200108071746.f77Hk0I31236@odiug.digicool.com>
Message-ID: <3B704CBE.88F7FC7A@ActiveState.com>

Guido van Rossum wrote:
> 
> ...
> 
> Nothing stops you from doing that -- the intention is that it's
> possible. 

I didn't ask the question properly. You can run multiple interpreters in
multiple threads but they share a GIL, right? I'm really asking whether
multiple interpreters can be as independent of each other as any other
bit of carefuly written re-entrant C code.

> ... Immutable objects (like strings and ints) and all built-in
> type objects (including extension types) are shared between all
> interpreters.  Interned strings are shared.  Exceptions are shared
> (there may actually be a bug related to initializing exceptions that I
> still have to track down).  Extension modules that have global state
> may cause additional confusion.  Operating system resources like file
> descriptors are shared.

It might be interesting if there were a module that allowed the Python
programmer to spawn new interpreters on new threads from within Python
and a way to share primitive data structures between them. My
understanding is that Perl and Tcl both use this "independent threads"
model which scales really well on SMP but seems to me to be pretty
brutal to program for. Nevertheless I can see how it would be useful
where all you care about is SMP performance.

http://search.cpan.org/doc/ABERGMAN/threads-shared-0.02/shared.pm
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From guido@python.org  Tue Aug  7 22:17:41 2001
From: guido@python.org (Guido van Rossum)
Date: Tue, 07 Aug 2001 17:17:41 -0400
Subject: [Python-Dev] Free threading
In-Reply-To: Your message of "Tue, 07 Aug 2001 13:17:02 PDT."
 <3B704CBE.88F7FC7A@ActiveState.com>
References: <LNBBLJKPBEHFEDALKOLCCEONLDAA.tim.one@home.com> <20010807101051.T626@xs4all.nl> <3B702661.DB18CD36@ActiveState.com> <200108071746.f77Hk0I31236@odiug.digicool.com>
 <3B704CBE.88F7FC7A@ActiveState.com>
Message-ID: <200108072117.RAA29094@cj20424-a.reston1.va.home.com>

> > Nothing stops you from doing that -- the intention is that it's
> > possible. 
> 
> I didn't ask the question properly. You can run multiple interpreters in
> multiple threads but they share a GIL, right? I'm really asking whether
> multiple interpreters can be as independent of each other as any other
> bit of carefuly written re-entrant C code.

The share the GIL to protect the other shared resources.  Since
immutable objects can be shared, there's no way to have completely
independent interpreters.

Anyway, I don't see the point: if you want two 100% independent
interpreters, you can get that easily by forking two separate
processes.  That's what separate processes are *for*...

> > ... Immutable objects (like strings and ints) and all built-in
> > type objects (including extension types) are shared between all
> > interpreters.  Interned strings are shared.  Exceptions are shared
> > (there may actually be a bug related to initializing exceptions that I
> > still have to track down).  Extension modules that have global state
> > may cause additional confusion.  Operating system resources like file
> > descriptors are shared.
> 
> It might be interesting if there were a module that allowed the Python
> programmer to spawn new interpreters on new threads from within Python
> and a way to share primitive data structures between them. My
> understanding is that Perl and Tcl both use this "independent threads"
> model which scales really well on SMP but seems to me to be pretty
> brutal to program for. Nevertheless I can see how it would be useful
> where all you care about is SMP performance.
> 
> http://search.cpan.org/doc/ABERGMAN/threads-shared-0.02/shared.pm

I bet the level of sharing exposed here can be implemented easily on
top of the mmap module.  Essentially, no matter what you do on an SMP
machine, it seems you have to minimize sharing because it slows things
down too much.  Every solution somehow ends up making private data (to
a process/thread) the default and makes you request sharing
explicitly.  In Python you won't be able to share *variables*, but it
should be easy enough to share objects via some memory-based RPC
protocol.

That sounds to me a quicker road to an efficient SMP solution than
trying to get rid of the GIL.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dan@sidhe.org  Tue Aug  7 22:44:59 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Tue, 07 Aug 2001 17:44:59 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <200108072135.XAA08800@core.inf.ethz.ch>
Message-ID: <5.1.0.14.0.20010807174100.023c1c58@tuatha.sidhe.org>

At 11:35 PM 8/7/2001 +0200, Samuele Pedroni wrote:
>The intention of my last meassage was not to shut down discussion with a
>pessimistic argument, I was just trying to pin down the potential problems,
>maybe the tone was a bit bitter...

Nope, not at all--I do intend to reply. (The tone was fine, BTW) The Real 
World just got in the way, as it has this annoying tendency to do. :)

As a quick response (this is from memory, so if it makes no sense don't be 
surprised), the current Parrot design's adequate for heavy OO, but could be 
a lot better. I'm working on that, and as soon as I've a better 
under-the-hood design I'll drop a note and everyone can savage it as 
appropriate. (And the more the better, honestly)

FWIW I've thrown the first partial draft of the Parrot assembly language 
spec out to the wolves. URL and details available to the interested.

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From jack@oratrix.nl  Tue Aug  7 23:07:10 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Wed, 08 Aug 2001 00:07:10 +0200
Subject: [Python-Dev] Using patch mgr for peer review - sensible or chicken?
Message-ID: <20010807220715.E68E714B630@oratrix.oratrix.nl>

Folks,
whenever I make a change to something essential, and something that
I'm not sure I understand for the full 100%, I tend to not check the
mods in directly but in stead put a patch into the patch manager. This
has happened recently with a setup.py and a configure.in patch, for
instance. My idea was that whoever felt responsible for the file in
question would see the patch email come by, have a look, and say "fine".

But whereas some patches get looked at quickly some can sit there for
days (or in some cases even months). This is a nuisance to me, as by
the time someone looks at the patch my development version of the file
has evolved again. Moreover, I get the impression that I'm the only
person with checkin permissions that does this, and that the rest of
the developer community just checks stuff in and counts on it being
(a) correct or (b) quickly fixed by someone else.

Am I too chicken and should I just check things in if I'm reasonably
convinced that they're okay?
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | ++++ see http://www.xs4all.nl/~tank/ ++++


From fdrake@acm.org  Tue Aug  7 23:06:40 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 7 Aug 2001 18:06:40 -0400 (EDT)
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <5.1.0.14.0.20010807174100.023c1c58@tuatha.sidhe.org>
References: <200108072135.XAA08800@core.inf.ethz.ch>
 <5.1.0.14.0.20010807174100.023c1c58@tuatha.sidhe.org>
Message-ID: <15216.26224.856076.269695@cj42289-a.reston1.va.home.com>

Dan Sugalski writes:
 > FWIW I've thrown the first partial draft of the Parrot assembly language 
 > spec out to the wolves. URL and details available to the interested.

  I would be surprised if there were many people on python-dev who
aren't interested.  I don't know how many aren't in the throws of
wrestling with "The Real World" though.
  Could you make the URL available on python-dev?  Failing that, I'd
like a change to read what you've written!
  Thanks!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From fdrake@acm.org  Tue Aug  7 23:17:24 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 7 Aug 2001 18:17:24 -0400 (EDT)
Subject: [Python-Dev] Using patch mgr for peer review - sensible or chicken?
In-Reply-To: <20010807220715.E68E714B630@oratrix.oratrix.nl>
References: <20010807220715.E68E714B630@oratrix.oratrix.nl>
Message-ID: <15216.26868.443867.736337@cj42289-a.reston1.va.home.com>

Jack Jansen writes:
 > But whereas some patches get looked at quickly some can sit there for
 > days (or in some cases even months). This is a nuisance to me, as by
 > the time someone looks at the patch my development version of the file
 > has evolved again. Moreover, I get the impression that I'm the only
 > person with checkin permissions that does this, and that the rest of

  You're not the only one to do this; the weakrefs implementation went
through several versions of patches before being checked in (but it
did receive comments and improvements from others).  I've seen others
do this as well.
  If you think a patch needs to be handled quicker, perhaps a note to
python-dev?  The SF patch manager helps us not lose patches, but it
doesn't do anything to help us set priorities, or give us extra time
to look at patches when we're just plain swamped.

 > Am I too chicken and should I just check things in if I'm reasonably
 > convinced that they're okay?

  If nobody responds to what you think is a reasonable patch, check it
in and we'll scream if it wasn't.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From jeremy@zope.com  Tue Aug  7 23:23:01 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Tue, 7 Aug 2001 18:23:01 -0400 (EDT)
Subject: [Python-Dev] interesting page of links on VM design
Message-ID: <15216.27205.438072.842721@slothrop.digicool.com>

http://www.ce.chalmers.se/~feldt/ruby/ideas/rubyvm/

Found this link by a circuitous route (David Simmons' post to
language-dev, leading to www.smallscript.net which discusses QKS work,
includes link to related topics).  May be interesting, related to
Parrot thread.

Jeremy



From jack@oratrix.nl  Wed Aug  8 01:45:05 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Wed, 08 Aug 2001 02:45:05 +0200
Subject: [Python-Dev] Using patch mgr for peer review - sensible or chicken?
In-Reply-To: Message by "Fred L. Drake, Jr." <fdrake@acm.org> ,
 Tue, 7 Aug 2001 18:17:24 -0400 (EDT) , <15216.26868.443867.736337@cj42289-a.reston1.va.home.com>
Message-ID: <20010808004510.3BC8B14B630@oratrix.oratrix.nl>

Recently, "Fred L. Drake, Jr." <fdrake@acm.org> said:
>   If you think a patch needs to be handled quicker, perhaps a note to
> python-dev?  The SF patch manager helps us not lose patches, but it
> doesn't do anything to help us set priorities, or give us extra time
> to look at patches when we're just plain swamped.

Ok, someone please look at patch 448261. It'll make the build process
on MacOSX a lot more palatable.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 


From tim.one@home.com  Wed Aug  8 01:48:48 2001
From: tim.one@home.com (Tim Peters)
Date: Tue, 7 Aug 2001 20:48:48 -0400
Subject: [Python-Dev] Using patch mgr for peer review - sensible or chicken?
In-Reply-To: <20010807220715.E68E714B630@oratrix.oratrix.nl>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEDBLEAA.tim.one@home.com>

[Jack Jansen]
> whenever I make a change to something essential, and something that
> I'm not sure I understand for the full 100%, I tend to not check the
> mods in directly but instead put a patch into the patch manager.

Good!  That's the way it's supposed to work.

> This has happened recently with a setup.py and a configure.in patch,
> for instance.  My idea was that whoever felt responsible for the file
> in question would see the patch email come by, have a look, and say
> "fine".

That part doesn't work:  you should assign it to whoever you think is best
qualified to judge the parts you're not sure about.  It won't necessarily
see action any quicker, but it feels good to have someone to blame <wink>.

> But whereas some patches get looked at quickly some can sit there for
> days (or in some cases even months).

Yup!  I imagine it's even worse for people we don't know well, who submit
odd patches nobody knows what to do with.

> This is a nuisance to me, as by the time someone looks at the patch
> my development version of the file has evolved again.

That went without saying, you know.

> Moreover, I get the impression that I'm the only person with checkin
> permissions that does this,

No, you're not.  "The rule" is that an expert in an area, with an
uncontroversial patch (and if they can't judge controversy in advance,
they're not expert in that area!), should just check something in.  Any
other case should go thru review.  For example, poor Martin hasn't had much
luck getting action from PythonLabs on some optimization patches, and I've
had to redo a UTF-16 patch from scratch (because it no longer applied) in
the 5 weeks I've been waiting for MAL to look at it (*told* you it felt good
to have someone to blame <wink>).

So everyone gets burned by this.  It's relatively rare for the
Python-Dev'ers, though, because by and large most of us stick to areas where
"I'm an expert, buzz off" applies.

> and that the rest of the developer community just checks stuff in and
> counts on it being (a) correct or (b) quickly fixed by someone else.

I *prefer* that approach when the only question is whether it will break
something on Windows.  I build Python every day anyway, so applying a patch
just to test Windows is just more work; it's easier to fix it myself than
apply a patch, explain what needs to change in the patch, then undo the
patch again.

> Am I too chicken

I forgot -- are you Dutch or German <wink>?

> and should I just check things in if I'm reasonably convinced that
> they're okay?

Depends on how sure you are.  If not "very", first assign your patch to
someone specific and give them a chance to disappoint you.



From paulp@ActiveState.com  Wed Aug  8 01:56:27 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Tue, 07 Aug 2001 17:56:27 -0700
Subject: [Python-Dev] Free threading
References: <LNBBLJKPBEHFEDALKOLCCEONLDAA.tim.one@home.com> <20010807101051.T626@xs4all.nl> <3B702661.DB18CD36@ActiveState.com> <200108071746.f77Hk0I31236@odiug.digicool.com>
 <3B704CBE.88F7FC7A@ActiveState.com> <200108072117.RAA29094@cj20424-a.reston1.va.home.com>
Message-ID: <3B708E3A.618374A2@ActiveState.com>

Guido van Rossum wrote:
> 
>...
> 
> Anyway, I don't see the point: if you want two 100% independent
> interpreters, you can get that easily by forking two separate
> processes.  That's what separate processes are *for*...

I am 95% in agreement with you and that's what I told the Perl guys.
That's why I see their "ithreads" solution as a little weird. But the
other 5% is when you are running in an already-threaded environment and
it is only the Python stuff that you want to be separate (for
performance). e.g. Apache, IIS, Sendmail, COM etc.

Also, Windows doesn't have "fork" so spawning a process means actually
running an executable from disk AFAIK. That's somewhat inefficient. But
I'll repeat that in the big picture I agree with you. The combination of
shared-data threads and independent-data processes is good enough for
most real-world apps.

>...
> I bet the level of sharing exposed here can be implemented easily on
> top of the mmap module.

Please excuse my ignorance: how do you do locking in an mmap solution?

>...
> That sounds to me a quicker road to an efficient SMP solution than
> trying to get rid of the GIL.

That's why I mentioned it. For whatever reason, the GIL project seems to
only be doable by Greg S. (or he's the only one with interest?) so I
thought a new approach might catch someone's interest.

Does your mild statement of approval indicate that you see the benefit
of having independent in-process interpreters? Or are you still thinking
about using real processes? If the former, it still seems that the
interpreter itself needs a little work so that it is safely reentrant in
the sense of any other C library. There especially needs to be a way to
NOT share a GIL and NOT share any other mutable data.

If you are talking about real processes, then "all" that's needed is an
efficient RPC mechanism.
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From guido@python.org  Wed Aug  8 05:09:55 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 08 Aug 2001 00:09:55 -0400
Subject: [Python-Dev] Free threading
In-Reply-To: Your message of "Tue, 07 Aug 2001 17:56:27 PDT."
 <3B708E3A.618374A2@ActiveState.com>
References: <LNBBLJKPBEHFEDALKOLCCEONLDAA.tim.one@home.com> <20010807101051.T626@xs4all.nl> <3B702661.DB18CD36@ActiveState.com> <200108071746.f77Hk0I31236@odiug.digicool.com> <3B704CBE.88F7FC7A@ActiveState.com> <200108072117.RAA29094@cj20424-a.reston1.va.home.com>
 <3B708E3A.618374A2@ActiveState.com>
Message-ID: <200108080409.AAA29799@cj20424-a.reston1.va.home.com>

[me]
> > Anyway, I don't see the point: if you want two 100% independent
> > interpreters, you can get that easily by forking two separate
> > processes.  That's what separate processes are *for*...

[PP]
> I am 95% in agreement with you and that's what I told the Perl guys.
> That's why I see their "ithreads" solution as a little weird. But the
> other 5% is when you are running in an already-threaded environment and
> it is only the Python stuff that you want to be separate (for
> performance). e.g. Apache, IIS, Sendmail, COM etc.

Yeah, but it's just not worth rewriting all of Python to get rid of
the GIL for *that*.  There are plenty of other ways to deal with it,
including pre-forked longer-running processes -- and remember, it's
only necessary if you need more than one CPU just for running your
Python code!  (E.g. Zope, which has a perfectly good Zope-specific
solution if you need multiple CPUs: ZEO.)

> Also, Windows doesn't have "fork" so spawning a process means actually
> running an executable from disk AFAIK. That's somewhat inefficient. But
> I'll repeat that in the big picture I agree with you. The combination of
> shared-data threads and independent-data processes is good enough for
> most real-world apps.

> >...
> > I bet the level of sharing exposed here can be implemented easily on
> > top of the mmap module.
> 
> Please excuse my ignorance: how do you do locking in an mmap solution?

Good question.  I've never used mmap myself. :-)  I know Unix shared
memory has locks and semaphores; the mmap module apparently doesn't
(possibly because Windows has a different philosophy there).

I would probably hide mmap and all other gory details in a module
whose interface is similar to that of Queue.  It could limit the data
type for items stored in the to strings, or it could use marshal or
pickle (or even XML :-) for more flexibility.

> >...
> > That sounds to me a quicker road to an efficient SMP solution than
> > trying to get rid of the GIL.
> 
> That's why I mentioned it. For whatever reason, the GIL project seems to
> only be doable by Greg S. (or he's the only one with interest?) so I
> thought a new approach might catch someone's interest.

IMO there's nothing about Greg here.  He did it once, that's all. :-)

> Does your mild statement of approval indicate that you see the benefit
> of having independent in-process interpreters?

My approval of what?  I like the independence of in-process
interpreters as far as it goes (and I would like to see the exception
bug fixed) but I don't think we need something more independent.  I
think I was arguing that for full independence of the GIL you should
fork a new process (Windows notwithstanding -- hey, it's not *that*
bad there unless you are trying to fork several per second, in which
case I would question your sanity anyway :-).

> Or are you still thinking about using real processes?

Yes.

> If the former, it still seems that the
> interpreter itself needs a little work so that it is safely reentrant in
> the sense of any other C library. There especially needs to be a way to
> NOT share a GIL and NOT share any other mutable data.

That would still be very hard to fix, and I don't think it's worth it.

> If you are talking about real processes, then "all" that's needed is an
> efficient RPC mechanism.

That's right, that's what my Queue-like idea above intended to be.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@home.com  Wed Aug  8 05:31:48 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 8 Aug 2001 00:31:48 -0400
Subject: [Python-Dev] Free threading
In-Reply-To: <200108072117.RAA29094@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEDHLEAA.tim.one@home.com>

[Guido]
> ...
> Anyway, I don't see the point: if you want two 100% independent
> interpreters, you can get that easily by forking two separate
> processes.  That's what separate processes are *for*...

So what is it that multiple interpreters within a single process *are* for?
I'm comfortable with threads, and comfortable with processes, but multiple
interpreters not-really-sharing yet not-really-isolated seem mostly an
opportunity to track down subtle bugs <wink>.



From skip@pobox.com (Skip Montanaro)  Wed Aug  8 05:36:06 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Tue, 7 Aug 2001 23:36:06 -0500
Subject: [Python-Dev] use co_flags to identify instruction set
Message-ID: <15216.49590.601115.957779@beluga.mojam.com>

All this talk of common backends for Python, Perl, and Ruby goaded me into
revisiting the Rattlesnake stuff I laid aside several years ago.  Assuming I
ever get anywhere with it, it would be nice if code objects could
distinguish instruction sets based on a flag in the PyCodeObject struct.
The co_flags field seems to have some room and be more-or-less the right
place for this stuff.  I propose that it be changed from signed to unsigned
int and that three bits be reserved to identify an instruction set.  Eight
possible instruction sets might seem a bit much, but I'd rather have a
little room for growth.  If we count the current instruction set, the
Rattlesnake (register) stuff I've been playing with, and Armin Rigo's Psyco
VM as distinct instruction sets, we've already used three of the possible
eight.  I'm still fiddling with a 1.5.2 code base and am currently only
using one bit in co_flags to distinguish the instruction set, but I do use
it to indirect through a two-element array of function pointers and call the
appropriate variant of eval_code2 (now eval_frame).

Just to whet peoples' appetites a bit...

I'm struggling to get conditional opcodes working at the moment, but have
had pretty good success eliding unnecessary loads and stores in straight
blocks of code.  Given this trivial function:

    def f(a):
      b = a + 4
      c = a + b
      return c

The Rattlesnake optimizer can convert it from

    >>    0 LOAD_FAST           0 (a)
          3 LOAD_CONST          1 (4)
          6 BINARY_ADD     
          7 STORE_FAST          1 (b)
         10 LOAD_FAST           0 (a)
         13 LOAD_FAST           1 (b)
         16 BINARY_ADD     
         17 STORE_FAST          2 (c)
         20 LOAD_FAST           2 (c)
         23 RETURN_VALUE   

to

    >>    0 (0336) LOAD_CONST_REG             %r4, 4
          3 (0077) BINARY_ADD_REG          b, a, %r4
          7 (0077) BINARY_ADD_REG            c, a, b
         11 (0075) RETURN_VALUE_REG                c

Needless to say, I expect it to run a bit faster than the original code.

Rattlesnake takes advantage of a property of frame objects that Tim pointed
out to me a long time ago, namely that the frame's locals and its temporary
stack space are contiguous and can just be treated as a single register
file.  In fact, the code above can just as easily be written as

    >>    0 (0336) LOAD_CONST_REG                 %r3, 4
          3 (0077) BINARY_ADD_REG          %r1, %r0, %r3
          7 (0077) BINARY_ADD_REG          %r2, %r0, %r1
         11 (0075) RETURN_VALUE_REG                  %r2

Skip


From guido@python.org  Wed Aug  8 05:46:45 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 08 Aug 2001 00:46:45 -0400
Subject: [Python-Dev] Free threading
In-Reply-To: Your message of "Wed, 08 Aug 2001 00:31:48 EDT."
 <LNBBLJKPBEHFEDALKOLCGEDHLEAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCGEDHLEAA.tim.one@home.com>
Message-ID: <200108080446.AAA31230@cj20424-a.reston1.va.home.com>

> [Guido]
> > ...
> > Anyway, I don't see the point: if you want two 100% independent
> > interpreters, you can get that easily by forking two separate
> > processes.  That's what separate processes are *for*...
> 
> So what is it that multiple interpreters within a single process *are* for?
> I'm comfortable with threads, and comfortable with processes, but multiple
> interpreters not-really-sharing yet not-really-isolated seem mostly an
> opportunity to track down subtle bugs <wink>.

It was always feature number one requested by folks coming from Tcl,
where creating multiple interpreters (threaded or not) is an
acceptable pastime for people of good manners.  But it seems to be
wasted on the average Pythoneer.

In Python, there's one good use I can think of: an application may
embed Python and occasionally want to run a good chunk of Python code.
If it needs each chunk of Python code to run completely independent
from each other chunk, creating a new interpreter for each chunk is a
good way to avoid that changes to e.g. sys.path or tabnanny's globals
made by one chunk affects the next chunk.  The hosting application
might want the execution to share memory with *it*, so these
interpreters have to live in the same process.

This works well if the next chunk is only started after the previous
chunk is done; but then using Py_Initialize() and Py_Finalize() is
enough (creating one interpreter at a time).  If the chunks need to
overlap in time, creating multiple interpreters is the right solution.

But apparently this habit has fallen in disuse -- from looking at the
code I believe there's a serious problem with exceptions, where
initializing the second interpreter messes up the identities of the
exception classes stored in global variables by the first interpreter,
and nobody has ever complained about this.  Or maybe they are still
using Python 1.5.2, which may not have the same problem.  Or maybe
this is the reason why PyApache has a reptabion of instability. :(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@home.com  Wed Aug  8 07:21:05 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 8 Aug 2001 02:21:05 -0400
Subject: [Python-Dev] Windows build broken, Unix dweeb should fix
Message-ID: <LNBBLJKPBEHFEDALKOLCGEDOLEAA.tim.one@home.com>

First

....\Python\bltinmodule.c(595) : warning C4013: 'S_ISDIR' undefined;
   assuming extern returning int

on the last line here:

	/* Test for existence or directory. */
	if (!stat(filename, &s)) {
		if (S_ISDIR(s.st_mode))

and later it fails to link cuz S_ISDIR doesn't exist.

Now bltinmodule.c doesn't even include <sys/stat.h>, so I don't know why
we'd *expect* the new code to work (although I know why *I* would expect it
to work <wink> -- see below).

getpath.c does import sys/stat.h, then hides this stuff 100 lines later:

#ifndef S_ISREG
#define S_ISREG(x) (((x) & S_IFMT) == S_IFREG)
#endif

#ifndef S_ISDIR
#define S_ISDIR(x) (((x) & S_IFMT) == S_IFDIR)
#endif

import.c doesn't even trust S_IFMT to work:

#if defined(PYCC_VACPP)
/* VisualAge C/C++ Failed to Define MountType Field in sys/stat.h */
#define S_IFMT (S_IFDIR|S_IFCHR|S_IFREG)
#endif

#ifndef S_ISDIR
#define S_ISDIR(mode) (((mode) & S_IFMT) == S_IFDIR)
#endif

Out-of-synch preprocessor-trick duplication like this is Major Evil, and I
don't want to see some bastard child of those duplicated into bltinmodule.c
too.  Windows doesn't define S_ISDIR, so at least that macro is needed.  The
only fellow doing an OS/2 port I know of now is Andrew MacIntyre, and I
happen to know that, as of Tuesday <wink>, he neither uses nor has access to
a VisualAge C compiler -- so no telling whether the PYCC_VACPP silliness is
still needed.  I'd drop it!

Who wants to move this crap into pyport.h?  I would, but I don't want to
break the Unix builds, or research 57 varieties of Unix spellings.  pyport.h
already #includes <sys/stat.h> on Windows, so for Windows it suffices to add
just the macros there (assuming these macros are worth the trouble of
defining at all -- S_ISDIR() is only used once in each of these files, and I
don't happen to find S_ISDIR(x) any clearer than its expansion).



From paulp@ActiveState.com  Wed Aug  8 07:29:20 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Tue, 07 Aug 2001 23:29:20 -0700
Subject: [Python-Dev] Free threading
References: <LNBBLJKPBEHFEDALKOLCCEONLDAA.tim.one@home.com> <20010807101051.T626@xs4all.nl> <3B702661.DB18CD36@ActiveState.com> <200108071746.f77Hk0I31236@odiug.digicool.com> <3B704CBE.88F7FC7A@ActiveState.com> <200108072117.RAA29094@cj20424-a.reston1.va.home.com>
 <3B708E3A.618374A2@ActiveState.com> <200108080409.AAA29799@cj20424-a.reston1.va.home.com>
Message-ID: <3B70DC3F.659E53A4@ActiveState.com>

Guido van Rossum wrote:
> 
>...
> 
> Yeah, but it's just not worth rewriting all of Python to get rid of
> the GIL for *that*.  

I'm not campaigning against the GIL. That's a separate issue.

I'm asking why all of the Python interpreters in a particular process
have to share *anything*. My naive view is that avoiding process-wide
data is just good practice in any library. Of course processes that WANT
to share data must share a GIL until Python is free-threaded.

For my own understanding I am curious, do "unrelated" Python
interpreters share state:

 a) because that makes the implementation of the Python interpreter
easier or 
 b) because it makes it easier to write extending and embedding apps
(don't have to pass around a context object)?
 c) that's just how it evolved.

> ... There are plenty of other ways to deal with it,
> including pre-forked longer-running processes -- and remember, it's
> only necessary if you need more than one CPU just for running your
> Python code!  (E.g. Zope, which has a perfectly good Zope-specific
> solution if you need multiple CPUs: ZEO.)

Yes, you can always work around it. But any non-reentrant libary can be
worked around. The only difference is that Python is a language so the
library provides an inherent mechanism for working around the issue. I
admit that in most cases this is a strong enough practical argument.

>...
> Good question.  I've never used mmap myself. :-)  I know Unix shared
> memory has locks and semaphores; the mmap module apparently doesn't
> (possibly because Windows has a different philosophy there).

So do you know of any fast, portable mechanism for cross-process
locking? That's the only thing between us and a nice queue-and-marshal
based IPC mechanism!

-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From loewis@informatik.hu-berlin.de  Wed Aug  8 08:13:32 2001
From: loewis@informatik.hu-berlin.de (Martin von Loewis)
Date: Wed, 8 Aug 2001 09:13:32 +0200 (MEST)
Subject: [Python-Dev] PEP comments, bugs, and patches
Message-ID: <200108080713.JAA21365@pandora.informatik.hu-berlin.de>

I just found that PEP 1 is confusing in its instructions of how to
submit feedback on a PEP. In one place, it says

#  A better strategy is to encourage public feedback directly to the
#  PEP author, who collects and integrates the comments back into the
#  PEP.

which suggest that proposed amendments to a PEP should be sent to the
author. Later, it says

# Please use the SourceForge bug manager[6] if you want to report
# problems with PEPs, or better yet, the SourceForge patch manager[2]
# for submitting patches to PEPs.

which suggest that proposed amendments should be sent as patches to
SF. Now, which of these procedures is preferred? Does the second mean
that every PEP author needs permission to modify the patch database?
Should every such patch be assigned to the PEP editor (i.e. Barry)?

In case you wonder, the specific patch in question is #448841.

Regards,
Martin



From martin@strakt.com  Wed Aug  8 08:14:04 2001
From: martin@strakt.com (Martin Sjögren)
Date: Wed, 8 Aug 2001 09:14:04 +0200
Subject: [Python-Dev] BEGIN_ALLOW_THREADS
In-Reply-To: <20010723142157.B16665@strakt.com>
References: <20010723142157.B16665@strakt.com>
Message-ID: <20010808091404.A11990@strakt.com>

On Mon, Jul 23, 2001 at 02:21:57PM +0200, Martin Sj=F6gren wrote:
> Is there a reason the Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS
> don't allow an argument specifying what variable to save the state to? =
I
> needed this myself so I wrote the following:
>=20
> #ifdef WITH_THREAD
> #  define MY_BEGIN_ALLOW_THREADS(st)    \
>     { st =3D PyEval_SaveThread(); }
> #  define MY_END_ALLOW_THREADS(st)      \
>     { PyEval_RestoreThread(st); st =3D NULL; }
> #else
> #  define MY_BEGIN_ALLOW_THREADS(st)
> #  define MY_END_ALLOW_THREADS(st)      { st =3D NULL; }
> #endif
>=20
> It works just fine but has one drawback: Whenever Py_BEGIN_ALLOW_THREAD=
S
> changes, I have to change my macros too.
>=20
> Wouldn't it be reasonable to supply two sets of macros, one that allows
> exactly this, and one that does what Py_BEGIN_ALLOW_THREADS currently
> does.

If you're wondering why I'm replying to my own age-old-mail, it is becaus=
e
I noticed the following snippet in python-announce-list:

"""
Martin Sj=F6gren wondered why the Py_BEGIN_ALLOW_THREADS and
Py_END_ALLOW_THREADS macros don't take an argument specifying a variable
to save the thread state to.  He said he needed this, but didn't explain
why.
"""

Looking back I see that, no, I did not motivate that one bit :)

Here's the scoop:

One of the functions (say foo()) in my extension module calls another
function (real_foo()) which might block since it does creepy things like
I/O.  Being a nice fella I surrounded the call with Py_BEGIN_ALLOW_THREAD=
S
and Py_END_ALLOW_THREADS so that other threads could jump in and do stuff.

The problem is that real_foo() might call a previously defined callback
(written in C) which in turn has to call a Python callback (what good is
an extension module if you have to write your callbacks in C?), but when =
I
try to PyEval_CallObject() it - KABOOM!  Segfault somewhere in the
interpreter (on PyFrame_New() or something like that).

After a lot of thinking (and a lot of help from python-list) I figured ou=
t
that the state I push away using Py_BEGIN_ALLOW_THREADS must be retrieved
again before the callback is called, or PyFrame_New() won't like me.
Problem is that that state variable is declared in Py_BEGIN_ALLOW_THREADS
and there's no way to reach it (and it also includes a naughty { with the
corresponding } in the Py_END_ALLOW_THREADS so it won't work to use them
anyway).

My solution: I wrote the two macros above.  This works for me, but
obviously I have to update my macros when the "real" macros in the Python
distribution change.


Okay, that's a more thorough explanation, I hope everybody are happy :)

Martin

--=20
Martin Sj=F6gren
  martin@strakt.com              ICQ : 41245059
  Phone: +46 (0)31 405242        Cell: +46 (0)739 169191
  GPG key: http://www.strakt.com/~martin/gpg.html


From loewis@informatik.hu-berlin.de  Wed Aug  8 08:23:29 2001
From: loewis@informatik.hu-berlin.de (Martin von Loewis)
Date: Wed, 8 Aug 2001 09:23:29 +0200 (MEST)
Subject: [Python-Dev] Windows build broken, Unix dweeb should fix
Message-ID: <200108080723.JAA21739@pandora.informatik.hu-berlin.de>

> Who wants to move this crap into pyport.h? 

I committed the execfile patch, so I'll also gonna do that part.
So just expect more breakage later :-)

Regards,
Martin


From mal@lemburg.com  Wed Aug  8 09:00:37 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 08 Aug 2001 10:00:37 +0200
Subject: [Python-Dev] PEP comments, bugs, and patches
References: <200108080713.JAA21365@pandora.informatik.hu-berlin.de>
Message-ID: <3B70F1A5.59F81B84@lemburg.com>

Martin von Loewis wrote:
> 
> I just found that PEP 1 is confusing in its instructions of how to
> submit feedback on a PEP. In one place, it says
> 
> #  A better strategy is to encourage public feedback directly to the
> #  PEP author, who collects and integrates the comments back into the
> #  PEP.
> 
> which suggest that proposed amendments to a PEP should be sent to the
> author. Later, it says
> 
> # Please use the SourceForge bug manager[6] if you want to report
> # problems with PEPs, or better yet, the SourceForge patch manager[2]
> # for submitting patches to PEPs.
> 
> which suggest that proposed amendments should be sent as patches to
> SF. Now, which of these procedures is preferred? Does the second mean
> that every PEP author needs permission to modify the patch database?
> Should every such patch be assigned to the PEP editor (i.e. Barry)?
> 
> In case you wonder, the specific patch in question is #448841.

Just to clarify this a bit: I told Bob (the contributor) to upload
the patch to SF so that I don't lose track of it. I wasn't aware
that there actually is a PEP patch guideline somewhere -- for
me the SF patch manager simply helps managing patch reviews, so
I thought it makes good sense to tell people to upload patches
for my PEPs to SF too. 

I don't think we need to make it a requirement though -- other PEP
authors may have different tracking methods.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/



From mal@lemburg.com  Wed Aug  8 10:08:50 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 08 Aug 2001 11:08:50 +0200
Subject: [Python-Dev] Free threading
References: <LNBBLJKPBEHFEDALKOLCGEDHLEAA.tim.one@home.com>
Message-ID: <3B7101A2.1D961A1@lemburg.com>

Tim Peters wrote:
> 
> [Guido]
> > ...
> > Anyway, I don't see the point: if you want two 100% independent
> > interpreters, you can get that easily by forking two separate
> > processes.  That's what separate processes are *for*...
> 
> So what is it that multiple interpreters within a single process *are* for?

This kind of application often comes up in ASP engines where a new
Python interpreter is started for each request.

> I'm comfortable with threads, and comfortable with processes, but multiple
> interpreters not-really-sharing yet not-really-isolated seem mostly an
> opportunity to track down subtle bugs <wink>.

mod_python et al. stumbled into these often enough ... it would be
a Good Thing for these engines to have *all* global data of the
interpreter on the heap rather than in static C globals.

Extensions would have to use the same logic though, so implementing
this won't be easy and will probably take a long time to get
standardized. Not sure whether it's worth the pain...

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From mwh@python.net  Wed Aug  8 10:13:37 2001
From: mwh@python.net (Michael Hudson)
Date: 08 Aug 2001 05:13:37 -0400
Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12
In-Reply-To: Michael Hudson's message of "30 Jul 2001 14:16:26 -0400"
References: <200107271949.PAA27171@cj20424-a.reston1.va.home.com> <2mn15pwg56.fsf@starship.python.net> <200107281357.JAA30859@cj20424-a.reston1.va.home.com> <2m3d7emged.fsf@starship.python.net>
Message-ID: <2mvgjy29tq.fsf@starship.python.net>

Michael Hudson <mwh@python.net> writes:

> Here's a fairly short pre-PEP on the issue.  If I haven't made any
> gross editorial blunders, can Barry give it a number and check the
> sucker in?

And here's a diff between what I posted and what I now have:

*** micro-pep1	Wed Aug  8 05:00:36 2001
--- micro-pep	Wed Aug  8 05:08:16 2001
***************
*** 1,13 ****
  PEP: XXXX
  Title: Supporting __future__ statements in simulated shells
! Version: $Version:$
  Author: Michael Hudson <mwh@python.net>
  Status: Draft
  Type: Standards Track
  Requires: 0236
  Created: 30-Jul-2001
  Python-Version: 2.2
! Post-History: 
  
  Abstract
  
--- 1,13 ----
  PEP: XXXX
  Title: Supporting __future__ statements in simulated shells
! Version: 2
  Author: Michael Hudson <mwh@python.net>
  Status: Draft
  Type: Standards Track
  Requires: 0236
  Created: 30-Jul-2001
  Python-Version: 2.2
! Post-History: 30-Jul-2001
  
  Abstract
  
***************
*** 24,39 ****
  Specification
  
      I propose adding a fourth, optional, "flags" argument to the
!     builtin "compile" function.  If this argument is omitted, there
!     will be no change in behaviour from that of Python 2.1.
  
      If it is present it is expected to be an integer, representing
      various possible compile time options as a bitfield.  The
      bitfields will have the same values as the PyCF_* flags #defined
!     in Include/pythonrun.h (at the time of writing there are only two
!     - PyCF_NESTED_SCOPES and PyCF_GENERATORS).  These are currently
!     not exposed to Python, so I propose adding them to codeop.py
!     (because it's already here, basically).
  
      XXX Should the supplied flags be or-ed with the flags of the
      calling frame, or do we override them?  I'm for the former,
--- 24,39 ----
  Specification
  
      I propose adding a fourth, optional, "flags" argument to the
!     builtin "compile" function.  If this argument is omitted,
!     there will be no change in behaviour from that of Python 2.1.
  
      If it is present it is expected to be an integer, representing
      various possible compile time options as a bitfield.  The
      bitfields will have the same values as the PyCF_* flags #defined
!     in Include/pythonrun.h (at the time of writing there are three -
!     PyCF_NESTED_SCOPES, PyCF_GENERATORS and PyCF_DIVISION).  These are
!     currently not exposed to Python, so I propose adding them to
!     codeop.py (because it's already here, basically).
  
      XXX Should the supplied flags be or-ed with the flags of the
      calling frame, or do we override them?  I'm for the former,
***************
*** 77,87 ****
      statement is added.  Such events will hopefully be very rare, so
      such a burden is unlikely to cause significant pain.
  
  Implementation
  
!     None yet; none of the above should be at all hard.  If this draft
!     is well received, I'll upload a patch to sf "soon" and point to it
!     here.
  
  Copyright
  
--- 77,107 ----
      statement is added.  Such events will hopefully be very rare, so
      such a burden is unlikely to cause significant pain.
  
+ Issues
+ 
+     Paul Prescod has reasonably complained about the choice of a
+     bitfield as the fourth argument to compile(), claiming it is
+     obscure and unpythonic.
+ 
+     There is also the thought of Jython compatibility; because Jython
+     has the ability to access any Java object without the PyObject
+     cruft needed in CPython, Jython already has a Python-visible
+     CompilerFlags object which has a boolean attribute
+     "compiler_flags", and will presumably have one fairly soon called
+     "generators".  This would be doable in CPython, but it would
+     require more hacking of deep magical bits of the interpreter and
+     require bumping the PYTHON_API_VERSION (OTOH, the division patch
+     just went in, so there can't be a freeze on the C API just
+     yet...).
+ 
  Implementation
  
!     I've uploaded a preliminary implementation as:
! 
!         http://sourceforge.net/tracker/?func=detail&atid=305470&aid=449043&group_id=5470
! 
!     I need to add docs and possibly implment a friendlier interface to
!     compile().
  
  Copyright
  

Basically I want to know if I have licence to go mucking around in the
PyEval_ interface and objects, doing things like making
PyCompilerFlags a PyObject and changing it's fields.

Cheers,
M.

-- 
59. In English every word can be verbed. Would that it were so in
    our programming languages.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html


From thomas@xs4all.net  Wed Aug  8 12:29:33 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Wed, 8 Aug 2001 13:29:33 +0200
Subject: [Python-Dev] PyChecker v0.8
In-Reply-To: <200108071237.IAA27282@cj20424-a.reston1.va.home.com>
References: <200108071237.IAA27282@cj20424-a.reston1.va.home.com>
Message-ID: <20010808132933.X626@xs4all.nl>

On Tue, Aug 07, 2001 at 08:37:59AM -0400, Guido van Rossum wrote:

> I just saw another announcement of PyChecker.  It's definitely growing
> to become a very useful (if chatty :-) tool!  How about we attempt to
> fix at least all the real bugs it detects in the standard library,
> such as the typos in urllib2.py ...  Please help!

Well, I'm not sure how you're running it, but Python 2.2 is breaking
PyChecker :-) First off, I get a ton of these:

/home/thomas/python/python-CVS/dist/src/Lib/copy.py:303: Object (l) has no
attribute (append)

where (from the code) 'l' is obviously a list. The problem is this:

_BUILTINS_ATTRS = { types.StringType : dir(''),
                    types.TypeType : dir(type(0)),
                    types.ListType : dir([]),
[etc]

I guess that should now be

def compat_dir(o):
    return dir(o) or dir(type(o)) # Or perhaps just a switch on python version

_BUILTINS_ATTRS = { types.StringType : compat_dir(''),
                    types.TypeType : compat_dir(type(0)),
[etc]

Looking at the error output, I suspect there's some more incompatibilities
in there, including one that causes pychecker to stop processing a couple of
modules.

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From gmcm@hypernet.com  Wed Aug  8 14:45:07 2001
From: gmcm@hypernet.com (Gordon McMillan)
Date: Wed, 8 Aug 2001 09:45:07 -0400
Subject: [Python-Dev] Free threading
In-Reply-To: <200108080409.AAA29799@cj20424-a.reston1.va.home.com>
References: Your message of "Tue, 07 Aug 2001 17:56:27 PDT."             <3B708E3A.618374A2@ActiveState.com>
Message-ID: <3B710A23.4832.3BFB3AA@localhost>

[Paul]
> > Please excuse my ignorance: how do you do locking in an mmap
> > solution?
[Guido]
> Good question.  I've never used mmap myself. :-)  I know Unix
> shared memory has locks and semaphores; the mmap module
> apparently doesn't (possibly because Windows has a different
> philosophy there).

Right, but Windows has named events and semaphores 
(cross process), so as long as your definition of "cross-
platform" is Windows / *nix, you're in good shape <wink>.


- Gordon


From gmcm@hypernet.com  Wed Aug  8 14:45:07 2001
From: gmcm@hypernet.com (Gordon McMillan)
Date: Wed, 8 Aug 2001 09:45:07 -0400
Subject: [Python-Dev] Free threading
In-Reply-To: <200108080446.AAA31230@cj20424-a.reston1.va.home.com>
References: Your message of "Wed, 08 Aug 2001 00:31:48 EDT."             <LNBBLJKPBEHFEDALKOLCGEDHLEAA.tim.one@home.com>
Message-ID: <3B710A23.22358.3BFB205@localhost>

[Guido]

Py_Initialize ... Py_Finalize, Py_Initialize ...

> But apparently this habit has fallen in disuse -- from looking at
> the code I believe there's a serious problem with exceptions,
> where initializing the second interpreter messes up the
> identities of the exception classes stored in global variables by
> the first interpreter, and nobody has ever complained about this.

Last week on c.l.py some embedder complained about this. 
On the second interpreter, he got an exception saying "this 
constructor takes no args", but the line in question was a 
function (taking no args).

- Gordon


From guido@python.org  Wed Aug  8 15:36:39 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 08 Aug 2001 10:36:39 -0400
Subject: [Python-Dev] Free threading
In-Reply-To: Your message of "Tue, 07 Aug 2001 23:29:20 PDT."
 <3B70DC3F.659E53A4@ActiveState.com>
References: <LNBBLJKPBEHFEDALKOLCCEONLDAA.tim.one@home.com> <20010807101051.T626@xs4all.nl> <3B702661.DB18CD36@ActiveState.com> <200108071746.f77Hk0I31236@odiug.digicool.com> <3B704CBE.88F7FC7A@ActiveState.com> <200108072117.RAA29094@cj20424-a.reston1.va.home.com> <3B708E3A.618374A2@ActiveState.com> <200108080409.AAA29799@cj20424-a.reston1.va.home.com>
 <3B70DC3F.659E53A4@ActiveState.com>
Message-ID: <200108081436.KAA32493@cj20424-a.reston1.va.home.com>

> I'm asking why all of the Python interpreters in a particular process
> have to share *anything*. My naive view is that avoiding process-wide
> data is just good practice in any library. Of course processes that WANT
> to share data must share a GIL until Python is free-threaded.

They share anything that's implemented as a global resource.  This
includes the caches and free lists for small strings and ints, the
'intern' database, etc.

> For my own understanding I am curious, do "unrelated" Python
> interpreters share state:
> 
>  a) because that makes the implementation of the Python interpreter
> easier or 
>  b) because it makes it easier to write extending and embedding apps
> (don't have to pass around a context object)?
>  c) that's just how it evolved.

All of the above.  Otherwise, *every* API would have to carry an
"interp" argument around (as does Tcl/Tk).

> > ... There are plenty of other ways to deal with it,
> > including pre-forked longer-running processes -- and remember, it's
> > only necessary if you need more than one CPU just for running your
> > Python code!  (E.g. Zope, which has a perfectly good Zope-specific
> > solution if you need multiple CPUs: ZEO.)
> 
> Yes, you can always work around it. But any non-reentrant libary can be
> worked around. The only difference is that Python is a language so the
> library provides an inherent mechanism for working around the issue. I
> admit that in most cases this is a strong enough practical argument.

Sorry, I don't understand what you mean.  What does Python being a
language buy you?

> >...
> > Good question.  I've never used mmap myself. :-)  I know Unix shared
> > memory has locks and semaphores; the mmap module apparently doesn't
> > (possibly because Windows has a different philosophy there).
> 
> So do you know of any fast, portable mechanism for cross-process
> locking? That's the only thing between us and a nice queue-and-marshal
> based IPC mechanism!

Sorry, don't look at me.  On Unix, there's something referred to as
"System V IPC".  For Windows, look at the various Win32 API calls to
create locks -- I believe you can name them and the names are global.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Aug  8 15:40:11 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 08 Aug 2001 10:40:11 -0400
Subject: [Python-Dev] BEGIN_ALLOW_THREADS
In-Reply-To: Your message of "Wed, 08 Aug 2001 09:14:04 +0200."
 <20010808091404.A11990@strakt.com>
References: <20010723142157.B16665@strakt.com>
 <20010808091404.A11990@strakt.com>
Message-ID: <200108081440.KAA32514@cj20424-a.reston1.va.home.com>

> One of the functions (say foo()) in my extension module calls another
> function (real_foo()) which might block since it does creepy things like
> I/O.  Being a nice fella I surrounded the call with Py_BEGIN_ALLOW_THREADS
> and Py_END_ALLOW_THREADS so that other threads could jump in and do stuff.
> 
> The problem is that real_foo() might call a previously defined callback
> (written in C) which in turn has to call a Python callback (what good is
> an extension module if you have to write your callbacks in C?), but when I
> try to PyEval_CallObject() it - KABOOM!  Segfault somewhere in the
> interpreter (on PyFrame_New() or something like that).
> 
> After a lot of thinking (and a lot of help from python-list) I figured out
> that the state I push away using Py_BEGIN_ALLOW_THREADS must be retrieved
> again before the callback is called, or PyFrame_New() won't like me.
> Problem is that that state variable is declared in Py_BEGIN_ALLOW_THREADS
> and there's no way to reach it (and it also includes a naughty { with the
> corresponding } in the Py_END_ALLOW_THREADS so it won't work to use them
> anyway).
> 
> My solution: I wrote the two macros above.  This works for me, but
> obviously I have to update my macros when the "real" macros in the Python
> distribution change.

Instead of asking us to add more macros, I recommend that you use the
lower-level C API that is used by the macros.  There's a bunch of ways
to save and restore the state.  The macros are just shortcuts (and a
handy way to make all the code disappear when threads don't exist).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Aug  8 15:47:37 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 08 Aug 2001 10:47:37 -0400
Subject: [Python-Dev] PEP comments, bugs, and patches
In-Reply-To: Your message of "Wed, 08 Aug 2001 09:13:32 +0200."
 <200108080713.JAA21365@pandora.informatik.hu-berlin.de>
References: <200108080713.JAA21365@pandora.informatik.hu-berlin.de>
Message-ID: <200108081447.KAA32556@cj20424-a.reston1.va.home.com>

> I just found that PEP 1 is confusing in its instructions of how to
> submit feedback on a PEP. In one place, it says
> 
> #  A better strategy is to encourage public feedback directly to the
> #  PEP author, who collects and integrates the comments back into the
> #  PEP.
> 
> which suggest that proposed amendments to a PEP should be sent to the
> author. Later, it says
> 
> # Please use the SourceForge bug manager[6] if you want to report
> # problems with PEPs, or better yet, the SourceForge patch manager[2]
> # for submitting patches to PEPs.
> 
> which suggest that proposed amendments should be sent as patches to
> SF. Now, which of these procedures is preferred? Does the second mean
> that every PEP author needs permission to modify the patch database?
> Should every such patch be assigned to the PEP editor (i.e. Barry)?

What's best may depend on the stage a PEP is in.  In the initial
stages of a PEP, when the author is actively working on it, it may be
more efficient to write the author than go through the (somewhat
heavier) SF mechanisms.  But later, when editing is less frantic, I
see the advantages of the SF tracker.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido@python.org  Wed Aug  8 16:04:03 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 08 Aug 2001 11:04:03 -0400
Subject: [Python-Dev] Free threading
In-Reply-To: Your message of "Wed, 08 Aug 2001 09:45:07 EDT."
 <3B710A23.22358.3BFB205@localhost>
References: Your message of "Wed, 08 Aug 2001 00:31:48 EDT." <LNBBLJKPBEHFEDALKOLCGEDHLEAA.tim.one@home.com>
 <3B710A23.22358.3BFB205@localhost>
Message-ID: <200108081504.LAA32734@cj20424-a.reston1.va.home.com>

> Last week on c.l.py some embedder complained about this. 
> On the second interpreter, he got an exception saying "this 
> constructor takes no args", but the line in question was a 
> function (taking no args).

And nobody filed a bug report?!?!?!

OK, I'm filing one now.

http://sourceforge.net/tracker/index.php?func=detail&aid=449151&group_id=5470&atid=105470

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Aug  8 16:31:29 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 08 Aug 2001 11:31:29 -0400
Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12
In-Reply-To: Your message of "08 Aug 2001 05:13:37 EDT."
 <2mvgjy29tq.fsf@starship.python.net>
References: <200107271949.PAA27171@cj20424-a.reston1.va.home.com> <2mn15pwg56.fsf@starship.python.net> <200107281357.JAA30859@cj20424-a.reston1.va.home.com> <2m3d7emged.fsf@starship.python.net>
 <2mvgjy29tq.fsf@starship.python.net>
Message-ID: <200108081531.LAA00619@cj20424-a.reston1.va.home.com>

Hi Michael,

Barry's on vacation, so I've done the honors.  It's checked in as PEP
264.

I like this a lot -- I hope we can add this to 2.2a2 (which was due
today but is postponed for at least a week).

I think the integral flags to compile() are fine, and 'or' is indeed
the right thing.  After reading the PEP, I understand the refactoring
that I complained about in a comment on the patch.

Good job!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@loewis.home.cs.tu-berlin.de  Wed Aug  8 16:55:30 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Wed, 8 Aug 2001 17:55:30 +0200
Subject: [Python-Dev] Free threading
Message-ID: <200108081555.f78FtUx01194@mira.informatik.hu-berlin.de>

> All of the above.  Otherwise, *every* API would have to carry an
> "interp" argument around (as does Tcl/Tk).

Not necessarily. The interp argument could be stored in thread-local
storage. Of course, using TLS opens another can of worms.

Regards,
Martin



From Samuele Pedroni <pedroni@inf.ethz.ch>  Wed Aug  8 17:21:23 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Wed, 8 Aug 2001 18:21:23 +0200 (MET DST)
Subject: [Python-Dev] use co_flags to identify instruction set
Message-ID: <200108081621.SAA03060@core.inf.ethz.ch>

> All this talk of common backends for Python, Perl, and Ruby goaded me into
> revisiting the Rattlesnake stuff I laid aside several years ago.  Assuming I
> ever get anywhere with it, it would be nice if code objects could
> distinguish instruction sets based on a flag in the PyCodeObject struct.
> The co_flags field seems to have some room and be more-or-less the right
> place for this stuff. 
I'm wondering whether this is a good idea, there are some tools (at least inspect) in the std 
lib that depend on code objects to just contain the old good pvm bytecode, PyChecker also
needs that (but we could have a flag to avoid generating strange code for its use...)

As marginal note: for this reason they fail miserably under Jython, but that's clearly
Jython fault.

My point is that the above examples are perfectly meaningful and legitimate, and it is a good 
tradition and practice (Self, Smalltalk) even if you're seeking for speed, to present and accept
code in a uniform manner, trough just one bytecode fortmat. Yes this way you waste memory,
but ir is just the typical  trade-off flexibility and speed vs. memory, if you have few memory 
you will simply disable your optimizer.

A different point is to define and always use a new bytecode format. Or to banish code
that rely on bytecode format for introspection ... I imagine that at some point someone will 
regret that.

We should consider that AFAIK Psyco don't really produce in one shot a new stream
of bytecode and on the other hand uses and keeps around  a fairly complicated graph structure 
together with the chunks of bytecode (that's why it can probably be applied only to hot spots). 
I wonder if that fits in the old PyCodeObject model anyway.
 
regards, Samuele Pedroni.



From dan@sidhe.org  Wed Aug  8 17:29:51 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Wed, 08 Aug 2001 12:29:51 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <15216.26224.856076.269695@cj42289-a.reston1.va.home.com>
References: <5.1.0.14.0.20010807174100.023c1c58@tuatha.sidhe.org>
 <200108072135.XAA08800@core.inf.ethz.ch>
 <5.1.0.14.0.20010807174100.023c1c58@tuatha.sidhe.org>
Message-ID: <5.1.0.14.0.20010808122734.01c04648@tuatha.sidhe.org>

At 06:06 PM 8/7/2001 -0400, Fred L. Drake, Jr. wrote:

>Dan Sugalski writes:
>  > FWIW I've thrown the first partial draft of the Parrot assembly language
>  > spec out to the wolves. URL and details available to the interested.
>
>   I would be surprised if there were many people on python-dev who
>aren't interested.  I don't know how many aren't in the throws of
>wrestling with "The Real World" though.

Okay, the docs in progress live in:

http://www.sidhe.org/~dan/perl

in this case we're looking specifically at the perl_assembly.html file. 
(The original is the .txt file, which is really in POD format, not plain 
text, but close enough) Some of the other docs that are in there might be 
of some interest.

If you can't get in, drop me a note--my ISP has threatened inbound port 80 
blocking courtesy of the CodeRed nonsense.

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From tim.one@home.com  Wed Aug  8 19:13:56 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 8 Aug 2001 14:13:56 -0400
Subject: [Python-Dev] FW: if-else with '|' operator - beginner's question/problem
Message-ID: <LNBBLJKPBEHFEDALKOLCOEGLLEAA.tim.one@home.com>

What a fascinating newbie confusion!  It surprised and tickled me.

FYI, in Icon this kind of thing "just works".  e1 | e2 | ... | en is "a
generator" there too, and acts like the Python

def g():
    for x in e1:
        yield x
    for x in e2:
        yield x
    ...
    for x in en:
        yield en

Then the expression y1 == y2 generates the cross-product of the y1 and y2
sequences, succeeding in the end iff some pair of generated values compares
equal.

No, I'm not suggesting Python do that too <wink>.  It's more in the way of
noting that no matter *how* odd a question looks to you, some peoples'
brains are so wired that it's hard for them to view it otherwise.

finally-understanding-vi-users<wink>-ly y'rs  - tim

-----Original Message-----
From: python-list-admin@python.org
Sent: Wednesday, August 08, 2001 1:26 PM
To: python-list@python.org
Subject: if-else with '|' operator - beginner's question/problem


Hi there, I wonder if someone could possibly tell me what is wrong with
the following statement. I'm extremely embarrased to ask but here
goes...

>>>  if (fname[1] == 'a'|'e'|'i'|'o'|'u'):
    vowel='true'

Traceback (innermost last):
  File "<pyshell#46>", line 1, in ?
    if (fname[1] == 'a'|'e'|'i'|'o'|'u'):
TypeError: bad operand type(s) for |

---
fname[1] gives 'e'

Thanks very much in advance!

Lee

--
http://mail.python.org/mailman/listinfo/python-list



From tim.one@home.com  Wed Aug  8 20:11:22 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 8 Aug 2001 15:11:22 -0400
Subject: [Python-Dev] PEP 264 (was RE: [Python-checkins] CVS: python/nondist/peps pep-0264.txt,NONE,1.1 pep-0000.txt,1.113,1.114)
In-Reply-To: <E15UVHj-0001Xr-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEHCLEAA.tim.one@home.com>

[Guido, checking in
    Michael Hudson's "Future statements in simulated shells".
 PEP
]
> ...
>     Paul Prescod has reasonably complained about the choice of a
>     bitfield as the fourth argument to compile(), claiming it is
>     obscure and unpythonic.

I've got a happy idea!  __future__.py is a real std module, and e.g.

    __future__.nested_scopes
    __future__.generators
    __future__.division

are honest-to-gosh Python objects, of class __future__._Feature.  That's the
perfect place to store feature-specific magic flag bits.  So, e.g., I'm
picturing

    compile(s1, s2, s3, __future__.generators.getflag())

and

    flags = 0
    for fname in list_of_feature_names:
        flags |= getattr(__future__, fname).getflag()
    compile(s1, s2, s3, flags)

etc.  For that matter, list-of-feature-strings -> flag_bits could be added
as a utility function to __future__.py.  It's Python -- there's nothing you
can't do with this in 5 seconds <wink>.



From skip@pobox.com (Skip Montanaro)  Wed Aug  8 21:29:55 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Wed, 8 Aug 2001 15:29:55 -0500
Subject: [Python-Dev] "not an acceptable base type"?
Message-ID: <15217.41283.463143.160922@beluga.mojam.com>

Are strings not yet subclassable or do I just need to "cvs up"?

    >>> class string(str):
    ...   pass
    ... 
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    TypeError: type 'str' is not an acceptable base type
    >>> "".__class__ == str
    1

Thx,

Skip


From guido@python.org  Wed Aug  8 21:33:53 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 08 Aug 2001 16:33:53 -0400
Subject: [Python-Dev] "not an acceptable base type"?
In-Reply-To: Your message of "Wed, 08 Aug 2001 15:29:55 CDT."
 <15217.41283.463143.160922@beluga.mojam.com>
References: <15217.41283.463143.160922@beluga.mojam.com>
Message-ID: <200108082033.QAA05704@cj20424-a.reston1.va.home.com>

> Are strings not yet subclassable or do I just need to "cvs up"?

Be patient.  We'll do this, but it's not yet done.  For now, you can
only subclass type, object, list, and dictionary.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@loewis.home.cs.tu-berlin.de  Wed Aug  8 21:34:47 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Wed, 8 Aug 2001 22:34:47 +0200
Subject: [Python-Dev] Any objections to adding threading.Timer?
Message-ID: <200108082034.f78KYlB16985@mira.informatik.hu-berlin.de>

In patch

https://sourceforge.net/tracker/index.php?func=detail&aid=428326&group_id=5470&atid=305470

a new threading.Timer class is proposed. Since the patch was submitted
two months ago, nobody seemed to care enough to comment on it. I like
it, and I'll commit the patch unless I hear otherwise.

Regards,
Martin



From skip@pobox.com (Skip Montanaro)  Wed Aug  8 21:43:31 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Wed, 8 Aug 2001 15:43:31 -0500
Subject: [Python-Dev] "not an acceptable base type"?
In-Reply-To: <200108082033.QAA05704@cj20424-a.reston1.va.home.com>
References: <15217.41283.463143.160922@beluga.mojam.com>
 <200108082033.QAA05704@cj20424-a.reston1.va.home.com>
Message-ID: <15217.42099.307124.333865@beluga.mojam.com>

    >> Are strings not yet subclassable or do I just need to "cvs up"?

    Guido> Be patient.  We'll do this, but it's not yet done.  For now, you
    Guido> can only subclass type, object, list, and dictionary.

Ah, thanks.  

S


From jeremy@zope.com  Wed Aug  8 21:49:02 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Wed, 8 Aug 2001 16:49:02 -0400 (EDT)
Subject: [Python-Dev] Any objections to adding threading.Timer?
In-Reply-To: <200108082034.f78KYlB16985@mira.informatik.hu-berlin.de>
References: <200108082034.f78KYlB16985@mira.informatik.hu-berlin.de>
Message-ID: <15217.42430.308688.74304@slothrop.digicool.com>

The idea sounded good in principle, but I didn't have time to look at
it.  I'll be happy with whatever you decide.

Jeremy



From mwh@python.net  Wed Aug  8 22:54:50 2001
From: mwh@python.net (Michael Hudson)
Date: 08 Aug 2001 17:54:50 -0400
Subject: [Python-Dev] PEP 264 (was RE: [Python-checkins] CVS: python/nondist/peps pep-0264.txt,NONE,1.1 pep-0000.txt,1.113,1.114)
In-Reply-To: "Tim Peters"'s message of "Wed, 8 Aug 2001 15:11:22 -0400"
References: <LNBBLJKPBEHFEDALKOLCGEHCLEAA.tim.one@home.com>
Message-ID: <2m8zgudxp1.fsf@starship.python.net>

Instant response, without that tedious-prone-to-problem-spotting
thinking: nice idea!

"Tim Peters" <tim.one@home.com> writes:

> [Guido, checking in
>     Michael Hudson's "Future statements in simulated shells".
>  PEP
> ]
> > ...
> >     Paul Prescod has reasonably complained about the choice of a
> >     bitfield as the fourth argument to compile(), claiming it is
> >     obscure and unpythonic.
> 
> I've got a happy idea!  __future__.py is a real std module, and e.g.
> 
>     __future__.nested_scopes
>     __future__.generators
>     __future__.division
> 
> are honest-to-gosh Python objects, of class __future__._Feature.  That's the
> perfect place to store feature-specific magic flag bits.  So, e.g., I'm
> picturing
> 
>     compile(s1, s2, s3, __future__.generators.getflag())

Or just 

    __future__.generators.compile_flag

presumably?

> and
> 
>     flags = 0
>     for fname in list_of_feature_names:
>         flags |= getattr(__future__, fname).getflag()
>     compile(s1, s2, s3, flags)
> 
> etc.  For that matter, list-of-feature-strings -> flag_bits could be
> added as a utility function to __future__.py.  It's Python --
> there's nothing you can't do with this in 5 seconds <wink>.

"list_of_feature_names" might as well be spelt "__future__.__all__",
I'd have thought.

There's still the icky issue of PyEval_MergeCompilerFlags to consider.
The fact that each future statement has two bits assigned to in - the
PyCF_* ones in pythonrun.h and the CO_* ones in compile.h that allow
__builtin__.compile to inherit the behaviour of surrounding code -
seems nasty.  We could just mandate that 

CO_FOO == PyCF_FOO << 12

Or add another flag to code objects.  Or do something else.

The possibility of fiddling with __future__.py had completely escaped
my mind - so thanks for pointing me at that!

Cheers,
M.

-- 
  If you don't use emacs, you're a pathetic, mewling, masochistic
  weakling and I can't be bothered to convert you.    -- Ron Echeverri


From aahz@rahul.net  Wed Aug  8 23:23:17 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Wed, 8 Aug 2001 15:23:17 -0700 (PDT)
Subject: [Python-Dev] Free threading
In-Reply-To: <200108081555.f78FtUx01194@mira.informatik.hu-berlin.de> from "Martin v. Loewis" at Aug 08, 2001 05:55:30 PM
Message-ID: <20010808222318.1D66699C81@waltz.rahul.net>

Martin v. Loewis wrote:
>Guido:
>> 
>> All of the above.  Otherwise, *every* API would have to carry an
>> "interp" argument around (as does Tcl/Tk).
> 
> Not necessarily. The interp argument could be stored in thread-local
> storage. Of course, using TLS opens another can of worms.

We need thread-local storage, anyway.  When I'm a little farther into my
work on Decimal, I'm going to take a hard look at this because context
needs to be global on a per-thread basis...
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From guido@python.org  Wed Aug  8 23:30:54 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 08 Aug 2001 18:30:54 -0400
Subject: [Python-Dev] Free threading
In-Reply-To: Your message of "Wed, 08 Aug 2001 15:23:17 PDT."
 <20010808222318.1D66699C81@waltz.rahul.net>
References: <20010808222318.1D66699C81@waltz.rahul.net>
Message-ID: <200108082230.SAA09334@cj20424-a.reston1.va.home.com>

> We need thread-local storage, anyway.  When I'm a little farther into my
> work on Decimal, I'm going to take a hard look at this because context
> needs to be global on a per-thread basis...

(Thick French accent:) "We already got one!"

In the C API, use PyThreadState_GetDict(): it returns a per-thread
dictionary.  The convention is that you pick a key unique to your
extension module (e.g. your full extension module name) and then store
your per-thread data under this key.

In Python, you can store instance variables on the current thread
object -- use threading.currentThread() to access it.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From nas@python.ca  Thu Aug  9 00:15:03 2001
From: nas@python.ca (Neil Schemenauer)
Date: Wed, 8 Aug 2001 16:15:03 -0700
Subject: [Python-Dev] "not an acceptable base type"?
In-Reply-To: <200108082033.QAA05704@cj20424-a.reston1.va.home.com>; from guido@python.org on Wed, Aug 08, 2001 at 04:33:53PM -0400
References: <15217.41283.463143.160922@beluga.mojam.com> <200108082033.QAA05704@cj20424-a.reston1.va.home.com>
Message-ID: <20010808161503.B2974@glacier.fnational.com>

Guido van Rossum wrote:
> > Are strings not yet subclassable or do I just need to "cvs up"?
> 
> Be patient.  We'll do this, but it's not yet done.  For now, you can
> only subclass type, object, list, and dictionary.

Is there something tricky about subclassing strings (and ints, floats,
...)?  I've considered spending some time trying to make other types
subclassable.  If each python-dev'er takes one it shouldn't take too
long. :-)

  Neil


From guido@python.org  Thu Aug  9 00:40:03 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 08 Aug 2001 19:40:03 -0400
Subject: [Python-Dev] "not an acceptable base type"?
In-Reply-To: Your message of "Wed, 08 Aug 2001 16:15:03 PDT."
 <20010808161503.B2974@glacier.fnational.com>
References: <15217.41283.463143.160922@beluga.mojam.com> <200108082033.QAA05704@cj20424-a.reston1.va.home.com>
 <20010808161503.B2974@glacier.fnational.com>
Message-ID: <200108082340.TAA12043@cj20424-a.reston1.va.home.com>

> Is there something tricky about subclassing strings (and ints, floats,
> ...)?  I've considered spending some time trying to make other types
> subclassable.  If each python-dev'er takes one it shouldn't take too
> long. :-)

The tricky thing is that you have to do most of the initialization in
tp_new (not tp_init), and yet you must allow a subclass to override
tp_alloc and tp_free.  Immutable types and types with their own free
list or caching definitely are trickier than the ones I did so far...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From simon@netthink.co.uk  Thu Aug  9 03:44:32 2001
From: simon@netthink.co.uk (Simon Cozens)
Date: Wed, 8 Aug 2001 19:44:32 -0700
Subject: [Python-Dev] Parrot -- let's move this thread.
In-Reply-To: <15216.26224.856076.269695@cj42289-a.reston1.va.home.com>
Message-ID: <20010808194432.B782@netthink.co.uk>

On Tue, Aug 07, 2001 at 06:06:40PM -0400, Fred L. Drake, Jr. wrote:
>   I would be surprised if there were many people on python-dev who
> aren't interested.

On the other hand, python-dev is for the development of Python, and
(for now at least, cackle cackle) Parrot isn't Python. Can we move
this thread to language-dev@netthink.co.uk, a generic list for the
implementors of various scripting languages. This is beneficial for
several reasons; primarily because it allows python-dev to get back
to work, but also because we have a lot of very Interesting People
on language-dev who can help scrutinise the ideas we're discussing.

Announcement of language-dev:
http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2001-07/msg00030.html

See also
http://aspn.activestate.com/ASPN/Mail/Browse/Threaded/language-dev

Simon


From DavidA@ActiveState.com  Thu Aug  9 01:20:29 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Wed, 08 Aug 2001 17:20:29 -0700
Subject: [Python-Dev] Parrot -- let's move this thread.
References: <20010808194432.B782@netthink.co.uk>
Message-ID: <3B71D74D.BCF20B86@ActiveState.com>

Simon Cozens wrote:

> implementors of various scripting languages. This is beneficial for
> several reasons; primarily because it allows python-dev to get back
> to work, but also because we have a lot of very Interesting People
> on language-dev who can help scrutinise the ideas we're discussing.

I especially found the post by David Simmons fascinating:


From DavidA@ActiveState.com  Thu Aug  9 01:21:53 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Wed, 08 Aug 2001 17:21:53 -0700
Subject: [Python-Dev] Parrot -- let's move this thread.
References: <20010808194432.B782@netthink.co.uk>
Message-ID: <3B71D7A1.2FDF099A@ActiveState.com>

Sigh -- hit the wrong key.

I meant to say:

I especially found the post by David Simmons interesting:

http://aspn.activestate.com/ASPN/Mail/Message/language-dev/722505

--david


From paulp@ActiveState.com  Thu Aug  9 02:11:41 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Wed, 08 Aug 2001 18:11:41 -0700
Subject: [Python-Dev] Common runtime sociology
Message-ID: <3B71E34D.7A2DD001@ActiveState.com>

It is my gut feeling that the only way to get to a common runtime (if it
is possible at all) is to take the development out of the hands of any
language group and put it in a trusted third party's hands. If any
language group feels like its concerns are going to get second-class
attention, they'll go back and focus on their own implementation.

So that's just my feeling...if the goal of some people is really to have
the different languages share implementation then the technical issues
are secondary to the project organization and leadership.

-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From dan@sidhe.org  Thu Aug  9 02:57:50 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Wed, 08 Aug 2001 21:57:50 -0400
Subject: [Python-Dev] Common runtime sociology
In-Reply-To: <3B71E34D.7A2DD001@ActiveState.com>
Message-ID: <5.1.0.14.0.20010808212559.0223fb60@tuatha.sidhe.org>

At 06:11 PM 8/8/2001 -0700, Paul Prescod wrote:
>It is my gut feeling that the only way to get to a common runtime (if it
>is possible at all) is to take the development out of the hands of any
>language group and put it in a trusted third party's hands. If any
>language group feels like its concerns are going to get second-class
>attention, they'll go back and focus on their own implementation.

I think it's possible for things to be done otherwise, but it puts a heavy 
burden on the implementation group to be actively engaged with the language 
groups and be as impartial as can be managed. You almost need to take the 
implementation group that starts this and pry them loose to be an entity in 
and of themselves, separate from the original language group. (FWIW, this 
is part of my Master Plan. Dunno if it's part of Simon's)

It's certainly the more difficult way to go than to start with an impartial 
third party. (Of course, then there's the problem of *finding* an impartial 
third party for something like this) For the record, if something like this 
does go through, I'm all for it and'll bow out of the way. (Any resemblance 
to dropped hot rocks is purely coincidental ;)

As for the common runtime, that's by far the least of the problems. I mean, 
we all manage with C, of all things, running on hardware that's not in the 
least forgiving of dynamic languages. We'd be hard pressed to find a worse 
environment to run on short of dragging out one of the old LISP or Forth 
systems...

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From skip@pobox.com (Skip Montanaro)  Thu Aug  9 03:41:04 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Wed, 8 Aug 2001 21:41:04 -0500
Subject: [Python-Dev] opcode dispatch - switch vs. function pointers
Message-ID: <15217.63552.143782.572894@beluga.mojam.com>

Popped over to the language-dev archive at ActiveState and came across this
interesting post about opcode dispatch:

    http://aspn.activestate.com/ASPN/Mail/Message/language-dev/724204

I think I'm going to experiment a little with dispatch via function pointers
in the Rattlesnake (once I get something that more-or-less works).  

Skip


From tim.one@home.com  Thu Aug  9 04:08:56 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 8 Aug 2001 23:08:56 -0400
Subject: [Python-Dev] Common runtime sociology
In-Reply-To: <3B71E34D.7A2DD001@ActiveState.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEIPLEAA.tim.one@home.com>

[Paul Prescod]
> It is my gut feeling that the only way to get to a common runtime (if it
> is possible at all) is to take the development out of the hands of any
> language group and put it in a trusted third party's hands.

Na.  There are two ways it can happen:

1. A well-funded project with this as one of its primary goals.

2. Any sort of project, funded or not, whose lead(s) is(are) obsessed
   with making it happen.

#1 may be close to MS .NET's approach, proving in part that the "common" in
"common runtime" is still a matter of manipulable definition <wink>.  Put in
many hands, it's the Industry Consortia model, famous for producing large
publicity/result ratios.

If it's just an afterthought, or a "nice to have", or even a "it's one of
the last things we'll give up -- but, ya, in the end it's expendable", it
really has slim chance of surviving the difficulties.

#2 is the best shot.  Alas, few people can *decide* to become an insane
visionary and make it stick <wink>.

craziness-is-a-gift-from-the-gods-ly y'rs  - tim



From esr@thyrsus.com  Thu Aug  9 04:21:47 2001
From: esr@thyrsus.com (Eric S. Raymond)
Date: Wed, 8 Aug 2001 23:21:47 -0400
Subject: [Python-Dev] Common runtime sociology
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEIPLEAA.tim.one@home.com>; from tim.one@home.com on Wed, Aug 08, 2001 at 11:08:56PM -0400
References: <3B71E34D.7A2DD001@ActiveState.com> <LNBBLJKPBEHFEDALKOLCKEIPLEAA.tim.one@home.com>
Message-ID: <20010808232147.A16038@thyrsus.com>

Tim Peters <tim.one@home.com>:
> #2 is the best shot.  Alas, few people can *decide* to become an insane
> visionary and make it stick <wink>.

I've managed it.  And Simon Cozens is staying over at my place tonight
(he's quietly hacking on the daybed downstairs, with our cat for company).
I think I'll show him this...

Yes.  Simon Cozens and I officially send you you a hearty:

<laugh role="mad scientist">BWAHAHAHAHAHAHA!</laugh>
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

According to the National Crime Survey administered by the Bureau of
the Census and the National Institute of Justice, it was found that
only 12 percent of those who use a gun to resist assault are injured,
as are 17 percent of those who use a gun to resist robbery. These
percentages are 27 and 25 percent, respectively, if they passively
comply with the felon's demands. Three times as many were injured if
they used other means of resistance.
        -- G. Kleck, "Policy Lessons from Recent Gun Control Research,"
		Law and Contemporary Problems 49, no. 1. (Winter 1986.): 35-62.


From tim.one@home.com  Thu Aug  9 04:47:37 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 8 Aug 2001 23:47:37 -0400
Subject: [Python-Dev] Common runtime sociology
In-Reply-To: <20010808232147.A16038@thyrsus.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEJCLEAA.tim.one@home.com>

[TIm]
> #2 is the best shot.  Alas, few people can *decide* to become an insane
> visionary and make it stick <wink>.

[Eric S. Raymond]
> I've managed it.

Damn straight, Eric!  That's one of the reasons I love you.  Born a Volvo
salesman, you spit in God's eye and decided to foment revolution instead.
And God is very pleased by this too, I hear (although His eye still burns,
and He does hold grudges).

> And Simon Cozens is staying over at my place tonight (he's quietly
> hacking on the daybed downstairs, with our cat for company).
> I think I'll show him this...
>
> Yes.  Simon Cozens and I officially send you you a hearty:
>
> <laugh role="mad scientist">BWAHAHAHAHAHAHA!</laugh>

I'll believe Simon has Crossed Over too if you can't find the cat Thursday
morning <meow>.



From esr@thyrsus.com  Thu Aug  9 04:59:40 2001
From: esr@thyrsus.com (Eric S. Raymond)
Date: Wed, 8 Aug 2001 23:59:40 -0400
Subject: [Python-Dev] Common runtime sociology
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEJCLEAA.tim.one@home.com>; from tim.one@home.com on Wed, Aug 08, 2001 at 11:47:37PM -0400
References: <20010808232147.A16038@thyrsus.com> <LNBBLJKPBEHFEDALKOLCEEJCLEAA.tim.one@home.com>
Message-ID: <20010808235940.A16661@thyrsus.com>

Tim Peters <tim.one@home.com>:
> I'll believe Simon has Crossed Over too if you can't find the cat Thursday
> morning <meow>.

Hey!  Hey!  Simon, being a gentleman of taste, *likes* cats -- and not for
breakfast either.  My wife Cathy observes, correctly, that if we can't find the
cat in the morning it will probably be because she has stowed away in his
luggage.

(The cat stowed away, I mean.  Not Cathy.  Cathy concedes that Simon is 
cute, but avers that he lacks my rugged masculine charm <wink>.)
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

Non-cooperation with evil is as much a duty as cooperation with good.
	-- Mohandas Gandhi


From tim.one@home.com  Thu Aug  9 05:31:30 2001
From: tim.one@home.com (Tim Peters)
Date: Thu, 9 Aug 2001 00:31:30 -0400
Subject: [Python-Dev] Free threading
In-Reply-To: <200108080446.AAA31230@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEJFLEAA.tim.one@home.com>

[Tim]
>> So what is it that multiple interpreters within a single process
> *are* for?

[Guidp]
> It was always feature number one requested by folks coming from Tcl,
> where creating multiple interpreters (threaded or not) is an
> acceptable pastime for people of good manners.  But it seems to be
> wasted on the average Pythoneer.

Well, I'm not a Tcl guy, but Tcl has historically had no threads at all, yet
did have an event model where the *illusion* of multiple threads could still
be put to good use.  I wouldn't fall over dead with surprise if that's all
there were to it:  Extreme Fear of Threads <wink>.

I believe I've seen three requests more common from Tcl land, least to most
common:

1. The Tcl event model indeed.

2. Tcl's more robust (than Python's) x-platform exec implementation
   (similar to Python os.system(), but with a platform-independent
   notation for pipes and redirection and envar substitution, and
   returning the exit status correctly even under command.com-based
   Windows flavors (where os.system() always returns 0)).  I looked
   at this once -- there are several thousand lines of exec support
   code in Tcl (which shouldn't be surprising, since they're basically
   supplying one of the hardest parts of a x-platform shell!).

3. Variants of Tcl's uplevel, i.e. gimmicks for fiddling locals
   "up the stack".  Maybe it's not called exactly "uplevel" in Tcl,
   but it's plain sick however it's spelled <wink>.

> In Python, there's one good use I can think of: an application may
> embed Python and occasionally want to run a good chunk of Python code.
> If it needs each chunk of Python code to run completely independent
> from each other chunk, creating a new interpreter for each chunk is a
> good way to avoid that changes to e.g. sys.path or tabnanny's globals
> made by one chunk affects the next chunk.  The hosting application
> might want the execution to share memory with *it*, so these
> interpreters have to live in the same process.
>
> This works well if the next chunk is only started after the previous
> chunk is done; but then using Py_Initialize() and Py_Finalize() is
> enough (creating one interpreter at a time).  If the chunks need to
> overlap in time, creating multiple interpreters is the right solution.

That's cool.  There isn't much code supporting multiple interpreters, so
there's little to gain by getting rid of it now.  I just wish it were
*used* -- and that it had a more compelling use case <wink>.



From aahz@rahul.net  Thu Aug  9 07:13:27 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Wed, 8 Aug 2001 23:13:27 -0700 (PDT)
Subject: [Python-Dev] Free threading
In-Reply-To: <200108082230.SAA09334@cj20424-a.reston1.va.home.com> from "Guido van Rossum" at Aug 08, 2001 06:30:54 PM
Message-ID: <20010809061327.24EC299C81@waltz.rahul.net>

Guido van Rossum wrote:
> Aahz:
>>
>> We need thread-local storage, anyway.  When I'm a little farther into my
>> work on Decimal, I'm going to take a hard look at this because context
>> needs to be global on a per-thread basis...
> 
> In the C API, use PyThreadState_GetDict(): it returns a per-thread
> dictionary.  The convention is that you pick a key unique to your
> extension module (e.g. your full extension module name) and then store
> your per-thread data under this key.
> 
> In Python, you can store instance variables on the current thread
> object -- use threading.currentThread() to access it.

It's not quite that simple, particularly if one is writing Python code
that permits use of the thread module instead of requiring threading.

But what I was really referring to was the annoyance of having every
method call pull in the thread-local information.  Seems to me that
there ought to be some way of making it happen magically.

Another issue is the problem of inheriting thread-local values when
creating a new thread.  That almost certainly requires a callback inside
the thread creation machinery.

Because I haven't thought much about this, there are probably other
issues I don't know about....
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From martin@strakt.com  Thu Aug  9 07:28:56 2001
From: martin@strakt.com (Martin Sjögren)
Date: Thu, 9 Aug 2001 08:28:56 +0200
Subject: [Python-Dev] BEGIN_ALLOW_THREADS
In-Reply-To: <200108081440.KAA32514@cj20424-a.reston1.va.home.com>
References: <20010723142157.B16665@strakt.com> <20010808091404.A11990@strakt.com> <200108081440.KAA32514@cj20424-a.reston1.va.home.com>
Message-ID: <20010809082856.A16148@strakt.com>

On Wed, Aug 08, 2001 at 10:40:11AM -0400, Guido van Rossum wrote:

[snip]

> Instead of asking us to add more macros, I recommend that you use the
> lower-level C API that is used by the macros.  There's a bunch of ways
> to save and restore the state.  The macros are just shortcuts (and a
> handy way to make all the code disappear when threads don't exist).

Well, that's exactly what I'm doing, isn't it?

Martin

--=20
Martin Sj=F6gren
  martin@strakt.com              ICQ : 41245059
  Phone: +46 (0)31 405242        Cell: +46 (0)739 169191
  GPG key: http://www.strakt.com/~martin/gpg.html


From thomas@xs4all.net  Thu Aug  9 09:28:12 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Thu, 9 Aug 2001 10:28:12 +0200
Subject: [Python-Dev] Common runtime sociology
In-Reply-To: <20010808235940.A16661@thyrsus.com>
References: <20010808232147.A16038@thyrsus.com> <LNBBLJKPBEHFEDALKOLCEEJCLEAA.tim.one@home.com> <20010808235940.A16661@thyrsus.com>
Message-ID: <20010809102812.D874@xs4all.nl>

On Wed, Aug 08, 2001 at 11:59:40PM -0400, Eric S. Raymond wrote:

> Simon, being a gentleman of taste, *likes* cats

Ah, but more importantly, does the cat like Simon ? That's how you recognize
*real* gentlemen. Especially if they sleep on the couch; if they sleep on
the couch, they're Bad. If they lie on the couch as an extra pillow,
providing bodyheat and attention-on-demand to the cat, they're acceptable.
If they pet the cat throughout the night, and feed her in the morning... Cat
heaven!

Sleepers'-hell-but-watch-the-cat-not-care-ly y'rs ;)

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From klm@zope.com  Thu Aug  9 16:15:03 2001
From: klm@zope.com (Ken Manheimer)
Date: Thu, 9 Aug 2001 11:15:03 -0400 (EDT)
Subject: [Python-Dev] Free threading (and tcl upvar)
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEJFLEAA.tim.one@home.com>
Message-ID: <Pine.LNX.4.21.0108090902050.890-100000@serenade.digicool.com>

On Thu, 9 Aug 2001, Tim Peters wrote:

> 2. Tcl's more robust (than Python's) x-platform exec implementation
>    (similar to Python os.system(), but with a platform-independent
>    notation for pipes and redirection and envar substitution, and
>    returning the exit status correctly even under command.com-based
>    Windows flavors (where os.system() always returns 0)).  I looked
>    at this once -- there are several thousand lines of exec support
>    code in Tcl (which shouldn't be surprising, since they're basically
>    supplying one of the hardest parts of a x-platform shell!).

I recall you reporting that.  It *would* be nice to have cross-platform
pipes and exec, though i understand maintaining such non-portable code is
too high a price to pay.  I bet we went over this then, but i can't help
but wonder whether an approach like tkinter, stealing the tcl code, would
be a manageable approach?  Mostly idle speculation, since i wouldn't have
the time nor expertise to implement such a thing myself.

> 3. Variants of Tcl's uplevel, i.e. gimmicks for fiddling locals
>    "up the stack".  Maybe it's not called exactly "uplevel" in Tcl,
>    but it's plain sick however it's spelled <wink>.

I think it's 'upvar'.  I probably remember because it had something to do
with my python roots.  Pardon me while i geeze a bit...

Back at nist i was looking for a scripting language for systems support
type stuff, bounced off perl, and tried using tcl for a trial task, to see
if i could use it.  I made a dns nested-domain traverser, so i could
automate a table-driven network service for all of nist's subdomains
(producing an exports file for a nist-wide NFS service).

I did it, but was not a happy camper - it was seriously painful to have to
manage recursion by, essentially, managing the stack with upvar, &c.  
Mike mclay happened to drop by around then, and i whined to him about my
tcl abomination um script - and he said, "have you looked at python?"  I
did, and, to make a long story short, lived happily ever after.-)

'upvar' *is* sick.

Thanks, mike.

Ken
klm@zope.com

[In a usenet news group, long long ago:]
> [Someone]:
>
> >Programming language preferences are largely a matter of taste, an area
> >where reasonable people will forever disagree.  I would like to think
> >that such disagreements could be predicated upon an understanding that
> >most of the issues involved are not decidable by logical proofs.
> 
> This is the 101 philosophy-student phenomena:  "I can't explain it with
> words".
> 
> - marcus@ee.pdx.edu (Marcus Daniels)





From simon@netthink.co.uk  Thu Aug  9 19:12:30 2001
From: simon@netthink.co.uk (Simon Cozens)
Date: Thu, 9 Aug 2001 11:12:30 -0700
Subject: [Python-Dev] Common runtime sociology
In-Reply-To: <5.1.0.14.0.20010808212559.0223fb60@tuatha.sidhe.org>
Message-ID: <20010809111229.D1266@netthink.co.uk>

On Wed, Aug 08, 2001 at 09:57:50PM -0400, Dan Sugalski wrote:
> I think it's possible for things to be done otherwise, but it puts a heavy 
> burden on the implementation group to be actively engaged with the language 
> groups and be as impartial as can be managed. 

I think technically, it's very easy; Python people might understand the
parallel that their interpeter per se isn't too concerned with the
behaviour of individual types - all the type-specific behaviour is
defined in the various vtables for the types. (Object/*.[ch]) 

It is, you know, possible to add Object/PerlVar.c to have an object
behave like a Perl variable. And since you can specialise the hell out
of what you do in Object/PerlVar.c while the main loop of the
interpreter is very generic, you can make Perl types run as fast as
Python ones or better. 

(The more I look at Python's VM, the more it seems to have many of the
ideas that we want in it...)

So *in* *a* *very* *restricted* sense*, I agree with Eric's proposal
that we should look at using the Python interpreter as a base for Parrot
- in design, if not in actual code. (Especially since we'll want to
change it to a register model.) There are a few things I've identified
that we should change from the way Python does stuff, (notably that we
should make types mutable, and we should try and see if we can remove
the "Type object" layer of abstraction, since that indirection slows us
down) but the core of what you've got looks really, really useful.

Now I've put myself on the line, it's time for Dan to come over and
correct me. ("Ouch!") :)

> You almost need to take the implementation group that starts this and
> pry them loose to be an entity in and of themselves, separate from the
> original language group. (FWIW, this is part of my Master Plan. Dunno
> if it's part of Simon's)
 
I was going to suggest a separate Parrot project, but thought that might
be going too far. Glad we're on the same wavelength.

Simon


From guido@python.org  Thu Aug  9 16:22:45 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 09 Aug 2001 11:22:45 -0400
Subject: [Python-Dev] Free threading (and tcl upvar)
In-Reply-To: Your message of "Thu, 09 Aug 2001 11:15:03 EDT."
 <Pine.LNX.4.21.0108090902050.890-100000@serenade.digicool.com>
References: <Pine.LNX.4.21.0108090902050.890-100000@serenade.digicool.com>
Message-ID: <200108091522.f79FMja02243@odiug.digicool.com>

> I recall you reporting that.  It *would* be nice to have cross-platform
> pipes and exec, though i understand maintaining such non-portable code is
> too high a price to pay.  I bet we went over this then, but i can't help
> but wonder whether an approach like tkinter, stealing the tcl code, would
> be a manageable approach?  Mostly idle speculation, since i wouldn't have
> the time nor expertise to implement such a thing myself.

Tcl is a glue language that encourages you to glue external processes
together.  This is much less emphasized in Python, so I doubt that it
will be as useful.  I don't think it's worth the maintenance
nightmare.  Copying the Tcl code doesn't mean we have no maintenance
-- the Tcl code undoubtedly uses lots of Tcl runtime convenience
routines, which we would have to adapt to the Python runtime
environment.

> 'upvar' *is* sick.

That's why we have sys._getframe(). ;-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From klm@zope.com  Thu Aug  9 16:43:32 2001
From: klm@zope.com (Ken Manheimer)
Date: Thu, 9 Aug 2001 11:43:32 -0400 (EDT)
Subject: [Python-Dev] Free threading (and tcl upvar)
In-Reply-To: <200108091522.f79FMja02243@odiug.digicool.com>
Message-ID: <Pine.LNX.4.21.0108091139100.30934-100000@serenade.digicool.com>

On Thu, 9 Aug 2001, Guido van Rossum wrote:

> > 'upvar' *is* sick.
> 
> That's why we have sys._getframe(). ;-)

We _have_ it, but most of the time we don't _need_ it - thank you.  !

(And now that i'm finally using python 2, i could probably reimplement my
dns walker similar to the way i did w/tcl, without having to resort to the
'raise' gambit to get a stack frame.  Woo woo!-)

Ken
klm@zope.com

"But i digress."



From barry@zope.com  Thu Aug  9 16:44:47 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 9 Aug 2001 11:44:47 -0400
Subject: [Python-Dev] Free threading
References: <LNBBLJKPBEHFEDALKOLCGEDHLEAA.tim.one@home.com>
 <200108080446.AAA31230@cj20424-a.reston1.va.home.com>
Message-ID: <15218.45039.403666.534610@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> It was always feature number one requested by folks coming
    GvR> from Tcl, where creating multiple interpreters (threaded or
    GvR> not) is an acceptable pastime for people of good manners.
    GvR> But it seems to be wasted on the average Pythoneer.

    GvR> In Python, there's one good use I can think of: an
    GvR> application may embed Python and occasionally want to run a
    GvR> good chunk of Python code.  If it needs each chunk of Python
    GvR> code to run completely independent from each other chunk,
    GvR> creating a new interpreter for each chunk is a good way to
    GvR> avoid that changes to e.g. sys.path or tabnanny's globals
    GvR> made by one chunk affects the next chunk.

funny!  in a previous life i wrote a large app that embedded tcl for
its configuration language.  i did exactly as guido describes for
exactly that reason!  in fact, in tcl, you could capture a complete
snapshot of the interpreter in tcl source, replay that into a fresh
interpreter and be back exactly where you started.  a cheapo (but
effective) "save-game" like mechanism.  haven't yet had the need to do
the same in python.

    GvR> But apparently this habit has fallen in disuse -- from
    GvR> looking at the code I believe there's a serious problem with
    GvR> exceptions, where initializing the second interpreter messes
    GvR> up the identities of the exception classes stored in global
    GvR> variables by the first interpreter, and nobody has ever
    GvR> complained about this.  Or maybe they are still using Python
    GvR> 1.5.2, which may not have the same problem.  Or maybe this is
    GvR> the reason why PyApache has a reptabion of instability. :(

i vaguely remember looking at this, and (i thought) fixing it, but
that memory's off-line through the weekend. :)

-barry


From jeremy@zope.com  Thu Aug  9 17:26:05 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 9 Aug 2001 12:26:05 -0400 (EDT)
Subject: [Python-Dev] Common runtime sociology
In-Reply-To: <20010809111229.D1266@netthink.co.uk>
References: <5.1.0.14.0.20010808212559.0223fb60@tuatha.sidhe.org>
 <20010809111229.D1266@netthink.co.uk>
Message-ID: <15218.47517.833096.75695@slothrop.digicool.com>

>>>>> "SC" == Simon Cozens <simon@netthink.co.uk> writes:

  SC> (The more I look at Python's VM, the more it seems to have many
  SC> of the ideas that we want in it...)

You're well on your way, Simon.  Once you realized that you don't need
Object/PerlVar.c, you will have reached enlightenment.

I don't think the cat will even entertain the notion of running away
with you before then.

Jeremy



From skip@pobox.com (Skip Montanaro)  Thu Aug  9 17:28:36 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Thu, 9 Aug 2001 11:28:36 -0500
Subject: [Python-Dev] optimizing non-local object access
Message-ID: <15218.47668.406742.453009@beluga.mojam.com>

I had a thought last night about optimizing access to globals in other
modules (e.g. "math.sin" or "string.uppercase").  I think it could actually
work to speed up all non-local object access.

The nice thing about module globals is that for the most part they are
"almost constant".  Once a module is imported, it's rare that its global
name bindings actually change.  I think it might suffice to create a
notification system that allows an executing frame to register its interest
in changes to a module's name bindings.

Let me make this idea concrete with an example.  Suppose I have the
following code:

    import math

    def sinseq(n):
        l = []
        for i in xrange(n):
            l.append(math.sin(i))
        return l

"math.sin" requires a global lookup and an attribute access each pass
through the loop.  Let's add an entry to the fastlocals array for it and an
entry to the local names called "math.sin".  The byte code compiler
generates a LOAD_FAST instruction anywhere math.sin is accessed, and at the
first point where it should be "live" (i.e., at the beginning of the
function) it simply inserts a TRACK_OBJECT instruction with that slot in
fastlocals as one argument and the "math.sin" slot in the local names as the
other.  When "math.sin" goes out of scope (i.e., at the end of the
function), it inserts an UNTRACK_OBJECT instruction with "math.sin" as its
argument.  The same can be done for "l.append" and xrange.  The bytecode for
the above function would look something like:

            TRACK_OBJECT        math.sin
            TRACK_OBJECT        xrange
    >>    0 BUILD_LIST          0 ('\000', '\000')
          3 STORE_FAST          1 (l)
            TRACK_OBJECT        l.append
          6 SETUP_LOOP         44 (to 53)
          9 LOAD_FAST           xrange
         12 LOAD_FAST           0 (n)
         15 CALL_FUNCTION       1 ('\001', '\000')
         18 LOAD_CONST          1 (0)
         21 FOR_LOOP           28 (to 52)
         24 STORE_FAST          2 (i)
         27 LOAD_FAST           l.append
            LOAD_FAST           math.sin
         39 LOAD_FAST           2 (i)
         42 CALL_FUNCTION       1 ('\001', '\000')
         45 CALL_FUNCTION       1 ('\001', '\000')
         48 POP_TOP        
         49 JUMP_ABSOLUTE      21 ('\025', '\000')
    >>   52 POP_BLOCK      
    >>   53 LOAD_FAST           1 (l)
            UNTRACK_OBJECT      l.append
            UNTRACK_OBJECT      xrange
            UNTRACK_OBJECT      math.sin
         56 RETURN_VALUE   


TRACK_OBJECT and UNTRACK_OBJECT are responsible for registering and
unregistering interest in an object.  TRACK_OBJECT also caches the
corresponding object in the fastlocals array.

<wave what="hands">

I think most or all of the work can be handled by PyObject_SetAttr and/or
PyDict_SetItem.  It seems to me that its greatest benefit would be to short
circuit access to global variables in other modules.  For the most part,
these don't change once an import is completed, so you'd effectively be
converting access to these objects into local variables, but you could use
it to track all non-local variables or attributes.

</wave>

stepping-behind-the-egg-proof-screen-ly y'rs,

Skip



From aahz@rahul.net  Thu Aug  9 17:33:27 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Thu, 9 Aug 2001 09:33:27 -0700 (PDT)
Subject: [Python-Dev] Any objections to adding threading.Timer?
In-Reply-To: <200108082034.f78KYlB16985@mira.informatik.hu-berlin.de> from "Martin v. Loewis" at Aug 08, 2001 10:34:47 PM
Message-ID: <20010809163328.6691299C86@waltz.rahul.net>

Martin v. Loewis wrote:
> 
> In patch
> 
> https://sourceforge.net/tracker/index.php?func=detail&aid=428326&group_id=5470&atid=305470
> 
> a new threading.Timer class is proposed. Since the patch was submitted
> two months ago, nobody seemed to care enough to comment on it. I like
> it, and I'll commit the patch unless I hear otherwise.

I'm not objecting, precisely, but I have an allergic reaction to the way
timed Event() works, even though there's really no other way to work it.
I think that encouraging people to use it is a Bad Idea.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From paulp@ActiveState.com  Thu Aug  9 17:40:38 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Thu, 09 Aug 2001 09:40:38 -0700
Subject: [Python-Dev] Common runtime sociology
References: <5.1.0.14.0.20010808212559.0223fb60@tuatha.sidhe.org>
 <20010809111229.D1266@netthink.co.uk> <15218.47517.833096.75695@slothrop.digicool.com>
Message-ID: <3B72BD06.687DC1CA@ActiveState.com>

Jeremy Hylton wrote:
> 
> >>>>> "SC" == Simon Cozens <simon@netthink.co.uk> writes:
> 
>   SC> (The more I look at Python's VM, the more it seems to have many
>   SC> of the ideas that we want in it...)
> 
> You're well on your way, Simon.  Once you realized that you don't need
> Object/PerlVar.c, you will have reached enlightenment.

;-)

Object/PerlVar.c is like the teddy bear you take with you to camp to
remind you of home. Or maybe its like the picture of your girlfriend you
take to remind you that you already HAVE a girlfriend. <0.5 wink>

-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From jeremy@zope.com  Thu Aug  9 17:50:47 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 9 Aug 2001 12:50:47 -0400 (EDT)
Subject: [Python-Dev] optimizing non-local object access
In-Reply-To: <15218.47668.406742.453009@beluga.mojam.com>
References: <15218.47668.406742.453009@beluga.mojam.com>
Message-ID: <15218.48999.964618.234723@slothrop.digicool.com>

I've been thinking about the same issues in my spare time.  I
certainly think the interpreter could be made faster by assuming that
most bindings are static most of the time.  Python has many places
where bindings can change unexpectedly, but most of the time they
don't change.

My worry about your approach is that the track-object opcodes could
add a lot of expense for objects only used once or twice.  If the
function uses math.sin inside a loop, it's an obvious win.  If it uses
it only once, it's not so clear.

The other approach I had been thinking about, which may have made it
into my pre-PEP post a few months ago, is to store module globals in a
table that can be accessed like fast locals.  Any reference to another
module would use the fast globals table.  For references to global
names in another module, the calling module could lookup the offset of
that name in the defining module's fast globals.

To be more concrete:  The math module would store the sin name in slot
X.  The first time the foobar module used math.sin it would lookup the
slot of sin in the math table.  The foobar module would store a
pointer to math's fast globals and the index of the sin slot.  Then
math.sin would be accessed via a single opcode that used the stored
information. 

This approach doesn't extend to instances in an obvious way, but I'd
be happy dealing only with modules for a start.  There's a fair amount
of Zope code that does things like this:

def hotspot_func(x, _tt=types.TupleType):
    ...

I'd like to get rid of the abuse of default args to make access to
TupleType fast.

I have been thinking some about how to make a similar strategy work
for instances.  I think we could apply some lessons from the Self
compiler and recompile generic bytecodes ("load the 'foo' attribute")
into more specific bytecodes ("load slot 12").  But this is more
speculation than anything else.

Jeremy




From Samuele Pedroni <pedroni@inf.ethz.ch>  Thu Aug  9 18:39:33 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Thu, 9 Aug 2001 19:39:33 +0200 (MET DST)
Subject: [Python-Dev] optimizing non-local object access
Message-ID: <200108091739.TAA10088@core.inf.ethz.ch>

[Jeremy Hylton]
> I've been thinking about the same issues in my spare time.  I
> certainly think the interpreter could be made faster by assuming that
> most bindings are static most of the time.  Python has many places
> where bindings can change unexpectedly, but most of the time they
> don't change.
> 
> My worry about your approach is that the track-object opcodes could
> add a lot of expense for objects only used once or twice.  If the
> function uses math.sin inside a loop, it's an obvious win.  If it uses
> it only once, it's not so clear.
I agree, and I wonder whether it is not going to explode (in term of effort and resources)
down a call graph.

> The other approach I had been thinking about, which may have made it
> into my pre-PEP post a few months ago, is to store module globals in a
> table that can be accessed like fast locals.  Any reference to another
> module would use the fast globals table.  For references to global
> names in another module, the calling module could lookup the offset of
> that name in the defining module's fast globals.
> 
I rembemer I liked it.

> I have been thinking some about how to make a similar strategy work
> for instances.  I think we could apply some lessons from the Self
> compiler and recompile generic bytecodes ("load the 'foo' attribute")
> into more specific bytecodes ("load slot 12").  But this is more
> speculation than anything else.
Some considerations about this:

Python vs. Smalltalk vs. Self

Smalltalk classes have rigid sets of slots, single inheritance and all slot are private (protected?),
so I imagine it is easy to number them.

In Self the set of slots of an object is also fixed (there are specific reflective methods to modify the 
set of slots) and also which are mutable etc. OTOH Self has dynamic inheritance (something like
the mutable __bases__ in Python). To deal with all this it uses customization (different version of
a single method are lazily compiled against the various concrete structural variations of objects that 
inherit it) and assume that things found up in the inheritance tree are constant but guards itself
about their changes

Python is even less rigid than Self and there is not an explicit notion of the set of slots that goes with
an object. So to achieve some improvement in slot access speed may be necessary to:
- set more rigid rules on slots (but then that's not Python anymore)

- find a way to detect some conservative approximation of the set of slots of a concrete class
- use customization (consumes memory) so you can exploit a fixed layout for the approximated slots
- use plain dictionary lookup for the slots missed that way

Not impossible, not trivial and surely a thing that will pollute code clarity, OTOH
whith this kind of optimizations in place, even just at the interp level, effective native
compilation is not that far ...

regards, Samuele Pedroni.




From jeremy@zope.com  Thu Aug  9 18:43:31 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 9 Aug 2001 13:43:31 -0400 (EDT)
Subject: [Python-Dev] optimizing non-local object access
In-Reply-To: <200108091739.TAA10088@core.inf.ethz.ch>
References: <200108091739.TAA10088@core.inf.ethz.ch>
Message-ID: <15218.52163.546008.296225@slothrop.digicool.com>

I've made some space for this project in my Wiki:
http://www.zope.org/Members/jeremy/CurrentAndFutureProjects/FastGlobals

It's got the text of the PEP that I posted and a pointer to Guido's
original message on the subject.  Feel free to contribute more info if
you're interested.  (The permissions may be wrong at the moment.  Let
me know if you run into trouble.)

Jeremy





From jeremy@zope.com  Thu Aug  9 18:49:29 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 9 Aug 2001 13:49:29 -0400 (EDT)
Subject: [Python-Dev] optimizing non-local object access
In-Reply-To: <200108091739.TAA10088@core.inf.ethz.ch>
References: <200108091739.TAA10088@core.inf.ethz.ch>
Message-ID: <15218.52521.776800.5873@slothrop.digicool.com>

>>>>> "SP" == Samuele Pedroni <pedroni@inf.ethz.ch> writes:

  SP> Python is even less rigid than Self and there is not an explicit
  SP> notion of the set of slots that goes with an object. So to

I agree that this is the key hurdle.

  SP> achieve some improvement in slot access speed may be necessary
  SP> to:
  SP> - set more rigid rules on slots (but then that's not Python
  SP>   anymore)

Not allowed :-).

  SP> - find a way to detect some conservative approximation of the
  SP>   set of slots of a concrete class

I think this can be done with better compiler support.

  SP> - use customization (consumes memory) so you can exploit a fixed
  SP>   layout for the approximated slots
  SP> - use plain dictionary lookup for the slots missed that way

Right.  

  SP> Not impossible, not trivial and surely a thing that will pollute
  SP> code clarity, OTOH whith this kind of optimizations in place,
  SP> even just at the interp level, effective native compilation is
  SP> not that far ...

It is a lot of work for instances, so I've been considering it
separately from the module globals issue.  But I think it is
possible.  Short of native-code, I wonder if we can extend the Python
VM with opcodes that take advantage of slots.  I'm not sure how
closely this idea is related to Pysco.

Imagine we didn't special case ints in BINARY_ADD.  If you're doing
mostly string addition, it's just slowing you down anyway :-).
Instead, a function that expected to see ints would check the types of
the relevant args.

def f(x):
    return 2*x

would become something like:

def f(x):
    if isinstance(x, int):
        return 2 * x_as_int
    else:
        return 2 * x

where the int branch could use opcodes that knew that x was an integer
and could call its __add__ method directly.

Jeremy





From skip@pobox.com (Skip Montanaro)  Thu Aug  9 18:58:37 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Thu, 9 Aug 2001 12:58:37 -0500
Subject: [Python-Dev] optimizing non-local object access
In-Reply-To: <15218.48999.964618.234723@slothrop.digicool.com>
References: <15218.47668.406742.453009@beluga.mojam.com>
 <15218.48999.964618.234723@slothrop.digicool.com>
Message-ID: <15218.53069.713724.34519@beluga.mojam.com>

    Jeremy> My worry about your approach is that the track-object opcodes
    Jeremy> could add a lot of expense for objects only used once or twice.
    Jeremy> If the function uses math.sin inside a loop, it's an obvious
    Jeremy> win.  If it uses it only once, it's not so clear.

Even if math.sin is used just once you swap a LOAD_GLOBAL/LOAD_ATTR pair for
a TRACK_OBJECT/LOAD_FAST/UNTRACK_OBJECT trio, so the hit you take shouldn't
be terrible.  (My assumption is that the register/unregister cost is fairly
low and the actual notification/update code will almost never be executed.)
You break even in total instructions executed with two accesses and win
after that.  In addition, this might be a strategy left for an optimization
pass that would only make the change if the LOAD_ATTR and/or LOAD_GLOBAL
instructions are executed in a loop.

    Jeremy> To be more concrete: The math module would store the sin name in
    Jeremy> slot X.  The first time the foobar module used math.sin it would
    Jeremy> lookup the slot of sin in the math table.  The foobar module
    Jeremy> would store a pointer to math's fast globals and the index of
    Jeremy> the sin slot.  Then math.sin would be accessed via a single
    Jeremy> opcode that used the stored information.

Unfortunately, the code that uses math.sin can't know that math is a module.
It might be an instance with a sin attribute.  Even worse, because of
Python's dynamic nature, what the name "math" is bound to can change.  You
can't assume it will always be bound to a module object, even if it is the
first time you set things up.  I think you have to work with names and name
bindings.  I don't think you can make assumptions about what the names are
bound to.

The handwaving bit in my post was there because I am not familiar enough
with the various possibilities for name rebinding.  Does it all boil down to
PyDict_SetItem or PyObject_SetAttr as I suspect?  Are those functions too
low-level, that is, have the names been forgetten completely at that point?
If so, perhaps STORE_GLOBAL and STORE_ATTR would have to be modified to use
PyDict_SetItemString and PyObject_SetAttrString instead.

Skip


From skip@pobox.com (Skip Montanaro)  Thu Aug  9 19:05:28 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Thu, 9 Aug 2001 13:05:28 -0500
Subject: [Python-Dev] optimizing non-local object access
In-Reply-To: <15218.52521.776800.5873@slothrop.digicool.com>
References: <200108091739.TAA10088@core.inf.ethz.ch>
 <15218.52521.776800.5873@slothrop.digicool.com>
Message-ID: <15218.53480.204468.854961@beluga.mojam.com>

    Jeremy> Imagine we didn't special case ints in BINARY_ADD.  If you're
    Jeremy> doing mostly string addition, it's just slowing you down anyway
    Jeremy> :-).  Instead, a function that expected to see ints would check
    Jeremy> the types of the relevant args.

    Jeremy> def f(x):
    Jeremy>     return 2*x

    Jeremy> would become something like:

    Jeremy> def f(x):
    Jeremy>     if isinstance(x, int):
    Jeremy>         return 2 * x_as_int
    Jeremy>     else:
    Jeremy>         return 2 * x

    Jeremy> where the int branch could use opcodes that knew that x was an
    Jeremy> integer and could call its __add__ method directly.

Yeah, but all you've really done is extract the current inline test for int
arguments into a few more PyVM instructions.  You've sacrificed a fairly
fast special case test for more passes around the interpreter loop.  I'm not
sure how this can be a win in the current virtual machine.

Skip






From jeremy@zope.com  Thu Aug  9 19:15:24 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 9 Aug 2001 14:15:24 -0400 (EDT)
Subject: [Python-Dev] optimizing non-local object access
In-Reply-To: <15218.53069.713724.34519@beluga.mojam.com>
References: <15218.47668.406742.453009@beluga.mojam.com>
 <15218.48999.964618.234723@slothrop.digicool.com>
 <15218.53069.713724.34519@beluga.mojam.com>
Message-ID: <15218.54076.512057.301420@slothrop.digicool.com>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

  Jeremy> My worry about your approach is that the track-object
  Jeremy> opcodes could add a lot of expense for objects only used
  Jeremy> once or twice.  If the function uses math.sin inside a loop,
  Jeremy> it's an obvious win.  If it uses it only once, it's not so
  Jeremy> clear.

  SM> Even if math.sin is used just once you swap a
  SM> LOAD_GLOBAL/LOAD_ATTR pair for a
  SM> TRACK_OBJECT/LOAD_FAST/UNTRACK_OBJECT trio, so the hit you take
  SM> shouldn't be terrible.  (My assumption is that the
  SM> register/unregister cost is fairly low and the actual
  SM> notification/update code will almost never be executed.)  You
  SM> break even in total instructions executed with two accesses and
  SM> win after that.

I'm assuming some kind of memory allocation is necessary to accomodate
an aribtrary number of handlers.  If you need to resize an array that
holds the pointers to the tracking callbacks, it could get expensive.

I also wonder if you have to pay for tracking changes whenever the
name is rebound in the module, or just when you need to use the name
again.

  SM>                  In addition, this might be a strategy left for
  SM> an optimization pass that would only make the change if the
  SM> LOAD_ATTR and/or LOAD_GLOBAL instructions are executed in a
  SM> loop.

Good point.

  Jeremy> To be more concrete: The math module would store the sin
  Jeremy> name in slot X.  The first time the foobar module used
  Jeremy> math.sin it would lookup the slot of sin in the math table.
  Jeremy> The foobar module would store a pointer to math's fast
  Jeremy> globals and the index of the sin slot.  Then math.sin would
  Jeremy> be accessed via a single opcode that used the stored
  Jeremy> information.

  SM> Unfortunately, the code that uses math.sin can't know that math
  SM> is a module.  It might be an instance with a sin attribute.
  SM> Even worse, because of Python's dynamic nature, what the name
  SM> "math" is bound to can change.  You can't assume it will always
  SM> be bound to a module object, even if it is the first time you
  SM> set things up.  I think you have to work with names and name
  SM> bindings.  I don't think you can make assumptions about what the
  SM> names are bound to.

No assumptions necessary.  The compiler only emits the new opcodes for
names bound by import or attributes thereof.  If the module name
('math') is rebound, the interpreter is responsible for reseting all
of the other bindings that depend on it ('math.sin').  If the object
'math', isn't a module (even though the compiler guessed it would be),
the opcodes fall back to the old implementation.

The first time math.sin is used, we do the following:

   - check if the math.sin fast globals entry is initialized
     (it won't be, but it could be marked uninitialized or 
      "don't use") 
   - check that math is indeed a module
   - lookup the sin slot in math
   - record the slot in the fast globals table

On future uses, the first step above will discover a valid binding.
If the name math is rebound, the interpreter marks as the fast globlas
refering to it as unitinitalized.

One advantage of this approach is that the work is shared across all
code in a module.  If many functions use math.sin, the first one
initializes the table and all the rest use it.

  SM> The handwaving bit in my post was there because I am not
  SM> familiar enough with the various possibilities for name
  SM> rebinding.  Does it all boil down to PyDict_SetItem or
  SM> PyObject_SetAttr as I suspect?  Are those functions too
  SM> low-level, that is, have the names been forgetten completely at
  SM> that point?  If so, perhaps STORE_GLOBAL and STORE_ATTR would
  SM> have to be modified to use PyDict_SetItemString and
  SM> PyObject_SetAttrString instead.

I think we hook in at the tp_getattr(o) level.  Module objects can
detect rebindings there and do whatever bookkeeping is necessary to
keep references to its name consistent.  I think this is the right
approach for either technique we're discussing.

Jeremy



From jeremy@zope.com  Thu Aug  9 19:22:02 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 9 Aug 2001 14:22:02 -0400 (EDT)
Subject: [Python-Dev] optimizing non-local object access
In-Reply-To: <15218.53480.204468.854961@beluga.mojam.com>
References: <200108091739.TAA10088@core.inf.ethz.ch>
 <15218.52521.776800.5873@slothrop.digicool.com>
 <15218.53480.204468.854961@beluga.mojam.com>
Message-ID: <15218.54474.374553.543595@slothrop.digicool.com>

I haven't thought through whether it's a win for integers.  I expect
it would be a win for instances, particularly when several methods are
called on the instances.  My assumption is that the cost of an extra
type dispatch pass through the interpreter loop is offset by the
ability to quickly resolve attribute names.  

This is at best half-baked, so I don't expect to be too convincing.  I
still need to convince myself.

Jeremy



From skip@pobox.com (Skip Montanaro)  Thu Aug  9 19:44:37 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Thu, 9 Aug 2001 13:44:37 -0500
Subject: [Python-Dev] optimizing non-local object access
In-Reply-To: <15218.54076.512057.301420@slothrop.digicool.com>
References: <15218.47668.406742.453009@beluga.mojam.com>
 <15218.48999.964618.234723@slothrop.digicool.com>
 <15218.53069.713724.34519@beluga.mojam.com>
 <15218.54076.512057.301420@slothrop.digicool.com>
Message-ID: <15218.55829.400113.482371@beluga.mojam.com>

Jeremy,

Perhaps we need a bake-off to try both approaches.  I think I can localize
my changes fairly well, but it will take me a little time to come
up-to-speed on changes to ceval.c since the 1.5.2 timeframe.  (I will have
to do that to migrate Rattlesnake to the current codebase and to work on
some notes about PyVM that might eventually turn into a PyVM reference
manual.)

wish-i-could-get-paid-for-this-ly y'rs,

Skip


From Samuele Pedroni <pedroni@inf.ethz.ch>  Thu Aug  9 19:50:33 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Thu, 9 Aug 2001 20:50:33 +0200 (MET DST)
Subject: [Python-Dev] optimizing non-local object access
Message-ID: <200108091850.UAA12818@core.inf.ethz.ch>

>   SP> - find a way to detect some conservative approximation of the
>   SP>   set of slots of a concrete class
> 
> I think this can be done with better compiler support.
> 

The simplest thing would be to detect all 'attr' that appear in the body of
the methods of a class statements in 'self.attr' form.  Class construction at
run-time will take the union of this together with the information computed
for base classes Why at runtime?  It avoids the trouble of
poor-man-to-full-fledged type-inference for the compiler.  It would be
reasonably effective and at most would require the programmer to put a
self.myFavoriteAttr = None in __init__.

>   SP> - use customization (consumes memory) so you can exploit a fixed
>   SP>   layout for the approximated slots
>   SP> - use plain dictionary lookup for the slots missed that way
> 
>   Right 
Clearly this should be played two-hands with method dispatch.

> 
>   SP> Not impossible, not trivial and surely a thing that will pollute
>   SP> code clarity, OTOH whith this kind of optimizations in place,
>   SP> even just at the interp level, effective native compilation is
>   SP> not that far ...
> 
> It is a lot of work for instances, so I've been considering it
> separately from the module globals issue.  But I think it is
> possible.  Short of native-code, I wonder if we can extend the Python
> VM with opcodes that take advantage of slots.

It is indeed a good idea, but the problem is that customising without native
compilation can have a very poor compilation-time vs.  runtime speedup
ratio.  OTOH we could customize only hot-spots.  But I think is worth trying
if we ever get at it.

>  I'm not sure how
> closely this idea is related to Pysco.

AFAIK it would be complementary, Armin Rigo pointed out that in principle
Psyco can layout objects at will, but I remarked that is way of dealing
with code comsumes a lot of memory so it is workable only if applied only to
hot spots, and then we pay for conversion when non-Pysco code calls Psyco code.

Psyco is doing some sort of customization (a generalisation of that), a
layout for instances that can be exploited in term of speed under
customization is a good thing for Psyco too IMHO.

> Imagine we didn't special case ints in BINARY_ADD.  If you're doing
> mostly string addition, it's just slowing you down anyway :-).
> Instead, a function that expected to see ints would check the types of
> the relevant args.
> 
> def f(x):
>     return 2*x
> 
> would become something like:
> 
> def f(x):
>     if isinstance(x, int):
>         return 2 * x_as_int
>     else:
>         return 2 * x
> 
> where the int branch could use opcodes that knew that x was an integer
> and could call its __add__ method directly.
AFAIK this is somehow what Psyco does, I have not checked the code but my understanding is that
it goes a hard way to do this kind of thing specifically when they can give you something.


Customization would mean the following:

class A:
  def __init__(self, a=2):
    self.a = a
    
  def double(self):
    self.a*=2
    
class B:
 def __init__(self, b=3):
   self.b = b
   
 def inc(self):
   self.b+=1
   
class C(A,B):
  def __init__(self,a,b):
    A.__init__(a)
    B.__init__(b)
    
  def mul(self):
    return a*b
    
under customization if both instances of A and C are used and in both cases
double is used there will be two versions of double to choose from, one that
expect self to be of concrete class A and one that expect it to be of
concrete class C.  Both these versions can be constructed knowing the exact
place of the slot a in the objects.

It works clearly best with good pure single dispatch oo code ;).

In any case both a more static layout for slots and some way to add hooks
that count how many times a code object is invoked, would be a good start 
point for a lot of possible nice experiments.

regards, Samuele.



From fdrake@acm.org  Thu Aug  9 19:50:00 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 9 Aug 2001 14:50:00 -0400 (EDT)
Subject: [Python-Dev] optimizing non-local object access
In-Reply-To: <15218.55829.400113.482371@beluga.mojam.com>
References: <15218.47668.406742.453009@beluga.mojam.com>
 <15218.48999.964618.234723@slothrop.digicool.com>
 <15218.53069.713724.34519@beluga.mojam.com>
 <15218.54076.512057.301420@slothrop.digicool.com>
 <15218.55829.400113.482371@beluga.mojam.com>
Message-ID: <15218.56152.976997.480495@cj42289-a.reston1.va.home.com>

Skip Montanaro writes:
 > to do that to migrate Rattlesnake to the current codebase and to work on
 > some notes about PyVM that might eventually turn into a PyVM reference
 > manual.)

  Just let me know when you're ready to check this in!  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From jeremy@zope.com  Thu Aug  9 19:55:28 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 9 Aug 2001 14:55:28 -0400 (EDT)
Subject: [Python-Dev] optimizing non-local object access
In-Reply-To: <15218.55829.400113.482371@beluga.mojam.com>
References: <15218.47668.406742.453009@beluga.mojam.com>
 <15218.48999.964618.234723@slothrop.digicool.com>
 <15218.53069.713724.34519@beluga.mojam.com>
 <15218.54076.512057.301420@slothrop.digicool.com>
 <15218.55829.400113.482371@beluga.mojam.com>
Message-ID: <15218.56480.865209.746007@slothrop.digicool.com>

A bake-off sounds like a great idea.  I probably won't get to this
until after 2.2 -- and wouldn't be able to get it into 2.2 at this
late date even if I had the time.  

Jeremy



From aahz@rahul.net  Thu Aug  9 21:00:13 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Thu, 9 Aug 2001 13:00:13 -0700 (PDT)
Subject: [Python-Dev] Common runtime sociology
In-Reply-To: <20010808232147.A16038@thyrsus.com> from "Eric S. Raymond" at Aug 08, 2001 11:21:47 PM
Message-ID: <20010809200014.2188399C8E@waltz.rahul.net>

Eric S. Raymond wrote:
> Tim Peters <tim.one@home.com>:
>> 
>> #2 is the best shot.  Alas, few people can *decide* to become an insane
>> visionary and make it stick <wink>.
> 
> I've managed it.  

I've heard enough comments from other people to only agree heartily on
the "insane" part.  ;-)
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From jepler@inetnebr.com  Thu Aug  9 21:05:51 2001
From: jepler@inetnebr.com (Jeff Epler)
Date: Thu, 9 Aug 2001 15:05:51 -0500
Subject: [Python-Dev] optimizing non-local object access
In-Reply-To: <15218.53480.204468.854961@beluga.mojam.com>; from skip@pobox.com on Thu, Aug 09, 2001 at 01:05:28PM -0500
References: <200108091739.TAA10088@core.inf.ethz.ch> <15218.52521.776800.5873@slothrop.digicool.com> <15218.53480.204468.854961@beluga.mojam.com>
Message-ID: <20010809150549.F9751@inetnebr.com>

On Thu, Aug 09, 2001 at 01:05:28PM -0500, Skip Montanaro wrote:
> Yeah, but all you've really done is extract the current inline test for int
> arguments into a few more PyVM instructions.  You've sacrificed a fairly
> fast special case test for more passes around the interpreter loop.  I'm not
> sure how this can be a win in the current virtual machine.

The real gain comes when you can hoist the type test out of the loop
(though that means proving the type condition is an invariant of the
loop!)

For instance, in the following, the type of x is invariant over the
loop, and the test can be moved entirely outside the loop.  Now, the
loop itself can operate on ints, and hopefully benefit from it (in the
most extreme case, the loop can become about 2 CPU (not VM) instructions
per line, given even a fairly rudimentary JIT.  Add a bit more to catch
the overflow in the 'x*3+1' expression (or even the 'r+1' expression),
whether to raise ValueError or silently become a long integer and bail
back to the slow version.

def f(x):
    """ The famous "hailstone" problem, assuming my memory is right ---
        nobody seems to know how to predict #iterations or highest
	intermediate value of x from the initial value, or has even
	proven that the algorithm completes for all values of x """

    r = 0
    while x > 1:
        r = r + 1
        if x%2 == 0:
            x=x/2   # Truncating integer division
        else:
            x=x*3+1
    return r        

def f(x):
    if isinstance(x, int):
        """run fully native code with x and r known to be platform ints"""
    else:
        """run python bytecode"""

The "hard" parts, as I see them, are detecting/deducing these type
invariants, even given hints from the programmer about the types of
parameters (a la 'f = optimize(f, (int,))'), and having a way to 
"bail" to the original bytecode version.  You could have
	if isinstance(x, int):
		try:
			""" fully native version """
		except ValueError: pass
	""" run python bytecode if there was no optimized version,
	    or this x happened to overflow platform int """
but that would not work if the code makes calls that have side-effects.
(mutate objects, produce output, ...)  Transmeta has patented their
methods to do this, in their dynamic binary translating CPUs (not that
their approaches are likely to have much relevance for the Python VM).

Jeff


From skip@pobox.com (Skip Montanaro)  Thu Aug  9 21:15:24 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Thu, 9 Aug 2001 15:15:24 -0500
Subject: [Python-Dev] optimizing non-local object access
In-Reply-To: <20010809150549.F9751@inetnebr.com>
References: <200108091739.TAA10088@core.inf.ethz.ch>
 <15218.52521.776800.5873@slothrop.digicool.com>
 <15218.53480.204468.854961@beluga.mojam.com>
 <20010809150549.F9751@inetnebr.com>
Message-ID: <15218.61276.654235.714590@beluga.mojam.com>

    Jeff> The real gain comes when you can hoist the type test out of the
    Jeff> loop (though that means proving the type condition is an invariant
    Jeff> of the loop!)

Even more difficult if the programmer can subclass ints, yes?.  Today, if I
see x=x/2 and I know x is an int, I can assume x/2 is also (or a float once
integer division returns floats).  Once ints can be subclassed, a devious
programmer could return something other than an int (say, a float, a
rational, or the string "x/2").

Skip


From neal@metaslash.com  Thu Aug  9 21:15:53 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Thu, 09 Aug 2001 16:15:53 -0400
Subject: [Python-Dev] Direction of PyChecker
Message-ID: <3B72EF79.7BE8F6C1@metaslash.com>

I'd like to solicit feedback on what everyone thinks about PyChecker.
Specifically, where should it go?  What should it include or not include?
What should the default warnings be?

I've seen some threads on c.l.p that checker might address.  
Do you think that there should be something added to the python 
FAQ/doc about checker?  I'd be willing to write something.

I'm close to exhausting my list of warnings/checks, let me know
if there are additional warnings/checks you can think of.

Thanks for your input,
Neal

PS I sent a patch to python-dev against 2.2a1, but it's 45k.  So it's awating
moderator approval.  Sorry if this isn't the right venue for patches.


From jepler@inetnebr.com  Thu Aug  9 21:17:45 2001
From: jepler@inetnebr.com (Jeff Epler)
Date: Thu, 9 Aug 2001 15:17:45 -0500
Subject: [Python-Dev] optimizing non-local object access
In-Reply-To: <15218.61276.654235.714590@beluga.mojam.com>; from skip@pobox.com on Thu, Aug 09, 2001 at 03:15:24PM -0500
References: <200108091739.TAA10088@core.inf.ethz.ch> <15218.52521.776800.5873@slothrop.digicool.com> <15218.53480.204468.854961@beluga.mojam.com> <20010809150549.F9751@inetnebr.com> <15218.61276.654235.714590@beluga.mojam.com>
Message-ID: <20010809151743.G9751@inetnebr.com>

On Thu, Aug 09, 2001 at 03:15:24PM -0500, Skip Montanaro wrote:
> 
>     Jeff> The real gain comes when you can hoist the type test out of the
>     Jeff> loop (though that means proving the type condition is an invariant
>     Jeff> of the loop!)
> 
> Even more difficult if the programmer can subclass ints, yes?.  Today, if I
> see x=x/2 and I know x is an int, I can assume x/2 is also (or a float once
> integer division returns floats).  Once ints can be subclassed, a devious
> programmer could return something other than an int (say, a float, a
> rational, or the string "x/2").

Then the test becomes 'x.__class__ is int', not 'isinstance(x, int)'.

Jeff


From martin@loewis.home.cs.tu-berlin.de  Thu Aug  9 21:12:39 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Thu, 9 Aug 2001 22:12:39 +0200
Subject: [Python-Dev] optimizing non-local object access
Message-ID: <200108092012.f79KCdv00872@mira.informatik.hu-berlin.de>

> I think we hook in at the tp_getattr(o) level.  Module objects can
> detect rebindings there and do whatever bookkeeping is necessary to
> keep references to its name consistent.  I think this is the right
> approach for either technique we're discussing.

Would that catch changes like

math.__dict__['sin'] = math.cos

??

Regards,
Martin


From martin@loewis.home.cs.tu-berlin.de  Thu Aug  9 21:21:15 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Thu, 9 Aug 2001 22:21:15 +0200
Subject: [Python-Dev] Any objections to adding threading.Timer?
In-Reply-To: <20010809163328.6691299C86@waltz.rahul.net> (aahz@rahul.net)
References: <20010809163328.6691299C86@waltz.rahul.net>
Message-ID: <200108092021.f79KLFI00918@mira.informatik.hu-berlin.de>

> I'm not objecting, precisely, but I have an allergic reaction to the way
> timed Event() works, even though there's really no other way to work it.
> I think that encouraging people to use it is a Bad Idea.

Can you elaborate in more detail what your concern about the timed
Event() is? This one itself looks fine; if anything, _Condition.wait
looks disturbing. Could you also kindly put your comments into

http://sourceforge.net/tracker/?group_id=5470&atid=305470&func=detail&aid=428326

There are a number of procedural objections with regard to this patch
(license, missing documentation); if you can add significant technical
objections, that might be a reason to reject it - or to find a better
alternative.

Regards,
Martin


From fdrake@acm.org  Thu Aug  9 21:28:28 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 9 Aug 2001 16:28:28 -0400 (EDT)
Subject: [Python-Dev] optimizing non-local object access
In-Reply-To: <15218.61276.654235.714590@beluga.mojam.com>
References: <200108091739.TAA10088@core.inf.ethz.ch>
 <15218.52521.776800.5873@slothrop.digicool.com>
 <15218.53480.204468.854961@beluga.mojam.com>
 <20010809150549.F9751@inetnebr.com>
 <15218.61276.654235.714590@beluga.mojam.com>
Message-ID: <15218.62060.653961.301524@cj42289-a.reston1.va.home.com>

Skip Montanaro writes:
 > see x=x/2 and I know x is an int, I can assume x/2 is also (or a float once
 > integer division returns floats).  Once ints can be subclassed, a devious

  When division can return a float, it isn't integer division any
more.  ;-)  We can always assume int//int is an integer, though.  But
only if we really have integers; subclasses are different.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From skip@pobox.com (Skip Montanaro)  Thu Aug  9 21:34:46 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Thu, 9 Aug 2001 15:34:46 -0500
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: <3B72EF79.7BE8F6C1@metaslash.com>
References: <3B72EF79.7BE8F6C1@metaslash.com>
Message-ID: <15218.62438.445125.366624@beluga.mojam.com>

    Neal> PS I sent a patch to python-dev against 2.2a1, but it's 45k.  So
    Neal> it's awating moderator approval.  Sorry if this isn't the right
    Neal> venue for patches.

Neal,

What's the patch cover?  Adding PyChecker to the core or something else?

Skip


From neal@metaslash.com  Thu Aug  9 20:33:52 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Thu, 09 Aug 2001 15:33:52 -0400
Subject: [Python-Dev] 2.2a1 patch from PyChecker
Message-ID: <3B72E5A0.DC2E8C4E@metaslash.com>

This is a multi-part message in MIME format.
--------------1899DB72A2CEC0DFFD96771A
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Attached is a patch to address some problems found by PyChecker.
Mostly unused imports/locals, etc.

There were a few methods that were removed because they were duplicated
(ConfigParser.py:has_option, gzip.py:writelines, locale.py:set_locale).
I deleted the first occurrence, but I'm not sure that was always right.

I have changed many unused locals to _ (e.g., for _ in range(3):).
This gets PyChecker to shut up about being unused.  It also defaults
to ignoring unused local variables if their name is 'empty' or 'unused'.

There are many more unused warnings.  Does anybody want me to change
the variables to empty, unused, _, or something different?  
Should the default unused names be expanded?

Note:  I could not run under Python 2.2a1 because dir([]) does not
return the methods.

Here are some other warnings I didn't fix because I wasn't sure what
should be done:

audiodev.py:214: No global (BUFFERSIZE) found
	(this is old, should BUFFERSIZE be defined, a better exception
	 raised, and/or deprecation warning?)

cgi.py:820: Overriden method (values) doesn't match signature in class (cgi.FormContentDict)

ihooks.py:491: Overriden method (reload) doesn't match signature in class (ihooks.BasicModuleImporter)

imaplib.py:1026: No global (j) found

ntpath.py:411: No global (WindowsError) found
	(I ran on unix, not sure if this is ok on windows.)

pdb.py:138: Overriden method (default) doesn't match signature in class (cmd.Cmd)

profile.py:179: No global (operator) found

--
Neal
--------------1899DB72A2CEC0DFFD96771A
Content-Type: text/plain; charset=us-ascii;
 name="py-2.2a1-patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="py-2.2a1-patch"

diff -C 5 Lib.orig/ConfigParser.py Lib/ConfigParser.py
*** Lib.orig/ConfigParser.py	Sat Jul 14 03:47:34 2001
--- Lib/ConfigParser.py	Mon Aug  6 20:26:14 2001
***************
*** 83,93 ****
  
      write(fp)
          write the configuration state in .ini format
  """
  
- import sys
  import string
  import re
  
  __all__ = ["NoSectionError","DuplicateSectionError","NoOptionError",
             "InterpolationError","InterpolationDepthError","ParsingError",
--- 83,92 ----
***************
*** 149,174 ****
                         % (section, option, rawval))
          self.option = option
          self.section = section
  
  class ParsingError(Error):
!     def __init__(self, filename):
!         Error.__init__(self, 'File contains parsing errors: %s' % filename)
          self.filename = filename
          self.errors = []
  
      def append(self, lineno, line):
          self.errors.append((lineno, line))
          self._msg = self._msg + '\n\t[line %2d]: %s' % (lineno, line)
  
  class MissingSectionHeaderError(ParsingError):
      def __init__(self, filename, lineno, line):
!         Error.__init__(
!             self,
              'File contains no section headers.\nfile: %s, line: %d\n%s' %
              (filename, lineno, line))
-         self.filename = filename
          self.lineno = lineno
          self.line = line
  
  
  
--- 148,174 ----
                         % (section, option, rawval))
          self.option = option
          self.section = section
  
  class ParsingError(Error):
!     def __init__(self, filename, msg=None):
! 	if msg is None:
! 	    msg = 'File contains parsing errors: %s' % filename
!         Error.__init__(self, msg)
          self.filename = filename
          self.errors = []
  
      def append(self, lineno, line):
          self.errors.append((lineno, line))
          self._msg = self._msg + '\n\t[line %2d]: %s' % (lineno, line)
  
  class MissingSectionHeaderError(ParsingError):
      def __init__(self, filename, lineno, line):
!         ParsingError.__init__(
!             self, filename,
              'File contains no section headers.\nfile: %s, line: %d\n%s' %
              (filename, lineno, line))
          self.lineno = lineno
          self.line = line
  
  
  
***************
*** 213,226 ****
              raise NoSectionError(section)
          opts.update(self.__defaults)
          if opts.has_key('__name__'):
              del opts['__name__']
          return opts.keys()
- 
-     def has_option(self, section, option):
-         """Return whether the given section has the given option."""
-         return option in self.options(section)
  
      def read(self, filenames):
          """Read and parse a filename or a list of filenames.
  
          Files that cannot be opened are silently ignored; this is
--- 213,222 ----
diff -C 5 Lib.orig/Cookie.py Lib/Cookie.py
*** Lib.orig/Cookie.py	Sat Jul  7 18:55:27 2001
--- Lib/Cookie.py	Mon Aug  6 20:26:31 2001
***************
*** 213,223 ****
  #     |----helps out font-lock
  
  #
  # Import our required modules
  #
! import string, sys
  from UserDict import UserDict
  
  try:
      from cPickle import dumps, loads
  except ImportError:
--- 213,223 ----
  #     |----helps out font-lock
  
  #
  # Import our required modules
  #
! import string
  from UserDict import UserDict
  
  try:
      from cPickle import dumps, loads
  except ImportError:
diff -C 5 Lib.orig/MimeWriter.py Lib/MimeWriter.py
*** Lib.orig/MimeWriter.py	Fri Feb  9 04:34:36 2001
--- Lib/MimeWriter.py	Mon Aug  6 20:27:58 2001
***************
*** 103,122 ****
          self._headers = []
  
      def startbody(self, ctype, plist=[], prefix=1):
          for name, value in plist:
              ctype = ctype + ';\n %s=\"%s\"' % (name, value)
!         self.addheader("Content-Type", ctype, prefix=prefix)
          self.flushheaders()
          self._fp.write("\n")
          return self._fp
  
      def startmultipartbody(self, subtype, boundary=None, plist=[], prefix=1):
          self._boundary = boundary or mimetools.choose_boundary()
          return self.startbody("multipart/" + subtype,
                                [("boundary", self._boundary)] + plist,
!                               prefix=prefix)
  
      def nextpart(self):
          self._fp.write("\n--" + self._boundary + "\n")
          return self.__class__(self._fp)
  
--- 103,122 ----
          self._headers = []
  
      def startbody(self, ctype, plist=[], prefix=1):
          for name, value in plist:
              ctype = ctype + ';\n %s=\"%s\"' % (name, value)
!         self.addheader("Content-Type", ctype, prefix)
          self.flushheaders()
          self._fp.write("\n")
          return self._fp
  
      def startmultipartbody(self, subtype, boundary=None, plist=[], prefix=1):
          self._boundary = boundary or mimetools.choose_boundary()
          return self.startbody("multipart/" + subtype,
                                [("boundary", self._boundary)] + plist,
!                               prefix)
  
      def nextpart(self):
          self._fp.write("\n--" + self._boundary + "\n")
          return self.__class__(self._fp)
  
diff -C 5 Lib.orig/StringIO.py Lib/StringIO.py
*** Lib.orig/StringIO.py	Fri Feb  9 08:37:37 2001
--- Lib/StringIO.py	Mon Aug  6 20:30:17 2001
***************
*** 182,192 ****
      print 'File length =', length
      f.seek(len(lines[0]))
      f.write(lines[1])
      f.seek(0)
      print 'First line =', `f.readline()`
-     here = f.tell()
      line = f.readline()
      print 'Second line =', `line`
      f.seek(-len(line), 1)
      line2 = f.read(len(line))
      if line != line2:
--- 182,191 ----
diff -C 5 Lib.orig/asyncore.py Lib/asyncore.py
*** Lib.orig/asyncore.py	Sat Jul  7 18:55:27 2001
--- Lib/asyncore.py	Mon Aug  6 20:31:22 2001
***************
*** 151,161 ****
          map=socket_map
      # timeout is in milliseconds
      timeout = int(timeout*1000)
      pollster = select.poll()
      if map:
-         l = []
          for fd, obj in map.items():
              flags = 0
              if obj.readable():
                  flags = select.POLLIN
              if obj.writable():
--- 151,160 ----
diff -C 5 Lib.orig/bdb.py Lib/bdb.py
*** Lib.orig/bdb.py	Sat Jul  7 18:55:27 2001
--- Lib/bdb.py	Mon Aug  6 20:43:40 2001
***************
*** 216,226 ****
          if not self.breaks.has_key(filename):
              self.breaks[filename] = []
          list = self.breaks[filename]
          if not lineno in list:
              list.append(lineno)
!         bp = Breakpoint(filename, lineno, temporary, cond)
  
      def clear_break(self, filename, lineno):
          filename = self.canonic(filename)
          if not self.breaks.has_key(filename):
              return 'There are no breakpoints in %s' % filename
--- 216,226 ----
          if not self.breaks.has_key(filename):
              self.breaks[filename] = []
          list = self.breaks[filename]
          if not lineno in list:
              list.append(lineno)
!         _ = Breakpoint(filename, lineno, temporary, cond)
  
      def clear_break(self, filename, lineno):
          filename = self.canonic(filename)
          if not self.breaks.has_key(filename):
              return 'There are no breakpoints in %s' % filename
diff -C 5 Lib.orig/cgi.py Lib/cgi.py
*** Lib.orig/cgi.py	Sat Jul  7 18:55:27 2001
--- Lib/cgi.py	Mon Aug  6 20:47:55 2001
***************
*** 846,856 ****
  
      Write minimal HTTP headers and dump all information provided to
      the script in HTML form.
  
      """
-     import traceback
      print "Content-type: text/html"
      print
      sys.stderr = sys.stdout
      try:
          form = FieldStorage()   # Replace with other classes to test those
--- 846,855 ----
diff -C 5 Lib.orig/codecs.py Lib/codecs.py
*** Lib.orig/codecs.py	Fri Jul  6 03:01:30 2001
--- Lib/codecs.py	Mon Aug  6 20:48:48 2001
***************
*** 5,15 ****
  
  (c) Copyright CNRI, All Rights Reserved. NO WARRANTY.
  
  """#"
  
! import struct, types, __builtin__
  
  ### Registry and builtin stateless codec functions
  
  try:
      from _codecs import *
--- 5,15 ----
  
  (c) Copyright CNRI, All Rights Reserved. NO WARRANTY.
  
  """#"
  
! import struct, __builtin__
  
  ### Registry and builtin stateless codec functions
  
  try:
      from _codecs import *
***************
*** 215,225 ****
          data = read(size)
          i = 0
          while 1:
              try:
                  object, decodedbytes = decode(data, self.errors)
!             except ValueError, why:
                  # This method is slow but should work under pretty much
                  # all conditions; at most 10 tries are made
                  i = i + 1
                  newdata = read(1)
                  if not newdata or i > 10:
--- 215,225 ----
          data = read(size)
          i = 0
          while 1:
              try:
                  object, decodedbytes = decode(data, self.errors)
!             except ValueError:
                  # This method is slow but should work under pretty much
                  # all conditions; at most 10 tries are made
                  i = i + 1
                  newdata = read(1)
                  if not newdata or i > 10:
diff -C 5 Lib.orig/codeop.py Lib/codeop.py
*** Lib.orig/codeop.py	Sat Apr  7 20:43:13 2001
--- Lib/codeop.py	Mon Aug  6 20:49:33 2001
***************
*** 50,65 ****
          if line and line[0] != '#':
              break               # Leave it alone
      else:
          source = "pass"         # Replace it with a 'pass' statement
  
!     err = err1 = err2 = None
      code = code1 = code2 = None
  
      try:
          code = compile(source, filename, symbol)
!     except SyntaxError, err:
          pass
  
      try:
          code1 = compile(source + "\n", filename, symbol)
      except SyntaxError, err1:
--- 50,65 ----
          if line and line[0] != '#':
              break               # Leave it alone
      else:
          source = "pass"         # Replace it with a 'pass' statement
  
!     err1 = err2 = None
      code = code1 = code2 = None
  
      try:
          code = compile(source, filename, symbol)
!     except SyntaxError:
          pass
  
      try:
          code1 = compile(source + "\n", filename, symbol)
      except SyntaxError, err1:
diff -C 5 Lib.orig/fileinput.py Lib/fileinput.py
*** Lib.orig/fileinput.py	Sat Jan 20 18:34:12 2001
--- Lib/fileinput.py	Mon Aug  6 20:53:16 2001
***************
*** 287,297 ****
      backup = 0
      opts, args = getopt.getopt(sys.argv[1:], "ib:")
      for o, a in opts:
          if o == '-i': inplace = 1
          if o == '-b': backup = a
!     for line in input(args, inplace=inplace, backup=backup):
          if line[-1:] == '\n': line = line[:-1]
          if line[-1:] == '\r': line = line[:-1]
          print "%d: %s[%d]%s %s" % (lineno(), filename(), filelineno(),
                                     isfirstline() and "*" or "", line)
      print "%d: %s[%d]" % (lineno(), filename(), filelineno())
--- 287,297 ----
      backup = 0
      opts, args = getopt.getopt(sys.argv[1:], "ib:")
      for o, a in opts:
          if o == '-i': inplace = 1
          if o == '-b': backup = a
!     for line in input(args, inplace, backup):
          if line[-1:] == '\n': line = line[:-1]
          if line[-1:] == '\r': line = line[:-1]
          print "%d: %s[%d]%s %s" % (lineno(), filename(), filelineno(),
                                     isfirstline() and "*" or "", line)
      print "%d: %s[%d]" % (lineno(), filename(), filelineno())
diff -C 5 Lib.orig/formatter.py Lib/formatter.py
*** Lib.orig/formatter.py	Sat Jul  7 18:55:27 2001
--- Lib/formatter.py	Mon Aug  6 20:53:57 2001
***************
*** 299,311 ****
      def send_literal_data(self, data): pass
  
  
  class AbstractWriter(NullWriter):
  
-     def __init__(self):
-         pass
- 
      def new_alignment(self, align):
          print "new_alignment(%s)" % `align`
  
      def new_font(self, font):
          print "new_font(%s)" % `font`
--- 299,308 ----
diff -C 5 Lib.orig/ftplib.py Lib/ftplib.py
*** Lib.orig/ftplib.py	Mon Apr  9 00:31:50 2001
--- Lib/ftplib.py	Mon Aug  6 20:54:20 2001
***************
*** 34,44 ****
  # Modified by Siebren to support docstrings and PASV.
  #
  
  import os
  import sys
- import string
  
  # Import SOCKS module if it exists, else standard socket module socket
  try:
      import SOCKS; socket = SOCKS; del SOCKS # import SOCKS as socket
      from socket import getfqdn; socket.getfqdn = getfqdn; del getfqdn
--- 34,43 ----
diff -C 5 Lib.orig/getpass.py Lib/getpass.py
*** Lib.orig/getpass.py	Tue Feb 27 16:23:31 2001
--- Lib/getpass.py	Mon Aug  6 20:56:40 2001
***************
*** 24,34 ****
      try:
          fd = sys.stdin.fileno()
      except:
          return default_getpass(prompt)
  
-     getpass = default_getpass
      old = termios.tcgetattr(fd)     # a copy to save
      new = old[:]
  
      new[3] = new[3] & ~termios.ECHO # 3 == 'lflags'
      try:
--- 24,33 ----
***************
*** 68,78 ****
  
  
  def _raw_input(prompt=""):
      # A raw_input() replacement that doesn't save the string in the
      # GNU readline history.
-     import sys
      prompt = str(prompt)
      if prompt:
          sys.stdout.write(prompt)
      line = sys.stdin.readline()
      if not line:
--- 67,76 ----
diff -C 5 Lib.orig/gettext.py Lib/gettext.py
*** Lib.orig/gettext.py	Tue Jan 23 10:35:04 2001
--- Lib/gettext.py	Mon Aug  6 20:57:07 2001
***************
*** 156,166 ****
          msgcount &= MASK
          masteridx &= MASK
          transidx &= MASK
          # Now put all messages from the .mo file buffer into the catalog
          # dictionary.
!         for i in xrange(0, msgcount):
              mlen, moff = unpack(ii, buf[masteridx:masteridx+8])
              moff &= MASK
              mend = moff + (mlen & MASK)
              tlen, toff = unpack(ii, buf[transidx:transidx+8])
              toff &= MASK
--- 156,166 ----
          msgcount &= MASK
          masteridx &= MASK
          transidx &= MASK
          # Now put all messages from the .mo file buffer into the catalog
          # dictionary.
!         for _ in xrange(0, msgcount):
              mlen, moff = unpack(ii, buf[masteridx:masteridx+8])
              moff &= MASK
              mend = moff + (mlen & MASK)
              tlen, toff = unpack(ii, buf[transidx:transidx+8])
              toff &= MASK
diff -C 5 Lib.orig/gopherlib.py Lib/gopherlib.py
*** Lib.orig/gopherlib.py	Fri Feb  9 05:10:02 2001
--- Lib/gopherlib.py	Mon Aug  6 20:57:23 2001
***************
*** 167,177 ****
      import getopt
      opts, args = getopt.getopt(sys.argv[1:], '')
      selector = DEF_SELECTOR
      type = selector[0]
      host = DEF_HOST
-     port = DEF_PORT
      if args:
          host = args[0]
          args = args[1:]
      if args:
          type = args[0]
--- 167,176 ----
diff -C 5 Lib.orig/gzip.py Lib/gzip.py
*** Lib.orig/gzip.py	Tue Mar 20 10:51:14 2001
--- Lib/gzip.py	Mon Aug  6 21:00:00 2001
***************
*** 137,149 ****
          if len(data) > 0:
              self.size = self.size + len(data)
              self.crc = zlib.crc32(data, self.crc)
              self.fileobj.write( self.compress.compress(data) )
  
-     def writelines(self,lines):
-         self.write(" ".join(lines))
- 
      def read(self, size=-1):
          if self.extrasize <= 0 and self.fileobj is None:
              return ''
  
          readsize = 1024
--- 137,146 ----
***************
*** 271,281 ****
          return 0
  
      def readline(self, size=-1):
          if size < 0: size = sys.maxint
          bufs = []
-         orig_size = size
          readsize = min(100, size)    # Read from the file in small chunks
          while 1:
              if size == 0:
                  return "".join(bufs) # Return resulting line
  
--- 268,277 ----
***************
*** 318,328 ****
  
  def _test():
      # Act like gzip; with -d, act like gunzip.
      # The input file is not deleted, however, nor are any other gzip
      # options or features supported.
-     import sys
      args = sys.argv[1:]
      decompress = args and args[0] == "-d"
      if decompress:
          args = args[1:]
      if not args:
--- 314,323 ----
diff -C 5 Lib.orig/httplib.py Lib/httplib.py
*** Lib.orig/httplib.py	Sat Jul  7 18:55:27 2001
--- Lib/httplib.py	Mon Aug  6 21:01:20 2001
***************
*** 588,598 ****
  
          msgbuf = []
          while 1:
              try:
                  buf = self.__ssl.read()
!             except socket.sslerror, msg:
                  break
              if buf == '':
                  break
              msgbuf.append(buf)
          return StringIO("".join(msgbuf))
--- 588,598 ----
  
          msgbuf = []
          while 1:
              try:
                  buf = self.__ssl.read()
!             except socket.sslerror:
                  break
              if buf == '':
                  break
              msgbuf.append(buf)
          return StringIO("".join(msgbuf))
diff -C 5 Lib.orig/ihooks.py Lib/ihooks.py
*** Lib.orig/ihooks.py	Fri Feb  9 05:17:30 2001
--- Lib/ihooks.py	Mon Aug  6 21:03:51 2001
***************
*** 489,503 ****
          return m
  
      def reload(self, module):
          name = module.__name__
          if '.' not in name:
!             return self.import_it(name, name, None, force_load=1)
          i = name.rfind('.')
          pname = name[:i]
          parent = self.modules[pname]
!         return self.import_it(name[i+1:], name, parent, force_load=1)
  
  
  default_importer = None
  current_importer = None
  
--- 489,503 ----
          return m
  
      def reload(self, module):
          name = module.__name__
          if '.' not in name:
!             return self.import_it(name, name, None, 1)
          i = name.rfind('.')
          pname = name[:i]
          parent = self.modules[pname]
!         return self.import_it(name[i+1:], name, parent, 1)
  
  
  default_importer = None
  current_importer = None
  
diff -C 5 Lib.orig/imaplib.py Lib/imaplib.py
*** Lib.orig/imaplib.py	Sat Jul  7 18:55:27 2001
--- Lib/imaplib.py	Mon Aug  6 21:05:16 2001
***************
*** 778,788 ****
              # Instead, send me details of the unexpected response and
              # I'll update the code in `_get_response()'.
  
              try:
                  self._get_response()
!             except self.abort, val:
                  if __debug__:
                      if self.debug >= 1:
                          print_log()
                  raise
  
--- 778,788 ----
              # Instead, send me details of the unexpected response and
              # I'll update the code in `_get_response()'.
  
              try:
                  self._get_response()
!             except self.abort:
                  if __debug__:
                      if self.debug >= 1:
                          print_log()
                  raise
  
***************
*** 1041,1051 ****
  
  
  
  if __name__ == '__main__':
  
!     import getopt, getpass, sys
  
      try:
          optlist, args = getopt.getopt(sys.argv[1:], 'd:')
      except getopt.error, val:
          pass
--- 1041,1051 ----
  
  
  
  if __name__ == '__main__':
  
!     import getopt, getpass
  
      try:
          optlist, args = getopt.getopt(sys.argv[1:], 'd:')
      except getopt.error, val:
          pass
diff -C 5 Lib.orig/locale.py Lib/locale.py
*** Lib.orig/locale.py	Mon Apr 16 12:04:10 2001
--- Lib/locale.py	Mon Aug  6 21:07:58 2001
***************
*** 63,80 ****
                  'mon_thousands_sep': '',
                  'frac_digits': 127,
                  'mon_decimal_point': '',
                  'int_frac_digits': 127}
  
-     def setlocale(category, value=None):
-         """ setlocale(integer,string=None) -> string.
-             Activates/queries locale processing.
-         """
-         if value is not None and value != 'C':
-             raise Error, '_locale emulation only supports "C" locale'
-         return 'C'
- 
      def strcoll(a,b):
          """ strcoll(string,string) -> int.
              Compares two strings according to the locale.
          """
          return cmp(a,b)
--- 63,72 ----
***************
*** 266,278 ****
      code = normalize(localename)
      if '.' in code:
          return code.split('.')[:2]
      elif code == 'C':
          return None, None
!     else:
!         raise ValueError, 'unknown locale: %s' % localename
!     return l
  
  def _build_localename(localetuple):
  
      """ Builds a locale code from the given tuple (language code,
          encoding).
--- 258,268 ----
      code = normalize(localename)
      if '.' in code:
          return code.split('.')[:2]
      elif code == 'C':
          return None, None
!     raise ValueError, 'unknown locale: %s' % localename
  
  def _build_localename(localetuple):
  
      """ Builds a locale code from the given tuple (language code,
          encoding).
diff -C 5 Lib.orig/mailbox.py Lib/mailbox.py
*** Lib.orig/mailbox.py	Sat Jul  7 18:55:27 2001
--- Lib/mailbox.py	Mon Aug  6 21:08:39 2001
***************
*** 256,268 ****
                  self.fp.seek(pos)
                  return
  
  
  def _test():
-     import time
      import sys
-     import os
  
      args = sys.argv[1:]
      if not args:
          for key in 'MAILDIR', 'MAIL', 'LOGNAME', 'USER':
              if os.environ.has_key(key):
--- 256,266 ----
diff -C 5 Lib.orig/multifile.py Lib/multifile.py
*** Lib.orig/multifile.py	Sun Mar 11 21:56:15 2001
--- Lib/multifile.py	Mon Aug  6 21:11:47 2001
***************
*** 25,36 ****
  it normally attempts in order to make seeks relative to the beginning of the
  current file part.  This may be useful when using MultiFile with a non-
  seekable stream object.
  """
  
- import sys
- 
  __all__ = ["MultiFile","Error"]
  
  class Error(Exception):
      pass
  
--- 25,34 ----
diff -C 5 Lib.orig/nturl2path.py Lib/nturl2path.py
*** Lib.orig/nturl2path.py	Sat Feb 17 22:30:53 2001
--- Lib/nturl2path.py	Mon Aug  6 21:14:48 2001
***************
*** 40,50 ****
                      becomes
  
              ///C|/foo/bar/spam.foo
      """
  
!     import string, urllib
      if not ':' in p:
          # No drive specifier, just convert slashes and quote the name
          if p[:2] == '\\\\':
          # path is something like \\host\path\on\remote\host
          # convert this to ////host/path/on/remote/host
--- 40,50 ----
                      becomes
  
              ///C|/foo/bar/spam.foo
      """
  
!     import urllib
      if not ':' in p:
          # No drive specifier, just convert slashes and quote the name
          if p[:2] == '\\\\':
          # path is something like \\host\path\on\remote\host
          # convert this to ////host/path/on/remote/host
diff -C 5 Lib.orig/pdb.py Lib/pdb.py
*** Lib.orig/pdb.py	Fri Feb  9 18:28:07 2001
--- Lib/pdb.py	Mon Aug  6 21:15:22 2001
***************
*** 214,224 ****
              else:
                  filename = f
              arg = arg[colon+1:].lstrip()
              try:
                  lineno = int(arg)
!             except ValueError, msg:
                  print '*** Bad lineno:', arg
                  return
          else:
              # no colon; can be lineno or function
              try:
--- 214,224 ----
              else:
                  filename = f
              arg = arg[colon+1:].lstrip()
              try:
                  lineno = int(arg)
!             except ValueError:
                  print '*** Bad lineno:', arg
                  return
          else:
              # no colon; can be lineno or function
              try:
diff -C 5 Lib.orig/pipes.py Lib/pipes.py
*** Lib.orig/pipes.py	Sat Jul  7 18:55:28 2001
--- Lib/pipes.py	Mon Aug  6 21:16:37 2001
***************
*** 57,67 ****
  
  For an example, see the function test() at the end of the file.
  """                                     # '
  
  
- import sys
  import re
  
  import os
  import tempfile
  import string
--- 57,66 ----
diff -C 5 Lib.orig/posixfile.py Lib/posixfile.py
*** Lib.orig/posixfile.py	Sat Jul  7 18:55:28 2001
--- Lib/posixfile.py	Thu Aug  9 15:25:13 2001
***************
*** 105,115 ****
  
          posix.dup2(self._file_.fileno(), fd)
          return posix.fdopen(fd, self._file_.mode)
  
      def flags(self, *which):
!         import fcntl
  
          if which:
              if len(which) > 1:
                  raise TypeError, 'Too many arguments'
              which = which[0]
--- 105,115 ----
  
          posix.dup2(self._file_.fileno(), fd)
          return posix.fdopen(fd, self._file_.mode)
  
      def flags(self, *which):
!         import fcntl, os
  
          if which:
              if len(which) > 1:
                  raise TypeError, 'Too many arguments'
              which = which[0]
***************
*** 142,152 ****
              if os.O_NDELAY & l_flags: which = which + 'n'
              if os.O_SYNC & l_flags: which = which + 's'
              return which
  
      def lock(self, how, *args):
!         import struct, fcntl
  
          if 'w' in how: l_type = fcntl.F_WRLCK
          elif 'r' in how: l_type = fcntl.F_RDLCK
          elif 'u' in how: l_type = fcntl.F_UNLCK
          else: raise TypeError, 'no type of lock specified'
--- 142,152 ----
              if os.O_NDELAY & l_flags: which = which + 'n'
              if os.O_SYNC & l_flags: which = which + 's'
              return which
  
      def lock(self, how, *args):
!         import struct, fcntl, os
  
          if 'w' in how: l_type = fcntl.F_WRLCK
          elif 'r' in how: l_type = fcntl.F_RDLCK
          elif 'u' in how: l_type = fcntl.F_UNLCK
          else: raise TypeError, 'no type of lock specified'
diff -C 5 Lib.orig/profile.py Lib/profile.py
*** Lib.orig/profile.py	Sat Jul  7 18:55:28 2001
--- Lib/profile.py	Mon Aug  6 21:18:22 2001
***************
*** 143,153 ****
          self.cur = None
          self.cmd = ""
  
          if not timer:
              if os.name == 'mac':
-                 import MacOS
                  self.timer = MacOS.GetTicks
                  self.dispatcher = self.trace_dispatch_mac
                  self.get_time = _get_time_mac
              elif hasattr(time, 'clock'):
                  self.timer = self.get_time = time.clock
--- 143,152 ----
diff -C 5 Lib.orig/pstats.py Lib/pstats.py
*** Lib.orig/pstats.py	Sat Jul  7 18:55:28 2001
--- Lib/pstats.py	Mon Aug  6 21:19:33 2001
***************
*** 183,193 ****
  
      def get_sort_arg_defs(self):
          """Expand all abbreviations that are unique."""
          if not self.sort_arg_dict:
              self.sort_arg_dict = dict = {}
-             std_list = dict.keys()
              bad_list = {}
              for word in self.sort_arg_dict_default.keys():
                  fragment = word
                  while fragment:
                      if not fragment:
--- 183,192 ----
diff -C 5 Lib.orig/pyclbr.py Lib/pyclbr.py
*** Lib.orig/pyclbr.py	Sun Feb 11 21:00:42 2001
--- Lib/pyclbr.py	Mon Aug  6 21:20:20 2001
***************
*** 51,61 ****
    It can't locate the parent.  It probably needs to have the same
    hairy logic that the import locator already does.  (This logic
    exists coded in Python in the freeze package.)
  """
  
- import os
  import sys
  import imp
  import re
  import string
  
--- 51,60 ----
***************
*** 201,211 ****
          f.close()
          _modules[module] = dict
          return dict
  
      _modules[module] = dict
-     imports = []
      classstack = [] # stack of (class, indent) pairs
      src = f.read()
      f.close()
  
      # To avoid having to stop the regexp at each newline, instead
--- 200,209 ----
diff -C 5 Lib.orig/random.py Lib/random.py
*** Lib.orig/random.py	Thu Feb 15 18:56:39 2001
--- Lib/random.py	Mon Aug  6 21:20:48 2001
***************
*** 582,592 ****
      sum = 0.0
      sqsum = 0.0
      smallest = 1e10
      largest = -1e10
      t0 = time.time()
!     for i in range(n):
          x = eval(code)
          sum = sum + x
          sqsum = sqsum + x*x
          smallest = min(x, smallest)
          largest = max(x, largest)
--- 582,592 ----
      sum = 0.0
      sqsum = 0.0
      smallest = 1e10
      largest = -1e10
      t0 = time.time()
!     for _ in range(n):
          x = eval(code)
          sum = sum + x
          sqsum = sqsum + x*x
          smallest = min(x, smallest)
          largest = max(x, largest)
***************
*** 623,633 ****
      s = getstate()
      jumpahead(N)
      r1 = random()
      # now do it the slow way
      setstate(s)
!     for i in range(N):
          random()
      r2 = random()
      if r1 != r2:
          raise ValueError("jumpahead test failed " + `(N, r1, r2)`)
  
--- 623,633 ----
      s = getstate()
      jumpahead(N)
      r1 = random()
      # now do it the slow way
      setstate(s)
!     for _ in range(N):
          random()
      r2 = random()
      if r1 != r2:
          raise ValueError("jumpahead test failed " + `(N, r1, r2)`)
  
diff -C 5 Lib.orig/rexec.py Lib/rexec.py
*** Lib.orig/rexec.py	Sat Jul  7 18:55:28 2001
--- Lib/rexec.py	Mon Aug  6 21:22:08 2001
***************
*** 366,376 ****
          tr = None
          return ty, va, tr
  
  
  def test():
!     import sys, getopt, traceback
      opts, args = getopt.getopt(sys.argv[1:], 'vt:')
      verbose = 0
      trusted = []
      for o, a in opts:
          if o == '-v':
--- 366,376 ----
          tr = None
          return ty, va, tr
  
  
  def test():
!     import getopt, traceback
      opts, args = getopt.getopt(sys.argv[1:], 'vt:')
      verbose = 0
      trusted = []
      for o, a in opts:
          if o == '-v':
***************
*** 387,397 ****
          r.modules['sys'].path.insert(0, "")
      fp = sys.stdin
      if args and args[0] != '-':
          try:
              fp = open(args[0])
!         except IOError, msg:
              print "%s: can't open file %s" % (sys.argv[0], `args[0]`)
              return 1
      if fp.isatty():
          print "*** RESTRICTED *** Python", sys.version
          print sys.copyright
--- 387,397 ----
          r.modules['sys'].path.insert(0, "")
      fp = sys.stdin
      if args and args[0] != '-':
          try:
              fp = open(args[0])
!         except IOError:
              print "%s: can't open file %s" % (sys.argv[0], `args[0]`)
              return 1
      if fp.isatty():
          print "*** RESTRICTED *** Python", sys.version
          print sys.copyright
diff -C 5 Lib.orig/rfc822.py Lib/rfc822.py
*** Lib.orig/rfc822.py	Tue Jul 17 00:19:05 2001
--- Lib/rfc822.py	Mon Aug  6 21:22:48 2001
***************
*** 631,641 ****
              elif self.field[self.pos] == '@':
                  self.pos = self.pos + 1
                  expectroute = 1
              elif self.field[self.pos] == ':':
                  self.pos = self.pos + 1
-                 expectaddrspec = 1
              else:
                  adlist = self.getaddrspec()
                  self.pos = self.pos + 1
                  break
              self.gotonext()
--- 631,640 ----
diff -C 5 Lib.orig/robotparser.py Lib/robotparser.py
*** Lib.orig/robotparser.py	Thu Feb 15 18:56:39 2001
--- Lib/robotparser.py	Mon Aug  6 21:22:59 2001
***************
*** 229,239 ****
          print "ok (%s)" % ac
      print
  
  def _test():
      global debug
-     import sys
      rp = RobotFileParser()
      debug = 1
  
      # robots.txt that exists, gotten to by redirection
      rp.set_url('http://www.musi-cal.com/robots.txt')
--- 229,238 ----
diff -C 5 Lib.orig/sched.py Lib/sched.py
*** Lib.orig/sched.py	Thu Feb 15 17:15:13 2001
--- Lib/sched.py	Mon Aug  6 21:23:12 2001
***************
*** 100,106 ****
              now = self.timefunc()
              if now < time:
                  self.delayfunc(time - now)
              else:
                  del q[0]
!                 void = apply(action, argument)
                  self.delayfunc(0)   # Let other threads run
--- 100,106 ----
              now = self.timefunc()
              if now < time:
                  self.delayfunc(time - now)
              else:
                  del q[0]
!                 _ = apply(action, argument)
                  self.delayfunc(0)   # Let other threads run
diff -C 5 Lib.orig/sgmllib.py Lib/sgmllib.py
*** Lib.orig/sgmllib.py	Mon Jul 16 17:39:41 2001
--- Lib/sgmllib.py	Mon Aug  6 21:23:43 2001
***************
*** 8,18 ****
  # and CDATA (character data -- only end tags are special).  RCDATA is
  # not supported at all.
  
  
  import re
- import string
  
  __all__ = ["SGMLParser"]
  
  # Regular expressions used for parsing
  
--- 8,17 ----
diff -C 5 Lib.orig/smtpd.py Lib/smtpd.py
*** Lib.orig/smtpd.py	Sun Apr 15 09:06:04 2001
--- Lib/smtpd.py	Mon Aug  6 21:25:08 2001
***************
*** 370,380 ****
  
  
  class MailmanProxy(PureProxy):
      def process_message(self, peer, mailfrom, rcpttos, data):
          from cStringIO import StringIO
-         import paths
          from Mailman import Utils
          from Mailman import Message
          from Mailman import MailList
          # If the message is to a Mailman mailing list, then we'll invoke the
          # Mailman script directly, without going through the real smtpd.
--- 370,379 ----
diff -C 5 Lib.orig/smtplib.py Lib/smtplib.py
*** Lib.orig/smtplib.py	Thu Feb 15 17:15:13 2001
--- Lib/smtplib.py	Mon Aug  6 21:25:37 2001
***************
*** 518,528 ****
  
  
  # Test the sendmail method, which tests most of the others.
  # Note: This always sends to localhost.
  if __name__ == '__main__':
!     import sys, rfc822
  
      def prompt(prompt):
          sys.stdout.write(prompt + ": ")
          return sys.stdin.readline().strip()
  
--- 518,528 ----
  
  
  # Test the sendmail method, which tests most of the others.
  # Note: This always sends to localhost.
  if __name__ == '__main__':
!     import sys
  
      def prompt(prompt):
          sys.stdout.write(prompt + ": ")
          return sys.stdin.readline().strip()
  
diff -C 5 Lib.orig/sunau.py Lib/sunau.py
*** Lib.orig/sunau.py	Sun Jan 14 20:36:40 2001
--- Lib/sunau.py	Mon Aug  6 21:26:36 2001
***************
*** 131,141 ****
  class Error(Exception):
      pass
  
  def _read_u32(file):
      x = 0L
!     for i in range(4):
          byte = file.read(1)
          if byte == '':
              raise EOFError
          x = x*256 + ord(byte)
      return x
--- 131,141 ----
  class Error(Exception):
      pass
  
  def _read_u32(file):
      x = 0L
!     for _ in range(4):
          byte = file.read(1)
          if byte == '':
              raise EOFError
          x = x*256 + ord(byte)
      return x
diff -C 5 Lib.orig/threading.py Lib/threading.py
*** Lib.orig/threading.py	Mon Apr  2 16:15:57 2001
--- Lib/threading.py	Mon Aug  6 21:30:23 2001
***************
*** 542,553 ****
  
  # Self-test code
  
  def _test():
  
-     import random
- 
      class BoundedQueue(_Verbose):
  
          def __init__(self, limit):
              _Verbose.__init__(self)
              self.mon = RLock()
--- 542,551 ----
***************
*** 605,616 ****
          def run(self):
              while self.count > 0:
                  item = self.queue.get()
                  print item
                  self.count = self.count - 1
- 
-     import time
  
      NP = 3
      QL = 4
      NI = 5
  
--- 603,612 ----
diff -C 5 Lib.orig/toaiff.py Lib/toaiff.py
*** Lib.orig/toaiff.py	Wed Feb 28 23:27:19 2001
--- Lib/toaiff.py	Mon Aug  6 21:31:07 2001
***************
*** 84,94 ****
          fname = filename
      try:
          ftype = sndhdr.whathdr(fname)
          if ftype:
              ftype = ftype[0] # All we're interested in
!     except IOError:
          if type(msg) == type(()) and len(msg) == 2 and \
                  type(msg[0]) == type(0) and type(msg[1]) == type(''):
              msg = msg[1]
          if type(msg) != type(''):
              msg = `msg`
--- 84,94 ----
          fname = filename
      try:
          ftype = sndhdr.whathdr(fname)
          if ftype:
              ftype = ftype[0] # All we're interested in
!     except IOError, msg:
          if type(msg) == type(()) and len(msg) == 2 and \
                  type(msg[0]) == type(0) and type(msg[1]) == type(''):
              msg = msg[1]
          if type(msg) != type(''):
              msg = `msg`
diff -C 5 Lib.orig/token.py Lib/token.py
*** Lib.orig/token.py	Fri Feb  9 19:20:16 2001
--- Lib/token.py	Mon Aug  6 21:31:28 2001
***************
*** 80,90 ****
      return x == ENDMARKER
  
  
  def main():
      import re
-     import string
      import sys
      args = sys.argv[1:]
      inFileName = args and args[0] or "Include/token.h"
      outFileName = "Lib/token.py"
      if len(args) > 1:
--- 80,89 ----
diff -C 5 Lib.orig/urllib2.py Lib/urllib2.py
*** Lib.orig/urllib2.py	Sat Jul  7 18:55:28 2001
--- Lib/urllib2.py	Mon Aug  6 21:36:33 2001
***************
*** 86,96 ****
  # ftp errors aren't handled cleanly
  # gopher can return a socket.error
  # check digest against correct (i.e. non-apache) implementation
  
  import socket
- import UserDict
  import httplib
  import re
  import base64
  import types
  import urlparse
--- 86,95 ----
***************
*** 435,445 ****
              newurl = headers['location']
          elif headers.has_key('uri'):
              newurl = headers['uri']
          else:
              return
!         nil = fp.read()
          fp.close()
  
          newurl = urlparse.urljoin(req.get_full_url(), newurl)
  
          # XXX Probably want to forget about the state of the current
--- 434,444 ----
              newurl = headers['location']
          elif headers.has_key('uri'):
              newurl = headers['uri']
          else:
              return
!         _ = fp.read()
          fp.close()
  
          newurl = urlparse.urljoin(req.get_full_url(), newurl)
  
          # XXX Probably want to forget about the state of the current
***************
*** 449,459 ****
          new.error_302_dict = {}
          if hasattr(req, 'error_302_dict'):
              if len(req.error_302_dict)>10 or \
                 req.error_302_dict.has_key(newurl):
                  raise HTTPError(req.get_full_url(), code,
!                                 self.inf_msg + msg, headers)
              new.error_302_dict.update(req.error_302_dict)
          new.error_302_dict[newurl] = newurl
          return self.parent.open(new)
  
      http_error_301 = http_error_302
--- 448,458 ----
          new.error_302_dict = {}
          if hasattr(req, 'error_302_dict'):
              if len(req.error_302_dict)>10 or \
                 req.error_302_dict.has_key(newurl):
                  raise HTTPError(req.get_full_url(), code,
!                                 self.inf_msg + msg, headers, None)
              new.error_302_dict.update(req.error_302_dict)
          new.error_302_dict[newurl] = newurl
          return self.parent.open(new)
  
      http_error_301 = http_error_302
***************
*** 658,668 ****
  
  class AbstractDigestAuthHandler:
  
      def __init__(self, passwd=None):
          if passwd is None:
!             passwd = HTTPPassowrdMgr()
          self.passwd = passwd
          self.add_password = self.passwd.add_password
          self.__current_realm = None
  
      def http_error_auth_reqed(self, authreq, host, req, headers):
--- 657,667 ----
  
  class AbstractDigestAuthHandler:
  
      def __init__(self, passwd=None):
          if passwd is None:
!             passwd = HTTPPasswordMgr()
          self.passwd = passwd
          self.add_password = self.passwd.add_password
          self.__current_realm = None
  
      def http_error_auth_reqed(self, authreq, host, req, headers):
diff -C 5 Lib.orig/urlparse.py Lib/urlparse.py
*** Lib.orig/urlparse.py	Wed Feb 28 23:27:19 2001
--- Lib/urlparse.py	Mon Aug  6 21:37:47 2001
***************
*** 53,63 ****
      cached = _parse_cache.get(key, None)
      if cached:
          return cached
      if len(_parse_cache) >= MAX_CACHE_SIZE: # avoid runaway growth
          clear_cache()
!     netloc = path = params = query = fragment = ''
      i = url.find(':')
      if i > 0:
          if url[:i] == 'http': # optimize the common case
              scheme = url[:i].lower()
              url = url[i+1:]
--- 53,63 ----
      cached = _parse_cache.get(key, None)
      if cached:
          return cached
      if len(_parse_cache) >= MAX_CACHE_SIZE: # avoid runaway growth
          clear_cache()
!     netloc = params = query = fragment = ''
      i = url.find(':')
      if i > 0:
          if url[:i] == 'http': # optimize the common case
              scheme = url[:i].lower()
              url = url[i+1:]
diff -C 5 Lib.orig/warnings.py Lib/warnings.py
*** Lib.orig/warnings.py	Wed Feb 28 23:27:19 2001
--- Lib/warnings.py	Mon Aug  6 21:38:04 2001
***************
*** 223,233 ****
      for item in filters: print item
      hello = "hello world"
      warn(hello); warn(hello); warn(hello); warn(hello)
      warn(hello, UserWarning)
      warn(hello, DeprecationWarning)
!     for i in range(3):
          warn(hello)
      filterwarnings("error", "", Warning, "", 0)
      try:
          warn(hello)
      except Exception, msg:
--- 223,233 ----
      for item in filters: print item
      hello = "hello world"
      warn(hello); warn(hello); warn(hello); warn(hello)
      warn(hello, UserWarning)
      warn(hello, DeprecationWarning)
!     for _ in range(3):
          warn(hello)
      filterwarnings("error", "", Warning, "", 0)
      try:
          warn(hello)
      except Exception, msg:
diff -C 5 Lib.orig/webbrowser.py Lib/webbrowser.py
*** Lib.orig/webbrowser.py	Thu Apr 12 18:07:27 2001
--- Lib/webbrowser.py	Mon Aug  6 21:39:15 2001
***************
*** 72,82 ****
          controller = copy.copy(controller)
          controller.name = browser
          controller.basename = os.path.basename(browser)
          register(browser, None, controller)
          return [None, controller]
!     ret
  
  #
  # Everything after this point initializes _browsers and _tryorder,
  # then disappears.  Some class definitions and instances remain
  # live through these globals, but only the minimum set needed to
--- 72,82 ----
          controller = copy.copy(controller)
          controller.name = browser
          controller.basename = os.path.basename(browser)
          register(browser, None, controller)
          return [None, controller]
!     return [None, None]
  
  #
  # Everything after this point initializes _browsers and _tryorder,
  # then disappears.  Some class definitions and instances remain
  # live through these globals, but only the minimum set needed to
diff -C 5 Lib.orig/xdrlib.py Lib/xdrlib.py
*** Lib.orig/xdrlib.py	Wed Feb 28 23:27:19 2001
--- Lib/xdrlib.py	Mon Aug  6 21:40:21 2001
***************
*** 212,222 ****
              list.append(item)
          return list
  
      def unpack_farray(self, n, unpack_item):
          list = []
!         for i in range(n):
              list.append(unpack_item())
          return list
  
      def unpack_array(self, unpack_item):
          n = self.unpack_uint()
--- 212,222 ----
              list.append(item)
          return list
  
      def unpack_farray(self, n, unpack_item):
          list = []
!         for _ in range(n):
              list.append(unpack_item())
          return list
  
      def unpack_array(self, unpack_item):
          n = self.unpack_uint()

--------------1899DB72A2CEC0DFFD96771A--



From neal@metaslash.com  Thu Aug  9 21:46:58 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Thu, 09 Aug 2001 16:46:58 -0400
Subject: [Python-Dev] Direction of PyChecker
References: <3B72EF79.7BE8F6C1@metaslash.com> <15218.62438.445125.366624@beluga.mojam.com>
Message-ID: <3B72F6C2.72E2DF48@metaslash.com>

Skip Montanaro wrote:
> 
>     Neal> PS I sent a patch to python-dev against 2.2a1, but it's 45k.  So
>     Neal> it's awating moderator approval.  Sorry if this isn't the right
>     Neal> venue for patches.

> What's the patch cover?  Adding PyChecker to the core or something else?

The patch is for the standard modules in Python-2.2a1/Lib.  I didn't
run PyChecker on any subdirs under Lib.  I should do that too.

Neal


From guido@python.org  Thu Aug  9 21:52:26 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 09 Aug 2001 16:52:26 -0400
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: Your message of "Thu, 09 Aug 2001 16:15:53 EDT."
 <3B72EF79.7BE8F6C1@metaslash.com>
References: <3B72EF79.7BE8F6C1@metaslash.com>
Message-ID: <200108092052.f79KqQg13802@odiug.digicool.com>

> I'd like to solicit feedback on what everyone thinks about PyChecker.

I love it when it works!

> Specifically, where should it go?  What should it include or not include?
> What should the default warnings be?

You'll never satisfy everyone.  Personally, I write too much code
without docstrings, so I find the complaints about missing docstrings
annoying.  I guess my preference would be to have the default mode ony
complain about things that have a high probability of being *coding*
errors -- undefined names and the like.  I forget if you're checking
string formatting conformance yet -- that would also be useful.

> I've seen some threads on c.l.p that checker might address.

Unfortunately the typical newbie doesn't have access to pychecker
yet...

> Do you think that there should be something added to the python 
> FAQ/doc about checker?  I'd be willing to write something.

Sure!  You can add a FAQ entry yourself -- the password is "Spam".

> I'm close to exhausting my list of warnings/checks, let me know
> if there are additional warnings/checks you can think of.

Python version compatibilities would be an interesting area to
explore.

I would like PyChecker to use a fundamentally different way of
scanning modules.  I believe that it currently imports the module and
then scans the module's __dict__ for classes and functions, and
analyses their bytecode.  That's fragile, dangerous (if you don't know
what a module does you may not want to import it), and sometimes
causes spurious errors, like when I have this at the global level:

    import foo
    foo.bar()

and no other references to the module foo anywhere in the code...  You
get an error about an unused foo module.

> Thanks for your input,
> Neal
> 
> PS I sent a patch to python-dev against 2.2a1, but it's 45k.  So
> it's awating moderator approval.  Sorry if this isn't the right
> venue for patches.

I've approved it, but in general, we'd all prefer it if you submitted
patches to the SourceForge patch manager -- that's what it's there
for.  If you want to discuss it here, post a URL. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Aug  9 22:20:20 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 09 Aug 2001 17:20:20 -0400
Subject: [Python-Dev] 2.2a1 patch from PyChecker
In-Reply-To: Your message of "Thu, 09 Aug 2001 15:33:52 EDT."
 <3B72E5A0.DC2E8C4E@metaslash.com>
References: <3B72E5A0.DC2E8C4E@metaslash.com>
Message-ID: <200108092120.f79LKNo13875@odiug.digicool.com>

> Attached is a patch to address some problems found by PyChecker.
> Mostly unused imports/locals, etc.

Thanks!

> There were a few methods that were removed because they were duplicated
> (ConfigParser.py:has_option, gzip.py:writelines, locale.py:set_locale).
> I deleted the first occurrence, but I'm not sure that was always right.

It'll take us some time to review all the patches.

> I have changed many unused locals to _ (e.g., for _ in range(3):).
> This gets PyChecker to shut up about being unused.  It also defaults
> to ignoring unused local variables if their name is 'empty' or 'unused'.

Argh!  I don't like using '_' for this, because '_' is already a
special built-in in interactive mode.  I don't even like the fact that
PyChecker warns about unused loop control variables at all -- I see
nothing wrong with code like

    for i in range(10): print "X"

and I would rather not have to write it differently specifically to
shut up PyChecker.  (In general, I find it annoying when a tool like
this requires one to change one's perfectly fine coding style because
of a blind spot in the tool. :-)

I'd be happy if there was an option "complain if a loop control
variable is not used" but it was off by default.

I

> There are many more unused warnings.  Does anybody want me to change
> the variables to empty, unused, _, or something different?  

No.

> Should the default unused names be expanded?

I tend to use "dummy" when I need a dummy variable.  But even in tuple
unpacks, I prefer to give all variables meaningful names even if I
don't use all.  To me, this:

    firstname, lastname, address, phone = get_employee()

is more readable than this:

    firstname, lastname, dummy, dummy = get_employee()

> Note:  I could not run under Python 2.2a1 because dir([]) does not
> return the methods.

But you can treat it as an instance, and get the methods by looking at
[].__class__.__dict__.  Read PEP 252 -- it explains how the
introspection interface will be rationalized.

> Here are some other warnings I didn't fix because I wasn't sure what
> should be done:
> 
> audiodev.py:214: No global (BUFFERSIZE) found
> 	(this is old, should BUFFERSIZE be defined, a better exception
> 	 raised, and/or deprecation warning?)

I still haven't figured that one out myself. :-(

> cgi.py:820: Overriden method (values) doesn't match signature in
> class (cgi.FormContentDict)

I'm afraid that one will have to remain -- it exists for b/w
compatibility (with 6 year old code :-).

> ihooks.py:491: Overriden method (reload) doesn't match signature in
> class (ihooks.BasicModuleImporter)

This is intentional.

> imaplib.py:1026: No global (j) found

Huh?  There's no 'j' on that line.

> ntpath.py:411: No global (WindowsError) found
> 	(I ran on unix, not sure if this is ok on windows.)

Yes, this exception is unique to Windows.

> pdb.py:138: Overriden method (default) doesn't match signature in
> class (cmd.Cmd)

This one I don't understand.  BTW, it would be nice if you could also
mention the file and line number of the other reference.

> profile.py:179: No global (operator) found

That looks like a bug in profile.py to me: it is clearly hoping
for "import operator". :-)

But in general: thanks for PyCecker, Neil, and thanks for the patch
set!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@zope.com  Thu Aug  9 22:53:15 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 9 Aug 2001 17:53:15 -0400 (EDT)
Subject: [Python-Dev] Re: optimizing non-local object access
In-Reply-To: <200108092012.f79KCdv00872@mira.informatik.hu-berlin.de>
References: <200108092012.f79KCdv00872@mira.informatik.hu-berlin.de>
Message-ID: <15219.1611.162859.963171@slothrop.digicool.com>

It would because __dict__ wouldn't be a dictionary; it would be a
dlict (dictionary-list hybrid).  This is the one compatibility issue
that has come up.  Code that expects module's to have a real
dictionary will break -- both in Python where it is probably quite
rare and at the C level where some API calls require real dicts.  The
__dict__ object will implement the mapping protocol.  It just won't be
a dict.

Jeremy



From jeremy@zope.com  Thu Aug  9 23:00:11 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 9 Aug 2001 18:00:11 -0400 (EDT)
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: <200108092052.f79KqQg13802@odiug.digicool.com>
References: <3B72EF79.7BE8F6C1@metaslash.com>
 <200108092052.f79KqQg13802@odiug.digicool.com>
Message-ID: <15219.2027.131396.703309@slothrop.digicool.com>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

  >> Specifically, where should it go?  What should it include or not
  >> include?  What should the default warnings be?

  GvR> You'll never satisfy everyone.  Personally, I write too much
  GvR> code without docstrings, so I find the complaints about missing
  GvR> docstrings annoying.  I guess my preference would be to have
  GvR> the default mode ony complain about things that have a high
  GvR> probability of being *coding* errors -- undefined names and the
  GvR> like.  I forget if you're checking string formatting
  GvR> conformance yet -- that would also be useful.

It would be much more helpful if the default setting identified real
bugs rather than coding style issues.  The doc string one is probably
the most glaring.  Add a -Wall option that turns on everything to
satisfy the pedants.

By way of example, I used pychecker on urllib2 yesterday.  It reported
78 bugs of which 3 were real.  Based on that feedback, I wouldn't use
the tool again, but would let other people sift through the noise and
report the bugs back to me.

Jeremy
 



From guido@python.org  Thu Aug  9 23:03:59 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 09 Aug 2001 18:03:59 -0400
Subject: [Python-Dev] Re: optimizing non-local object access
In-Reply-To: Your message of "Thu, 09 Aug 2001 17:53:15 EDT."
 <15219.1611.162859.963171@slothrop.digicool.com>
References: <200108092012.f79KCdv00872@mira.informatik.hu-berlin.de>
 <15219.1611.162859.963171@slothrop.digicool.com>
Message-ID: <200108092204.f79M41S14253@odiug.digicool.com>

[Jeremy]
> It would because __dict__ wouldn't be a dictionary; it would be a
> dlict (dictionary-list hybrid).  This is the one compatibility issue
> that has come up.  Code that expects module's to have a real
> dictionary will break -- both in Python where it is probably quite
> rare and at the C level where some API calls require real dicts.  The
> __dict__ object will implement the mapping protocol.  It just won't be
> a dict.

In 2.2, if you don't use __getitem__ for access-by-number, you could
subclass dictionary and add a separate set of APIs to access the
variables by number (and to assign numbers to them in the first
place).  Then it would still behave like a real dictionary enough to
be usable for the PyDict_* C API functions.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From akuchlin@mems-exchange.org  Thu Aug  9 23:09:03 2001
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Thu, 9 Aug 2001 18:09:03 -0400
Subject: [Python-Dev] 2.2a1 patch from PyChecker
In-Reply-To: <200108092120.f79LKNo13875@odiug.digicool.com>; from guido@python.org on Thu, Aug 09, 2001 at 05:20:20PM -0400
References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108092120.f79LKNo13875@odiug.digicool.com>
Message-ID: <20010809180903.A31343@ute.cnri.reston.va.us>

On Thu, Aug 09, 2001 at 05:20:20PM -0400, Guido van Rossum wrote:
>> cgi.py:820: Overriden method (values) doesn't match signature in
>> class (cgi.FormContentDict)
>
>I'm afraid that one will have to remain -- it exists for b/w
>compatibility (with 6 year old code :-).

That reminds me: should we put warnings in cgi.py for instantiations
of FormContent and other old classes, so we can delete them in Python
2.3?  One problem: if a class doesn't have an __init__ method,
subclasses that have their own __init__ methods won't call it.  Maybe
this could be done by adding 'self.warn()' to all the methods of an
outdated class, and defining a warn() method that raises a warning.

>> imaplib.py:1026: No global (j) found
>Huh?  There's no 'j' on that line.

Bug in the 2.1 version; the CVS code has been changed.

--amk


From jeremy@zope.com  Thu Aug  9 23:13:51 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 9 Aug 2001 18:13:51 -0400 (EDT)
Subject: [Python-Dev] Re: optimizing non-local object access
In-Reply-To: <200108092204.f79M41S14253@odiug.digicool.com>
References: <200108092012.f79KCdv00872@mira.informatik.hu-berlin.de>
 <15219.1611.162859.963171@slothrop.digicool.com>
 <200108092204.f79M41S14253@odiug.digicool.com>
Message-ID: <15219.2847.865125.543986@slothrop.digicool.com>

Have I mentioned that I love this type-class unification work :-)

Jeremy



From mwh@python.net  Thu Aug  9 23:36:15 2001
From: mwh@python.net (Michael Hudson)
Date: 09 Aug 2001 18:36:15 -0400
Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12
In-Reply-To: Guido van Rossum's message of "Wed, 08 Aug 2001 11:31:29 -0400"
References: <200107271949.PAA27171@cj20424-a.reston1.va.home.com> <2mn15pwg56.fsf@starship.python.net> <200107281357.JAA30859@cj20424-a.reston1.va.home.com> <2m3d7emged.fsf@starship.python.net> <2mvgjy29tq.fsf@starship.python.net> <200108081531.LAA00619@cj20424-a.reston1.va.home.com>
Message-ID: <2mg0b0uahs.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> Hi Michael,
> 
> Barry's on vacation, so I've done the honors.  It's checked in as PEP
> 264.

Cool, thanks.

> I like this a lot -- I hope we can add this to 2.2a2 (which was due
> today but is postponed for at least a week).

Well, I'm off sunseeking in a week; I'd hope it can get it done by then.

> I think the integral flags to compile() are fine,

Noted.

> and 'or' is indeed the right thing.

Hmm.  While I agree that 'or'ing is the best thing to do by default,
it would be nice if there was some way of requesting a blank starting
point (this is the other unresolved issue in PEP 236, after all).
Maybe it's not worth it.

> After reading the PEP, I understand the refactoring that I
> complained about in a comment on the patch.

OK.

> Good job!

It's not done yet (and I thought this was a dead simple change!).

I've uploaded a new patch, which takes Tim's suggestion of using the
_Feature objects and runs with it.  I'd appreciate it if you and Tim
could cast an eye over it - if you think the apprach is sound then
I'll update my PEP and draft some changes to 236.

Cheers,
M.

-- 
  Presumably pronging in the wrong place zogs it.
                                        -- Aldabra Stoddart, ucam.chat


From fdrake@acm.org  Thu Aug  9 23:58:07 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 9 Aug 2001 18:58:07 -0400 (EDT)
Subject: [Python-Dev] Re: optimizing non-local object access
In-Reply-To: <15219.1611.162859.963171@slothrop.digicool.com>
References: <200108092012.f79KCdv00872@mira.informatik.hu-berlin.de>
 <15219.1611.162859.963171@slothrop.digicool.com>
Message-ID: <15219.5503.655419.896502@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > that has come up.  Code that expects module's to have a real
 > dictionary will break -- both in Python where it is probably quite
 > rare and at the C level where some API calls require real dicts.  The
 > __dict__ object will implement the mapping protocol.  It just won't be
 > a dict.

  This would certainly break many of the module initialization
functions.  Sounds like everyone should consider using PyModule_Add*()
there.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From guido@python.org  Fri Aug 10 01:34:56 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 09 Aug 2001 20:34:56 -0400
Subject: [Python-Dev] 2.2a1 patch from PyChecker
In-Reply-To: Your message of "Thu, 09 Aug 2001 18:09:03 EDT."
 <20010809180903.A31343@ute.cnri.reston.va.us>
References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108092120.f79LKNo13875@odiug.digicool.com>
 <20010809180903.A31343@ute.cnri.reston.va.us>
Message-ID: <200108100034.UAA15749@cj20424-a.reston1.va.home.com>

> >> cgi.py:820: Overriden method (values) doesn't match signature in
> >> class (cgi.FormContentDict)
> >
> >I'm afraid that one will have to remain -- it exists for b/w
> >compatibility (with 6 year old code :-).
> 
> That reminds me: should we put warnings in cgi.py for instantiations
> of FormContent and other old classes, so we can delete them in Python
> 2.3?

Sure, but if I believe what folks on c.l.py say about the habits of
the average CGI coder, they'll probably have to remain until python
3.0 -- few will bother to check the warnings. :-) / :-(

> One problem: if a class doesn't have an __init__ method,
> subclasses that have their own __init__ methods won't call it.  Maybe
> this could be done by adding 'self.warn()' to all the methods of an
> outdated class, and defining a warn() method that raises a warning.

Good idea!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Aug 10 01:38:28 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 09 Aug 2001 20:38:28 -0400
Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12
In-Reply-To: Your message of "09 Aug 2001 18:36:15 EDT."
 <2mg0b0uahs.fsf@starship.python.net>
References: <200107271949.PAA27171@cj20424-a.reston1.va.home.com> <2mn15pwg56.fsf@starship.python.net> <200107281357.JAA30859@cj20424-a.reston1.va.home.com> <2m3d7emged.fsf@starship.python.net> <2mvgjy29tq.fsf@starship.python.net> <200108081531.LAA00619@cj20424-a.reston1.va.home.com>
 <2mg0b0uahs.fsf@starship.python.net>
Message-ID: <200108100038.UAA15791@cj20424-a.reston1.va.home.com>

> I've uploaded a new patch, which takes Tim's suggestion of using the
> _Feature objects and runs with it.  I'd appreciate it if you and Tim
> could cast an eye over it - if you think the apprach is sound then
> I'll update my PEP and draft some changes to 236.

I hope Tim can do this -- I need to take a break from chores.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@zope.com  Fri Aug 10 02:47:28 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 9 Aug 2001 21:47:28 -0400
Subject: [Python-Dev] Free threading (and tcl upvar)
References: <Pine.LNX.4.21.0108090902050.890-100000@serenade.digicool.com>
 <200108091522.f79FMja02243@odiug.digicool.com>
Message-ID: <15219.15664.450890.783449@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> That's why we have sys._getframe(). ;-)

btw, when you need it, sys._getframe() is a godsend.  thank the genius
who came up with that one <wink>.

i've been using it in mailman in my _() wrapper (for marking
translatable strings), and because you have to use %()s named
interpolation instead of %s positional interpolation, using
_getframe() to "get at" the locals one frame up makes the call site
very clean and compact, with (imo) no loss of clarity or feeling of
"too much magic".

i'm planning on writing a paper for ipcx on this stuff.

-barry


From jeremy@zope.com  Fri Aug 10 03:04:11 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 9 Aug 2001 22:04:11 -0400 (EDT)
Subject: [Python-Dev] Free threading (and tcl upvar)
In-Reply-To: <15219.15664.450890.783449@anthem.wooz.org>
References: <Pine.LNX.4.21.0108090902050.890-100000@serenade.digicool.com>
 <200108091522.f79FMja02243@odiug.digicool.com>
 <15219.15664.450890.783449@anthem.wooz.org>
Message-ID: <15219.16667.375672.59499@slothrop.digicool.com>

>>>>> "BAW" == Barry A Warsaw <barry@zope.com> writes:
>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

  GvR> That's why we have sys._getframe(). ;-)

  BAW> btw, when you need it, sys._getframe() is a godsend.  thank the
  BAW> genius who came up with that one <wink>.

It takes all kinds, doesn't it?  For every genius who comes up with
something like sys._getframe(), we have some fool who comes up with
something like "print >>".

Jeremy



From martin@loewis.home.cs.tu-berlin.de  Fri Aug 10 07:00:46 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Fri, 10 Aug 2001 08:00:46 +0200
Subject: [Python-Dev] optimizing non-local object access
Message-ID: <200108100600.f7A60kD01107@mira.informatik.hu-berlin.de>

[Martin]
> Would that catch changes like [accessing __dict__]

[Jeremy]
> It would because __dict__ wouldn't be a dictionary

[Guido]

> In 2.2, if you don't use __getitem__ for access-by-number, you could
> subclass dictionary and add a separate set of APIs to access the
> variables by number (and to assign numbers to them in the first
> place).  Then it would still behave like a real dictionary enough to
> be usable for the PyDict_* C API functions.

But then, changes that happen through PyDict_SetItem would not be
tracked for the dlict objects, would they (assuming that the dlict
objects keep an additional array per dictionary which needs to be
updated)?

Regards,
Martin


From tim.one@home.com  Fri Aug 10 07:10:25 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 10 Aug 2001 02:10:25 -0400
Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12
In-Reply-To: <2mg0b0uahs.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOENELEAA.tim.one@home.com>

[Michael Hudson]
> Hmm.  While I agree that 'or'ing is the best thing to do by default,
> it would be nice if there was some way of requesting a blank starting
> point (this is the other unresolved issue in PEP 236, after all).
> Maybe it's not worth it.

Well, there's no reason at all to believe that whatever future statements
IDLE and doctest (for examples) happen to use in their own implementations
are also appropriate for the user-code they're simulating.  So the problem
isn't solved in full unless that connection can be broken (is that hard?
offhand it *sounds* like it just needs another yes/no argument).

OTOH, IDLE and doctest (for examples) can easily enough be written to use no
future-stmts at all of their own, so that code compiled from them gets a
blank starting point.  Whether that remains easy down the road depends on
how silly we get in introducing stupid future stmts <0.9 wink>.



From tim.one@home.com  Fri Aug 10 07:34:04 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 10 Aug 2001 02:34:04 -0400
Subject: [Python-Dev] BEGIN_ALLOW_THREADS
In-Reply-To: <20010809082856.A16148@strakt.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMENFLEAA.tim.one@home.com>

[Guido]
> Instead of asking us to add more macros, I recommend that you use the
> lower-level C API that is used by the macros.  There's a bunch of ways
> to save and restore the state.  The macros are just shortcuts (and a
> handy way to make all the code disappear when threads don't exist).

[Martin Sjögren]
> Well, that's exactly what I'm doing, isn't it?

You are using macros, yes <wink>.  What Guido is telling you is that the set
of macros we already have is adequate for everything we do, so it would just
bloat our code base with stuff we never ever test if we were add additional
stuff we never use.  He's also telling you that you can write your macros in
such a way that you don't have to change them across Python releases, and it
was *that* complaint you seemed keenest about.  That one you can solve on
your own.



From mwh@python.net  Fri Aug 10 09:17:13 2001
From: mwh@python.net (Michael Hudson)
Date: 10 Aug 2001 04:17:13 -0400
Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12
In-Reply-To: "Tim Peters"'s message of "Fri, 10 Aug 2001 02:10:25 -0400"
References: <LNBBLJKPBEHFEDALKOLCOENELEAA.tim.one@home.com>
Message-ID: <2m8zgsfhx2.fsf@starship.python.net>

"Tim Peters" <tim.one@home.com> writes:

> [Michael Hudson]
> > Hmm.  While I agree that 'or'ing is the best thing to do by default,
> > it would be nice if there was some way of requesting a blank starting
> > point (this is the other unresolved issue in PEP 236, after all).
> > Maybe it's not worth it.
> 
> Well, there's no reason at all to believe that whatever future statements
> IDLE and doctest (for examples) happen to use in their own implementations
> are also appropriate for the user-code they're simulating.  So the problem
> isn't solved in full unless that connection can be broken (is that hard?
> offhand it *sounds* like it just needs another yes/no argument).

It's a matter of interface, really.  It's certainly not at all
technically hard.  Maybe:

   compile(text, filename, symbol[, flags[, dont_inherit]])

I worry that saying "you don't inherit behaviour from surrounding code
as soon as you pass a flag" might get really, really confusing at
times.

> OTOH, IDLE and doctest (for examples) can easily enough be written
> to use no future-stmts at all of their own, so that code compiled
> from them gets a blank starting point.  Whether that remains easy
> down the road depends on how silly we get in introducing stupid
> future stmts <0.9 wink>.

This is indeed the issue.

we-need-a-tritfield-ly y'rs
M.

-- 
  Important data should not be entrusted to Pinstripe, as it may
  eat it and make loud belching noises.
   -- from the announcement of the beta of "Pinstripe" aka. Redhat 7.0


From jack@oratrix.nl  Fri Aug 10 10:23:38 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Fri, 10 Aug 2001 11:23:38 +0200
Subject: [Python-Dev] struct initializer in gcmodule.c - Allowed?
Message-ID: <20010810092338.CF134303181@snelboot.oratrix.nl>

A recent change to gcmodule.c (function gc_referents_for()) gave it a 
structure initializer for an auto variable:
    struct referents refs = {objs, 0};

My compiler chokes on this, and my memory of the C standard also says this is 
not allowed. But then, my memory may have been superseded by more recent C 
standards...
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | ++++ see http://www.xs4all.nl/~tank/ ++++




From guido@python.org  Fri Aug 10 14:49:12 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 10 Aug 2001 09:49:12 -0400
Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12
In-Reply-To: Your message of "10 Aug 2001 04:17:13 EDT."
 <2m8zgsfhx2.fsf@starship.python.net>
References: <LNBBLJKPBEHFEDALKOLCOENELEAA.tim.one@home.com>
 <2m8zgsfhx2.fsf@starship.python.net>
Message-ID: <200108101349.JAA16834@cj20424-a.reston1.va.home.com>

> It's a matter of interface, really.  It's certainly not at all
> technically hard.  Maybe:
> 
>    compile(text, filename, symbol[, flags[, dont_inherit]])

Occam sez: let's add the dont_inherit argument when we have found a
real use for it.

My bigger worry about this interface is that the flags accepted should
be carefully checked to be from the small set related to future
statements.  It would be harmful if the user could set flags like
CO_OPTIMIZED, CO_GENERATOR, or CO_VARARGS this way!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From nas@python.ca  Fri Aug 10 15:04:04 2001
From: nas@python.ca (Neil Schemenauer)
Date: Fri, 10 Aug 2001 07:04:04 -0700
Subject: [Python-Dev] struct initializer in gcmodule.c - Allowed?
In-Reply-To: <20010810092338.CF134303181@snelboot.oratrix.nl>; from jack@oratrix.nl on Fri, Aug 10, 2001 at 11:23:38AM +0200
References: <20010810092338.CF134303181@snelboot.oratrix.nl>
Message-ID: <20010810070404.A6727@glacier.fnational.com>

Jack Jansen wrote:
> A recent change to gcmodule.c (function gc_referents_for()) gave it a 
> structure initializer for an auto variable:
>     struct referents refs = {objs, 0};
> 
> My compiler chokes on this, and my memory of the C standard also says this is 
> not allowed. But then, my memory may have been superseded by more recent C 
> standards...

I'm working on fixing this.  That structure is not really needed so I'm
removing it.

  Neil


From mwh@python.net  Fri Aug 10 15:08:50 2001
From: mwh@python.net (Michael Hudson)
Date: 10 Aug 2001 10:08:50 -0400
Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12
In-Reply-To: Guido van Rossum's message of "Fri, 10 Aug 2001 09:49:12 -0400"
References: <LNBBLJKPBEHFEDALKOLCOENELEAA.tim.one@home.com> <2m8zgsfhx2.fsf@starship.python.net> <200108101349.JAA16834@cj20424-a.reston1.va.home.com>
Message-ID: <2mwv4c2ej1.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> > It's a matter of interface, really.  It's certainly not at all
> > technically hard.  Maybe:
> > 
> >    compile(text, filename, symbol[, flags[, dont_inherit]])
> 
> Occam sez: let's add the dont_inherit argument when we have found a
> real use for it.

Fair enough.

> My bigger worry about this interface is that the flags accepted should
> be carefully checked to be from the small set related to future
> statements.  It would be harmful if the user could set flags like
> CO_OPTIMIZED, CO_GENERATOR, or CO_VARARGS this way!

Wrong set of flags!

There are two complementary sets of flags here:

(1) The PyCF_* ones, defined in Include/pythonrun.h
(2) The CO_* ones defined in Include/compile.h

The proposed fourth argument to compile() should be a combination of
the former set.

I only use the latter to tell whether a __future__ statement was used
in the text compiled (which is a bit horrible, but no better way
springs to mind).

There might be value in checking the flags passed to compile() anyway,
but I can't see it being dangerous.

Cheers,
M.

-- 
  You owe The Oracle a TV with an 'intelligence' control - I've 
  tried 'brightness' but that didn't work.
                                      -- Internet Oracularity #1192-01


From guido@python.org  Fri Aug 10 15:19:58 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 10 Aug 2001 10:19:58 -0400
Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12
In-Reply-To: Your message of "10 Aug 2001 10:08:50 EDT."
 <2mwv4c2ej1.fsf@starship.python.net>
References: <LNBBLJKPBEHFEDALKOLCOENELEAA.tim.one@home.com> <2m8zgsfhx2.fsf@starship.python.net> <200108101349.JAA16834@cj20424-a.reston1.va.home.com>
 <2mwv4c2ej1.fsf@starship.python.net>
Message-ID: <200108101419.KAA17114@cj20424-a.reston1.va.home.com>

> > My bigger worry about this interface is that the flags accepted should
> > be carefully checked to be from the small set related to future
> > statements.  It would be harmful if the user could set flags like
> > CO_OPTIMIZED, CO_GENERATOR, or CO_VARARGS this way!
> 
> Wrong set of flags!
> 
> There are two complementary sets of flags here:
> 
> (1) The PyCF_* ones, defined in Include/pythonrun.h
> (2) The CO_* ones defined in Include/compile.h
> 
> The proposed fourth argument to compile() should be a combination of
> the former set.
> 
> I only use the latter to tell whether a __future__ statement was used
> in the text compiled (which is a bit horrible, but no better way
> springs to mind).
> 
> There might be value in checking the flags passed to compile() anyway,
> but I can't see it being dangerous.

Ah, good.  Makes sense.  (I think I may have been confused by your
original patch, which AFAICR *did* manipulate the co_flags field of
the code object, which contains the CO_* flags.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Fri Aug 10 16:05:08 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 10 Aug 2001 11:05:08 -0400
Subject: [Python-Dev] Direction of PyChecker
References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com>
Message-ID: <3B73F824.82A53539@metaslash.com>

Guido van Rossum wrote:

> You'll never satisfy everyone.  Personally, I write too much code
> without docstrings, so I find the complaints about missing docstrings
> annoying.  I guess my preference would be to have the default mode ony
> complain about things that have a high probability of being *coding*
> errors -- undefined names and the like.  I forget if you're checking
> string formatting conformance yet -- that would also be useful.

I know I can't satisfy everyone.  I'm not even sure who PyChecker
should be geared to by default--newbie, intermediate, or advanced user.
A bunch of the warnings annoy me too. :-)

ISTM that it probably should be geared to newbies by default.  Since,
they are the ones that would likely benefit the most.  It's also
easier for more advanced users to change defaults.  I don't really
want to do turn on all the warnings, even though I've tended towards that.
I'm trying to figure out what would be best.

There is a single option -e/--errors that will disable all the doc string,
style, and code complexity warnings.  I think there's about 40 config
options.  So it can be configured, but would take a little work.

There is checks for format strings.  It should be pretty good now.

> Unfortunately the typical newbie doesn't have access to pychecker
> yet...

True, but that can change. :-)

> Sure!  You can add a FAQ entry yourself -- the password is "Spam".

Added 4.101.  Pretty short, let me know if you have comments.

> Python version compatibilities would be an interesting area to
> explore.

Just starting to do that.  There is a check for integer division now.
I don't really know of version incompatibilities, other than new features
such as methods on strings, list comprehension, etc.

Right now string methods are checked based on the interpreter running.
So if you run PyChecker in 1.5.2 and do ''.strip(), it should complain.
If you run PyChecker in 2.0+ and do ''.strip(), it should NOT complain.

> I would like PyChecker to use a fundamentally different way of
> scanning modules.  I believe that it currently imports the module and
> then scans the module's __dict__ for classes and functions, and
> analyses their bytecode.  That's fragile, dangerous (if you don't know
> what a module does you may not want to import it), and sometimes
> causes spurious errors, like when I have this at the global level:

Yes, this is one of the worst problems I know of.  I plan on changing this.

Neal


From Samuele Pedroni <pedroni@inf.ethz.ch>  Fri Aug 10 16:16:07 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Fri, 10 Aug 2001 17:16:07 +0200 (MET DST)
Subject: [Python-Dev] Direction of PyChecker
Message-ID: <200108101516.RAA15086@core.inf.ethz.ch>

Hi.

[Neal Norwitz]
> 
> > I would like PyChecker to use a fundamentally different way of
> > scanning modules.  I believe that it currently imports the module and
> > then scans the module's __dict__ for classes and functions, and
> > analyses their bytecode.  That's fragile, dangerous (if you don't know
> > what a module does you may not want to import it), and sometimes
> > causes spurious errors, like when I have this at the global level:
> 
> Yes, this is one of the worst problems I know of.  I plan on changing this.
> 
Let me know what are your plans about that, will you stick with some bytecode
based approach?

I'm consindering the possibility of adding a special mode to jython (based on
hooks and on porting the CPython tools/compiler) in which you can retrieve
the PVM bytecode of code objects, then there would be still to make PyChecker
java classes aware, but it could run under jython. (Don't know when I will
get at doing this)

It seems a nice tool, I would like the jython users to be able to use it too.

regards, Samuele Pedroni.



From guido@python.org  Fri Aug 10 16:15:14 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 10 Aug 2001 11:15:14 -0400
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: Your message of "Fri, 10 Aug 2001 11:05:08 EDT."
 <3B73F824.82A53539@metaslash.com>
References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com>
 <3B73F824.82A53539@metaslash.com>
Message-ID: <200108101515.LAA18907@cj20424-a.reston1.va.home.com>

> > I would like PyChecker to use a fundamentally different way of
> > scanning modules.  I believe that it currently imports the module and
> > then scans the module's __dict__ for classes and functions, and
> > analyses their bytecode.  That's fragile, dangerous (if you don't know
> > what a module does you may not want to import it), and sometimes
> > causes spurious errors, like when I have this at the global level:
> 
> Yes, this is one of the worst problems I know of.  I plan on changing this.

If you don't want to give up scanning the bytecode (if that's what you
do), you could use compile() on the source code to get the bytecode
for the entire module.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Fri Aug 10 16:17:56 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 10 Aug 2001 11:17:56 -0400
Subject: [Python-Dev] 2.2a1 patch from PyChecker
References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108092120.f79LKNo13875@odiug.digicool.com>
Message-ID: <3B73FB24.369B5EFD@metaslash.com>

Guido van Rossum wrote:

> > I have changed many unused locals to _ (e.g., for _ in range(3):).
> > This gets PyChecker to shut up about being unused.  It also defaults
> > to ignoring unused local variables if their name is 'empty' or 'unused'.
> 
> Argh!  I don't like using '_' for this, because '_' is already a
> special built-in in interactive mode.  I don't even like the fact that
> PyChecker warns about unused loop control variables at all -- I see
> nothing wrong with code like
> 
>     for i in range(10): print "X"
> 
> and I would rather not have to write it differently specifically to
> shut up PyChecker.  (In general, I find it annoying when a tool like
> this requires one to change one's perfectly fine coding style because
> of a blind spot in the tool. :-)

Agreed.  I try to warn about things that people may care about.

> I'd be happy if there was an option "complain if a loop control
> variable is not used" but it was off by default.

I looked at adding this check, but decided against it initially.
I'll think about adding it.

> > Should the default unused names be expanded?
> 
> I tend to use "dummy" when I need a dummy variable.  But even in tuple

I added 'dummy' as another variable name to ignore if unused.

> unpacks, I prefer to give all variables meaningful names even if I
> don't use all.  To me, this:
> 
>     firstname, lastname, address, phone = get_employee()
> 
> is more readable than this:
> 
>     firstname, lastname, dummy, dummy = get_employee()

I agree, in fact, the first line with real names, does not warn by default.
There is an option to warn if address and phone are unused, -t/--tuple,
but it must be manually enabled.

> > Note:  I could not run under Python 2.2a1 because dir([]) does not
> > return the methods.
> 
> But you can treat it as an instance, and get the methods by looking at
> [].__class__.__dict__.  Read PEP 252 -- it explains how the
> introspection interface will be rationalized.

Ok, I'll take a look.

> > imaplib.py:1026: No global (j) found
> 
> Huh?  There's no 'j' on that line.

I have:

 1025: l = map(lambda x:'%s: "%s"' % (x[0], x[1][0] and '" "'.join(x[1]) or ''
), l)
 1026: _mesg('untagged responses dump:%s%s' % (t, j(l, t)))
                                                  ^^^^^^^

> > pdb.py:138: Overriden method (default) doesn't match signature in
> > class (cmd.Cmd)
> 
> This one I don't understand.  BTW, it would be nice if you could also
> mention the file and line number of the other reference.

Good idea.

> But in general: thanks for PyCecker, Neil, and thanks for the patch
> set!

Thanks for the feedback and feel free to make suggestions.

PyChecker is still an experiment at this point, but it seems
that many people have found it useful.  My goal is to make it
as useful as possible and help promote the use of python
(or eliminate bogus complaints about python, like type checking, etc).

Neal


From guido@python.org  Fri Aug 10 16:19:15 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 10 Aug 2001 11:19:15 -0400
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: Your message of "Fri, 10 Aug 2001 17:16:07 +0200."
 <200108101516.RAA15086@core.inf.ethz.ch>
References: <200108101516.RAA15086@core.inf.ethz.ch>
Message-ID: <200108101519.LAA18940@cj20424-a.reston1.va.home.com>

> I'm consindering the possibility of adding a special mode to jython
> (based on hooks and on porting the CPython tools/compiler) in which
> you can retrieve the PVM bytecode of code objects, then there would
> be still to make PyChecker java classes aware, but it could run
> under jython. (Don't know when I will get at doing this)

But is it really a good idea to codify PVM bytecode as the way to do
introspection?  I'd much rather have a standard library API that
parses a module and gives you a list of variables, functions, classes,
etc., and lets you drill down into the symbol table of each of those.
(I realize more is needed, you need to be able to inspect every
statement too -- that can be done by providing access to an AST.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From cgw@alum.mit.edu  Fri Aug 10 16:21:58 2001
From: cgw@alum.mit.edu (Charles G Waldman)
Date: Fri, 10 Aug 2001 10:21:58 -0500
Subject: [Python-Dev] ++x oddnes
Message-ID: <15219.64534.411625.193775@nyx.dyndns.org>

Sorry if this is old news.  I don't remember anybody reporting this,
but I don't follow c.l.py as closely as I used to.

I've been writing a lot of C++ lately and my mind is stuck in limbo
somewhere between C++ and Python.  I inadvertently wrote the following:

x = 0
while x < 10:
      # do something
      ++x


Interestingly, this runs, it just never terminates!  Looks like
somehow ++x is being parsed as "+0 + x".  Shouldn't this construct
be flagged as a syntax error?  

This happens with both 2.1 and 2.2


From guido@python.org  Fri Aug 10 16:23:37 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 10 Aug 2001 11:23:37 -0400
Subject: [Python-Dev] ++x oddnes
In-Reply-To: Your message of "Fri, 10 Aug 2001 10:21:58 CDT."
 <15219.64534.411625.193775@nyx.dyndns.org>
References: <15219.64534.411625.193775@nyx.dyndns.org>
Message-ID: <200108101523.LAA19012@cj20424-a.reston1.va.home.com>

> Sorry if this is old news.  I don't remember anybody reporting this,
> but I don't follow c.l.py as closely as I used to.

It's new to me. :-)

> I've been writing a lot of C++ lately and my mind is stuck in limbo
> somewhere between C++ and Python.  I inadvertently wrote the following:
> 
> x = 0
> while x < 10:
>       # do something
>       ++x
> 
> Interestingly, this runs, it just never terminates!  Looks like
> somehow ++x is being parsed as "+0 + x".  Shouldn't this construct
> be flagged as a syntax error?  
> 
> This happens with both 2.1 and 2.2

IMO not worth it changing the language.  Train yourself to write x++
instead. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Fri Aug 10 16:38:48 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 10 Aug 2001 11:38:48 -0400
Subject: [Python-Dev] Direction of PyChecker
References: <3B72EF79.7BE8F6C1@metaslash.com>
 <200108092052.f79KqQg13802@odiug.digicool.com> <15219.2027.131396.703309@slothrop.digicool.com>
Message-ID: <3B740008.2CF200C2@metaslash.com>

Jeremy Hylton wrote:

> It would be much more helpful if the default setting identified real
> bugs rather than coding style issues.  The doc string one is probably
> the most glaring.  Add a -Wall option that turns on everything to
> satisfy the pedants.

This is generally true for more advanced users, but not necessarily
for newbies.  There is the -e/--errors which eliminates many warnings.

> By way of example, I used pychecker on urllib2 yesterday.  It reported
> 78 bugs of which 3 were real.  Based on that feedback, I wouldn't use
> the tool again, but would let other people sift through the noise and
> report the bugs back to me.

I ran checker w/--errors on urllib2.  There were some warnings which
should have been excluded--I'll have to fix those.  But of the 12 warnings
produced there seems to be one real bug, which you may have caught:

	urllib2.py:1054: No global (OpenerDirectory) found

I didn't investigate the other warnings closely.

It's not always easy to know if there should really be a warning or not.
For example, an unused variable could be a real bug, even though often
it is not.  No class attribute warnings are likely bugs, but could depend 
a lot on how one codes.

I don't know if there is a good middle ground.  That's part of the
reason that you can specify a .pycheckrc file, rather than keep entering
command line arguments.  Also, there should probably be some more
general classes of command line args, like -e/--errors.  Suggestions?

Neal


From neal@metaslash.com  Fri Aug 10 16:43:52 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 10 Aug 2001 11:43:52 -0400
Subject: [Python-Dev] Direction of PyChecker
References: <200108101516.RAA15086@core.inf.ethz.ch> <200108101519.LAA18940@cj20424-a.reston1.va.home.com>
Message-ID: <3B740138.98E75C5F@metaslash.com>

Guido van Rossum wrote:
> 
> [Samuele Pedroni]
> > I'm consindering the possibility of adding a special mode to jython
> > (based on hooks and on porting the CPython tools/compiler) in which
> > you can retrieve the PVM bytecode of code objects, then there would
> > be still to make PyChecker java classes aware, but it could run
> > under jython. (Don't know when I will get at doing this)
> 
> But is it really a good idea to codify PVM bytecode as the way to do
> introspection?  I'd much rather have a standard library API that
> parses a module and gives you a list of variables, functions, classes,
> etc., and lets you drill down into the symbol table of each of those.
> (I realize more is needed, you need to be able to inspect every
> statement too -- that can be done by providing access to an AST.)

While it would make my life easier to modify jython, I tend to agree
with Guido.  Ideally, checker would use ASTs.  I really want checker
to work with jython, but it's too much work to do right now.

The reason why I chose to use the PVM bytecode was because it was
faster and easier to get something.  I started trying to use ASTs
but it would have taken a lot longer to get anywhere.

I would like to support pychecker under jython, just not sure of
the best or fastest way to get there.

Neal


From thomas@xs4all.net  Fri Aug 10 16:54:14 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Fri, 10 Aug 2001 17:54:14 +0200
Subject: [Python-Dev] ++x oddnes
In-Reply-To: <15219.64534.411625.193775@nyx.dyndns.org>
References: <15219.64534.411625.193775@nyx.dyndns.org>
Message-ID: <20010810175414.E874@xs4all.nl>

On Fri, Aug 10, 2001 at 10:21:58AM -0500, Charles G Waldman wrote:

>       ++x

> Interestingly, this runs, it just never terminates!  Looks like
> somehow ++x is being parsed as "+0 + x".

Actually, it's being parsed as '+(+x)':

          6 LOAD_FAST                0 (x)
          9 UNARY_POSITIVE      
         10 UNARY_POSITIVE      

The same goes for '--x'. If it were parsed as '+0 + x', 'x++' would also be
parsed as 'x + +0'.

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From arigo@ulb.ac.be  Fri Aug 10 16:54:57 2001
From: arigo@ulb.ac.be (Armin Rigo)
Date: Fri, 10 Aug 2001 17:54:57 +0200 (CEST)
Subject: [Python-Dev] Psyco update
Message-ID: <Pine.LNX.4.33L2.0108101745000.5771-100000@magma.unil.ch>

Hello everybody,

Just a quick update on the Python Specializing Compiler.

I started to work on a quite different prototype one month ago. This is
all very experimental, so I will delay the technical explanations for some
more time. The motivations are as follows: the previous implementation
loosed WAY too much time in compilation proper: around 0.2 seconds for a
10-15 lines function! I am not sure that writing Psyco in very optimized C
would have been enough to be of much help in anything but the most
particular cases.

Because of this I am re-thinking the way it could work. While I did not
abandon the base ideas, I switched to a slightly different approach which,
I think, could produce slightly less efficient machine code but incredibly
more quickly. This work is based on research articles describing
techniques to produce code at run-time [XXX references]. With the
execution of only 4 to 6 processor instructions, they output one
instruction of the dynamic code. Compare it with a typical C compiler,
which takes thousands times more clock cycles to run than the number of
instructions it emits! Or course, I do not question that it outputs more
optimized code than what can be done with 4 to 6 instructions; however,
this has the great advantage that even if some dynamically produced code
is only used a few times, even only once, the overhead of having produced
it in the first place is very low -- maybe even lower than the time that
would have been required to load it from a disk file if it had been
precompiled, and in the worst cases similar to the current Python VM
interpreter.


A bientot,

Armin.



From nas@python.ca  Fri Aug 10 17:01:11 2001
From: nas@python.ca (Neil Schemenauer)
Date: Fri, 10 Aug 2001 09:01:11 -0700
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: <3B740138.98E75C5F@metaslash.com>; from neal@metaslash.com on Fri, Aug 10, 2001 at 11:43:52AM -0400
References: <200108101516.RAA15086@core.inf.ethz.ch> <200108101519.LAA18940@cj20424-a.reston1.va.home.com> <3B740138.98E75C5F@metaslash.com>
Message-ID: <20010810090111.A7248@glacier.fnational.com>

Neal Norwitz wrote:
> The reason why I chose to use the PVM bytecode was because it was
> faster and easier to get something.  I started trying to use ASTs
> but it would have taken a lot longer to get anywhere.

Couldn't you use Tools/compiler to generate a nice AST?  

  Neil


From DavidA@ActiveState.com  Fri Aug 10 18:01:05 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Fri, 10 Aug 2001 10:01:05 -0700
Subject: [Python-Dev] ++x oddnes
References: <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com>
Message-ID: <3B741351.C7C2E2B8@ActiveState.com>

Guido van Rossum wrote:

> IMO not worth it changing the language.  Train yourself to write x++
> instead. :-)

Sounds like an excellent one for PyChecker =)

-- David Ascher


From jeremy@zope.com  Fri Aug 10 19:09:28 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 10 Aug 2001 14:09:28 -0400 (EDT)
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: <3B740008.2CF200C2@metaslash.com>
References: <3B72EF79.7BE8F6C1@metaslash.com>
 <200108092052.f79KqQg13802@odiug.digicool.com>
 <15219.2027.131396.703309@slothrop.digicool.com>
 <3B740008.2CF200C2@metaslash.com>
Message-ID: <15220.9048.366116.587916@slothrop.digicool.com>

>>>>> "NN" == Neal Norwitz <neal@metaslash.com> writes:

  NN> Jeremy Hylton wrote:
  >> It would be much more helpful if the default setting identified
  >> real bugs rather than coding style issues.  The doc string one is
  >> probably the most glaring.  Add a -Wall option that turns on
  >> everything to satisfy the pedants.

  NN> This is generally true for more advanced users, but not
  NN> necessarily for newbies.  There is the -e/--errors which
  NN> eliminates many warnings.

Do you mean that it is better for newbies if the tool emits many
warnings though they may be spurious?  I would think the reverse: A
newbie is going to have a lot more trouble deciding that the bug
reported by PyChecker is, in fact, not a bug at all.  They haven't
developed the expertise or confidence to disagree with the tool.

In part, it's a question of first impressions.  If the user sees a
very high number of false positives the first time they see the tool,
they're not going to see it as very useful.

  >> By way of example, I used pychecker on urllib2 yesterday.  It
  >> reported 78 bugs of which 3 were real.  Based on that feedback, I
  >> wouldn't use the tool again, but would let other people sift
  >> through the noise and report the bugs back to me.

  NN> I ran checker w/--errors on urllib2.  There were some warnings
  NN> which should have been excluded--I'll have to fix those.  But of
  NN> the 12 warnings produced there seems to be one real bug, which
  NN> you may have caught:

I don't know what I'm doing differently today, but I don't see any of
the warnings about classes without doc strings and the like.  Could
something have changed very recently?  Was I imagining the output I
saw the other day?

  NN> 	urllib2.py:1054: No global (OpenerDirectory) found

Yes.  This was one of the typos it found.

  NN> It's not always easy to know if there should really be a warning
  NN> or not.  For example, an unused variable could be a real bug,
  NN> even though often it is not.  No class attribute warnings are
  NN> likely bugs, but could depend a lot on how one codes.

There seem to be a lot of spurious complaints related to class
hierarchies.  I see a bunch of complaints that a sublcass doesn't have
an __init__() -- but subclasses don't need to have an __init__ and
often shouldn't.  There are also complaints about attribute used by an
abstract base class but only defined in the subclass.

Jeremy



From Samuele Pedroni <pedroni@inf.ethz.ch>  Fri Aug 10 19:21:19 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Fri, 10 Aug 2001 20:21:19 +0200 (MET DST)
Subject: [Python-Dev] Direction of PyChecker
Message-ID: <200108101821.UAA28722@core.inf.ethz.ch>


[GvR]
> But is it really a good idea to codify PVM bytecode as the way to do
> introspection? 
No (but I don't know - for example - what is the tradition under Smalltalk impls)
but it seems that people like to do it that way (inspect also uses bytecode in some
places but maybe there is OK, because is starting from low-level to give an high-level
API). Doesn't pydoc also use imports? 

On the other hand the new nested scope rules don't make life easy for someone using
ASTs.  At least Jeremy and I should come out with a really interoperable symtable module.
At the moment such a beast does not exist under jython and the CPython part is under-specified.

I imagine that a problem is the very late runtime semantics of python, it is very clear,
but then just executing a module you are sure of what are the most likely values of some
stuff.

> I'd much rather have a standard library API that
> parses a module and gives you a list of variables, functions, classes,
> etc., and lets you drill down into the symbol table of each of those.
> (I realize more is needed, you need to be able to inspect every
> statement too -- that can be done by providing access to an AST.)
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)


regards, Samuele Pedroni.



From neal@metaslash.com  Fri Aug 10 20:15:57 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 10 Aug 2001 15:15:57 -0400
Subject: [Python-Dev] Direction of PyChecker
References: <200108101516.RAA15086@core.inf.ethz.ch> <200108101519.LAA18940@cj20424-a.reston1.va.home.com> <3B740138.98E75C5F@metaslash.com> <20010810090111.A7248@glacier.fnational.com>
Message-ID: <3B7432ED.12A117C1@metaslash.com>

Neil Schemenauer wrote:
> 
> Neal Norwitz wrote:
> > The reason why I chose to use the PVM bytecode was because it was
> > faster and easier to get something.  I started trying to use ASTs
> > but it would have taken a lot longer to get anywhere.
> 
> Couldn't you use Tools/compiler to generate a nice AST?

I probably could.  Recently, I looked at the tool in 2.2.
But I didn't know about the compiler when I started.

I looked at pylint (from 1999) and the ASTs that were generated
from parser, etc.  But it was going to require too much work
at the time.  And I was just trying to figure out if static
analysis was even possible.

I've considered the benefits/drawbacks of converting checker 
to use ASTs.  The drawback is how much time it will take to convert, 
so I haven't done anything....yet.

Neal


From neal@metaslash.com  Fri Aug 10 20:29:45 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 10 Aug 2001 15:29:45 -0400
Subject: [Python-Dev] Direction of PyChecker
References: <3B72EF79.7BE8F6C1@metaslash.com>
 <200108092052.f79KqQg13802@odiug.digicool.com>
 <15219.2027.131396.703309@slothrop.digicool.com>
 <3B740008.2CF200C2@metaslash.com> <15220.9048.366116.587916@slothrop.digicool.com>
Message-ID: <3B743629.E1220DBB@metaslash.com>

Jeremy Hylton wrote:
> 
> >>>>> "NN" == Neal Norwitz <neal@metaslash.com> writes:
> 
>   NN> Jeremy Hylton wrote:
>   >> It would be much more helpful if the default setting identified
>   >> real bugs rather than coding style issues.  The doc string one is
>   >> probably the most glaring.  Add a -Wall option that turns on
>   >> everything to satisfy the pedants.
> 
>   NN> This is generally true for more advanced users, but not
>   NN> necessarily for newbies.  There is the -e/--errors which
>   NN> eliminates many warnings.
> 
> Do you mean that it is better for newbies if the tool emits many
> warnings though they may be spurious?  I would think the reverse: A
> newbie is going to have a lot more trouble deciding that the bug
> reported by PyChecker is, in fact, not a bug at all.  They haven't
> developed the expertise or confidence to disagree with the tool.

You've got a very good point here.  I'm not conviced, but I'm very
close.  So keep working. :-)

I agree that the doc strings should not be warned about by default.
That will be changed.  As I said before, I think that the modules/classes
in the standard library should have them.  And docstrings are definitely
doc, not code.

> In part, it's a question of first impressions.  If the user sees a
> very high number of false positives the first time they see the tool,
> they're not going to see it as very useful.

I agree.

>   >> By way of example, I used pychecker on urllib2 yesterday.  It
>   >> reported 78 bugs of which 3 were real.  Based on that feedback, I
>   >> wouldn't use the tool again, but would let other people sift
>   >> through the noise and report the bugs back to me.
> 
>   NN> I ran checker w/--errors on urllib2.  There were some warnings
>   NN> which should have been excluded--I'll have to fix those.  But of
>   NN> the 12 warnings produced there seems to be one real bug, which
>   NN> you may have caught:
> 
> I don't know what I'm doing differently today, but I don't see any of
> the warnings about classes without doc strings and the like.  Could
> something have changed very recently?  Was I imagining the output I
> saw the other day?

I didn't change it yet! :-)  The default (currently) is to warn about
missing docstrings for modules and classes, not functions or methods.
This will change shortly (missing docstrings will not generate a warning
by default).

I also added some config options, so now the # of warnings on urllib2 dropped
when using --errors.

>   NN> It's not always easy to know if there should really be a warning
>   NN> or not.  For example, an unused variable could be a real bug,
>   NN> even though often it is not.  No class attribute warnings are
>   NN> likely bugs, but could depend a lot on how one codes.
> 
> There seem to be a lot of spurious complaints related to class
> hierarchies.  I see a bunch of complaints that a sublcass doesn't have
> an __init__() -- but subclasses don't need to have an __init__ and
> often shouldn't.  

Lacking an __init__() should be off by default.

> There are also complaints about attribute used by an
> abstract base class but only defined in the subclass.

This is true.  While python doesn't require setting, I think this feature
can be dangerous.  It seems better to init the attr to None. 

In urllib2, there are 6 warnings about 2 different attributes. 
I can provide a config option to turn that off too.

Neal


From guido@python.org  Fri Aug 10 20:45:19 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 10 Aug 2001 15:45:19 -0400
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: Your message of "Fri, 10 Aug 2001 15:29:45 EDT."
 <3B743629.E1220DBB@metaslash.com>
References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> <15219.2027.131396.703309@slothrop.digicool.com> <3B740008.2CF200C2@metaslash.com> <15220.9048.366116.587916@slothrop.digicool.com>
 <3B743629.E1220DBB@metaslash.com>
Message-ID: <200108101945.PAA24012@cj20424-a.reston1.va.home.com>

> As I said before, I think that the modules/classes
> in the standard library should have them.

You can help by submitting some patches (one module at a time please :-).

> And docstrings are definitely
> doc, not code.

Agreed, but I don;'t understand why you said this.

> > There are also complaints about attribute used by an
> > abstract base class but only defined in the subclass.
> 
> This is true.  While python doesn't require setting, I think this feature
> can be dangerous.  It seems better to init the attr to None. 

Agreed.  The initialization to None can have a comment explaining what
the subclass should do.

Ditto for methods: in an abstract base class, the methods should be
defined (so the signature is known) but raise NotImplementedError.

> In urllib2, there are 6 warnings about 2 different attributes. 

How come PyChecker didn't find the other typo there?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Fri Aug 10 20:56:25 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 10 Aug 2001 15:56:25 -0400
Subject: [Python-Dev] Direction of PyChecker
References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> <15219.2027.131396.703309@slothrop.digicool.com> <3B740008.2CF200C2@metaslash.com> <15220.9048.366116.587916@slothrop.digicool.com>
 <3B743629.E1220DBB@metaslash.com> <200108101945.PAA24012@cj20424-a.reston1.va.home.com>
Message-ID: <3B743C69.2BC9B090@metaslash.com>

Guido van Rossum wrote:
> 
> > As I said before, I think that the modules/classes
> > in the standard library should have them.
> 
> You can help by submitting some patches (one module at a time please :-).

I'll do my best.  And this time, I'll post to SF, with link here. :-)

> > And docstrings are definitely
> > doc, not code.
> 
> Agreed, but I don;'t understand why you said this.

I was just pointing out that docstrings should not be warned about
and that they won't be in the future (about 5 minutes :-).

> > > There are also complaints about attribute used by an
> > > abstract base class but only defined in the subclass.
> >
> > This is true.  While python doesn't require setting, I think this feature
> > can be dangerous.  It seems better to init the attr to None.
> 
> Agreed.  The initialization to None can have a comment explaining what
> the subclass should do.
> 
> Ditto for methods: in an abstract base class, the methods should be
> defined (so the signature is known) but raise NotImplementedError.

I will try to create patches for a lot of these warnings as well then.
I think there are already comments, so I will just init to None.

> > In urllib2, there are 6 warnings about 2 different attributes.
> 
> How come PyChecker didn't find the other typo there?

Not sure.  What was broken?  Was it in 2.2a1 or cvs?  (I ran against 2.2a1.)

Neal


From nas@python.ca  Fri Aug 10 21:21:05 2001
From: nas@python.ca (Neil Schemenauer)
Date: Fri, 10 Aug 2001 13:21:05 -0700
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: <3B7432ED.12A117C1@metaslash.com>; from neal@metaslash.com on Fri, Aug 10, 2001 at 03:15:57PM -0400
References: <200108101516.RAA15086@core.inf.ethz.ch> <200108101519.LAA18940@cj20424-a.reston1.va.home.com> <3B740138.98E75C5F@metaslash.com> <20010810090111.A7248@glacier.fnational.com> <3B7432ED.12A117C1@metaslash.com>
Message-ID: <20010810132104.B7903@glacier.fnational.com>

Neal Norwitz wrote:
> I looked at pylint (from 1999) and the ASTs that were generated
> from parser, etc.  But it was going to require too much work
> at the time.

The tree generated by Tools/compiler is much nicer than the one
generated by parser.

  Neil


From jeremy@zope.com  Fri Aug 10 21:52:06 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 10 Aug 2001 16:52:06 -0400 (EDT)
Subject: [Python-Dev] future and present
Message-ID: <15220.18806.202695.349346@slothrop.digicool.com>

If a script has
    from __future__ import nested_scopes
and is run by Python 2.2, should the interpreter complain?  The PEP
says "the compiler knows which features names have been defined, and
raises a compile-time error if a future_statement contains a feature
not known to it."  I think I'd interpret this as: nested_scopes is
known to 2.2 (and redundant since the feature is now mandatory),
therefore the compiler does nothing.

Jeremy




From guido@python.org  Fri Aug 10 22:01:11 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 10 Aug 2001 17:01:11 -0400
Subject: [Python-Dev] future and present
In-Reply-To: Your message of "Fri, 10 Aug 2001 16:52:06 EDT."
 <15220.18806.202695.349346@slothrop.digicool.com>
References: <15220.18806.202695.349346@slothrop.digicool.com>
Message-ID: <200108102101.RAA27718@cj20424-a.reston1.va.home.com>

> If a script has
>     from __future__ import nested_scopes
> and is run by Python 2.2, should the interpreter complain?

Of course not!  The whole point of the future statement is that it
should be harmless once you're *in* that future (unless the future
feature ends up retracted -- think of it as an alternate universe, a
side effect of time machine misuse :-).

> The PEP says "the compiler knows which features names have been
> defined, and raises a compile-time error if a future_statement
> contains a feature not known to it."  I think I'd interpret this as:
> nested_scopes is known to 2.2 (and redundant since the feature is
> now mandatory), therefore the compiler does nothing.

Correct.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@home.com  Fri Aug 10 22:18:14 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 10 Aug 2001 17:18:14 -0400
Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12
In-Reply-To: <2m8zgsfhx2.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEAJLFAA.tim.one@home.com>

Man, I'm having a hard time following this!  Perhaps people are talking
passed each other?

[Michael Hudson]
> It's a matter of interface, really.  It's certainly not at all
> technically hard.  Maybe:
>
>    compile(text, filename, symbol[, flags[, dont_inherit]])

Sounds good -- let's do it.

> I worry that saying "you don't inherit behaviour from surrounding code
> as soon as you pass a flag" might get really, really confusing at
> times.

I would too, but did someone suggest that?  Not me.  If someone did, they're
confusing issues to no good end.

[Guido]
> Occam sez: let's add the dont_inherit argument when we have found a
> real use for it.

It's trivial to add, impossible to work around its absence if necessary, and
is an inherent part of the problem space (as Michael said, the future-PEP
anticipated the problem).  It seems as simple as possible without leaving
problems unaddressed.

[Michael]
> There are two complementary sets of flags here:
>
> (1) The PyCF_* ones, defined in Include/pythonrun.h
> (2) The CO_* ones defined in Include/compile.h
>
> The proposed fourth argument to compile() should be a combination of
> the former set.
>
> I only use the latter to tell whether a __future__ statement was used
> in the text compiled (which is a bit horrible, but no better way
> springs to mind).

If there were a better way, we would have used it instead of the goofy CO_*
flags everywhere else too <wink>.

> There might be value in checking the flags passed to compile() anyway,
> but I can't see it being dangerous.

It's so easy to make this bulletproof that there's no point discussing
whether it *should* be made bulletproof.  I'll see to it that it's
bulletproof in the end, so if it's a hangup don't worry about it now.

[Guido]
> (I think I may have been confused by your original patch, which AFAICR
*did*
> manipulate the co_flags field of the code object, which contains the CO_*
flags.

Right, it did, and the newer scheme is safer.



From DavidA@ActiveState.com  Fri Aug 10 22:31:29 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Fri, 10 Aug 2001 14:31:29 -0700
Subject: [Python-Dev] future and present
References: <15220.18806.202695.349346@slothrop.digicool.com> <200108102101.RAA27718@cj20424-a.reston1.va.home.com>
Message-ID: <3B7452B1.CC57F1@ActiveState.com>

Guido van Rossum wrote:
> 
> > If a script has
> >     from __future__ import nested_scopes
> > and is run by Python 2.2, should the interpreter complain?
> 
> Of course not!  The whole point of the future statement is that it
> should be harmless once you're *in* that future (unless the future
> feature ends up retracted -- think of it as an alternate universe, a
> side effect of time machine misuse :-).

Although there could be a warning (off by default) in 2.3 or whenever to
let users know that the code is no longer needed.  People tend to cut &
paste and reuse code from others, without always knowing why.  Noops
should be culled from code as a general rule =).

--david


From guido@python.org  Fri Aug 10 22:31:22 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 10 Aug 2001 17:31:22 -0400
Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12
In-Reply-To: Your message of "Fri, 10 Aug 2001 17:18:14 EDT."
 <LNBBLJKPBEHFEDALKOLCIEAJLFAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCIEAJLFAA.tim.one@home.com>
Message-ID: <200108102131.RAA28517@cj20424-a.reston1.va.home.com>

OK, dont_inherit is good.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Fri Aug 10 22:33:39 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 10 Aug 2001 17:33:39 -0400 (EDT)
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: <20010810132104.B7903@glacier.fnational.com>
References: <200108101516.RAA15086@core.inf.ethz.ch>
 <200108101519.LAA18940@cj20424-a.reston1.va.home.com>
 <3B740138.98E75C5F@metaslash.com>
 <20010810090111.A7248@glacier.fnational.com>
 <3B7432ED.12A117C1@metaslash.com>
 <20010810132104.B7903@glacier.fnational.com>
Message-ID: <15220.21299.541723.697757@cj42289-a.reston1.va.home.com>

Neil Schemenauer writes:
 > The tree generated by Tools/compiler is much nicer than the one
 > generated by parser.

  An excellent reason for the compiler package to be documented!

  Jeremy?


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From guido@python.org  Fri Aug 10 22:35:14 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 10 Aug 2001 17:35:14 -0400
Subject: [Python-Dev] future and present
In-Reply-To: Your message of "Fri, 10 Aug 2001 14:31:29 PDT."
 <3B7452B1.CC57F1@ActiveState.com>
References: <15220.18806.202695.349346@slothrop.digicool.com> <200108102101.RAA27718@cj20424-a.reston1.va.home.com>
 <3B7452B1.CC57F1@ActiveState.com>
Message-ID: <200108102135.RAA28904@cj20424-a.reston1.va.home.com>

> Although there could be a warning (off by default) in 2.3 or whenever to
> let users know that the code is no longer needed.  People tend to cut &
> paste and reuse code from others, without always knowing why.  Noops
> should be culled from code as a general rule =).

Agreed.

This could also be a PyChecker feature. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@zope.com  Fri Aug 10 22:43:10 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Fri, 10 Aug 2001 17:43:10 -0400 (EDT)
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: <15220.21299.541723.697757@cj42289-a.reston1.va.home.com>
References: <200108101516.RAA15086@core.inf.ethz.ch>
 <200108101519.LAA18940@cj20424-a.reston1.va.home.com>
 <3B740138.98E75C5F@metaslash.com>
 <20010810090111.A7248@glacier.fnational.com>
 <3B7432ED.12A117C1@metaslash.com>
 <20010810132104.B7903@glacier.fnational.com>
 <15220.21299.541723.697757@cj42289-a.reston1.va.home.com>
Message-ID: <15220.21870.462055.990576@slothrop.digicool.com>

I'll work on it next week.  Really!  I will.

Jeremy



From fdrake@acm.org  Fri Aug 10 22:47:12 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 10 Aug 2001 17:47:12 -0400 (EDT)
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: <15220.21870.462055.990576@slothrop.digicool.com>
References: <200108101516.RAA15086@core.inf.ethz.ch>
 <200108101519.LAA18940@cj20424-a.reston1.va.home.com>
 <3B740138.98E75C5F@metaslash.com>
 <20010810090111.A7248@glacier.fnational.com>
 <3B7432ED.12A117C1@metaslash.com>
 <20010810132104.B7903@glacier.fnational.com>
 <15220.21299.541723.697757@cj42289-a.reston1.va.home.com>
 <15220.21870.462055.990576@slothrop.digicool.com>
Message-ID: <15220.22112.452257.158074@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > I'll work on it next week.  Really!  I will.

  I'm sure you'll have it done by the end of Monday, so you don't have
to deal with my staring at you accusingly all day Tuesday.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From tim.one@home.com  Fri Aug 10 22:57:48 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 10 Aug 2001 17:57:48 -0400
Subject: [Python-Dev] future and present
In-Reply-To: <15220.18806.202695.349346@slothrop.digicool.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEAPLFAA.tim.one@home.com>

[Jeremy]
> If a script has
>     from __future__ import nested_scopes
> and is run by Python 2.2, should the interpreter complain?

No:

    Else MandatoryRelease records when the feature became part of the
    language; in releases at or after that, modules no longer need

        from __future__ import FeatureName

    to use the feature in question, but may continue to use such imports.

[David Ascher]
> Although there could be a warning (off by default) in 2.3 or whenever
> to let users know that the code is no longer needed.  People tend to
> cut & paste and reuse code from others, without always knowing why.
> Noops should be culled from code as a general rule =).

The syntax of future statments was designed so that a trival sed script can
find them reliably.  Heck, even ActivePerl could find them <wink>.  They
have to begin in the first column, and have to start with "from __future__".

I expect that many people will *want* to keep obsolete future stmts in
forever, as a protection against trying to run code under earlier versions
of the interpreter -- the semantics were designed so that such back-porting
always complains (if to a pre-future version of Python, because the "from
__future__ import ..." stmt raises a runtime ImportError due to the absence
of module __future__.py (which has a reserved name); or if to a post-future
pre-feature version, a compile-time error).

People insanely <wink> concerned about this can query the MandatoryRelease
fields of the future-features they import, raising an exception of their
choosing if the MandatoryRelease is <= sys.version_info; e.g.,

from __future__ import division
assert division.getMandatoryRelease() > sys.version_info

That will complain the first time they run it under a Python where new
division is "the rule".



From tim.one@home.com  Sat Aug 11 00:00:41 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 10 Aug 2001 19:00:41 -0400
Subject: [Python-Dev] Somebody stopped running tests?
Message-ID: <LNBBLJKPBEHFEDALKOLCMEBCLFAA.tim.one@home.com>

Over the last 12 hours, 4 tests started failing on Windows:

    4 tests failed: test_descr test_future test_generators test_scope

Would someone on Linux please confirm or deny it's not unique to Windows?  I
don't want to spend a lot of time tracking these down if the responsible
parties see them on their boxes too.



From tim.one@home.com  Sat Aug 11 00:17:38 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 10 Aug 2001 19:17:38 -0400
Subject: [Python-Dev] RE: Somebody stopped running tests?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEBCLFAA.tim.one@home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEBDLFAA.tim.one@home.com>

[Tim said]
> Over the last 12 hours, 4 tests started failing on Windows:
>
>     4 tests failed: test_descr test_future test_generators test_scope
>
> Would someone on Linux please confirm or deny it's not unique to
> Windows?  I don't want to spend a lot of time tracking these down
> if the responsible parties see them on their boxes too.

But only in release mode.  In a debug build, these pass.  This strongly
suggests a change "somewhere" that accidentally relies on an uninitialized
value.  Under the release build, I get oodles of bogus warnings related to
scope, like

../lib/test/test_scope.py:401: SyntaxWarning:
    local name 'x' in 'f1' shadows use of 'x' as global in nested
    scope 'f2'

so best guess is it's Jeremy's recent future-flag fiddling patches.



From aahz@rahul.net  Sat Aug 11 00:18:09 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Fri, 10 Aug 2001 16:18:09 -0700 (PDT)
Subject: [Python-Dev] Any objections to adding threading.Timer?
In-Reply-To: <200108092021.f79KLFI00918@mira.informatik.hu-berlin.de> from "Martin v. Loewis" at Aug 09, 2001 10:21:15 PM
Message-ID: <20010810231810.3FDD499C82@waltz.rahul.net>

[sorry for the delay, network connectivity problems]

Martin v. Loewis wrote:
> Aahz:
>> 
>> I'm not objecting, precisely, but I have an allergic reaction to the way
>> timed Event() works, even though there's really no other way to work it.
>> I think that encouraging people to use it is a Bad Idea.
> 
> Can you elaborate in more detail what your concern about the timed
> Event() is? This one itself looks fine; if anything, _Condition.wait
> looks disturbing. Could you also kindly put your comments into
> 
> http://sourceforge.net/tracker/?group_id=5470&atid=305470&func=detail&aid=428326

Hrm.  I'll put my comments there if you ask, but my objection is more
general than this specific feature.  As you note, I'm actually objecting
to the whole Condition.wait() feature.  The main problem with this kind
of timeout is the way the granularity works; there was a recent post to
c.l.py where someone was complaining that it took a full second to
respond to the Event() getting set.

Fred: I think I did send a note to python-docs, but maybe I should file
an SF doc bug?

At the same time, when you really do want timeout functionality, there
isn't any other way to do it in a cross-platform way.  I think most
people claim they want a timeout when they really shouldn't, so I'd like
to make the feature less accessible.

> There are a number of procedural objections with regard to this patch
> (license, missing documentation); if you can add significant technical
> objections, that might be a reason to reject it - or to find a better
> alternative.

Well, I can't quite call this a "significant" technical objection, given
that this is the only good cross-platform scheme in a threaded
environment.  I'm just whining because I don't like it anyway.  ;-)
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From pedroni@inf.ethz.ch  Sat Aug 11 00:44:39 2001
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Sat, 11 Aug 2001 01:44:39 +0200
Subject: [Python-Dev] Direction of PyChecker
References: <200108101516.RAA15086@core.inf.ethz.ch> <200108101519.LAA18940@cj20424-a.reston1.va.home.com> <3B740138.98E75C5F@metaslash.com>
Message-ID: <00b601c121f6$6d35d340$8a73fea9@newmexico>

> Guido van Rossum wrote:
> >
> > [Samuele Pedroni]
> > > I'm consindering the possibility of adding a special mode to jython
> > > (based on hooks and on porting the CPython tools/compiler) in which
> > > you can retrieve the PVM bytecode of code objects, then there would
> > > be still to make PyChecker java classes aware, but it could run
> > > under jython. (Don't know when I will get at doing this)
> >
> > But is it really a good idea to codify PVM bytecode as the way to do
> > introspection?  I'd much rather have a standard library API that
> > parses a module and gives you a list of variables, functions, classes,
> > etc., and lets you drill down into the symbol table of each of those.
> > (I realize more is needed, you need to be able to inspect every
> > statement too -- that can be done by providing access to an AST.)
>
> While it would make my life easier to modify jython, I tend to agree
> with Guido.  Ideally, checker would use ASTs.  I really want checker
> to work with jython, but it's too much work to do right now.
>
> The reason why I chose to use the PVM bytecode was because it was
> faster and easier to get something.  I started trying to use ASTs
> but it would have taken a lot longer to get anywhere.
>
> I would like to support pychecker under jython, just not sure of
> the best or fastest way to get there.

There have been  a misunderestanding and don't like at all to play the role of
the ominous sinner,
ignorant of language design.

I was just proposing a hack, not codifying anything at all,
vs. what at the moment is just discussing about The Right Thing, aka wishful
thinking.

regards, Samuele Pedroni.



From tim_one@email.msn.com  Sat Aug 11 02:10:22 2001
From: tim_one@email.msn.com (Tim Peters)
Date: Fri, 10 Aug 2001 21:10:22 -0400
Subject: [Python-Dev] RE: Somebody stopped running tests?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEBDLFAA.tim.one@home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEBGLFAA.tim_one@email.msn.com>

> Over the last 12 hours, 4 tests started failing on Windows:
>
>     4 tests failed: test_descr test_future test_generators test_scope
> ...
> But only in release mode.  In a debug build, these pass.  This strongly
> suggests a change "somewhere" that accidentally relies on an
> uninitialized value.

And so it was.  "Fixed" in current CVS, but Jeremy needs to fix it a right
way.

hacking-a-hack-that-wasn't-fully-unhacked-ly y'rs  - tim



From barry@zope.com  Sat Aug 11 04:08:53 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 10 Aug 2001 23:08:53 -0400
Subject: [Python-Dev] Free threading (and tcl upvar)
References: <Pine.LNX.4.21.0108090902050.890-100000@serenade.digicool.com>
 <200108091522.f79FMja02243@odiug.digicool.com>
 <15219.15664.450890.783449@anthem.wooz.org>
 <15219.16667.375672.59499@slothrop.digicool.com>
Message-ID: <15220.41413.209107.815162@anthem.wooz.org>

>>>>> "JH" == Jeremy Hylton <jeremy@zope.com> writes:

    JH> It takes all kinds, doesn't it?  For every genius who comes up
    JH> with something like sys._getframe(), we have some fool who
    JH> comes up with something like "print >>".

there is indeed a fine line between genius and insanity.



From guido@python.org  Sat Aug 11 05:06:14 2001
From: guido@python.org (Guido van Rossum)
Date: Sat, 11 Aug 2001 00:06:14 -0400
Subject: [Python-Dev] Any objections to adding threading.Timer?
In-Reply-To: Your message of "Fri, 10 Aug 2001 16:18:09 PDT."
 <20010810231810.3FDD499C82@waltz.rahul.net>
References: <20010810231810.3FDD499C82@waltz.rahul.net>
Message-ID: <200108110406.AAA04467@cj20424-a.reston1.va.home.com>

> As you note, I'm actually objecting to the whole Condition.wait()
> feature.  The main problem with this kind of timeout is the way the
> granularity works; there was a recent post to c.l.py where someone
> was complaining that it took a full second to respond to the Event()
> getting set.

Hm, maybe we should think about adding some low-level thing somewhere
so that wait(timeout=...) can be implemented more efficiently?  I
agree that it's ugly now -- busy-waiting with a sleep time doubling
until it reaches 1 second...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz@rahul.net  Sat Aug 11 05:42:36 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Fri, 10 Aug 2001 21:42:36 -0700 (PDT)
Subject: [Python-Dev] Any objections to adding threading.Timer?
In-Reply-To: <200108110406.AAA04467@cj20424-a.reston1.va.home.com> from "Guido van Rossum" at Aug 11, 2001 12:06:14 AM
Message-ID: <20010811044236.9F2BA99C82@waltz.rahul.net>

Guido van Rossum wrote:
>Aahz:
>> 
>> As you note, I'm actually objecting to the whole Condition.wait()
>> feature.  The main problem with this kind of timeout is the way the
>> granularity works; there was a recent post to c.l.py where someone
>> was complaining that it took a full second to respond to the Event()
>> getting set.
> 
> Hm, maybe we should think about adding some low-level thing somewhere
> so that wait(timeout=...) can be implemented more efficiently?  I
> agree that it's ugly now -- busy-waiting with a sleep time doubling
> until it reaches 1 second...

Unfortunately, I don't think this is possible unless we add some
platform-specific code to thread.c, and you'll need to ask Uncle Timmy
for that.  To the extent that I'm an expert on threading, I'm only an
expert on Python threads.

Unless someone is aware of some brilliantly simple solution, I think we
need to live with the ugliness for now, but I'd prefer to discourage
further efforts in the ugly direction.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From martin@loewis.home.cs.tu-berlin.de  Sat Aug 11 08:43:05 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sat, 11 Aug 2001 09:43:05 +0200
Subject: [Python-Dev] future and present
Message-ID: <200108110743.f7B7h5702904@mira.informatik.hu-berlin.de>

> Although there could be a warning (off by default) in 2.3 or
> whenever to let users know that the code is no longer needed.
> People tend to cut & paste and reuse code from others, without
> always knowing why.  Noops should be culled from code as a general
> rule =).

Actually, Tim promised to write a tool to remove unneeded future
imports, when the feature was first discussed. Maybe he did not
actually promise to write such a tool, but only explained how easy it
would be to write, given that it is introspective :-)

Regards,
Martin


From martin@loewis.home.cs.tu-berlin.de  Sat Aug 11 09:06:35 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sat, 11 Aug 2001 10:06:35 +0200
Subject: [Python-Dev] Any objections to adding threading.Timer?
In-Reply-To: <20010810231810.3FDD499C82@waltz.rahul.net> (aahz@rahul.net)
References: <20010810231810.3FDD499C82@waltz.rahul.net>
Message-ID: <200108110806.f7B86Zp03107@mira.informatik.hu-berlin.de>

> Hrm.  I'll put my comments there if you ask, but my objection is more
> general than this specific feature.  As you note, I'm actually objecting
> to the whole Condition.wait() feature.  The main problem with this kind
> of timeout is the way the granularity works; there was a recent post to
> c.l.py where someone was complaining that it took a full second to
> respond to the Event() getting set.

Please do. Correct me if I'm wrong, but the timeout of the Timer won't
be precise; that clearly is undesirable. It also seems that this would
be easy to correct: If endtime-_time() < delay, you have to decrease
delay. 

What then is left is that cancellation of the timer may take up to a
second, that should be documented. I think it is possible to do better
than that on some systems; maybe somebody feels inclined to improve
the code on, say, UNIX.

Regards,
Martin


From just@letterror.com  Sat Aug 11 09:17:31 2001
From: just@letterror.com (Just van Rossum)
Date: Sat, 11 Aug 2001 10:17:31 +0200
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: <200108092052.f79KqQg13802@odiug.digicool.com>
Message-ID: <20010811101734-r01010700-bede4223-0910-010c@10.0.0.23>

A couple of days ago, Guido wrote:

> I would like PyChecker to use a fundamentally different way of
> scanning modules.  I believe that it currently imports the module and
> then scans the module's __dict__ for classes and functions, and
> analyses their bytecode.  That's fragile, dangerous (if you don't know
> what a module does you may not want to import it), [ ... ]

[Off topic] The same seems to be true for pydoc in http-serving mode.
It just imports any .py on sys.path, when doco is requested for it.
Although pydoc is *really* cool, this aspect scares me a bit...
(I guess if the port pydoc is listening to is visible from the outside
world, it better run as a user with very limited permissions, right?)

(Hm, that makes me think that it may be a good practice to name scripts
that are not supposed to be imported so that they *can't* be imported?)

Just


From guido@python.org  Sat Aug 11 13:52:16 2001
From: guido@python.org (Guido van Rossum)
Date: Sat, 11 Aug 2001 08:52:16 -0400
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: Your message of "Sat, 11 Aug 2001 10:17:31 +0200."
 <20010811101734-r01010700-bede4223-0910-010c@10.0.0.23>
References: <20010811101734-r01010700-bede4223-0910-010c@10.0.0.23>
Message-ID: <200108111252.IAA05424@cj20424-a.reston1.va.home.com>

> [Off topic] The same seems to be true for pydoc in http-serving mode.
> It just imports any .py on sys.path, when doco is requested for it.
> Although pydoc is *really* cool, this aspect scares me a bit...
> (I guess if the port pydoc is listening to is visible from the outside
> world, it better run as a user with very limited permissions, right?)

Yes, this bothers me greatly.  I hope that someone will fix this.

> (Hm, that makes me think that it may be a good practice to name scripts
> that are not supposed to be imported so that they *can't* be imported?)

Indeedy!  (Or make sure they're noy on sys.path. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@home.com  Sat Aug 11 18:29:40 2001
From: tim.one@home.com (Tim Peters)
Date: Sat, 11 Aug 2001 13:29:40 -0400
Subject: [Python-Dev] Any objections to adding threading.Timer?
In-Reply-To: <20010811044236.9F2BA99C82@waltz.rahul.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCMECHLFAA.tim.one@home.com>

[Aahz]
> Unfortunately, I don't think this is possible unless we add some
> platform-specific code to thread.c, and you'll need to ask Uncle Timmy
> for that.  To the extent that I'm an expert on threading, I'm only an
> expert on Python threads.

AFAIK there are no reliable x-platform timeout facilities.  Even on Windows,
the current scheme sucks in (at least) three ways:

1. time.time() updates only 18.2 times per second on Windows, so
   55ms is the finest granularity possible via any time.time()-based
   approach.

2. time.sleep() on Windows truncates the argument to an integer
   number of milliseconds (due to limitations of the Win32 Sleep),
   so any delay value < 0.001 acts like Sleep(0), i.e. returns
   immediately if no other threads are waiting (that's how Win32
   Sleep(0) works).

3. As Glyph so visibly discovered on c.l.py recently, time.time() can
   "go backwards", and especially on laptops with Intel SpeedStep
   CPUs.

All that said, I'd still change the timeout loop from

                endtime = _time() + timeout
                delay = 0.000001 # 1 usec
                while 1:
                    gotit = waiter.acquire(0)
                    if gotit or _time() >= endtime:
                        break
                    _sleep(delay)
                    if delay < 1.0:
                        delay = delay * 2.0

to

                endtime = _time() + timeout
                delay = 0.001 # 1 ms
                while 1:
                    gotit = waiter.acquire(0)
                    if gotit:
                        break
                    remaining = endtime - _time()
                    if remaining < 0:
                        break
                    delay = min(delay * 2, remaining, .05)
                    _sleep(delay)



From tim.one@home.com  Sat Aug 11 18:39:39 2001
From: tim.one@home.com (Tim Peters)
Date: Sat, 11 Aug 2001 13:39:39 -0400
Subject: [Python-Dev] future and present
In-Reply-To: <200108110743.f7B7h5702904@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCIECILFAA.tim.one@home.com>

[Martin v. Loewis]
> Actually, Tim promised to write a tool to remove unneeded future
> imports, when the feature was first discussed. Maybe he did not
> actually promise to write such a tool, but only explained how easy it
> would be to write, given that it is introspective :-)

Given that David is the only person to date to ask for this, and not even
because he personally wants it, I now view this as a value-added proposition
for ActiveState to sell <wink>.

it-really-is-a-finger-exercise-ly y'rs  - tim



From paulp@ActiveState.com  Sat Aug 11 18:49:41 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Sat, 11 Aug 2001 10:49:41 -0700
Subject: [Python-Dev] xmlrpclib Server
Message-ID: <3B757035.6894450E@ActiveState.com>

What would people think about renaming xmlrpclib.Server to
xmlrpclib.RPCProxy or ServerProxy or something like that?

I find it confusing that the *client* software creates a Server object.
I would expect xmlrpclib.Server() to create a true XML-RPC Server. I
like the way that SOAPPy works:

 * SOAPProxy creates a client-side proxy object for a server
 * SOAPServer creates a SOAP server.
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From martin@loewis.home.cs.tu-berlin.de  Sat Aug 11 19:46:19 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sat, 11 Aug 2001 20:46:19 +0200
Subject: [Python-Dev] xmlrpclib Server
Message-ID: <200108111846.f7BIkJZ04391@mira.informatik.hu-berlin.de>

> What would people think about renaming xmlrpclib.Server to
> xmlrpclib.RPCProxy or ServerProxy or something like that?

The time machine at work? It is already called ServerProxy.

Regards,
Martin


From paulp@ActiveState.com  Sat Aug 11 21:16:04 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Sat, 11 Aug 2001 13:16:04 -0700
Subject: [Python-Dev] xmlrpclib Server
References: <200108111846.f7BIkJZ04391@mira.informatik.hu-berlin.de>
Message-ID: <3B759284.463FBE9A@ActiveState.com>

"Martin v. Loewis" wrote:
> 
> > What would people think about renaming xmlrpclib.Server to
> > xmlrpclib.RPCProxy or ServerProxy or something like that?
> 
> The time machine at work? It is already called ServerProxy.

You are right! Doing a diff of the version I have hanging around and the
CVS version, I see:

< class ServerProxy:
<
< Server = ServerProxy
---
> class Server:

But the TeX documentation still says "Server". Would anyone object to
changing the doc to make ServerProxy official?
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From neal@metaslash.com  Sat Aug 11 21:17:39 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Sat, 11 Aug 2001 16:17:39 -0400
Subject: [Python-Dev] More bugs in 2.2a1
Message-ID: <3B7592E3.892D1995@metaslash.com>

Here's more warnings from pychecker.  There are a lot from plat-irix6 &
plat-unixware7, not sure if those are real problems or not.

Neal
--

curses:
    has_key.py:176: Module (_curses) re-imported
    textpad.py:72: No global (ERR) found

distutils:
    bcppcompiler.py:359: No global (UnknownFileError) found
    dist.py:583: Variable (klass) used before being set
    fancy_getopt.py:277: No global (value) found
    file_util.py:218: No global (errno) found

distutils.command:
    bdist_rpm.py:170: No global (DistutilsOptionsError) found
    bdist_wininst.py:217: Module (base64) re-imported
    config.py:277: No global (exe) found
    install.py:389: No global (DistutilsPlatformError) found

encodings:
    charmap.py:28: No global (strict) found

lib-tk:
    Tix.py:81: self is not first method argument
    Tix.py:331: No global (value) found

plat-irix6:
    FILE.py:32: No global (__major) found
    FILE.py:34: No global (__minor) found
    FILE.py:50: No global (splhi) found
    FILE.py:52: No global (getsr) found
    FILE.py:52: No global (issplhi) found
    FILE.py:66: No global (mutex_unlock) found
    FILE.py:91: No global (spinlock_init) found
    FILE.py:99: No global (PZERO) found
    FILE.py:99: No global (mutex_lock) found
    FILE.py:101: No global (mutex_trylock) found
    FILE.py:105: No global (sv_signal) found
    FILE.py:107: No global (sv_broadcast) found
    FILE.py:109: No global (sv_destroy) found
    FILE.py:113: No global (mrunlock) found
    FILE.py:115: No global (cmrpromote) found
    FILE.py:117: No global (mrdemote) found
    FILE.py:119: No global (mrfree) found
    FILE.py:123: No global (ismrlocked) found

plat-unixware7:
    IN.py:75: No global (IN6_ADDR_EQUAL_L) found
    IN.py:77: No global (IN6_ADDR_COPY_L) found
    IN.py:162: No global (REMQUE) found
    IN.py:440: No global (CYCLES) found
    IN.py:440: No global (CYCLES_BETWEEN) found
    IN.py:482: No global (_ATOMIC_INT_READ) found
    IN.py:490: No global (DISABLE) found
    IN.py:494: No global (ENABLE) found
    IN.py:522: No global (LOCK) found
    IN.py:524: No global (TRYLOCK) found
    IN.py:526: No global (LOCK_SH) found
    IN.py:528: No global (RW_RDLOCK) found
    IN.py:530: No global (RW_WRLOCK) found
    IN.py:534: No global (B_FALSE) found
    IN.py:540: No global (crholdn) found
    IN.py:542: No global (crfreen) found
    IN.py:546: No global (sizeof) found
    IN.py:546: No global (uint) found
    IN.py:597: No global (B_TRUE) found
    IN.py:601: No global (TICKS) found
    IN.py:601: No global (TICKS_BETWEEN) found
    IN.py:617: No global (engine) found
    IN.py:638: No global (STRM_PUTCNT) found
    IN.py:638: No global (l) found
    IN.py:718: No global (caddr_t) found
    IN.py:832: No global (canput) found
    IN.py:834: No global (canputnext) found

xml.sax:
    __init__.py:78: Module (sys) re-imported


From mal@lemburg.com  Sat Aug 11 15:36:01 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 11 Aug 2001 16:36:01 +0200
Subject: [Python-Dev] ++x oddnes
References: <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> <3B741351.C7C2E2B8@ActiveState.com>
Message-ID: <3B7542D1.3E38E579@lemburg.com>

David Ascher wrote:
> 
> Guido van Rossum wrote:
> 
> > IMO not worth it changing the language.  Train yourself to write x++
> > instead. :-)
> 
> Sounds like an excellent one for PyChecker =)

Why ? ++x can be put to some real use: I have a counter
type which actually uses ++x to increment the counter.

Note that the interpreter sees ++x as +(+x), that is the "+" is 
interpreted as unary + -- perfectly legal Python if you ask me.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/




From paulp@ActiveState.com  Sun Aug 12 00:52:49 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Sat, 11 Aug 2001 16:52:49 -0700
Subject: [Python-Dev] ++x oddnes
References: <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> <3B741351.C7C2E2B8@ActiveState.com> <3B7542D1.3E38E579@lemburg.com>
Message-ID: <3B75C551.DFDF915C@ActiveState.com>

"M.-A. Lemburg" wrote:
> 
>..
> 
> Why ? ++x can be put to some real use: I have a counter
> type which actually uses ++x to increment the counter.

Do you implement a matching "x++"? What's wrong with x.inc()?

> Note that the interpreter sees ++x as +(+x), that is the "+" is
> interpreted as unary + -- perfectly legal Python if you ask me.

Legal but perverse. Your users will expect +++x and x++ to be equally
valid and have their C semantics. Plus you are using a Python construct
that means one thing to mean something totally different just because it
means the totally different thing in other languages. In Python that
syntax would usually be side-effect free (and useless).
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From DavidA@ActiveState.com  Sun Aug 12 01:11:09 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Sat, 11 Aug 2001 17:11:09 -0700
Subject: [Python-Dev] ++x oddnes
References: <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> <3B741351.C7C2E2B8@ActiveState.com> <3B7542D1.3E38E579@lemburg.com>
Message-ID: <3B75C99D.303E52EC@ActiveState.com>

"M.-A. Lemburg" wrote:
> 
> David Ascher wrote:
> >
> > Guido van Rossum wrote:
> >
> > > IMO not worth it changing the language.  Train yourself to write x++
> > > instead. :-)
> >
> > Sounds like an excellent one for PyChecker =)
> 
> Why ? ++x can be put to some real use: I have a counter
> type which actually uses ++x to increment the counter.

The point of linters is to find things that are likely to be errors.  I
claim that ++foo is likely to be an error resulting from the corruption
of young minds by C++ more often than deliberate and appropriate.  

But you could turn it off when you used PyChecker =).

--david


From tim.one@home.com  Sun Aug 12 02:24:15 2001
From: tim.one@home.com (Tim Peters)
Date: Sat, 11 Aug 2001 21:24:15 -0400
Subject: [Python-Dev] New regrtest.py gimmick
Message-ID: <LNBBLJKPBEHFEDALKOLCKEDGLFAA.tim.one@home.com>

Since checkin email isn't working, I'm sending this to Python-Dev.

More than once I've been burned by tests getting skipped on Windows when
they shouldn't have been.  Indeed, for at least one release we didn't ship
the XML libraries on Windows because I had no idea the associated tests
weren't *supposed* to get skipped.

regrtest.py now prints this for me (note the last line):

...
138 tests OK.
25 tests skipped: test_al test_cd test_cl test_commands test_crypt
    test_dbm test_dl test_fcntl test_fork1 test_gdbm test_gl test_grp
    test_imgfile test_largefile test_linuxaudiodev test_mhlib test_nis
    test_openpty test_poll test_pty test_pwd test_signal
    test_socketserver test_sunaudiodev test_timing
Those skips are all expected on win32.


For you it will print something like:

...
MMM tests OK.
NN tests skipped: XXX YYY ...
Ask someone to teach regrtest.py about which tests are
expected to get skipped on [sys.platform].


You do that by changing the new _ExpectedSkips.__init__ near the bottom of
regrtest.py.

If sys.platform isn't enough to determine the set of skipped tests, then
sys.platform isn't particularly useful <0.9 wink>.



From m.favas@per.dem.csiro.au  Sun Aug 12 02:42:02 2001
From: m.favas@per.dem.csiro.au (Mark Favas)
Date: Sun, 12 Aug 2001 09:42:02 +0800
Subject: [Python-Dev] New regrtest.py gimmick
Message-ID: <3B75DEEA.7B12E66A@per.dem.csiro.au>

[Tim tests for expected platform-specific skips]

... just a slight change needed for the new gimmick to be happy on other
platforms <wink>

Instead of:

        e = _ExpectedSkips()
        if e.isvalid():
            surprise = _Set(skipped) - e.getexpected()
            plat = sys.platform #<-------------------------------------
            if surprise:
                print count(len(surprise), "skip"), \
                      "unexpected on", plat + ":", \
                      " ".join(surprise.tolist())
            else:
                print "Those skips are all expected on", plat + "."
        else:
            print "Ask someone to teach regrtest.py about which tests
are"
            print "expected to get skipped on", plat + "."

try:

        e = _ExpectedSkips()
        plat = sys.platform #<+++++++++++++++++++++++++++++++++++++++++
        if e.isvalid():
            surprise = _Set(skipped) - e.getexpected()
            if surprise:
                print count(len(surprise), "skip"), \
                      "unexpected on", plat + ":", \
                      " ".join(surprise.tolist())
            else:
                print "Those skips are all expected on", plat + "."
        else:
            print "Ask someone to teach regrtest.py about which tests
are"
            print "expected to get skipped on", plat + "."

-- 
Mark Favas  -   m.favas@per.dem.csiro.au
CSIRO, Private Bag No 5, Wembley, Western Australia 6913, AUSTRALIA


From tim.one@home.com  Sun Aug 12 03:02:34 2001
From: tim.one@home.com (Tim Peters)
Date: Sat, 11 Aug 2001 22:02:34 -0400
Subject: [Python-Dev] RE: New regrtest.py gimmick
In-Reply-To: <3B75DEEA.7B12E66A@per.dem.csiro.au>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEDILFAA.tim.one@home.com>

]Mark Favas]
> ... just a slight change needed for the new gimmick to be happy on other
> platforms <wink>

What other platforms?  If you were running on Windows like you're supposed
to, this wouldn't have bothered you <wink>.

the-suggested-change-has-been-gratefully-checked-in-ly y'rs  - tim



From tim.one@home.com  Sun Aug 12 05:05:37 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 12 Aug 2001 00:05:37 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <5.1.0.14.0.20010801020813.022a56b8@24.8.96.48>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEDPLFAA.tim.one@home.com>

[Dan Sugalski, on
 Wednesday, August 01, 2001 2:20 AM]
> ...
> Ouch, I'd bet that hurts. Has anyone timed the difference between
> making lots of getc calls and making a few larger reads and managing
> the buffers internally? I can see it going either way, and another
> data point would be useful to have.

There's been lots of this in Python-Dev, like in this thread:

    http://aspn.activestate.com/ASPN/Mail/Message/600485

I'll quote the high-order bit:

    My line-at-a-time test case used (rounding to nearest whole integers)
    30 seconds in Python and 6 in Perl.  The result of testing many
    changes to Python's implementation was that the excess 24 seconds
    broke down like so:

    17   spent inside internal MS threadsafe getc() lock/unlock
             routines
     5   uncertain, but evidence suggests much of it due to MS
             malloc/realloc (Perl does its own memory mgmt)
     2   for not copying directly out of the platform FILE*
             implementation struct in a highly optimized loop (like
             Perl does)

    My last checkin to fileobject.c reclaimed 17 seconds on Win98SE
    while remaining threadsafe, via a combination of locking per line
    instead of per character, and invoking realloc much less often
    (only for lines exceeding 200 chars).

Note that thread overhead is overwhelmingly the biggest hangup.  Python has
two threadsafe input tricks now:

1. On platforms that have flockfile(), funlockfile(), and
   getc_unlocked(), the last is used in a loop bracketed by the first
   two.

2. At least on Windows, which doesn't have those, we use the platform
   fgets() in an excruciating way, tricking it into letting us read
   lines with embedded null bytes.

Oddly enough, in the timing reports I saw, approach #1 was never faster than
approach #2, and on at least one platform (Tru64, IIRC) was slower.

Of course fgets() is a primitive in std C because they *wanted* to make it
possible for vendors to optimize it (in the ways Perl does), but it appears
very few vendors do optimize it.  On Windows it's the same old
getc()-in-a-loop, but they lock/unlock the stream only once per fgets call
(using internal stream functions that aren't exposed).

The "2 seconds for not copying directly ... like Perl does" I reported above
came from hacking together a thread-unsafe line input routine that used the
same FILE* tricks Perl uses.  That is, thread-unsafe getc-in-a-loop was 2
seconds slower than using thread-unsafe FILE* tricks.  That's significant in
absolute terms, but was lost in the noise compared to the other stuff we
were fighting.



From guido@python.org  Sun Aug 12 05:11:56 2001
From: guido@python.org (Guido van Rossum)
Date: Sun, 12 Aug 2001 00:11:56 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: Your message of "Sun, 12 Aug 2001 00:05:37 EDT."
 <LNBBLJKPBEHFEDALKOLCEEDPLFAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCEEDPLFAA.tim.one@home.com>
Message-ID: <200108120411.AAA08638@cj20424-a.reston1.va.home.com>

I don't have context for this thread anymore, so perhaps this is
already what's being discussed:

Perhaps a less ambitious (than Parrot) but still very useful project
would be a portable stdio-like library that has the features dynamic
languages need?  This could be based upon a few low-level calls like
read() and write() on Unix.  These tend to exist in most platforms C
libraries (even on Windows) because they're so darned useful.    For
full "portability" it should be possible to implement these yourself
if the platform doesn't have them (or if they are not fast enough!).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@home.com  Sun Aug 12 05:23:42 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 12 Aug 2001 00:23:42 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <20010802103006.H1667@netthink.co.uk>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEEALFAA.tim.one@home.com>

[Simon Cozens
 on Thursday, August 02, 2001 1:30 PM]

> ...
> Oh, and here's our divide, for comparison:
>
> PP(pp_divide)
> {
>     dSP; dATARGET; tryAMAGICbin(div,opASSIGN);
>     {
>       dPOPPOPnnrl;
>       NV value;
>       if (right == 0.0)
>         DIE(aTHX_ "Illegal division by zero");
>       value = left / right;
>       PUSHn( value );
>       RETURN;
>     }
> }
>
> See, this is why we need a new interpreter. :)

Yes, I've tried to read the Perl source before <snarl/wink>.  What does "/"
mean here?  You're using native C arithmetic, or is this a C++ overload?
Even simple arithmetic is long-winded in Python, because, for example, the
int operations check for overflow, while the float operations allow for
catching IEEE-754 exceptions (or SIGFPE in general) and translating them
into Python-level exceptions.  We also go thru piles and piles of code
coercing among different numeric types.  If you were to step thru the Python
line

    x = 3.0 / 4

in a debugger, it's a race on Windows between your finger getting numb and
the OS crashing ...



From tim.one@home.com  Sun Aug 12 09:49:40 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 12 Aug 2001 04:49:40 -0400
Subject: [Python-Dev] future and present
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIECILFAA.tim.one@home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEEFLFAA.tim.one@home.com>

There's a new Tools/scripts/cleanfuture.py to delete (or just remove the
appropriate parts of) obsolete future statements.

If you want any changes, make 'em yourself.  In particular, some Unixoid
will probably want to change it to fiddle file permissions.

Note that this serves as a nice example of why generators are truly
useful -- it doesn't use generators itself, but uses the .next() method of
the generator interface to tokenize.py.  This lets it parse files in a
natural way without building godawful state machines by hand, yet runs very
fast because future stmts can only appear near the start of a file (and so
tokenize.py only has to work until the driver determines no more future
stmts can appear).



From mwh@python.net  Sun Aug 12 10:22:40 2001
From: mwh@python.net (Michael Hudson)
Date: 12 Aug 2001 05:22:40 -0400
Subject: [Python-Dev] Re: [Python-checkins] testing
In-Reply-To: Guido van Rossum's message of "Sat, 11 Aug 2001 21:56:44 -0400"
References: <200108120156.VAA06262@cj20424-a.reston1.va.home.com>
Message-ID: <2mu1zd8wf3.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> warning: somehow the python checkin mail is dropped on the floor.

And bugs email, and a message I just sent myself from sf.net's web
form all seem to be disappearing too.

Mail sent to <username>@users.sourceforge.net still seems to work, so
at least *something* still works over there...

Cheers,
M.


From martin@loewis.home.cs.tu-berlin.de  Sun Aug 12 10:30:06 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sun, 12 Aug 2001 11:30:06 +0200
Subject: [Python-Dev] CVS build breakage: snprintf finds its way into socketmodule.c
In-Reply-To: <etdvgkino79.fsf@opus.mit.edu> (message from Alex Coventry on 24
 Jul 2001 15:04:26 -0400)
References: <200107240638.f6O6cPn03340@mira.informatik.hu-berlin.de> <etdvgkino79.fsf@opus.mit.edu>
Message-ID: <200108120930.f7C9U6902163@mira.informatik.hu-berlin.de>

> In PySocket_getaddrinfo, would it make sense to increase the allocation
> of pbuf from 10 characters to, say, 30 characters, in case 
> 
> sprintf(pbuf, "%ld", PyInt_AsLong(pobj));
> 
> gets run on a 64-bit machine?

It isn't strictly necessary, since the port number should never be
that large; it doesn't hurt to increase the size, either, so I did
that.

Thanks,
Martin



From mal@lemburg.com  Sun Aug 12 11:47:38 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sun, 12 Aug 2001 12:47:38 +0200
Subject: [Python-Dev] ++x oddness
References: <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> <3B741351.C7C2E2B8@ActiveState.com> <3B7542D1.3E38E579@lemburg.com> <3B75C551.DFDF915C@ActiveState.com>
Message-ID: <3B765ECA.1E806DD7@lemburg.com>

Paul Prescod wrote:
> 
> "M.-A. Lemburg" wrote:
> >
> >..
> >
> > Why ? ++x can be put to some real use: I have a counter
> > type which actually uses ++x to increment the counter.
> 
> Do you implement a matching "x++"? What's wrong with x.inc()?

x++ does not work. x.inc() would work too, but it involves a
method lookup and function call, which "++x" doesn't.
 
> > Note that the interpreter sees ++x as +(+x), that is the "+" is
> > interpreted as unary + -- perfectly legal Python if you ask me.
> 
> Legal but perverse. Your users will expect +++x and x++ to be equally
> valid and have their C semantics. Plus you are using a Python construct
> that means one thing to mean something totally different just because it
> means the totally different thing in other languages. In Python that
> syntax would usually be side-effect free (and useless).

Well, it is not necessarily side-effect free, since Python does call
a method slot on the underlying object and that slot can change
of course change state.

Nevermind, the syntax is only an experiment -- I just wanted to
point out that "++x" is legal in Python (you can place any number
of + before the object, BTW).

Some examples:

>>> +++++1
1
>>> ------2
2
>>> --2
2
>>> -2
-2
>>> ---2
-2
>>> ----2
2

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/




From guido@python.org  Sun Aug 12 15:53:29 2001
From: guido@python.org (Guido van Rossum)
Date: Sun, 12 Aug 2001 10:53:29 -0400
Subject: [Python-Dev] CVS commit permission for Andrew MacIntyre
Message-ID: <200108121453.KAA15888@cj20424-a.reston1.va.home.com>

The SF patch manager has two large patch sets for OS/2 from Andrew I
MacIntyre.  By and large they are benign.  Rather than applying these
myself, I suggest giving Andrew checkin permission.  Does anybody have
an objection?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From simon@netthink.co.uk  Sun Aug 12 20:25:39 2001
From: simon@netthink.co.uk (Simon Cozens)
Date: Sun, 12 Aug 2001 20:25:39 +0100
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <200108120411.AAA08638@cj20424-a.reston1.va.home.com>; from guido@python.org on Sun, Aug 12, 2001 at 12:11:56AM -0400
References: <LNBBLJKPBEHFEDALKOLCEEDPLFAA.tim.one@home.com> <200108120411.AAA08638@cj20424-a.reston1.va.home.com>
Message-ID: <20010812202539.C1231@deep-dark-truthful-mirror>

On Sun, Aug 12, 2001 at 12:11:56AM -0400, Guido van Rossum wrote:
> Perhaps a less ambitious (than Parrot) but still very useful project
> would be a portable stdio-like library that has the features dynamic
> languages need?  

I hope you do not think that I'm blowing Perl's trumpet again, but I see
one of my roles here as sharing knowledge about things that Python has
that Perl needs and vice versa.

Perl 5.7.x has such a library, and it's my expectation (another sideways
glance at Dan) that if we did Parrot, it would include this library.
PerlIO attempts to be a very portable stdio (it's running on everything
that Perl runs on, AFAIK) and it supports "line disciplines" in the sfio
sense: you can stuff processing modules between the file and the
filehandle. This allows us to do anything between altering buffering
semantics, and decompressing and decrypting files on the fly.

It'll need to be somewhat rewritten for Perl 6, at which point I'm
hoping it'll be done in such a way that it'll make it easier to extract
from the Perl mess around it and be made available to other projects.

-- 
Putting heated bricks close to the news.admin.net-abuse.* groups.
        -- Megahal (trained on asr), 1998-11-06


From tim.one@home.com  Sun Aug 12 20:42:44 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 12 Aug 2001 15:42:44 -0400
Subject: [Python-Dev] CVS commit permission for Andrew MacIntyre
In-Reply-To: <200108121453.KAA15888@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEFBLFAA.tim.one@home.com>

[Guido van Rossum]
> The SF patch manager has two large patch sets for OS/2 from Andrew I
> MacIntyre.  By and large they are benign.  Rather than applying these
> myself, I suggest giving Andrew checkin permission.  Does anybody have
> an objection?

No objection here.  BTW, I asked Andrew to upload his OS/2 patches -- but I
had no idea they'd be so large <wink>!



From thomas@xs4all.net  Sun Aug 12 21:25:47 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Sun, 12 Aug 2001 22:25:47 +0200
Subject: [Python-Dev] CVS commit permission for Andrew MacIntyre
In-Reply-To: <200108121453.KAA15888@cj20424-a.reston1.va.home.com>
References: <200108121453.KAA15888@cj20424-a.reston1.va.home.com>
Message-ID: <20010812222547.I874@xs4all.nl>

On Sun, Aug 12, 2001 at 10:53:29AM -0400, Guido van Rossum wrote:

> The SF patch manager has two large patch sets for OS/2 from Andrew I
> MacIntyre.  By and large they are benign.  Rather than applying these
> myself, I suggest giving Andrew checkin permission.  Does anybody have
> an objection?

None, though I'd suggest subscribing him to python-dev too, for the reasons
I gave when I found out Just had checkin privs but wasn't on py-dev.

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From guido@python.org  Sun Aug 12 21:15:05 2001
From: guido@python.org (Guido van Rossum)
Date: Sun, 12 Aug 2001 16:15:05 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: Your message of "Sun, 12 Aug 2001 20:25:39 BST."
 <20010812202539.C1231@deep-dark-truthful-mirror>
References: <LNBBLJKPBEHFEDALKOLCEEDPLFAA.tim.one@home.com> <200108120411.AAA08638@cj20424-a.reston1.va.home.com>
 <20010812202539.C1231@deep-dark-truthful-mirror>
Message-ID: <200108122015.QAA16295@cj20424-a.reston1.va.home.com>

> Perl 5.7.x has such a library, and it's my expectation (another sideways
> glance at Dan) that if we did Parrot, it would include this library.
> PerlIO attempts to be a very portable stdio (it's running on everything
> that Perl runs on, AFAIK) and it supports "line disciplines" in the sfio
> sense: you can stuff processing modules between the file and the
> filehandle. This allows us to do anything between altering buffering
> semantics, and decompressing and decrypting files on the fly.

Great!

> It'll need to be somewhat rewritten for Perl 6, at which point I'm
> hoping it'll be done in such a way that it'll make it easier to extract
> from the Perl mess around it and be made available to other projects.

I hope that even if the Parrot goal is to support all languages,
PerlIO can be made a separate and independent project, not relying on
Perl internals.  (And I'd change the name, but maybe that's asking too
much. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dan@sidhe.org  Sun Aug 12 21:10:38 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Sun, 12 Aug 2001 16:10:38 -0400 (EDT)
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <200108122015.QAA16295@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.LNX.4.21.0108121606420.1411-100000@tuatha.sidhe.org>

On Sun, 12 Aug 2001, Guido van Rossum wrote:

> > Perl 5.7.x has such a library, and it's my expectation (another sideways
> > glance at Dan) that if we did Parrot, it would include this library.
> > PerlIO attempts to be a very portable stdio (it's running on everything
> > that Perl runs on, AFAIK) and it supports "line disciplines" in the sfio
> > sense: you can stuff processing modules between the file and the
> > filehandle. This allows us to do anything between altering buffering
> > semantics, and decompressing and decrypting files on the fly.
> 
> Great!

Two of the other design requirements are good async I/O support and
good event handling support. (So there's something to handle the completed
async I/Os) 

Rumors that we're just filing the serial numbers off the VMS I/O system
are, of course, wildly exaggerated. ;-)
 
> > It'll need to be somewhat rewritten for Perl 6, at which point I'm
> > hoping it'll be done in such a way that it'll make it easier to extract
> > from the Perl mess around it and be made available to other projects.
> 
> I hope that even if the Parrot goal is to support all languages,
> PerlIO can be made a separate and independent project, not relying on
> Perl internals.  (And I'd change the name, but maybe that's asking too
> much. :-)

Nah, a name change would certainly be in order--we'd have to call it the
ParrotIO system. :) Making the core bits modular is one of the goals, for
no other reason than to make it easy to have someone responsible for its
maintenance and development that doesn't need to be intimately familiar
with the rest of the interpreter system. And so it'll still function
properly when we compile down to native code and don't actually *have* an
interpreter handy.

It ought to be an interesting project. Suggestions for features are, of
course, always welcome.

					Dan



From jack@oratrix.nl  Sun Aug 12 22:31:31 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Sun, 12 Aug 2001 23:31:31 +0200
Subject: [Python-Dev] Any NeXT users out there?
Message-ID: <20010812213136.8DA5C162E06@oratrix.oratrix.nl>

Are there any NeXT Python users left out there?

The reason I'm asking is that we're changing the --with-next-framework
support beyond all recognition to make it work on Mac OS X. I am
probably breaking what was there previously, BUT I'm pretty convinced
that that didn't work anyway as it stood. At least, according to all
rumour Mac OS X frameworks are very very similar to NeXT frameworks,
and the support that was in configure.in and Makefile.pre.in was
really no more than building a .dylib. There was no support to then
install this dylib (but that could be/have been in a separate script),
and moreover Python would never have been able to find the .dylib
(again, as far as I can tell with a few weeks of OSX experience and no
NeXT experience).
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | ++++ see http://www.xs4all.nl/~tank/ ++++


From jack@oratrix.nl  Sun Aug 12 23:17:19 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Mon, 13 Aug 2001 00:17:19 +0200
Subject: [Python-Dev] Feedback wanted on --with-next-framework patch
Message-ID: <20010812221724.35589162E06@oratrix.oratrix.nl>

I've put a patch to configure.in, Makefile.pre.in and
Modules/getpath.c into the patch manager, number 450351 (will pasting
URLs here work? The url is
https://sourceforge.net/tracker/index.php?func=detail&aid=450350&group_id=5470&atid=305470

I've assigned it to Martin because he's been helpful with configure
stuff in the past, but as we've added quite a few targets to the
Makefile (used for framework builds only) I can't be sure we didn't
break things on other platforms.

"we", by the way, is myself and Tony Lownds, who did a large part of
the work.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | ++++ see http://www.xs4all.nl/~tank/ ++++


From martin@loewis.home.cs.tu-berlin.de  Sun Aug 12 23:34:48 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Mon, 13 Aug 2001 00:34:48 +0200
Subject: [Python-Dev] Any NeXT users out there?
Message-ID: <200108122234.f7CMYml01363@mira.informatik.hu-berlin.de>

> The reason I'm asking is that we're changing the
> --with-next-framework support beyond all recognition to make it
> work on Mac OS X. I am probably breaking what was there previously,
> BUT I'm pretty convinced that that didn't work anyway as it stood.

Do you really need to call the option --with-next-framework? First, I
think it should be an --enable option, since you are not offering to
use or not to use some external tool or library. Furthermore, I find
it very confusing to have to invoke incantations involving NeXT when
compiling software for Mac OS. That Steve Jobs is head of the company
is *not* enough reason :-) Finally, if you are going to break what it
currently does, you may just as well remove the current code.

I don't know whether NeXT(Step?) should be supported anymore, but I
definitely know that MacOS should not be the vehicle to keep the NeXT
code forever. I also know that system specific code simpler if you
know what system it is intended to work on.

Out of curiosity: Just what is a framework? (Should that be Jack, what
is a framework ?-)

Regards,
Martin


From paulp@ActiveState.com  Mon Aug 13 00:41:51 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Sun, 12 Aug 2001 16:41:51 -0700
Subject: [Python-Dev] Parrot -- should life imitate satire?
References: <Pine.LNX.4.21.0108121606420.1411-100000@tuatha.sidhe.org>
Message-ID: <3B77143F.F9158ED1@ActiveState.com>

Dan Sugalski wrote:
> 
>...
> 
> Nah, a name change would certainly be in order--we'd have to call it the
> ParrotIO system. :) Making the core bits modular is one of the goals, for
> no other reason than to make it easy to have someone responsible for its
> maintenance and development that doesn't need to be intimately familiar
> with the rest of the interpreter system. And so it'll still function
> properly when we compile down to native code and don't actually *have* an
> interpreter handy.

Would it make sense to poll both communities for people interested in
helping with this effort and maybe making it a standalone project
(perhaps on SourceForge?)? 

In fact, there are many bits of the C library that Python people would
like to replace or improve but we don't have the resources alone. Maybe
if all of the scripting language groups got together we would be able to
do it. I'm not talking about the larger project of sharing bytecodes and
so forth, but I could see that we probably have very similar needs with
respect to files, memory allocation, networking, event loops, process
handling and maybe threads. And the other scripting languages are
probably not too far off either.
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From esr@thyrsus.com  Mon Aug 13 00:47:33 2001
From: esr@thyrsus.com (Eric S. Raymond)
Date: Sun, 12 Aug 2001 19:47:33 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <3B77143F.F9158ED1@ActiveState.com>; from paulp@ActiveState.com on Sun, Aug 12, 2001 at 04:41:51PM -0700
References: <Pine.LNX.4.21.0108121606420.1411-100000@tuatha.sidhe.org> <3B77143F.F9158ED1@ActiveState.com>
Message-ID: <20010812194733.A8000@thyrsus.com>

Paul Prescod <paulp@ActiveState.com>:
> Would it make sense to poll both communities for people interested in
> helping with this effort and maybe making it a standalone project
> (perhaps on SourceForge?)? 
> 
> In fact, there are many bits of the C library that Python people would
> like to replace or improve but we don't have the resources alone. Maybe
> if all of the scripting language groups got together we would be able to
> do it. I'm not talking about the larger project of sharing bytecodes and
> so forth, but I could see that we probably have very similar needs with
> respect to files, memory allocation, networking, event loops, process
> handling and maybe threads. And the other scripting languages are
> probably not too far off either.

I think this is a good idea in itself.  And would also be a good pilot
project for the Parrot unification.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

The direct use of physical force is so poor a solution to the problem of
limited resources that it is commonly employed only by small children and
great nations.
	-- David Friedman


From sterling@designmultimedia.com  Mon Aug 13 12:49:23 2001
From: sterling@designmultimedia.com (Sterling Hughes)
Date: Mon, 13 Aug 2001 07:49:23 -0400 (EDT)
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <3B77143F.F9158ED1@ActiveState.com>
Message-ID: <Pine.LNX.4.33L2.0108130746000.1252-100000@localhost.localdomain>

On Sun, 12 Aug 2001, Paul Prescod wrote:

> Dan Sugalski wrote:
> >
> >...
> >
> > Nah, a name change would certainly be in order--we'd have to call it the
> > ParrotIO system. :) Making the core bits modular is one of the goals, for
> > no other reason than to make it easy to have someone responsible for its
> > maintenance and development that doesn't need to be intimately familiar
> > with the rest of the interpreter system. And so it'll still function
> > properly when we compile down to native code and don't actually *have* an
> > interpreter handy.
>
> Would it make sense to poll both communities for people interested in
> helping with this effort and maybe making it a standalone project
> (perhaps on SourceForge?)?
>
> In fact, there are many bits of the C library that Python people would
> like to replace or improve but we don't have the resources alone. Maybe
> if all of the scripting language groups got together we would be able to
> do it. I'm not talking about the larger project of sharing bytecodes and
> so forth, but I could see that we probably have very similar needs with
> respect to files, memory allocation, networking, event loops, process
> handling and maybe threads. And the other scripting languages are
> probably not too far off either.
>
    I know that there are some PHP developers who are also thinking
    along the same lines -- myself for one. :)

    Also, another area this might reach is into the extension space,
    we're all developing the same extension for each of the different
    languages, it would be great to find a way to pool the resources of all
    the extension developers to create one extension together (ie, a singular
    XSLT extension, used in Perl, Python, PHP, etc.)

    -Sterling




From simon@netthink.co.uk  Mon Aug 13 01:06:33 2001
From: simon@netthink.co.uk (Simon Cozens)
Date: Mon, 13 Aug 2001 01:06:33 +0100
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <3B77143F.F9158ED1@ActiveState.com>; from paulp@ActiveState.com on Sun, Aug 12, 2001 at 04:41:51PM -0700
References: <Pine.LNX.4.21.0108121606420.1411-100000@tuatha.sidhe.org> <3B77143F.F9158ED1@ActiveState.com>
Message-ID: <20010813010633.A6537@deep-dark-truthful-mirror>

On Sun, Aug 12, 2001 at 04:41:51PM -0700, Paul Prescod wrote:
> Would it make sense to poll both communities for people interested in
> helping with this effort and maybe making it a standalone project
> (perhaps on SourceForge?)? 

Yes. But you'll have to talk to Nick (<nick@ing-simmons.net>) if you want to
play with the current code, since it's his baby. 

> In fact, there are many bits of the C library that Python people would
> like to replace or improve but we don't have the resources alone.

*nod*. Perl reimplements far too much of C for my liking. (aton, sprintf,
malloc [1] and friends.) 

I keep telling 'em to use glib.

[1] ... but then any C program over a given size reimplements malloc ...
-- 
The best book on programming for the layman is "Alice in Wonderland";
but that's because it's the best book on anything for the layman. 
    - Alan Perlis


From paulp@ActiveState.com  Mon Aug 13 02:43:35 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Sun, 12 Aug 2001 18:43:35 -0700
Subject: [Python-Dev] Free threading
References: <LNBBLJKPBEHFEDALKOLCAEJFLEAA.tim.one@home.com>
Message-ID: <3B7730C7.FE3983A2@ActiveState.com>

Tim Peters wrote:
> 
>...
> 
> Well, I'm not a Tcl guy, but Tcl has historically had no threads at all, yet
> did have an event model where the *illusion* of multiple threads could still
> be put to good use.  I wouldn't fall over dead with surprise if that's all
> there were to it:  Extreme Fear of Threads <wink>.

No, Tcl has real threads now. Tcl and Perl both share a model that the
Perl guys call "Ithreads". (for interpreter threads or independent
threads, I guess) where each interpreter is firewalled from the other
unless you ask to share information. Tcl's threads were driven by the
AOL guys' need to get massive scalability.

>...
> That's cool.  There isn't much code supporting multiple interpreters, so
> there's little to gain by getting rid of it now.  I just wish it were
> *used* -- and that it had a more compelling use case <wink>.

Why do people use mod_python, fcgi, mod_snake, mod_php and all of those
other things instead of pure CGI? The usual claim is that they dislike
the cost of forking a process and loading the interpreter code. Now
imagine a user with a 20 processor machine. She isn't going to be happy
with the price of forking processes and using IPC for information
sharing either. On the other hand, she isn't going to be happy with a
shared GIL.

Free threading helps, but if sharing data has a performance cost (e.g.
by requiring reference count operations to be locked, or requiring
mutexes on dictionaries) then you might not want to pay that cost
either. The Perl guys convinced me of that much.

The most popular "embedded scripting languages" (PHP and VBScript/ASP)
use this totally independent thread model. As far as I know, neither has
a concept of sharing information between threads. To a PHP programmer,
that's what SQL and browser cookies are for. :-)
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From swm@linuxworld.com.au  Mon Aug 13 03:04:32 2001
From: swm@linuxworld.com.au (Gavin Sherry)
Date: Mon, 13 Aug 2001 12:04:32 +1000 (EST)
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <Pine.LNX.4.33L2.0108130746000.1252-100000@localhost.localdomain>
Message-ID: <Pine.LNX.4.21.0108131149210.32372-100000@linuxworld.com.au>

On Mon, 13 Aug 2001, Sterling Hughes wrote:

> On Sun, 12 Aug 2001, Paul Prescod wrote:
> 
> > Dan Sugalski wrote:
> > >
> > >...
> > >
> > > Nah, a name change would certainly be in order--we'd have to call it the
> > > ParrotIO system. :) Making the core bits modular is one of the goals, for
> > > no other reason than to make it easy to have someone responsible for its
> > > maintenance and development that doesn't need to be intimately familiar
> > > with the rest of the interpreter system. And so it'll still function
> > > properly when we compile down to native code and don't actually *have* an
> > > interpreter handy.
> >
> > Would it make sense to poll both communities for people interested in
> > helping with this effort and maybe making it a standalone project
> > (perhaps on SourceForge?)?
> >


>     I know that there are some PHP developers who are also thinking
>     along the same lines -- myself for one. :)
> 
>     Also, another area this might reach is into the extension space,
>     we're all developing the same extension for each of the different
>     languages, it would be great to find a way to pool the resources of all
>     the extension developers to create one extension together (ie, a singular
>     XSLT extension, used in Perl, Python, PHP, etc.)

By introducing another layer of abstraction we will most probably decrease
the performance of all languages[1]. That is, since the universal
interface to, given your example, the Sablotron XSLT library is the API
and Perl, Python and PHP have internals different enough that there would
have to be a  certain amount of glue and translation work between the
structures of the extension layer and the internals of the interpretter.

Building extensions to these languages involves a lot of hack work I
agree. The reason for this is that it is merely connecting the wires
between an API and the language of interest. Anything more than that and
you'll be damaging the performance of Perl, Python, PHP...

[1] The situation where you could perhaps increase performance is the
converse of what is being suggested. Presumably, if it is in the interest
of language developers to have a unified extension layer, then the point
of different which language developers want to maintain with with other
languages is the syntax/grammar of the language. 

As I see it, along this line of thinking it makes sense to look at the
feasibility of a language system (internals and extension layer) which
handles plugable scripting engines. As such, the majority of work which
goes into language development can be concentrated while allowing, to the
users, automony in their choice of language.

> 
>     -Sterling
> 
> 

Gavin



From rasmus@php.net  Mon Aug 13 03:30:54 2001
From: rasmus@php.net (Rasmus Lerdorf)
Date: Sun, 12 Aug 2001 19:30:54 -0700 (PDT)
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <Pine.LNX.4.21.0108131149210.32372-100000@linuxworld.com.au>
Message-ID: <Pine.LNX.4.33.0108121917170.1239-100000@rasmus.php.net>

> By introducing another layer of abstraction we will most probably decrease
> the performance of all languages[1]. That is, since the universal
> interface to, given your example, the Sablotron XSLT library is the API
> and Perl, Python and PHP have internals different enough that there would
> have to be a  certain amount of glue and translation work between the
> structures of the extension layer and the internals of the interpretter.

It should still be possible to come up with some guidelines for writing a
single extension that supports multiple scripting languages.  Especially
for something like Sablotron which isn't that complex.  You feed it some
XML and some XSL and it spits back a result.  Most of the work in writing
the extension is figuring out how the user-visible API should look and
then translating that API along with its function/method calls and
argument type mangling to whatever the thing that is being glued is
expecting.

Note that I am not talking about runtime binary compatibility here.  I am
talking about source compatibility where potentially big chunks of code
would be very different across the different languages.  In an approach
like this I don't see a performance issue.  It is more of an education
issue actually.

And with a bit of effort I bet we could come up with enough common ground,
or at least some nifty macro tricks, where people wouldn't feel
overwhelmed by the task of supporting multiple scripting languages when
they sat down to write some new backend library.

Perhaps it would be a good experiment to take some really simple thing and
write Perl, Python, PHP, Ruby, and Tcl extensions for it and compare
notes.  See where the overlap is, see where the biggest divergence is, and
figure out if some of that divergence could be eliminated with some clever
tricks.

> [1] The situation where you could perhaps increase performance is the
> converse of what is being suggested. Presumably, if it is in the interest
> of language developers to have a unified extension layer, then the point
> of different which language developers want to maintain with with other
> languages is the syntax/grammar of the language.
>
> As I see it, along this line of thinking it makes sense to look at the
> feasibility of a language system (internals and extension layer) which
> handles plugable scripting engines. As such, the majority of work which
> goes into language development can be concentrated while allowing, to the
> users, automony in their choice of language.

Sure, but now you are back on the holy grail and CLR stuff again.  Baby
steps...

-Rasmus



From skip@pobox.com (Skip Montanaro)  Sat Aug 11 03:43:44 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Fri, 10 Aug 2001 21:43:44 -0500
Subject: [Python-Dev] Silly pystones question
Message-ID: <15220.39904.966751.603159@beluga.mojam.com>

In pystones.Proc0, most of the variables declared global are never assigned.
Was that just slavish adherence to the style of code from which it was
adapted or was the code changed at some point in the past, making the global
declarations superfluous?  Seems to me that the only time the global keyword
should be used is when an assignment to the variable occurs in the code and
you want to make sure you're modifying the global name.

This is obviously a silly piece of code in which to question this technique,
but I was looking at it to gauge the effects of possible global object
access optimization.  My first assumption was that a variable declared
global wouldn't be subject to any optimization, but now I realize I'd have
to look for variables that were operands of LOAD_GLOBAL instructions but not
STORE_GLOBAL instructions.

Skip



From otaylor@redhat.com  Mon Aug 13 04:05:43 2001
From: otaylor@redhat.com (Owen Taylor)
Date: 12 Aug 2001 23:05:43 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: Rasmus Lerdorf's message of "Sun, 12 Aug 2001 19:30:54 -0700 (PDT)"
References: <Pine.LNX.4.33.0108121917170.1239-100000@rasmus.php.net>
Message-ID: <ybelmkooe0o.fsf@fresnel.labs.redhat.com>

Rasmus Lerdorf <rasmus@php.net> writes:

> > By introducing another layer of abstraction we will most probably decrease
> > the performance of all languages[1]. That is, since the universal
> > interface to, given your example, the Sablotron XSLT library is the API
> > and Perl, Python and PHP have internals different enough that there would
> > have to be a  certain amount of glue and translation work between the
> > structures of the extension layer and the internals of the interpretter.
> 
> It should still be possible to come up with some guidelines for writing a
> single extension that supports multiple scripting languages.  Especially
> for something like Sablotron which isn't that complex.  You feed it some
> XML and some XSL and it spits back a result.  Most of the work in writing
> the extension is figuring out how the user-visible API should look and
> then translating that API along with its function/method calls and
> argument type mangling to whatever the thing that is being glued is
> expecting.
> 
> Note that I am not talking about runtime binary compatibility here.  I am
> talking about source compatibility where potentially big chunks of code
> would be very different across the different languages.  In an approach
> like this I don't see a performance issue.  It is more of an education
> issue actually.
> 
> And with a bit of effort I bet we could come up with enough common ground,
> or at least some nifty macro tricks, where people wouldn't feel
> overwhelmed by the task of supporting multiple scripting languages when
> they sat down to write some new backend library.

I think knowledge of the rules for writing bindable libraries is
much more useful than macro tricks.

Writing down the rules would be a big project, though the set
of rules isn't that big. A few important ones are:

 - User data stored with a structure should have an associated
   notifier.

 - Callback functions must take not just a function, but 
   a function,user data, destroy notifier combination.
  
 - Data structures should be one of:

   * Immutable and refcounted. 
   * Copy by value.
   * Copy by reference, reference counted, with destroy
     notification and a user data facility for maintaining 
     persistant language <=> library associates.

 - Keep the C simple. Opaque structure with accessors is
   the ideal, structures initialized with magic macros
   one of the worse offendors.

Etc. If you follow these rules, you get a a library where writing
bindings for it is a brain-dead task. But still task.  A library like
GObject that standardizes memory management, and object-oriented
structures helps quite a bit, but I'm certainly of the opinion that
the eventual goal has to be

 - write a library using tools that produce an introspectable
   descriptions of its interfaces.
  
 - Language bindings are automatic and can be done without
   a compilation step.

Regards,
                                        Owen


From otaylor@redhat.com  Mon Aug 13 04:10:02 2001
From: otaylor@redhat.com (Owen Taylor)
Date: 12 Aug 2001 23:10:02 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: Owen Taylor's message of "12 Aug 2001 23:05:43 -0400"
References: <Pine.LNX.4.33.0108121917170.1239-100000@rasmus.php.net>
 <ybelmkooe0o.fsf@fresnel.labs.redhat.com>
Message-ID: <ybehevcodth.fsf@fresnel.labs.redhat.com>

Owen Taylor <otaylor@redhat.com> writes:

> Rasmus Lerdorf <rasmus@php.net> writes:
> 
> > > By introducing another layer of abstraction we will most probably decrease
> > > the performance of all languages[1]. That is, since the universal
> > > interface to, given your example, the Sablotron XSLT library is the API
> > > and Perl, Python and PHP have internals different enough that there would
> > > have to be a  certain amount of glue and translation work between the
> > > structures of the extension layer and the internals of the interpretter.
> > 
> > It should still be possible to come up with some guidelines for writing a
> > single extension that supports multiple scripting languages.  Especially
> > for something like Sablotron which isn't that complex.  You feed it some
> > XML and some XSL and it spits back a result.  Most of the work in writing
> > the extension is figuring out how the user-visible API should look and
> > then translating that API along with its function/method calls and
> > argument type mangling to whatever the thing that is being glued is
> > expecting.
> > 
> > Note that I am not talking about runtime binary compatibility here.  I am
> > talking about source compatibility where potentially big chunks of code
> > would be very different across the different languages.  In an approach
> > like this I don't see a performance issue.  It is more of an education
> > issue actually.
> > 
> > And with a bit of effort I bet we could come up with enough common ground,
> > or at least some nifty macro tricks, where people wouldn't feel
> > overwhelmed by the task of supporting multiple scripting languages when
> > they sat down to write some new backend library.
> 
> I think knowledge of the rules for writing bindable libraries is
> much more useful than macro tricks.

[...]

> Etc. If you follow these rules, you get a a library where writing
> bindings for it is a brain-dead task. But still task.  A library like
> GObject that standardizes memory management, and object-oriented
> structures helps quite a bit, but I'm certainly of the opinion that
> the eventual goal has to be
> 
>  - write a library using tools that produce an introspectable
>    descriptions of its interfaces.
>   
>  - Language bindings are automatic and can be done without
>    a compilation step.

Oh yes, and the thing I forgot to mention:

 Write a library and the language bindings will come. The problem
 is getting a vast number of language bindings (the list of 38 
 language  bindings for GTK+ at http://erik.bagfors.nu/gnome/languages.html
 is almost certainly not even complete). The problem is keeping language 
 bindings complete and up-to-date. This is the biggest reason
 why automation is crucual.

Regards,
                                        Owen


From rasmus@php.net  Mon Aug 13 04:17:13 2001
From: rasmus@php.net (Rasmus Lerdorf)
Date: Sun, 12 Aug 2001 20:17:13 -0700 (PDT)
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <ybelmkooe0o.fsf@fresnel.labs.redhat.com>
Message-ID: <Pine.LNX.4.33.0108122011440.1239-100000@rasmus.php.net>

>  - write a library using tools that produce an introspectable
>    descriptions of its interfaces.
>
>  - Language bindings are automatic and can be done without
>    a compilation step.

Of course, but this doesn't help much for all the legacy libraries out
there that we all have to deal with.  We all have the rather thankless
task of having to figure out stuff like liboci8.  It would be very nice if
that work could be centralized into a single effort.

But I agree, if such rules were written down and actually abided by in
third-party libraries, our lives would be much easier.

Perhaps something like swig is as far as we can go on the legacy libs
though.

-Rasmus



From sterling@designmultimedia.com  Mon Aug 13 16:21:30 2001
From: sterling@designmultimedia.com (Sterling Hughes)
Date: Mon, 13 Aug 2001 11:21:30 -0400 (EDT)
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <ybelmkooe0o.fsf@fresnel.labs.redhat.com>
Message-ID: <Pine.LNX.4.33L2.0108131110520.1252-100000@localhost.localdomain>

On 12 Aug 2001, Owen Taylor wrote:

>
> Rasmus Lerdorf <rasmus@php.net> writes:
>
> > > By introducing another layer of abstraction we will most probably decrease
> > > the performance of all languages[1]. That is, since the universal
> > > interface to, given your example, the Sablotron XSLT library is the API
> > > and Perl, Python and PHP have internals different enough that there would
> > > have to be a  certain amount of glue and translation work between the
> > > structures of the extension layer and the internals of the interpretter.
> >
> > It should still be possible to come up with some guidelines for writing a
> > single extension that supports multiple scripting languages.  Especially
> > for something like Sablotron which isn't that complex.  You feed it some
> > XML and some XSL and it spits back a result.  Most of the work in writing
> > the extension is figuring out how the user-visible API should look and
> > then translating that API along with its function/method calls and
> > argument type mangling to whatever the thing that is being glued is
> > expecting.
> >
> > Note that I am not talking about runtime binary compatibility here.  I am
> > talking about source compatibility where potentially big chunks of code
> > would be very different across the different languages.  In an approach
> > like this I don't see a performance issue.  It is more of an education
> > issue actually.
> >
> > And with a bit of effort I bet we could come up with enough common ground,
> > or at least some nifty macro tricks, where people wouldn't feel
> > overwhelmed by the task of supporting multiple scripting languages when
> > they sat down to write some new backend library.
>
> I think knowledge of the rules for writing bindable libraries is
> much more useful than macro tricks.
>
> Writing down the rules would be a big project, though the set
> of rules isn't that big. A few important ones are:
>
>  - User data stored with a structure should have an associated
>    notifier.
>
>  - Callback functions must take not just a function, but
>    a function,user data, destroy notifier combination.
>
>  - Data structures should be one of:
>
>    * Immutable and refcounted.
>    * Copy by value.
>    * Copy by reference, reference counted, with destroy
>      notification and a user data facility for maintaining
>      persistant language <=> library associates.
>
>  - Keep the C simple. Opaque structure with accessors is
>    the ideal, structures initialized with magic macros
>    one of the worse offendors.
>


    Yes! Yes! Yes!

    Its surprising how many library authors seem to ignore this.

> Etc. If you follow these rules, you get a a library where writing
> bindings for it is a brain-dead task. But still task.  A library like
> GObject that standardizes memory management, and object-oriented
> structures helps quite a bit, but I'm certainly of the opinion that
> the eventual goal has to be
>
>  - write a library using tools that produce an introspectable
>    descriptions of its interfaces.
>
>  - Language bindings are automatic and can be done without
>    a compilation step.


    Not quite sure I understand you here -- write a library using tools
    that produce an introspectable descriptions of its interface???
    context?

    Are you talking about a library that is written so that it can be
    analyzed by a program and then extension bindings for the individual
    language can be created?

    -Sterling



From sterling@designmultimedia.com  Mon Aug 13 16:40:23 2001
From: sterling@designmultimedia.com (Sterling Hughes)
Date: Mon, 13 Aug 2001 11:40:23 -0400 (EDT)
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <Pine.LNX.4.33.0108121917170.1239-100000@rasmus.php.net>
Message-ID: <Pine.LNX.4.33L2.0108131122020.1252-100000@localhost.localdomain>

On Sun, 12 Aug 2001, Rasmus Lerdorf wrote:

> > By introducing another layer of abstraction we will most probably decrease
> > the performance of all languages[1]. That is, since the universal
> > interface to, given your example, the Sablotron XSLT library is the API
> > and Perl, Python and PHP have internals different enough that there would
> > have to be a  certain amount of glue and translation work between the
> > structures of the extension layer and the internals of the interpretter.
>
> It should still be possible to come up with some guidelines for writing a
> single extension that supports multiple scripting languages.  Especially
> for something like Sablotron which isn't that complex.  You feed it some
> XML and some XSL and it spits back a result.  Most of the work in writing
> the extension is figuring out how the user-visible API should look and
> then translating that API along with its function/method calls and
> argument type mangling to whatever the thing that is being glued is
> expecting.
>

    Well -- Sablotron is more of an annoyance in this regard than
    anything -- if you want to support SAX, Scheme, error handling, log
    handling and such, you need to register a million callbacks and then
    have each of these callbacks either do something useful, or call
    some user space function. puhhhh.

    The PHP-XSLT extension, all in all is 2525 lines of code, Makefiles,
    and explanations, 2525 more than I would've liked to write
    (http://cvs.php.net/cvs.php/php4/ext/xslt)...

    Using PHP-Sablotron as an example is something where having a
    couple of minds is better...  The first version of the extension
    (http://cvs.php.net/cvs.php/php4/ext/sablot), left, uhhmm, something
    to be desired -- the api was a bit hard to understand, and it made
    code maintainability within the extension something close to hell.
    Perhaps if there were a couple more minds working on the extension,
    some of the mistakes of the original version might not have been
    present (perhaps if I had slept the weekend that I wrote it, that
    might have happened as well, who knows? :)

    Anyway, just counting now -- 4530 lines of "stuff" :) have been
    written on the PHP end of things for bring sablotron support in
    (actually, the newer XSLT extension will also support libxslt and
    Xalan, so some of its bloat for those extra libraries, but..)  I
    imagine that on the Perl and Python ends at least 1,000 lines of
    code have been written on each side.  Wouldn't it be spiffy if this
    only had to be done once?  Hell, if there's a singular extension
    api, as you say, vendors may just write to that spec, and we don't
    have to worry about writing it at all!

> Note that I am not talking about runtime binary compatibility here.  I am
> talking about source compatibility where potentially big chunks of code
> would be very different across the different languages.  In an approach
> like this I don't see a performance issue.  It is more of an education
> issue actually.
>
> And with a bit of effort I bet we could come up with enough common ground,
> or at least some nifty macro tricks, where people wouldn't feel
> overwhelmed by the task of supporting multiple scripting languages when
> they sat down to write some new backend library.
>
> Perhaps it would be a good experiment to take some really simple thing and
> write Perl, Python, PHP, Ruby, and Tcl extensions for it and compare
> notes.  See where the overlap is, see where the biggest divergence is, and
> figure out if some of that divergence could be eliminated with some clever
> tricks.
>

    I think this would be interesting at least -- I'll write the PHP
    extension (Hell, I already have,
    http://cvs.php.net/cvs.php/php4/ext/zip, it don't get easier than
    that :)

    -Sterling




From tim.one@home.com  Mon Aug 13 04:46:06 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 12 Aug 2001 23:46:06 -0400
Subject: [Python-Dev] Silly pystones question
In-Reply-To: <15220.39904.966751.603159@beluga.mojam.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEGBLFAA.tim.one@home.com>

[Skip Montanaro]
> In pystones.Proc0, most of the variables declared global are
> never assigned.  Was that just slavish adherence to the style of code
> from which it was adapted or was the code changed at some point in
> the past, making the global declarations superfluous?

You're a developer, Skip:  it's time you learned how to use CVS <wink>:

<http://cvs.sf.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Lib/test/pysto
ne.py>

The evidence there says it's always been this way.

> Seems to me that the only time the global keyword should be used is
> when an assignment to the variable occurs in the code and you want
> to make sure you're modifying the global name.

Well, like any other declaration, it can also have documentation value; and
sometimes you'll see every function in a module repeat the same global decl.
"just for consistency" (I guess).

> This is obviously a silly piece of code in which to question this
> technique, but I was looking at it to gauge the effects of possible
> global object access optimization.  My first assumption was that a
> variable declared global wouldn't be subject to any optimization, but
> now I realize I'd have to look for variables that were operands of
> LOAD_GLOBAL instructions but not STORE_GLOBAL instructions.

Until The Rules change, any global (or builtin) is subject to rebinding at
any time, even if there's no STORE_GLOBAL in sight.  The "danger" across
call sites is clear enough (a Python global is like a C extern), but in the
presence of threads another thread can reach into the module and rebind its
globals between any pair of byte codes (which C worms around via X3J11
refusing to say anthing at all about threads <0.9 wink>).



From tim.one@home.com  Mon Aug 13 05:26:05 2001
From: tim.one@home.com (Tim Peters)
Date: Mon, 13 Aug 2001 00:26:05 -0400
Subject: [Python-Dev] Free threading
In-Reply-To: <3B7730C7.FE3983A2@ActiveState.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEGCLFAA.tim.one@home.com>

[Tim]
> Well, I'm not a Tcl guy, but Tcl has historically had no threads at
> all ...

[Paul Prescod]
> No, Tcl has real threads now.

"Now", yes, not "historically".  When Guido said that multiple interpreters
were the #1 request from Tcl-land, I'm sure he was talking about pre-thread
Tcl-land.

> Tcl and Perl both share a model that the Perl guys call "Ithreads".
> (for interpreter threads or independent threads, I guess) where each
> interpreter is firewalled from the other unless you ask to share
> information. Tcl's threads were driven by the AOL guys' need to get
> massive scalability.

You can't get massive scalability using OS threads, unless the OS is an
oddball specifically designed for that.  Worming around the bloat of Windows
and Linux (etc) OS threads is Stackless Python's natural domain (simulated
threads).  The catch, of course, is they that don't run in parallel!  Or
maybe by "massive" you mean a few dozen?  I worked in the "massively
parallel supercomputer" biz, so I'm thinking more on the order of a few
thousands of dozens.  I bet you're not.

> ...
> Why do people use mod_python, fcgi, mod_snake, mod_php and all of those
> other things instead of pure CGI?

Beats me why people bother with web programming at all <wink>.

> The usual claim is that they dislike the cost of forking a process
> and loading the interpreter code.  Now imagine a user with a 20
> processor machine. She isn't going to be happy with the price of
> forking processes and using IPC for information sharing either. On
> the other hand, she isn't going to be happy with a shared GIL.

The cost of creating 20 processes is trivial "even on Windows" if you only
need to do that once.  Process-based (as opposed to thread-based) solutions
are unnatural on Windows, though, and part of this argument seems cultural
in related ways.

> Free threading helps, but if sharing data has a performance cost (e.g.
> by requiring reference count operations to be locked, or requiring
> mutexes on dictionaries) then you might not want to pay that cost
> either. The Perl guys convinced me of that much.

IIRC, Greg's fabled free-threading version of Python took a speed hit of
about a factor of 2 (for a program using only 1 thread, compared to that
same program without the free-threading patches).

> The most popular "embedded scripting languages" (PHP and VBScript/ASP)
> use this totally independent thread model. As far as I know, neither has
> a concept of sharing information between threads. To a PHP programmer,
> that's what SQL and browser cookies are for. :-)

Well, the idea that threads don't share information is foreign to every
intense belief about the world *I've* ever been paid to adopt <wink>, and
I'm not enough of a Windows geek to believe "threads are always the answer"
even so.  A pool of worker processes using OS-specific IPC as needed works
great in my real-life experience, and if information sharing is rare, works
especially great because it's not *fighting* the OS and C libraries tooth
and nail.

What of your hypothetical user earlier, who "isn't going to be happy with
the price of ... using IPC for information sharing either"?  That is, in
what sense do isolated threads leave her happy about her information sharing
needs?  If she's happy to communicate via database transactions and queries,
fear of IPC being too expensive wouldn't be rational.



From otaylor@redhat.com  Mon Aug 13 05:46:27 2001
From: otaylor@redhat.com (Owen Taylor)
Date: 13 Aug 2001 00:46:27 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: Sterling Hughes's message of "Mon, 13 Aug 2001 11:21:30 -0400 (EDT)"
References: <Pine.LNX.4.33L2.0108131110520.1252-100000@localhost.localdomain>
Message-ID: <ybe4rrcmusc.fsf@fresnel.labs.redhat.com>

Sterling Hughes <sterling@designmultimedia.com> writes:

> > Etc. If you follow these rules, you get a a library where writing
> > bindings for it is a brain-dead task. But still task.  A library like
> > GObject that standardizes memory management, and object-oriented
> > structures helps quite a bit, but I'm certainly of the opinion that
> > the eventual goal has to be
> >
> >  - write a library using tools that produce an introspectable
> >    descriptions of its interfaces.
> >
> >  - Language bindings are automatic and can be done without
> >    a compilation step.
> 
> 
>     Not quite sure I understand you here -- write a library using tools
>     that produce an introspectable descriptions of its interface???
>     context?
> 
>     Are you talking about a library that is written so that it can be
>     analyzed by a program and then extension bindings for the individual
>     language can be created?

Sorry for the vagueness, just trying to be very generic.

There are multiple ways of "automatically" getting information
about the interfaces library/module, some being:

 - Generate the headers for the library from an IDL file,
   that can later be used (directly or via a typelib)
   for generating language bindings. (example XPCOM) 

 - Produce interface information as part of the compilation process.
   (example .NET)

 - Parse header files to determine the interface. (examples: SWIG,
   most GTK+ bindings) 

Once you have that information, you can:

 - Compile an extension module for a particular language.

 - Generate bindings on the fly using dynamic invocation
   (XptCall, libffi, etc.)

In that space of possiiblities, I tend to favor avoiding
header parsing, and dynamic bindings rather than static
bindings. 

Regards,
                                        Owen


From jack@oratrix.nl  Mon Aug 13 10:52:50 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Mon, 13 Aug 2001 11:52:50 +0200
Subject: [Python-Dev] Any NeXT users out there?
In-Reply-To: Message by "Martin v. Loewis" <martin@loewis.home.cs.tu-berlin.de>
 ,
 Mon, 13 Aug 2001 00:34:48 +0200 , <200108122234.f7CMYml01363@mira.informatik.hu-berlin.de>
Message-ID: <20010813095250.7F568303181@snelboot.oratrix.nl>

> Do you really need to call the option --with-next-framework? First, I
> think it should be an --enable option, since you are not offering to
> use or not to use some external tool or library.

I'm unsure, I've never fully understood the intention behind --with and 
--enable. For instance, why is it "--enable-unicode" and "--with-threads"?

I agree that the "next" bit should probably go, leaving "--with-framework" or 
"--enable-framework".

> Furthermore, I find
> it very confusing to have to invoke incantations involving NeXT when
> compiling software for Mac OS. That Steve Jobs is head of the company
> is *not* enough reason :-)

The alternative is of course that we also provide a 
"--with-previous-framework". Any suggestions as to what it should do? :-)

> Out of curiosity: Just what is a framework? (Should that be Jack, what
> is a framework ?-)

On a low level a framework is really little more than a directory containing a 
collection of directories and symlinks, with a bit of support in the 
compiler/linker (-framework option) and the dynamic loader (searching for 
frameworks over a search path).

On a high level it is a very nifty way to package a shared library complete 
with all its supporting files. These supporting files range from things used 
at runtime (like the python Lib directory) to the C header files needed when 
you want to link against the library to the documentation. Everything is in a 
single directory, so install/uninstall is a breeze with no danger of missing 
things. Also, the structure of a framework allows you to have older, 
incompatible versions of the library (and support documents) in the framework 
as well, with these older versions only used by programs that were linked 
against the framework when that version was current.

A framework is really an instance of a bundle. The other type of bundle is an 
Application Bundle, which is (you guessed it) a directory containing an 
application binary together with all of its support files. A very nifty 
feature is that you can put the frameworks that an application uses into the 
application bundle, thereby forestalling problems like Windows dll-hell. The 
application will even pick up newer, compatible, versions of the framework 
from the system directories if they happen to be there.

The framework patch in sourceforge is the first step. The next step is to 
create a Python application bundle. We can then use a single trick in the 
Python main() program, where it will check to see whether it's being run from 
an application bundle and, if so, whether there's a magic file in the bundle 
(__main__.py comes to mind). If there is Python will run that file as the 
script. This will give Python users "freeze" functionality without using a 
compiler. (MacPython users have enjoyed this functionality for quite some time 
already, with BuildApplet and BuildApplication).

And, of course, if we take the Python application bundle with the __main__.py 
in it and stuff the Python framework into its Frameworks subdirectory we have 
a complete standalone application that can be drag-drop installed without 
having to touch a C compiler, without having to install Python, and which 
won't even interfere if you happen to have an incompatible version of Python 
installed already. And you can uninstall it by dragging it to the trash 
without fear that you will wreck your existing Python installation in the 
process!

Once all this is in place (1-2 weeks, I think) I would really like to try it 
with a Real World application. Zope is the first one that comes to mind, but 
there may be other options. I'd like to hear if someone wants to put some work 
into this.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 




From thomas@xs4all.net  Mon Aug 13 11:09:10 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Mon, 13 Aug 2001 12:09:10 +0200
Subject: [Python-Dev] Any NeXT users out there?
In-Reply-To: <20010813095250.7F568303181@snelboot.oratrix.nl>
References: <martin@loewis.home.cs.tu-berlin.de> <200108122234.f7CMYml01363@mira.informatik.hu-berlin.de> <20010813095250.7F568303181@snelboot.oratrix.nl>
Message-ID: <20010813120910.J874@xs4all.nl>

On Mon, Aug 13, 2001 at 11:52:50AM +0200, Jack Jansen wrote:

> > Do you really need to call the option --with-next-framework? First, I
> > think it should be an --enable option, since you are not offering to
> > use or not to use some external tool or library.

> I'm unsure, I've never fully understood the intention behind --with and 
> --enable. For instance, why is it "--enable-unicode" and "--with-threads"?

Like Martin explained earlier, '--with' is for options that require or might
require an external package or library, and '--enable' is for enabling a
feature wholy contained in The Software. So you have
--enable-unicode(=type) and --with-threads(=external_threads_package).

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From guido@python.org  Mon Aug 13 11:39:15 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Aug 2001 06:39:15 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: Your message of "Sun, 12 Aug 2001 19:47:33 EDT."
 <20010812194733.A8000@thyrsus.com>
References: <Pine.LNX.4.21.0108121606420.1411-100000@tuatha.sidhe.org> <3B77143F.F9158ED1@ActiveState.com>
 <20010812194733.A8000@thyrsus.com>
Message-ID: <200108131039.GAA17322@cj20424-a.reston1.va.home.com>

Can I make a request?  Can we *not* crosspost between language-dev and
python-dev?  The whole point of language-dev is to keep the traffic
*out* of the language-specific lists.  Please trim your "to" lines.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Aug 13 11:44:00 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Aug 2001 06:44:00 -0400
Subject: [Python-Dev] Silly pystones question
In-Reply-To: Your message of "Fri, 10 Aug 2001 21:43:44 CDT."
 <15220.39904.966751.603159@beluga.mojam.com>
References: <15220.39904.966751.603159@beluga.mojam.com>
Message-ID: <200108131044.GAA17367@cj20424-a.reston1.va.home.com>

Whatever you do, *don't* succumb to the temptation of "fixing"
pystone.  It's a benchmark and unless it stays the same there's no way
to compare to previous versions.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jack@oratrix.nl  Mon Aug 13 13:42:30 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Mon, 13 Aug 2001 14:42:30 +0200
Subject: [Python-Dev] Any NeXT users out there?
In-Reply-To: Message by Thomas Wouters <thomas@xs4all.net> ,
 Mon, 13 Aug 2001 12:09:10 +0200 , <20010813120910.J874@xs4all.nl>
Message-ID: <20010813124231.6C945303181@snelboot.oratrix.nl>

> On Mon, Aug 13, 2001 at 11:52:50AM +0200, Jack Jansen wrote:
> 
> > > Do you really need to call the option --with-next-framework? First, I
> > > think it should be an --enable option, since you are not offering to
> > > use or not to use some external tool or library.
> 
> > I'm unsure, I've never fully understood the intention behind --with and 
> > --enable. For instance, why is it "--enable-unicode" and "--with-threads"?
> 
> Like Martin explained earlier, '--with' is for options that require or might
> require an external package or library, and '--enable' is for enabling a
> feature wholy contained in The Software. So you have
> --enable-unicode(=type) and --with-threads(=external_threads_package).

Ah, thanks, that means that it should with "--with-framework", because there's 
external stuff involved.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 




From thomas@xs4all.net  Mon Aug 13 14:24:13 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Mon, 13 Aug 2001 15:24:13 +0200
Subject: [Python-Dev] Any NeXT users out there?
In-Reply-To: <20010813124231.6C945303181@snelboot.oratrix.nl>
References: <thomas@xs4all.net> <20010813120910.J874@xs4all.nl> <20010813124231.6C945303181@snelboot.oratrix.nl>
Message-ID: <20010813152413.N872@xs4all.nl>

On Mon, Aug 13, 2001 at 02:42:30PM +0200, Jack Jansen wrote:

> > Like Martin explained earlier, '--with' is for options that require or might
> > require an external package or library, and '--enable' is for enabling a
> > feature wholy contained in The Software. So you have
> > --enable-unicode(=type) and --with-threads(=external_threads_package).

> Ah, thanks, that means that it should with "--with-framework", because there's 
> external stuff involved.

There's almost always external stuff involved :) the question is whether you
need to be able to say
'--with-framework=<external_library_to_link_with_or_directory_find_auxiliary_files_in>',
or '--enable-framework=<style_of_framework>'. Maybe the first is the case,
but from what I've seen, it's more like the second.

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From guido@python.org  Mon Aug 13 14:40:06 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Aug 2001 09:40:06 -0400
Subject: [Python-Dev] yield without future statement?
Message-ID: <200108131340.JAA17834@cj20424-a.reston1.va.home.com>

On c.l.py, there's a small discussion going on the necessity of a
future statement to enable the yield keyword.  The general opinion
seems to be that the future statement is redundant, since previous
uses of yield will generally result in a syntax error -- and there
aren't very many uses of yield in the first place.  The other uses of
future (nested scopes and division) were needed because these features
cause *silent* failure -- but with yield, you'd have to work really
really hard to cause a silent failure.

Opinions?  I'd love to get rid of this particular future statement.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jack@oratrix.nl  Mon Aug 13 14:44:17 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Mon, 13 Aug 2001 15:44:17 +0200
Subject: [Python-Dev] Any NeXT users out there?
In-Reply-To: Message by Thomas Wouters <thomas@xs4all.net> ,
 Mon, 13 Aug 2001 15:24:13 +0200 , <20010813152413.N872@xs4all.nl>
Message-ID: <20010813134417.99842303181@snelboot.oratrix.nl>

> There's almost always external stuff involved :) the question is whether you
> need to be able to say
> '--with-framework=<external_library_to_link_with_or_directory_find_auxiliary_files_in>',
> or '--enable-framework=<style_of_framework>'. Maybe the first is the case,
> but from what I've seen, it's more like the second.

Hey, I'm easy! --enable-framework it is, so.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 




From skip@pobox.com (Skip Montanaro)  Mon Aug 13 14:47:33 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Mon, 13 Aug 2001 08:47:33 -0500
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: <200108101515.LAA18907@cj20424-a.reston1.va.home.com>
References: <3B72EF79.7BE8F6C1@metaslash.com>
 <200108092052.f79KqQg13802@odiug.digicool.com>
 <3B73F824.82A53539@metaslash.com>
 <200108101515.LAA18907@cj20424-a.reston1.va.home.com>
Message-ID: <15223.55925.912142.77948@beluga.mojam.com>

    >> [ importing to scan snipped ]
    >> 
    >> Yes, this is one of the worst problems I know of.  I plan on changing
    >> this.

    Guido> If you don't want to give up scanning the bytecode (if that's
    Guido> what you do), you could use compile() on the source code to get
    Guido> the bytecode for the entire module.

I think we may need to start thinking about some standard interface to byte
code scanning.  (I keep thinking about Rattlesnake.)  Perhaps there are
standard tasks people are performing that could be abstracted into a
standard module (for example, getting a list of global variables that are
set by a function, or getting first set and first read points for particular
variables).  One complaint I saw voiced about the Rattlesnake idea was that
code that scans byte code would break.

Skip



From guido@python.org  Mon Aug 13 14:49:14 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Aug 2001 09:49:14 -0400
Subject: [Python-Dev] Any NeXT users out there?
In-Reply-To: Your message of "Mon, 13 Aug 2001 15:24:13 +0200."
 <20010813152413.N872@xs4all.nl>
References: <thomas@xs4all.net> <20010813120910.J874@xs4all.nl> <20010813124231.6C945303181@snelboot.oratrix.nl>
 <20010813152413.N872@xs4all.nl>
Message-ID: <200108131349.JAA17882@cj20424-a.reston1.va.home.com>

I've got to say, the distinction between --with and --enable, no
matter how thoretically pure, in practice seems extremely fuzzy.

Quoting Neil Schemenauer quoting the autoconf docs
(http://mail.python.org/pipermail/python-dev/2001-January/011938.html):

|     If a software package has optional compile-time features, the
|     user can give `configure' command line options to specify
|     whether to compile them. The options have one of these forms:
| 
|         --enable-FEATURE[=ARG]
|         --disable-FEATURE
| 
|     Some packages require, or can optionally use, other software
|     packages which are already installed.  The user can give
|     `configure' command line options to specify which such
|     external software to use.  The options have one of these
|     forms:
| 
|         --with-package[=ARG]
|         --without-package

Now, for example, should it be --with-ipv6 or --enable-ipv6?  I would
think --with, since this is clearly depending on optional external
software.  Yet, it is one of the few configure options that use
--enable.

I am seriously considering standardizing on --with, ignoring the
autoconf docs.  Given how vague the distinction is, I'm sure most
users will find the distinction completely arbitrary even if we
applied the rules correctly.  They would be better served with a
consistent rule they can remember, like "always use --with/--without".

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com (Skip Montanaro)  Mon Aug 13 14:53:23 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Mon, 13 Aug 2001 08:53:23 -0500
Subject: [Python-Dev] ++x oddnes
In-Reply-To: <3B7542D1.3E38E579@lemburg.com>
References: <15219.64534.411625.193775@nyx.dyndns.org>
 <200108101523.LAA19012@cj20424-a.reston1.va.home.com>
 <3B741351.C7C2E2B8@ActiveState.com>
 <3B7542D1.3E38E579@lemburg.com>
Message-ID: <15223.56275.968603.294087@beluga.mojam.com>

    mal> David Ascher wrote:
    >> 
    >> Guido van Rossum wrote:
    >> 
    >> > IMO not worth it changing the language.  Train yourself to write x++
    >> > instead. :-)
    >> 
    >> Sounds like an excellent one for PyChecker =)

    mal> Why ? ++x can be put to some real use: I have a counter type which
    mal> actually uses ++x to increment the counter.

(I've been away for a few days, so perhaps this has been covered already.
I'm still catching up.)

How?  ++x compiles to

         LOAD_FAST         x
         UNARY_POSITIVE      
         UNARY_POSITIVE      

I don't see any incrementing going on...

Skip


From guido@python.org  Mon Aug 13 14:56:51 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Aug 2001 09:56:51 -0400
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: Your message of "Mon, 13 Aug 2001 08:47:33 CDT."
 <15223.55925.912142.77948@beluga.mojam.com>
References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> <3B73F824.82A53539@metaslash.com> <200108101515.LAA18907@cj20424-a.reston1.va.home.com>
 <15223.55925.912142.77948@beluga.mojam.com>
Message-ID: <200108131356.JAA17936@cj20424-a.reston1.va.home.com>

> I think we may need to start thinking about some standard interface to byte
> code scanning.  (I keep thinking about Rattlesnake.)  Perhaps there are
> standard tasks people are performing that could be abstracted into a
> standard module (for example, getting a list of global variables that are
> set by a function, or getting first set and first read points for particular
> variables).  One complaint I saw voiced about the Rattlesnake idea was that
> code that scans byte code would break.

Code that scans bytecode is well aware that it can break.  People have
often asked me about the stability of the bytecode, and I've always
responded that it changes in every release, and is considered an
internal implementation detail.

A bytecode scanning API might be a good idea, but it might also end up
being restrictive: it's hard to design such an API without making
*some* assumptions about the general bytecode architecture, and a
future implementation might violate those assumptions.  Jython is a
case in point: it doesn't support Python's bytecode at all -- and
that's right, because it has no use for it.

For tools like PyChecker, a higher level is more appropriate: give it
an abstract syntax tree that can answer questions related to the
symbol table, so that you can tell for example whether 'x = y' assigns
a local to a global or vice versa -- and if a local is involved, in
which function it is defined.

For Rattlesnake, a bytecode accessor would of course still be needed
-- but in its capacity as an implementation detail of CPython.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From akuchlin@mems-exchange.org  Mon Aug 13 15:01:17 2001
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Mon, 13 Aug 2001 10:01:17 -0400
Subject: [Python-Dev] More bugs in 2.2a1
In-Reply-To: <3B7592E3.892D1995@metaslash.com>; from neal@metaslash.com on Sat, Aug 11, 2001 at 04:17:39PM -0400
References: <3B7592E3.892D1995@metaslash.com>
Message-ID: <20010813100116.A791@ute.cnri.reston.va.us>

On Sat, Aug 11, 2001 at 04:17:39PM -0400, Neal Norwitz wrote:
>Here's more warnings from pychecker.  There are a lot from plat-irix6 &
>plat-unixware7, not sure if those are real problems or not.

The plat-* problems all seem to be macros converted from C to Python.
if a user ever calls them, they'll only get a NameError for their
trouble, but since no one is likely to call them, this can probably
just be ignored.

I've checked in fixes for the actual bugs reported in the other
modules; thanks!  The only one I haven't done is the following one;
there's definitely at least one bug here (item #2), but I don't know
Tk well enough to fix it.

>lib-tk:
>    Tix.py:81: self is not first method argument
>    Tix.py:331: No global (value) found

--amk




From guido@python.org  Mon Aug 13 15:02:21 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Aug 2001 10:02:21 -0400
Subject: [Python-Dev] ++x oddnes
In-Reply-To: Your message of "Mon, 13 Aug 2001 08:53:23 CDT."
 <15223.56275.968603.294087@beluga.mojam.com>
References: <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> <3B741351.C7C2E2B8@ActiveState.com> <3B7542D1.3E38E579@lemburg.com>
 <15223.56275.968603.294087@beluga.mojam.com>
Message-ID: <200108131402.KAA17976@cj20424-a.reston1.va.home.com>

> How?  ++x compiles to
> 
>          LOAD_FAST         x
>          UNARY_POSITIVE      
>          UNARY_POSITIVE      
> 
> I don't see any incrementing going on...

I'm guessing that MAL used a perverse implementation trick, where for
a certain *mutable* counter type '+x' returns an object that contains
a reference to x, and the unary plus operation on *that* object
increments the counter it references.

    class C:
        def __init__(self):
            self.counter = 0
        def inc(self):
            self.counter += 1
        def __str__(self):
            return str(self.counter)
        def __pos__(self):
            return Ref(self)

    class Ref:
        def __init__(self, arg):
            self.referent = arg
        def __pos__(self):
            self.referent.inc()
            return self.referent

    a = C()
    print a             # 0
    print ++a   # 1
    print a             # 1

Twisted, sick, call it what you want -- but please don't get used to
this.  The likelihood that I'll ever add a ++ operator is very small,
but not zero. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jack@oratrix.nl  Mon Aug 13 15:06:43 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Mon, 13 Aug 2001 16:06:43 +0200
Subject: [Python-Dev] Any NeXT users out there?
In-Reply-To: Message by Guido van Rossum <guido@python.org> ,
 Mon, 13 Aug 2001 09:49:14 -0400 , <200108131349.JAA17882@cj20424-a.reston1.va.home.com>
Message-ID: <20010813140643.9B5C1303181@snelboot.oratrix.nl>

> I've got to say, the distinction between --with and --enable, no
> matter how thoretically pure, in practice seems extremely fuzzy.

... Seeing that I'm in good company wrt my with/enable confusion I'll chime in 
one last time, there's a couple of cases where with is used that are plainly 
wrong. --with-suffix is probably the silliest one, as it is provided by 
autoconf itself.

A distinction that I could understand is if 'with' options may require some 
external stuff (or may be required by some external stuff) and 'enable' were 
purely internal options. With that definition unicode, gc and malloc are the 
only 'enable' style options.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 




From guido@python.org  Mon Aug 13 15:09:50 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Aug 2001 10:09:50 -0400
Subject: [Python-Dev] More bugs in 2.2a1
In-Reply-To: Your message of "Mon, 13 Aug 2001 10:01:17 EDT."
 <20010813100116.A791@ute.cnri.reston.va.us>
References: <3B7592E3.892D1995@metaslash.com>
 <20010813100116.A791@ute.cnri.reston.va.us>
Message-ID: <200108131409.KAA18039@cj20424-a.reston1.va.home.com>

> I've checked in fixes for the actual bugs reported in the other
> modules; thanks!

Thanks, Andrew!

(It really sucks that SF doesn't send checkin mail any more; I've
complained but they are still asleep I assume.)

> The only one I haven't done is the following one;
> there's definitely at least one bug here (item #2), but I don't know
> Tk well enough to fix it.
> 
> >lib-tk:
> >    Tix.py:81: self is not first method argument

According to line 78, Form.form is an alias for the config method,
which definitely requires a self argument.  I'll go ahead and fix it.

> >    Tix.py:331: No global (value) found

Looks like a copy-and-paste error from __setitem__.  I'll fix this
too.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fredrik@pythonware.com  Mon Aug 13 15:12:15 2001
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Mon, 13 Aug 2001 16:12:15 +0200
Subject: [Python-Dev] ++x oddnes
References: <15219.64534.411625.193775@nyx.dyndns.org>        <200108101523.LAA19012@cj20424-a.reston1.va.home.com>        <3B741351.C7C2E2B8@ActiveState.com>        <3B7542D1.3E38E579@lemburg.com> <15223.56275.968603.294087@beluga.mojam.com>
Message-ID: <062401c12401$f5b80690$0900a8c0@spiff>

skip wrote:
> How?  ++x compiles to
>
>          LOAD_FAST         x
>          UNARY_POSITIVE
>          UNARY_POSITIVE
>
> I don't see any incrementing going on...

try this:

class StupidCounter:
    count = 0
    def __pos__(self):
        self.count = self.count + 1
        return 0 # ignore extra __pos__ calls
    def __int__(self):
        return self.count
    def __repr__(self):
        return repr(self.count)

c = StupidCounter()
print c
++c
print c

</F>




From skip@pobox.com (Skip Montanaro)  Mon Aug 13 15:43:40 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Mon, 13 Aug 2001 09:43:40 -0500
Subject: [Python-Dev] CVS commit permission for Andrew MacIntyre
In-Reply-To: <20010812222547.I874@xs4all.nl>
References: <200108121453.KAA15888@cj20424-a.reston1.va.home.com>
 <20010812222547.I874@xs4all.nl>
Message-ID: <15223.59292.919952.827402@beluga.mojam.com>

    Thomas> None, though I'd suggest subscribing him to python-dev too, for
    Thomas> the reasons I gave when I found out Just had checkin privs but
    Thomas> wasn't on py-dev.

Perhaps a (one) requirement for checkin privileges should be a python-dev
subscription?

Skip


From guido@python.org  Mon Aug 13 15:57:34 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Aug 2001 10:57:34 -0400
Subject: [Python-Dev] CVS commit permission for Andrew MacIntyre
In-Reply-To: Your message of "Mon, 13 Aug 2001 09:43:40 CDT."
 <15223.59292.919952.827402@beluga.mojam.com>
References: <200108121453.KAA15888@cj20424-a.reston1.va.home.com> <20010812222547.I874@xs4all.nl>
 <15223.59292.919952.827402@beluga.mojam.com>
Message-ID: <200108131457.f7DEvYQ07335@odiug.digicool.com>

> Perhaps a (one) requirement for checkin privileges should be a python-dev
> subscription?

Sounds good to me.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com (Skip Montanaro)  Mon Aug 13 15:59:34 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Mon, 13 Aug 2001 09:59:34 -0500
Subject: [Python-Dev] Silly pystones question
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEGBLFAA.tim.one@home.com>
References: <15220.39904.966751.603159@beluga.mojam.com>
 <LNBBLJKPBEHFEDALKOLCGEGBLFAA.tim.one@home.com>
Message-ID: <15223.60246.928262.733591@beluga.mojam.com>

    Tim> You're a developer, Skip: it's time you learned how to use CVS
    Tim> <wink>:

    Tim> <http://cvs.sf.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Lib/test/pystone.py>

Thanks for the pointer to the CVS browsing stuff.  In my own defense, I
wrote that note while I was way offline in a rural cottage overlooking Lake
Michigan, so I couldn't run my usual "cvs annotate".  ;-)

    Tim> The evidence there says it's always been this way.

Thanks.

    Tim> Until The Rules change, any global (or builtin) is subject to
    Tim> rebinding at any time, even if there's no STORE_GLOBAL in sight.
    Tim> The "danger" across call sites is clear enough (a Python global is
    Tim> like a C extern), but in the presence of threads another thread can
    Tim> reach into the module and rebind its globals between any pair of
    Tim> byte codes (which C worms around via X3J11 refusing to say anthing
    Tim> at all about threads <0.9 wink>).

Yes, I understand this.  I was thinking (initially, at least) of only
applying such access optimizations in functions that themselves don't set
the global in question.  For example, in pystones, any references to the
various Proc* functions would be candidates for such optimizations.  This is
only to make my life easier.

The real thing here is that while I can "see" what I want with global
tracking, I'm a bit at a loss how to implement it cleanly.  I think what I
need to do at this point is write a PEP and solicit some feedback on
implementation possibilities.

Skip


From skip@pobox.com (Skip Montanaro)  Mon Aug 13 16:02:56 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Mon, 13 Aug 2001 10:02:56 -0500
Subject: [Python-Dev] Silly pystones question
In-Reply-To: <200108131044.GAA17367@cj20424-a.reston1.va.home.com>
References: <15220.39904.966751.603159@beluga.mojam.com>
 <200108131044.GAA17367@cj20424-a.reston1.va.home.com>
Message-ID: <15223.60448.58595.728950@beluga.mojam.com>

    Guido> Whatever you do, *don't* succumb to the temptation of "fixing"
    Guido> pystone.  It's a benchmark and unless it stays the same there's
    Guido> no way to compare to previous versions.

I had no intention of doing anything to it, I was just wondering... ;-)

Skip



From jeremy@zope.com  Mon Aug 13 16:07:32 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Mon, 13 Aug 2001 11:07:32 -0400 (EDT)
Subject: [Python-Dev] RE: Somebody stopped running tests?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEBGLFAA.tim_one@email.msn.com>
References: <LNBBLJKPBEHFEDALKOLCGEBDLFAA.tim.one@home.com>
 <LNBBLJKPBEHFEDALKOLCKEBGLFAA.tim_one@email.msn.com>
Message-ID: <15223.60724.456033.218059@slothrop.digicool.com>

>>>>> "TP" == Tim Peters <tim_one@email.msn.com> writes:

  >> Over the last 12 hours, 4 tests started failing on Windows:
  >>
  >> 4 tests failed: test_descr test_future test_generators test_scope
  >> ...  But only in release mode.  In a debug build, these pass.
  >> This strongly suggests a change "somewhere" that accidentally
  >> relies on an uninitialized value.

  TP> And so it was.  "Fixed" in current CVS, but Jeremy needs to fix
  TP> it a right way.

Fixed now -- thought I'd mention it in the absence of any checkin
mail.  I had started down the path of removing all the
is-nested-scopes-enabled-or-not code, then decided to cleanup the
handling for future statements first.  I accidentally checked in one
of the former changes with the latter checkin.

Jeremy





From skip@pobox.com (Skip Montanaro)  Mon Aug 13 16:09:43 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Mon, 13 Aug 2001 10:09:43 -0500
Subject: [Python-Dev] ++x oddnes
In-Reply-To: <062401c12401$f5b80690$0900a8c0@spiff>
References: <15219.64534.411625.193775@nyx.dyndns.org>
 <200108101523.LAA19012@cj20424-a.reston1.va.home.com>
 <3B741351.C7C2E2B8@ActiveState.com>
 <3B7542D1.3E38E579@lemburg.com>
 <15223.56275.968603.294087@beluga.mojam.com>
 <062401c12401$f5b80690$0900a8c0@spiff>
Message-ID: <15223.60855.1990.594442@beluga.mojam.com>

    >> I don't see any incrementing going on...

    Fredrik> try this:

    Fredrik> class StupidCounter:
    Fredrik>     count = 0
    Fredrik>     def __pos__(self):
    Fredrik>         self.count = self.count + 1
    Fredrik>         return 0 # ignore extra __pos__ calls
    Fredrik>     def __int__(self):
    Fredrik>         return self.count
    Fredrik>     def __repr__(self):
    Fredrik>         return repr(self.count)

    Fredrik> c = StupidCounter()
    Fredrik> print c
    Fredrik> ++c
    Fredrik> print c

Thanks, I understand now.  Of course +c also increments the counter...  

S





From esr@thyrsus.com  Mon Aug 13 16:08:51 2001
From: esr@thyrsus.com (Eric S. Raymond)
Date: Mon, 13 Aug 2001 11:08:51 -0400
Subject: [Python-Dev] yield without future statement?
In-Reply-To: <200108131340.JAA17834@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Aug 13, 2001 at 09:40:06AM -0400
References: <200108131340.JAA17834@cj20424-a.reston1.va.home.com>
Message-ID: <20010813110851.B21008@thyrsus.com>

Guido van Rossum <guido@python.org>:
> On c.l.py, there's a small discussion going on the necessity of a
> future statement to enable the yield keyword.  The general opinion
> seems to be that the future statement is redundant, since previous
> uses of yield will generally result in a syntax error -- and there
> aren't very many uses of yield in the first place.  The other uses of
> future (nested scopes and division) were needed because these features
> cause *silent* failure -- but with yield, you'd have to work really
> really hard to cause a silent failure.
> 
> Opinions?  I'd love to get rid of this particular future statement.

I think you safely can.  I buy this argument.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

Non-cooperation with evil is as much a duty as cooperation with good.
	-- Mohandas Gandhi


From jeremy@zope.com  Mon Aug 13 16:23:04 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Mon, 13 Aug 2001 11:23:04 -0400 (EDT)
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: <200108131356.JAA17936@cj20424-a.reston1.va.home.com>
References: <3B72EF79.7BE8F6C1@metaslash.com>
 <200108092052.f79KqQg13802@odiug.digicool.com>
 <3B73F824.82A53539@metaslash.com>
 <200108101515.LAA18907@cj20424-a.reston1.va.home.com>
 <15223.55925.912142.77948@beluga.mojam.com>
 <200108131356.JAA17936@cj20424-a.reston1.va.home.com>
Message-ID: <15223.61656.410876.775564@slothrop.digicool.com>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

  GvR> For tools like PyChecker, a higher level is more appropriate:
  GvR> give it an abstract syntax tree that can answer questions
  GvR> related to the symbol table, so that you can tell for example
  GvR> whether 'x = y' assigns a local to a global or vice versa --
  GvR> and if a local is involved, in which function it is defined.

I'm not sure how much time I can make for it in the 2.2 release
schedule, but I think a high-level source analysis module would be a
valuable addition to the compiler package.  We've got pychecker and
pydoc as to drive the requirements.  Do you think it's worth making
time for in the 2.2 release schedule?  

Jeremy



From guido@python.org  Mon Aug 13 16:46:31 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Aug 2001 11:46:31 -0400
Subject: [Python-Dev] 2.2a1 patch from PyChecker
In-Reply-To: Your message of "Thu, 09 Aug 2001 15:33:52 EDT."
 <3B72E5A0.DC2E8C4E@metaslash.com>
References: <3B72E5A0.DC2E8C4E@metaslash.com>
Message-ID: <200108131546.f7DFkVk07612@odiug.digicool.com>

Hi Neal,

Going through your patches, I notice a few areas where I disagree.

- You want to change all occurrences of "except <exc>, <var>:" where
  <var> is unused.  I don't -- often it's more regular to leave <var>
  in.

- You want to change all occurrences of "func(name=value)" where name
  is an argument without a default.  I disagree that a default is
  necessary for an argument to be acceptable as a keyword argument.

- You want to change "for i in <expr>" into "for _ in <expr>" when i
  is unused.  I disagree (we've been through this).

Also, I can't figure out why you want to delete the setlocale() method
from the locale.py module.

Apart from these, I'm applying many of these, and Andrew has already
applied many more.  So thanks again!

One thing.  Next time please use a recent CVS.  Martin had already
fixed many of the missing imports.  (If others agree, we could give
you checkin permission so you could check in non-controversial fixes
yourself.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh@python.net  Mon Aug 13 16:43:43 2001
From: mwh@python.net (Michael Hudson)
Date: 13 Aug 2001 11:43:43 -0400
Subject: [Python-Dev] Silly pystones question
In-Reply-To: Skip Montanaro's message of "Mon, 13 Aug 2001 09:59:34 -0500"
References: <15220.39904.966751.603159@beluga.mojam.com> <LNBBLJKPBEHFEDALKOLCGEGBLFAA.tim.one@home.com> <15223.60246.928262.733591@beluga.mojam.com>
Message-ID: <2mwv48nexc.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

> The real thing here is that while I can "see" what I want with
> global tracking, I'm a bit at a loss how to implement it cleanly.  I
> think what I need to do at this point is write a PEP and solicit
> some feedback on implementation possibilities.

Go for it!  I'm sure there are plenty of folks here who would do their
best to pick holes in and suggest improvements in any scheme you come
up with.  And the importance of having something concrete cannot
possibly be overstated - I'm sure each one of us has enough handwavy
ideas on this sort of topic for us all.  I know I do.

Cheers,
M.

-- 
  Two things I learned for sure during a particularly intense acid
  trip in my own lost youth: (1) everything is a trivial special case
  of something else; and, (2) death is a bunch of blue spheres.
                                             -- Tim Peters, 1 May 1998


From guido@python.org  Mon Aug 13 17:04:01 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Aug 2001 12:04:01 -0400
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: Your message of "Mon, 13 Aug 2001 11:23:04 EDT."
 <15223.61656.410876.775564@slothrop.digicool.com>
References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> <3B73F824.82A53539@metaslash.com> <200108101515.LAA18907@cj20424-a.reston1.va.home.com> <15223.55925.912142.77948@beluga.mojam.com> <200108131356.JAA17936@cj20424-a.reston1.va.home.com>
 <15223.61656.410876.775564@slothrop.digicool.com>
Message-ID: <200108131604.f7DG41E08086@odiug.digicool.com>

>   GvR> For tools like PyChecker, a higher level is more appropriate:
>   GvR> give it an abstract syntax tree that can answer questions
>   GvR> related to the symbol table, so that you can tell for example
>   GvR> whether 'x = y' assigns a local to a global or vice versa --
>   GvR> and if a local is involved, in which function it is defined.
> 
> I'm not sure how much time I can make for it in the 2.2 release
> schedule, but I think a high-level source analysis module would be a
> valuable addition to the compiler package.  We've got pychecker and
> pydoc as to drive the requirements.  Do you think it's worth making
> time for in the 2.2 release schedule?  
> 
> Jeremy

I think this falls in the "would be nice" category.  I hope that
non-PythonLabs folks might contribute something of this sort.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com (Skip Montanaro)  Mon Aug 13 17:04:16 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Mon, 13 Aug 2001 11:04:16 -0500
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: <15223.61656.410876.775564@slothrop.digicool.com>
References: <3B72EF79.7BE8F6C1@metaslash.com>
 <200108092052.f79KqQg13802@odiug.digicool.com>
 <3B73F824.82A53539@metaslash.com>
 <200108101515.LAA18907@cj20424-a.reston1.va.home.com>
 <15223.55925.912142.77948@beluga.mojam.com>
 <200108131356.JAA17936@cj20424-a.reston1.va.home.com>
 <15223.61656.410876.775564@slothrop.digicool.com>
Message-ID: <15223.64128.590894.72129@beluga.mojam.com>

    GvR> For tools like PyChecker, a higher level is more appropriate: give
    GvR> it an abstract syntax tree that can answer questions related to the
    GvR> symbol table, so that you can tell for example whether 'x = y'
    GvR> assigns a local to a global or vice versa -- and if a local is
    GvR> involved, in which function it is defined.

    Jeremy> I'm not sure how much time I can make for it in the 2.2 release
    Jeremy> schedule, but I think a high-level source analysis module would
    Jeremy> be a valuable addition to the compiler package.  We've got
    Jeremy> pychecker and pydoc as to drive the requirements.  Do you think
    Jeremy> it's worth making time for in the 2.2 release schedule?

I don't think so.  Designing features to suit the needs of just two tools
may result in a module that's suited for them.  Pydoc and pychecker work
now.  I suggest working on such a module for 2.3 instead, where you'll have
a fair bit of time to properly analyze the requirements.

Skip



From martin@loewis.home.cs.tu-berlin.de  Mon Aug 13 17:01:56 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Mon, 13 Aug 2001 18:01:56 +0200
Subject: [Python-Dev] Any NeXT users out there?
In-Reply-To: <20010813095250.7F568303181@snelboot.oratrix.nl> (message from
 Jack Jansen on Mon, 13 Aug 2001 11:52:50 +0200)
References: <20010813095250.7F568303181@snelboot.oratrix.nl>
Message-ID: <200108131601.f7DG1uB01479@mira.informatik.hu-berlin.de>

> I'm unsure, I've never fully understood the intention behind --with and 
> --enable. For instance, why is it "--enable-unicode" and "--with-threads"?

I think the eventual choice is yours, given Guido's objections to
gettext manual; my objections where primarily backed by this manual.

> The alternative is of course that we also provide a 
> "--with-previous-framework". Any suggestions as to what it should do? :-)

Based on your current patch, I think it is save to remove any claim
that this does something useful on the Next. It may do, but that would
be purely coincidental.

> On a high level it is a very nifty way to package a shared library
> complete with all its supporting files. These supporting files range
> from things used at runtime (like the python Lib directory) to the C
> header files needed when you want to link against the library to the
> documentation.

Ok. Then my question is: Why is it desirable to build libpython as a
framework? You give some rationale below, but I fail to see the
point. I'll pick what I think are your arguments to build a Python
framework, and try to see what it gives to the end user - compared to
the option of linking libpython into the Python interpreter.

> Everything is in a single directory, so install/uninstall is a
> breeze with no danger of missing things.

In case of Python, "everything" is the header files, and the .py
library, right?

Couldn't you also get this by making Python an Application Bundle
(which you intend to do anyway)?

> Also, the structure of a framework allows you to have older,
> incompatible versions of the library (and support documents) in the
> framework as well, with these older versions only used by programs
> that were linked against the framework when that version was
> current.

Again, why is that desirable? If you have multiple versions of Python
installed, if each version has libpython linked statically, and comes
with its own set of header files and .py files: there wouldn't be any
older, incompatible versions of the library that you may need to
defend against.

> The framework patch in sourceforge is the first step. The next step
> is to create a Python application bundle. We can then use a single
> trick in the Python main() program, where it will check to see
> whether it's being run from an application bundle and, if so,
> whether there's a magic file in the bundle (__main__.py comes to
> mind). If there is Python will run that file as the script. This
> will give Python users "freeze" functionality without using a
> compiler.

For that second step, do you really need the first one? I agree that
'freezing' Python applications by just putting a Python installation
into a subdirectory is cool (if I understand right how such a frozen
application would work), however, I fail to see the role of the
framework in here.

You may wonder why I question that feature so much. To me, it is
similar to building a shared libpython.so on Unix. I think this will
cause many problems, and in no way it should be done just for a single
platform (there were patches to do this for Linux, I believe, and such
a patch to make it work specifically on UnixWare only even made it
into 2.1). I think it is desirable that the feature "shared libpython"
is implemented as uniformly as possible across systems. 

If Mac OS X wants to do it differently from everybody else, you need a
very good reason.

The only platform that already has a shared pythonXY.dll has a good
reason: you cannot export symbols from the application to extension
modules on Windows. Yet, this is also the only platform where the API
version magic is meaningless, since extension modules break with every
new Python version exactly because libpython is shared.

Regards,
Martin


From m@moshez.org  Mon Aug 13 17:12:19 2001
From: m@moshez.org (Moshe Zadka)
Date: Mon, 13 Aug 2001 19:12:19 +0300 (IDT)
Subject: [Python-Dev] ++x oddnes
In-Reply-To: <062401c12401$f5b80690$0900a8c0@spiff>
References: <062401c12401$f5b80690$0900a8c0@spiff>, <15219.64534.411625.193775@nyx.dyndns.org>        <200108101523.LAA19012@cj20424-a.reston1.va.home.com>        <3B741351.C7C2E2B8@ActiveState.com>        <3B7542D1.3E38E579@lemburg.com> <15223.56275.968603.294087@beluga.mojam.com>
Message-ID: <20010813161219.F0B38392B@darjeeling>

On Mon, 13 Aug 2001, "Fredrik Lundh" <fredrik@pythonware.com> wrote:

> try this:
> 
> class StupidCounter:
>     count = 0
>     def __pos__(self):
>         self.count = self.count + 1
>         return 0 # ignore extra __pos__ calls
>     def __int__(self):
>         return self.count
>     def __repr__(self):
>         return repr(self.count)
> 
> c = StupidCounter()
> print c
> ++c
> print c

Everybody is ignoring the obvious correct way to do it:

class StupidCounter:
    count = 0
    def __pos__(self):
        self.count = self.count+0.5
        return self
    def __repr__(self): return str(int(self.count))
    def __int__(self): return int(self.count)

Let me just add, muhahahahahahahahahahahahaha!!!!!!
-- 
The Official Moshe Zadka FAQ: http://moshez.geek
The Official Moshe Zadka FAQ For Dummies: http://moshez.org
Read the FAQ


From jeremy@zope.com  Mon Aug 13 17:24:02 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Mon, 13 Aug 2001 12:24:02 -0400 (EDT)
Subject: [Python-Dev] yield without future statement?
In-Reply-To: <200108131340.JAA17834@cj20424-a.reston1.va.home.com>
References: <200108131340.JAA17834@cj20424-a.reston1.va.home.com>
Message-ID: <15223.65314.231423.190925@slothrop.digicool.com>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

  GvR> On c.l.py, there's a small discussion going on the necessity of
  GvR> a future statement to enable the yield keyword.  The general
  GvR> opinion seems to be that the future statement is redundant,
  GvR> since previous uses of yield will generally result in a syntax
  GvR> error -- and there aren't very many uses of yield in the first
  GvR> place.  The other uses of future (nested scopes and division)
  GvR> were needed because these features cause *silent* failure --
  GvR> but with yield, you'd have to work really really hard to cause
  GvR> a silent failure.

I does seem almost comical that the introduction of yield as a keyword
requires a future statement but the descr branch changes do not.  Talk
about silent failures -- dir() no longers work, method resolution for
multiple inheritance changes, changes to the str names of types.  Why
don't these require a future statement?

Jeremy



From Samuele Pedroni <pedroni@inf.ethz.ch>  Mon Aug 13 17:29:37 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Mon, 13 Aug 2001 18:29:37 +0200 (MET DST)
Subject: [Python-Dev] yield without future statement?
Message-ID: <200108131629.SAA06332@core.inf.ethz.ch>

[Jeremy Hylton]
> I does seem almost comical that the introduction of yield as a keyword
> requires a future statement but the descr branch changes do not.  Talk
> about silent failures -- dir() no longers work, method resolution for
> multiple inheritance changes, changes to the str names of types.  Why
> don't these require a future statement?
> 
About dir, in Jython it already worked like it works now in CPython,
so that's was just an implemenation detail :-).

regards.



From fredrik@pythonware.com  Mon Aug 13 17:55:22 2001
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Mon, 13 Aug 2001 18:55:22 +0200
Subject: [Python-Dev] ++x oddnes
References: <062401c12401$f5b80690$0900a8c0@spiff>, <15219.64534.411625.193775@nyx.dyndns.org>        <200108101523.LAA19012@cj20424-a.reston1.va.home.com>        <3B741351.C7C2E2B8@ActiveState.com>        <3B7542D1.3E38E579@lemburg.com> <15223.56275.968603.294087@beluga.mojam.com> <20010813161219.F0B38392B@darjeeling>
Message-ID: <009c01c12418$bf3437d0$4ffa42d5@hagrid>

moshe wrote:

> Everybody is ignoring the obvious correct way to do it:
> 
> class StupidCounter:
>     count = 0
>     def __pos__(self):
>         self.count = self.count+0.5
>         return self
>     def __repr__(self): return str(int(self.count))
>     def __int__(self): return int(self.count)

that was my first implementation (TooStupidCounter).  but
to avoid an endless float vs. integer thread, I decided to post
the plain StupidCounter version instead.

(SlightlyLessStupidCounter is same as StupidCounter,
but returns self.count instead of 0)

</F>



From mal@lemburg.com  Mon Aug 13 19:36:47 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 13 Aug 2001 20:36:47 +0200
Subject: [Python-Dev] ++x oddness
References: <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> <3B741351.C7C2E2B8@ActiveState.com> <3B7542D1.3E38E579@lemburg.com>
 <15223.56275.968603.294087@beluga.mojam.com> <200108131402.KAA17976@cj20424-a.reston1.va.home.com>
Message-ID: <3B781E3F.5C433915@lemburg.com>

Guido van Rossum wrote:
> 
> > How?  ++x compiles to
> >
> >          LOAD_FAST         x
> >          UNARY_POSITIVE
> >          UNARY_POSITIVE
> >
> > I don't see any incrementing going on...
> 
> I'm guessing that MAL used a perverse implementation trick, where for
> a certain *mutable* counter type '+x' returns an object that contains
> a reference to x, and the unary plus operation on *that* object
> increments the counter it references.

Well, not quite that complicated: the type counts the number of calls
to __pos__ and then increments the value in every second call.

It was just an implementation joke and proof of concept thingie.
The idea behind it was to be able to increment the counter (which
is the most often used action on a counter after all) without having
to go through the whole Python method lookup and call machinery.
 
> Twisted, sick, call it what you want -- but please don't get used to
> this.  The likelihood that I'll ever add a ++ operator is very small,
> but not zero. :-)

Ah, but then you'll have to add it as post-fix operator: x++ (which is
illegal today). The difference between ++x and x++ isn't needed in
Python anyway, so this should work out nicely.

Hmm, x++ could map directly to the tp_iternext slot on iterator 
objects... ;-)

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/




From neal@metaslash.com  Mon Aug 13 20:04:44 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Mon, 13 Aug 2001 15:04:44 -0400
Subject: [Python-Dev] Direction of PyChecker
References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> <3B73F824.82A53539@metaslash.com> <200108101515.LAA18907@cj20424-a.reston1.va.home.com>
 <15223.55925.912142.77948@beluga.mojam.com> <200108131356.JAA17936@cj20424-a.reston1.va.home.com>
Message-ID: <3B7824CC.5452CDCE@metaslash.com>

Guido van Rossum wrote:

> For tools like PyChecker, a higher level is more appropriate: give it
> an abstract syntax tree that can answer questions related to the
> symbol table, so that you can tell for example whether 'x = y' assigns

I agree ASTs are more appropriate, but I've been working on adding
features.  My plan is to move to ASTs eventually.

Neal


From neal@metaslash.com  Mon Aug 13 20:18:28 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Mon, 13 Aug 2001 15:18:28 -0400
Subject: [Python-Dev] 2.2a1 patch from PyChecker
References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108131546.f7DFkVk07612@odiug.digicool.com>
Message-ID: <3B782804.26FB66BC@metaslash.com>

Guido van Rossum wrote:

> Going through your patches, I notice a few areas where I disagree.

Ok.  I won't make these changes in the future.

> Also, I can't figure out why you want to delete the setlocale() method
> from the locale.py module.

In my version (2.2a1), there was was something like this:

	try:
		from x import *
	except ImportError:
		def setlocale(): pass

	def setlocale(): pass

So it seems to me the setlocale in the except clause (which is the one
that should have been deleted), will always be replaced with one outside
try/except.  Am I missing something or has the code changed?

> Apart from these, I'm applying many of these, and Andrew has already
> applied many more.  So thanks again!

Great.  I will try to start working on the doc strings.

> One thing.  Next time please use a recent CVS.

Ok.

Neal


From akuchlin@mems-exchange.org  Mon Aug 13 20:22:39 2001
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Mon, 13 Aug 2001 15:22:39 -0400
Subject: [Python-Dev] 2.2a1 patch from PyChecker
In-Reply-To: <3B782804.26FB66BC@metaslash.com>; from neal@metaslash.com on Mon, Aug 13, 2001 at 03:18:28PM -0400
References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108131546.f7DFkVk07612@odiug.digicool.com> <3B782804.26FB66BC@metaslash.com>
Message-ID: <20010813152239.B5318@ute.cnri.reston.va.us>

On Mon, Aug 13, 2001 at 03:18:28PM -0400, Neal Norwitz wrote:
>So it seems to me the setlocale in the except clause (which is the one
>that should have been deleted), will always be replaced with one outside
>try/except.  Am I missing something or has the code changed?

There's a sneaky bit where the code does '_setlocale = setlocale', and
the second setlocale() definition uses _setlocale.  So PyChecker is
correct in reporting that setlocale() is being redefined, but this
can't be removed without more extensive changes.  I'd leave that
decision up to the maintainer of locale.py.

--amk


From guido@python.org  Mon Aug 13 21:05:10 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Aug 2001 16:05:10 -0400
Subject: [Python-Dev] 2.2a1 patch from PyChecker
In-Reply-To: Your message of "Mon, 13 Aug 2001 15:18:28 EDT."
 <3B782804.26FB66BC@metaslash.com>
References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108131546.f7DFkVk07612@odiug.digicool.com>
 <3B782804.26FB66BC@metaslash.com>
Message-ID: <200108132005.f7DK5AE08644@odiug.digicool.com>

> > Also, I can't figure out why you want to delete the setlocale() method
> > from the locale.py module.
> 
> In my version (2.2a1), there was was something like this:
> 
> 	try:
> 		from x import *
> 	except ImportError:
> 		def setlocale(): pass
> 
> 	def setlocale(): pass
> 
> So it seems to me the setlocale in the except clause (which is the one
> that should have been deleted), will always be replaced with one outside
> try/except.  Am I missing something or has the code changed?

You're right.  I think that module is laid out a bit confusing.  I
think it's up to Martin von Loewis to make the cut -- it's basically
his module.  :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Aug 13 21:47:55 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Aug 2001 16:47:55 -0400
Subject: [Python-Dev] yield without future statement?
In-Reply-To: Your message of "Mon, 13 Aug 2001 12:24:02 EDT."
 <15223.65314.231423.190925@slothrop.digicool.com>
References: <200108131340.JAA17834@cj20424-a.reston1.va.home.com>
 <15223.65314.231423.190925@slothrop.digicool.com>
Message-ID: <200108132047.f7DKltE08941@odiug.digicool.com>

> I does seem almost comical that the introduction of yield as a keyword
> requires a future statement but the descr branch changes do not.

We discussed this extensively at a PythonLabs meeting today.  Unless
we get a real revolt about this from the users, we'll keep the future
generators statement because that's the least work.

> Talk about silent failures -- dir() no longers work, method
> resolution for multiple inheritance changes, changes to the str
> names of types.  Why don't these require a future statement?

The latest proposal is to let dir() return more rather than less: it
will return the instance variable names *plus* all attributes defined
by the class *and* its base classes.

The MRO change is not incompatible: the MRO for classic classes is
unchanged.  You get the new MRO only when you explicitly request new
classes (by inheriting from 'object' or specifying '__metaclass__ =
type').

I don't know how I would make the name of the string object vary based
on a future statement, and I doubt that this particular breakage will
bother those affected much.

In general, the unification introduces new or changed features only
when they are explicitly requested.  dir() and type("").__name__ are
the rare exceptions.

BTW, please read the introduction at

  http://www.python.org/2.2/descrintro.html

--Guido van Rossum (home page: http://www.python.org/~guido/)


From thomas@xs4all.net  Mon Aug 13 21:45:30 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Mon, 13 Aug 2001 22:45:30 +0200
Subject: [Python-Dev] 2.2a1 patch from PyChecker
In-Reply-To: <200108131546.f7DFkVk07612@odiug.digicool.com>
References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108131546.f7DFkVk07612@odiug.digicool.com>
Message-ID: <20010813224530.L874@xs4all.nl>

On Mon, Aug 13, 2001 at 11:46:31AM -0400, Guido van Rossum wrote:

> (If others agree, we could give you checkin permission so you could check
> in non-controversial fixes yourself.)

If that meant PyChecker would be a standard part of the Python distribution,
I would be more than all for it ;) But I would like to point out that most
modules have their own maintainer, and they may have good reasons to code
like they do, so if Neal does get commit privs, he shouldn't start fixing
bugs on his own anyway (unless, of course, they're clearly wrong.)

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From guido@python.org  Mon Aug 13 21:54:55 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Aug 2001 16:54:55 -0400
Subject: [Python-Dev] 2.2a1 patch from PyChecker
In-Reply-To: Your message of "Mon, 13 Aug 2001 22:45:30 +0200."
 <20010813224530.L874@xs4all.nl>
References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108131546.f7DFkVk07612@odiug.digicool.com>
 <20010813224530.L874@xs4all.nl>
Message-ID: <200108132054.f7DKste09020@odiug.digicool.com>

> > (If others agree, we could give you checkin permission so you could check
> > in non-controversial fixes yourself.)
> 
> If that meant PyChecker would be a standard part of the Python distribution,
> I would be more than all for it ;)

That's a separate step -- but definitely worth considering.  For now,
I expect that PyChecker's release cycle is so much faster than
Python's that it's better to keep the two separate, but maybe a year
from now PyChecker has grown up enough to do this.  (We could put a
PyChecker snapshot in the Tools directory around release time, but
that's not much use if it keeps changing.)

> But I would like to point out that most
> modules have their own maintainer, and they may have good reasons to code
> like they do, so if Neal does get commit privs, he shouldn't start fixing
> bugs on his own anyway (unless, of course, they're clearly wrong.)

I'm sure Neal will know his limits.  :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From thomas@xs4all.net  Mon Aug 13 21:50:43 2001
From: thomas@xs4all.net (Thomas Wouters)
Date: Mon, 13 Aug 2001 22:50:43 +0200
Subject: [Python-Dev] yield without future statement?
In-Reply-To: <200108131340.JAA17834@cj20424-a.reston1.va.home.com>
References: <200108131340.JAA17834@cj20424-a.reston1.va.home.com>
Message-ID: <20010813225043.M874@xs4all.nl>

On Mon, Aug 13, 2001 at 09:40:06AM -0400, Guido van Rossum wrote:

> On c.l.py, there's a small discussion going on the necessity of a
> future statement to enable the yield keyword.  The general opinion
> seems to be that the future statement is redundant, since previous
> uses of yield will generally result in a syntax error -- and there
> aren't very many uses of yield in the first place.  The other uses of
> future (nested scopes and division) were needed because these features
> cause *silent* failure -- but with yield, you'd have to work really
> really hard to cause a silent failure.

I'm still against -- sorry! But it isn't just about silently breaking code,
it's also about upgrade paths. I've said this before, but I don't feel
comfortable upgrading our system-wide Python version, which thousands of
users might use, without there being a clear warning that things will break.
The user might not even notice that Python was upgraded, and not understand
why the use of 'yield' as an identifier suddenly causes a syntax error. With
a release where the use of yield generates a warning, that's their own
fault. (And yes, CGI scripts dump their warnings in the Apache errorlog, but
we have our own way of dealing with that :)

-- 
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!


From skip@pobox.com (Skip Montanaro)  Mon Aug 13 23:08:52 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Mon, 13 Aug 2001 17:08:52 -0500
Subject: [Python-Dev] yield without future statement?
In-Reply-To: <200108132047.f7DKltE08941@odiug.digicool.com>
References: <200108131340.JAA17834@cj20424-a.reston1.va.home.com>
 <15223.65314.231423.190925@slothrop.digicool.com>
 <200108132047.f7DKltE08941@odiug.digicool.com>
Message-ID: <15224.20468.160433.254687@beluga.mojam.com>

    Guido> The latest proposal is to let dir() return more rather than less:
    Guido> it will return the instance variable names *plus* all attributes
    Guido> defined by the class *and* its base classes.

While that may seem like a step in the right direction, it will still break
things and not obviate the need for some sort of warning.  Also, one might
argue that for interactive use it might be less helpful than the current
behavior for users to be overwhelmed with dir() output.  I have these two
functions available to me in interactive sessions:

    def dir(o=globals,showall=0):
        if not showall and hasattr(o, "__all__"):
            x = list(o.__all__)
            x.sort()
            return x
        from __builtin__ import dir
        return dir(o)

    def dirall(o, showall=0):
        attrs = dir(o, showall)
        if hasattr(o, "__bases__"):
            for b in o.__bases__:
                attrs.extend(dirall(b, showall))
        if hasattr(o, "__class__") and o != o.__class__:
            attrs.extend(dirall(o.__class__, showall))
        adict = {}
        for a in attrs:
            adict[a] = 1
        attrs = adict.keys()
        attrs.sort()
        return attrs

If I execute "dirall(gtk.GtkRadioButton())" I get a list with 175 entries.
Granted, that's an extreme, but it suggests that in some cases the output of
a recursive dir won't be all that helpful.

What might be useful is to add a recursive flag to dir so that people can
ask for everything.  That way current behaviour would be preserved but users
could get everything if they wanted to.

Skip




From ping@lfw.org  Mon Aug 13 23:33:19 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Mon, 13 Aug 2001 15:33:19 -0700 (PDT)
Subject: [Python-Dev] classmethod() and staticmethod()
In-Reply-To: <200108132005.f7DK5AE08644@odiug.digicool.com>
Message-ID: <Pine.LNX.4.32.0108131503070.662-100000@ziggy.localdomain.fake>

I'm very happy with the design for type-class unification -- it looks
like an excellent move towards making Python more consistent and flexible.

However, the mechanism for declaring class and static methods:

    class MyClass:
        def method(x, y):                      # part 1
            ...

        method = staticmethod(method)          # part 2

...sticks out as awkward and strange to read.  Kevin Smith and Paul
Prescod commented earlier on this, and i agree with their opinion that
a more direct declaration would be a better way to do this.

I don't especially care what keyword or syntax is used to declare a
static method, just that we require it to occur right where the method
is being declared, and that it not be a "normal" expression (i.e. it
must be invalid syntax in Python versions < 2.2).

The two-part declaration above is unfortunate because:

    (a) A human reader doesn't immediately see what kind of method
        is being declared, and has to read ahead to figure out.

    (b) An automated documentation or syntax checking utility can't
        immediately see what kind of method is being declared, and
        has to scan ahead to figure out.  Same thing for colourizing
        editors, IDEs that try to help you fix errors, etc.

    (c) It is possible for deceptive code to come about by accident.
        In much the same way that braces and indentation can get out
        of sync and confuse a reader, the appearance of parts 1 and 2
        can get out of sync with the semantics of the code.

        (If someone forgets to write part 2, or inadvertently forgets
        to type in "self" as the first argument to an instance method,
        then part 1 will look like a static method even though it's
        an instance method.  If someone writes part 2, but by habit
        forgets to omit "self" as the first argument, then part 1 will
        look like an instance method even though it's a static method.
        It would be much better to design this in the first place so
        that such mess-ups are not possible.)

    (d) The new declaration is syntactically correct in older Pythons.
        Someone unfamiliar with this new static-method mechanism will
        not be given very useful ideas about what is wrong.  With an old
        version of Python, they'll get an error seeming to indicate
        that "staticmethod" is an undeclared function, and wonder what
        they need to import.  I think it would be better for them to get
        a syntax error on the line where the method is declared, clearly
        indicating that a fundamentally new feature is being used and
        a new version of Python is required.



-- ?!ng



From greg@cosc.canterbury.ac.nz  Tue Aug 14 00:26:55 2001
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 14 Aug 2001 11:26:55 +1200 (NZST)
Subject: [Python-Dev] yield without future statement?
In-Reply-To: <200108131340.JAA17834@cj20424-a.reston1.va.home.com>
Message-ID: <200108132326.LAA02534@s454.cosc.canterbury.ac.nz>

Guido:

> The general opinion
> seems to be that the future statement is redundant, since previous
> uses of yield will generally result in a syntax error

I thought the future statement was also to give users a
release or two of warning before a semantics-changing
feature becomes mandatory. Maybe I was wrong about the 
intention?

I don't have particularly strong feelings either way in
this case, but people more sensitive to code breakage
might.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Tue Aug 14 01:18:32 2001
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 14 Aug 2001 12:18:32 +1200 (NZST)
Subject: [Python-Dev] dir (was: yield without future statement?)
In-Reply-To: <15224.20468.160433.254687@beluga.mojam.com>
Message-ID: <200108140018.MAA02548@s454.cosc.canterbury.ac.nz>

> If I execute "dirall(gtk.GtkRadioButton())" I get a list with 175
> entries.

Seems to me that, for interactive use, it would be better to
have a function that formats its output into categories, so
you can see which are instance attributes and which are class
attributes, and which superclass they come from.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From guido@python.org  Tue Aug 14 02:58:03 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Aug 2001 21:58:03 -0400
Subject: [Python-Dev] classmethod() and staticmethod()
In-Reply-To: Your message of "Mon, 13 Aug 2001 15:33:19 PDT."
 <Pine.LNX.4.32.0108131503070.662-100000@ziggy.localdomain.fake>
References: <Pine.LNX.4.32.0108131503070.662-100000@ziggy.localdomain.fake>
Message-ID: <200108140158.VAA18746@cj20424-a.reston1.va.home.com>

> I'm very happy with the design for type-class unification -- it looks
> like an excellent move towards making Python more consistent and flexible.

Thanks!

> However, the mechanism for declaring class and static methods:
> 
>     class MyClass:
>         def method(x, y):                      # part 1
>             ...
> 
>         method = staticmethod(method)          # part 2
> 
> ...sticks out as awkward and strange to read.  Kevin Smith and Paul
> Prescod commented earlier on this, and i agree with their opinion that
> a more direct declaration would be a better way to do this.

Understood, and agreed.  It is my plan to eventually provide syntax
for this.  However, right now I need to concentrate on getting the
underlying mechanisms to work.  The unification is a lot of work, and
I'd like to finish the run-time before I start thinking about
syntactic sugar.  I can't promise that I'll change the syntax before
2.2 is released -- there will be plenty of time after that, and there
are plenty of more important things to do before I can make the
release.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido@python.org  Tue Aug 14 03:25:19 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Aug 2001 22:25:19 -0400
Subject: [Python-Dev] yield without future statement?
In-Reply-To: Your message of "Mon, 13 Aug 2001 17:08:52 CDT."
 <15224.20468.160433.254687@beluga.mojam.com>
References: <200108131340.JAA17834@cj20424-a.reston1.va.home.com> <15223.65314.231423.190925@slothrop.digicool.com> <200108132047.f7DKltE08941@odiug.digicool.com>
 <15224.20468.160433.254687@beluga.mojam.com>
Message-ID: <200108140225.WAA18816@cj20424-a.reston1.va.home.com>

>     Guido> The latest proposal is to let dir() return more rather than less:
>     Guido> it will return the instance variable names *plus* all attributes
>     Guido> defined by the class *and* its base classes.
> 
> While that may seem like a step in the right direction, it will still break
> things and not obviate the need for some sort of warning.

Unclear.  If you look it up, you'll find that dir() is documented very
vaguely.  It returns a list of strings, and the strings in that list
are attributes of the object -- that's about it.  If someone wrote
code that relied on this returning a specific set of strings, they get
what they deserve.

Suppose this idea had started differently: as an improvement to dir(),
to make it return all the attributes (or at least all the ones that
are reasonably discoverable).  I don't think anyone would complain.

> Also, one might
> argue that for interactive use it might be less helpful than the current
> behavior for users to be overwhelmed with dir() output.  I have these two
> functions available to me in interactive sessions:
> 
>     def dir(o=globals,showall=0):
>         if not showall and hasattr(o, "__all__"):
>             x = list(o.__all__)
>             x.sort()
>             return x
>         from __builtin__ import dir
>         return dir(o)
> 
>     def dirall(o, showall=0):
>         attrs = dir(o, showall)
>         if hasattr(o, "__bases__"):
>             for b in o.__bases__:
>                 attrs.extend(dirall(b, showall))
>         if hasattr(o, "__class__") and o != o.__class__:
>             attrs.extend(dirall(o.__class__, showall))
>         adict = {}
>         for a in attrs:
>             adict[a] = 1
>         attrs = adict.keys()
>         attrs.sort()
>         return attrs
> 
> If I execute "dirall(gtk.GtkRadioButton())" I get a list with 175 entries.
> Granted, that's an extreme, but it suggests that in some cases the output of
> a recursive dir won't be all that helpful.

If interactive behavior changes, that can't be claimed to be "breaking
old code".  It doesn't take very long to get used to a different way
of introspecting objects.

> What might be useful is to add a recursive flag to dir so that people can
> ask for everything.  That way current behaviour would be preserved but users
> could get everything if they wanted to.

But "current behavior" is inconsistent.  For some objects it includes
instance variables and methods.  For other objects it includes only
instance variables.  How useful is that?

If an object has 175 methods, you need a class browser -- neither a
recursive dir() nor a non-recursive dir() is very useful in that case.

I would prefer to see help() improved to the point where it can be
used for this purpose.

I think I've heard just about every argument there is about dir().  In
2.1 and before, it is an ill-defined mess of a function.  We can
either try to give it a useful well-defined meaning, or deprecate it
altogether.  The only two useful well-defined meanings that I can see
are either a sorted list of the keys of the argument's __dict__ (as
implemented in Python 2.2a1), or a sorted list of *all* the
discoverable attributes of the object.  I actually prefer the first
for myself, but this received lots of complaints about how returning
[] for list instances would break code, so I am now proposing the
second well-defined meaning.  I find it hard to imagine how returning
*more* names would still break code -- that code cannot have been very
robust in the first place.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz@rahul.net  Tue Aug 14 03:31:11 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Mon, 13 Aug 2001 19:31:11 -0700 (PDT)
Subject: [Python-Dev] 2.2a1: classmethod() and class attributes
In-Reply-To: <200108132047.f7DKltE08941@odiug.digicool.com> from "Guido van Rossum" at Aug 13, 2001 04:47:55 PM
Message-ID: <20010814023111.3273A99C80@waltz.rahul.net>

Guido van Rossum wrote:
> 
> BTW, please read the introduction at
>   http://www.python.org/2.2/descrintro.html

Okay, I did that.

It's not particularly convenient for me to try out 2.2a1, so I'm just
going by what's written.  One little hole that I don't see an answer to
is what happens when you do this:

class C(object):
    x = 0
    def foo(cls):
        cls.x += 1
    foo = classmethod(foo)

C.foo()
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From barry@zope.com  Tue Aug 14 03:49:05 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 13 Aug 2001 22:49:05 -0400
Subject: [Python-Dev] 2.2a1 patch from PyChecker
References: <3B72E5A0.DC2E8C4E@metaslash.com>
 <200108092120.f79LKNo13875@odiug.digicool.com>
 <20010809180903.A31343@ute.cnri.reston.va.us>
Message-ID: <15224.37281.601490.99294@anthem.wooz.org>

>>>>> "AK" == Andrew Kuchling <akuchlin@mems-exchange.org> writes:

    AK> That reminds me: should we put warnings in cgi.py for
    AK> instantiations of FormContent and other old classes, so we can
    AK> delete them in Python 2.3?  One problem: if a class doesn't
    AK> have an __init__ method, subclasses that have their own
    AK> __init__ methods won't call it.  Maybe this could be done by
    AK> adding 'self.warn()' to all the methods of an outdated class,
    AK> and defining a warn() method that raises a warning.

Which reminds me: do you think you'll one day resurrect PEP 222?

-Barry


From guido@python.org  Tue Aug 14 04:59:29 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 13 Aug 2001 23:59:29 -0400
Subject: [Python-Dev] 2.2a1: classmethod() and class attributes
In-Reply-To: Your message of "Mon, 13 Aug 2001 19:31:11 PDT."
 <20010814023111.3273A99C80@waltz.rahul.net>
References: <20010814023111.3273A99C80@waltz.rahul.net>
Message-ID: <200108140359.XAA19714@cj20424-a.reston1.va.home.com>

> One little hole that I don't see an answer to is what happens when
> you do this:
> 
> class C(object):
>     x = 0
>     def foo(cls):
>         cls.x += 1
>     foo = classmethod(foo)
> 
> C.foo()

As written, you get TypeError: can't set type attributes.

If you add __dynamic__ = 1 to the class statement, it works as
expected.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com (Skip Montanaro)  Tue Aug 14 05:04:04 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Mon, 13 Aug 2001 23:04:04 -0500
Subject: [Python-Dev] yield without future statement?
In-Reply-To: <200108140225.WAA18816@cj20424-a.reston1.va.home.com>
References: <200108131340.JAA17834@cj20424-a.reston1.va.home.com>
 <15223.65314.231423.190925@slothrop.digicool.com>
 <200108132047.f7DKltE08941@odiug.digicool.com>
 <15224.20468.160433.254687@beluga.mojam.com>
 <200108140225.WAA18816@cj20424-a.reston1.va.home.com>
Message-ID: <15224.41780.829026.859816@beluga.mojam.com>

    Guido> Suppose this idea had started differently: as an improvement to
    Guido> dir(), to make it return all the attributes (or at least all the
    Guido> ones that are reasonably discoverable).  I don't think anyone
    Guido> would complain.

Yeah, but there have already been complaints about the change to what dir()
returns for list instances in the context of the 2.2 class/type unification.
I can only suppose there are going to be complaints about other changes
(from other people) if dir() changes in other ways.  That was the main point
of my post.  I probably should have avoided the issue of how verbose the
output might be.

most-people-don't-read-the-docs-ly y'rs,

Skip


From dgoodger@bigfoot.com  Tue Aug 14 05:29:39 2001
From: dgoodger@bigfoot.com (David Goodger)
Date: Tue, 14 Aug 2001 00:29:39 -0400
Subject: [Python-Dev] Re: [Python-checkins] testing
Message-ID: <B79E2172.16068%dgoodger@bigfoot.com>

I'm not sure if this is the problem, but here goes...

I was unable to set up syncmail for my projects; the executable permissions
wouldn't stick on CVSROOT/syncmail. I solved it by adding an explicit
/usr/bin/python to CVSROOT/loginfo::

    DEFAULT /usr/bin/python $CVSROOT/CVSROOT/syncmail ...

syncmail no longer needs to be executable.

Hope this helps.

-- 
David Goodger    dgoodger@bigfoot.com    Open-source projects:
 - Python Docstring Processing System: http://docstring.sourceforge.net
 - reStructuredText: http://structuredtext.sourceforge.net
 - The Go Tools Project: http://gotools.sourceforge.net



From dgoodger@bigfoot.com  Tue Aug 14 05:44:52 2001
From: dgoodger@bigfoot.com (David Goodger)
Date: Tue, 14 Aug 2001 00:44:52 -0400
Subject: [Python-Dev] AST mining (was Re: Direction of PyChecker)
Message-ID: <B79E2503.1606C%dgoodger@bigfoot.com>

>From the PyChecker AST discussion, it seems we may have a common goal. For
the Docstring Processing System, I am looking into gleaning information from
the abstract syntax tree. From the working notes, under "Docstring
Extractor":

    We need code that scans a parsed Python module, and returns an
    ordered tree containing the names, docstrings (including
    additional docstrings), and additional info (in parentheses below)
    of all of the following objects:

    - packages
    - modules
    - module attributes (+ values)
    - classes (+ inheritance)
    - class attributes (+ values)
    - instance attributes (+ values)
    - methods (+ formal parameters)
    - functions (+ formal parameters)

    In order to evaluate interpreted text cross-references, namespaces
    for each of the above will also be required.

I'd be very interested in pooling efforts to make this easier. I know almost
nothing about ASTs now, but that could change in a hurry :-).

-- 
David Goodger    dgoodger@bigfoot.com    Open-source projects:
 - Python Docstring Processing System: http://docstring.sourceforge.net
 - reStructuredText: http://structuredtext.sourceforge.net
 - The Go Tools Project: http://gotools.sourceforge.net



From guido@python.org  Tue Aug 14 05:43:29 2001
From: guido@python.org (Guido van Rossum)
Date: Tue, 14 Aug 2001 00:43:29 -0400
Subject: [Python-Dev] yield without future statement?
In-Reply-To: Your message of "Mon, 13 Aug 2001 23:04:04 CDT."
 <15224.41780.829026.859816@beluga.mojam.com>
References: <200108131340.JAA17834@cj20424-a.reston1.va.home.com> <15223.65314.231423.190925@slothrop.digicool.com> <200108132047.f7DKltE08941@odiug.digicool.com> <15224.20468.160433.254687@beluga.mojam.com> <200108140225.WAA18816@cj20424-a.reston1.va.home.com>
 <15224.41780.829026.859816@beluga.mojam.com>
Message-ID: <200108140443.AAA19887@cj20424-a.reston1.va.home.com>

>     Guido> Suppose this idea had started differently: as an improvement to
>     Guido> dir(), to make it return all the attributes (or at least all the
>     Guido> ones that are reasonably discoverable).  I don't think anyone
>     Guido> would complain.
> 
> Yeah, but there have already been complaints about the change to what dir()
> returns for list instances in the context of the 2.2 class/type unification.
> I can only suppose there are going to be complaints about other changes
> (from other people) if dir() changes in other ways.

But the proposed change would revert the change that people complained
about: it would return all the methods of a list, like before.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@home.com  Tue Aug 14 06:26:19 2001
From: tim.one@home.com (Tim Peters)
Date: Tue, 14 Aug 2001 01:26:19 -0400
Subject: [Python-Dev] AST mining (was Re: Direction of PyChecker)
In-Reply-To: <B79E2503.1606C%dgoodger@bigfoot.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEIPLFAA.tim.one@home.com>

[David Goodger]
> ...
> For the Docstring Processing System, I am looking into gleaning
> information from the abstract syntax tree.
> ...
>     - packages
>     - modules
>     - module attributes (+ values)
>     - classes (+ inheritance)
>     - class attributes (+ values)
>     - instance attributes (+ values)
>     - methods (+ formal parameters)
>     - functions (+ formal parameters)
> ...
> I'd be very interested in pooling efforts to make this easier. I
> know almost nothing about ASTs now, but that could change in a
> hurry :-).

Let me suggest you don't really want an AST -- you want an object model for
Python source that answers the questions above directly.  An AST may be an
effective (under the covers) implementation technique to get such info, but
if you don't want to wait for people to argue about "the right" AST and "the
right" tree-based query language to make it better than completely useless
<wink>, you can answer all the stuff above by building on tokenize.py now.

BTW, exploiting generators in 2.2 can make a tokenize-based approach much
more pleasant than before; see the newish Tools/scripts/cleanfuture.py (in
CVS) for an example of how easily 1-token lookahead parsing can be coded
now.



From tim.one@home.com  Tue Aug 14 06:36:49 2001
From: tim.one@home.com (Tim Peters)
Date: Tue, 14 Aug 2001 01:36:49 -0400
Subject: [Python-Dev] yield without future statement?
In-Reply-To: <15224.41780.829026.859816@beluga.mojam.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEJALFAA.tim.one@home.com>

[Skip Montanaro]
> Yeah, but there have already been complaints about the change to
> what dir() returns for list instances in the context of the 2.2
> class/type unification.

I'm finding it impossible to keep up with all the discussions, but I believe
I've seen every msg related to this one.  I believe they were all wondering
why dir([]) or dir({}) returned [] under 2.2a1 at *an interactive prompt*.
Indeed, I beat everyone to that by asking Guido the same thing perhaps a
month ago <wink>.

> I can only suppose there are going to be complaints about other
> changes (from other people) if dir() changes in other ways.

Sure.  And now that Guido has suggested a richer dir(), there will be
complaints too if we leave dir() alone!  The richer dir() attracted some
real enthusiasm.

But dir() simply has no well-defined purpose beyond dir(module), so IMO
complaints about dir() are declarations of preference-- and nothing more
than that --and so unavoidable.



From skip@pobox.com (Skip Montanaro)  Tue Aug 14 06:57:44 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Tue, 14 Aug 2001 00:57:44 -0500
Subject: [Python-Dev] Proposed PEP: Optimizing Global Variable and Attribute Access
Message-ID: <15224.48600.621401.101492@beluga.mojam.com>

Here's a first stab at a PEP for optimizing global variable and attribute
access.  Barry, when you get a chance, could you assign it a number and let
me know about any PEP-related formatting mistakes I made?  (It would be nice
if PEP 1 referenced a piece of boilerplate people could use.  While the PEP
Style section lists the headers, I wasn't sure what the Version and
Last-Modified headers were actually supposed to look like.  Also, it wasn't
clear what I was supposed to put in the Post-History: header.)

Skip

------------------------------------------------------------------------------
PEP: 
Title: Optimizing Global Variable and Attribute Access
Version: $Revision:$
Author: skip@pobox.com (Skip Montanaro)
Status: Draft
Type: Standards Track
Python-Version: 2.3
Created: 13-Aug-2001
Post-History:

Abstract

    The bindings for most global variables and attributes of other modules
    typically never change during the execution of a Python program, but
    because of Python's dynamic nature, code which accesses such global
    objects must run through a full lookup each time the object is needed.
    This PEP proposes a mechanism that allows code that accesses most global
    objects to treat them as local objects and places the burden of updating
    references on the code that changes the name bindings of such objects.

Introduction

    Consider the workhorse function sre_compile._compile.  It is the
    internal compilation function for the sre module.  It consists almost
    entirely of a loop over the elements of the pattern being compiled,
    comparing opcodes with known constant values and appending tokens to an
    outpu list.  Most of the comparisons are with constants imported from
    the sre_constants module.  This means there are lots of LOAD_GLOBAL
    bytecodes in the compiled output of this module.  Just by reading the
    code it's apparent that the author intended LITERAL, NOT_LITERAL,
    OPCODES and many other symbols to be constants.  Still, each time they
    are involved in an expression, they must be looked up anew.

    Most global accesses are actually to objects that are "almost
    constants".  This includes global variables in the current module as
    well as the attributes of other imported modules.  Since they rarely
    change, it seems reasonable to place the burden of updating references
    to such objects on the code that changes the name bindings.  If
    sre_constants.LITERAL is changed to refer to another object, perhaps it
    would be worthwhile for the code that modifies the sre_constants module
    dict to correct any active references to that object.  By doing so, in
    many cases global variables and the attributes of many objects could be
    cached as local variables.  If the bindings between the names given to
    the objects and the objects themselves changes rarely, the cost of
    keeping track of such objects should be low and the potential payoff
    fairly large.

Proposed Change

    I propose that the Python virtual machine be modified to include
    TRACK_OBJECT and UNTRACK_OBJECT opcodes.  TRACK_OBJECT would associate a
    global name or attribute of a global name with a slot in the local
    variable array and perform an initial lookup of the associated object to
    fill in the slot with a valid value.  The association it creates would
    be noted by the code responsible for changing the name-to-object binding
    to cause the associated local variable to be updated.  The
    UNTRACK_OBJECT opcode would delete any association between the name and
    the local variable slot.

Rationale

    Global variables and attributes rarely change.  For example, once a
    function imports the math module, the binding between the name "math"
    and the module it refers to aren't likely to change.  Similarly, if the
    function that uses the math module refers to its "sin" attribute, it's
    unlikely to change.  Still, every time the module wants to call the
    math.sin function, it must first execute a pair of instructions:

        LOAD_GLOBAL     math
        LOAD_ATTR       sin

    If the client module always assumed that math.sin was a local constant
    and it was the responsibility of "external forces" outside the function
    to keep the reference correct, we might have code like this:

        TRACK_OBJECT       math.sin
        ...
        LOAD_FAST          math.sin
        ...
        UNTRACK_OBJECT     math.sin

    If the LOAD_FAST was in a loop the payoff in reduced global loads and
    attribute lookups could be significant.

    This technique could, in theory, be applied to any global variable
    access or attribute lookup.  Consider this code:

        l = []
        for i in range(10):
            l.append(math.sin(i))
        return l

    Even though l is a local variable, you still pay the cost of loading
    l.append ten times in the loop.  The compiler (or an optimizer) could
    recognize that both math.sin and l.append are being called in the loop
    and decide to generate the tracked local code, avoiding it for the
    builtin range() function because it's only called once during loop
    setup.

    According to a post to python-dev by Marc-Andre Lemburg [1], LOAD_GLOBAL
    opcodes account for over 7% of all instructions executed by the Python
    virtual machine.  This can be a very expensive instruction, at least
    relative to a LOAD_FAST instruction, which is a simple array index and
    requires no extra function calls by the virtual machine.  I believe many
    LOAD_GLOBAL instructions and LOAD_GLOBAL/ LOAD_ATTR pairs could be
    converted to LOAD_FAST instructions.

    Code that uses global variables heavily often resorts to various tricks
    to avoid global variable and attribute lookup.  The aforementioned
    sre_compile._compile function caches the append method of the growing
    output list.  Many people commonly abuse functions' default argument
    feature to cache global variable lookups.  Both of these schemes are
    hackish and rarely address all the available opportunities for
    optimization.  (For example, sre_compile._compile does not cache the two
    globals that it uses most frequently: the builtin len function and the
    global OPCODES array that it imports from sre_constants.py.

Discussion

    Jeremy Hylton has an alternate proposal on the table [2].  His proposal
    seeks to create a hybrid dictionary/list object for use in global name
    lookups that would make global variable access look more like local
    variable access.  While there is no C code available to examine, the
    Python implementation given in his proposal still appears to require
    dictionary key lookup.  It doesn't appear that his proposal could
    speed local variable attribute lookup, which might be worthwhile in some
    situations.

Backwards Compatibility

    I don't believe there will be any serious issues of backward
    compatibility.  Obviously, Python bytecode that contains TRACK_OBJECT
    opcodes could not be executed by earlier versions of the interpreter,
    but breakage at the bytecode level is often assumed between versions.

Implementation

    TBD.  This is where I need help.  I believe there should be either a
    central name/location registry or the code that modifies object
    attributes should be modified, but I'm not sure the best way to go about
    this.  If you look at the code that implements the STORE_GLOBAL and
    STORE_ATTR opcodes, it seems likely that some changes will be required
    to PyDict_SetItem and PyObject_SetAttr or their String variants.
    Ideally, there'd be a fairly central place to localize these changes.
    If you begin considering tracking attributes of local variables you get
    into issues of modifying STORE_FAST as well, which could be a problem,
    since the name bindings for local variables are changed much more
    frequently.  (I think an optimizer could avoid inserting the tracking
    code for the attributes for any local variables where the variable's
    name binding changes.)

Performance

    I believe (though I have no code to prove it at this point), that
    implementing TRACK_OBJECT will generally not be much more expensive than
    a single LOAD_GLOBAL instruction or a LOAD_GLOBAL/LOAD_ATTR pair.  An
    optimizer should be able to avoid converting LOAD_GLOBAL and
    LOAD_GLOBAL/LOAD_ATTR to the new scheme unless the object access
    occurred within a loop.  Further down the line, a register-oriented
    replacement for the current Python virtual machine [3] could conceivably
    eliminate most of the LOAD_FAST instructions as well.

    The number of tracked objects should be relatively small.  All active
    frames of all active threads could conceivably be tracking objects, but
    this seems small compared to the number of functions defined in a given
    application.

References

    [1] http://mail.python.org/pipermail/python-dev/2000-July/007609.html

    [2] http://www.zope.org/Members/jeremy/CurrentAndFutureProjects/FastGlobalsPEP

    [3] http://www.musi-cal.com/~skip/python/rattlesnake20010813.tar.gz

Copyright

    This document has been placed in the public domain.


Local Variables:
mode: indented-text
indent-tabs-mode: nil
End:




From martin@loewis.home.cs.tu-berlin.de  Tue Aug 14 09:05:01 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Tue, 14 Aug 2001 10:05:01 +0200
Subject: [Python-Dev] 2.2a1: classmethod() and class attributes
Message-ID: <200108140805.f7E851j01484@mira.informatik.hu-berlin.de>

> If you add __dynamic__ = 1 to the class statement, it works as
> expected.

Is that documented anywhere yet? I searched all three documents (the
PEPs, and the intro), and could not find any mentioning of
__dynamic__.

Regards,
Martin



From gmcm@hypernet.com  Tue Aug 14 13:11:13 2001
From: gmcm@hypernet.com (Gordon McMillan)
Date: Tue, 14 Aug 2001 08:11:13 -0400
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: <15223.61656.410876.775564@slothrop.digicool.com>
References: <200108131356.JAA17936@cj20424-a.reston1.va.home.com>
Message-ID: <3B78DD21.31888.224FE356@localhost>

Jeremy wrote:

[byte code scanning -> AST analysis]

> ....  We've got
> pychecker and pydoc as to drive the requirements.  

Don't forget freeze/modulefinder.py.

- Gordon


From jack@oratrix.nl  Tue Aug 14 13:43:03 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Tue, 14 Aug 2001 14:43:03 +0200
Subject: [Python-Dev] Any NeXT users out there?
In-Reply-To: Message by "Martin v. Loewis" <martin@loewis.home.cs.tu-berlin.de>
 ,
 Mon, 13 Aug 2001 18:01:56 +0200 , <200108131601.f7DG1uB01479@mira.informatik.hu-berlin.de>
Message-ID: <20010814124304.01511303181@snelboot.oratrix.nl>

> > The alternative is of course that we also provide a 
> > "--with-previous-framework". Any suggestions as to what it should do? :-)
> 
> Based on your current patch, I think it is save to remove any claim
> that this does something useful on the Next. It may do, but that would
> be purely coincidental.

That was a joke:-)

But seriously: if noone speaks up within the next couple of hours I will rip 
out NeXT support. I'll also add an option (--enable-toolbox-glue?) to rip out 
the glue code for MacPython/Carbon modules, thereby increasing the chances 
that this will all work on a bare Darwin without MacOSX.

> > On a high level it is a very nifty way to package a shared library
> > complete with all its supporting files. These supporting files range
> > from things used at runtime (like the python Lib directory) to the C
> > header files needed when you want to link against the library to the
> > documentation.
> 
> Ok. Then my question is: Why is it desirable to build libpython as a
> framework?

Sharing is the answer. When everything is finished an average Python user will 
have at least the interpreter application (the bundled drag-and-drop one) and 
the IDE installed. There may be auxiliary apps like the current MacPython's 
BuildApplet and BuildApplication. Eventually a slightly specialised 
interpreter that does the Cocoa/NIB magic on startup so that creating Cocoa 
apps in Python should be (almost?) as easy as creating them in ObjC or Java.

They may have the commandline interpreter too, if they're one of the 5% of mac 
users who aren't scared sh*tless by a command line. They may have Zope. They 
may have any of a number of other apps that are yet to be created that are 
implemented in Python.

> > Everything is in a single directory, so install/uninstall is a
> > breeze with no danger of missing things.
> 
> In case of Python, "everything" is the header files, and the .py
> library, right?

Yep. This is all you would need to either embed Python or extend it. So even 
for many developers there would be no reason to get a source distribution.
 
> Couldn't you also get this by making Python an Application Bundle
> (which you intend to do anyway)?

I've looked at that, but I don't think you can get the sharing explained above 
by that. Also, the development tools all know about frameworks, and if we'd 
want to pick stuff up from an application bundle we would have to hand-craft 
things.

> > Also, the structure of a framework allows you to have older,
> > incompatible versions of the library (and support documents) in the
> > framework as well, with these older versions only used by programs
> > that were linked against the framework when that version was
> > current.
> 
> Again, why is that desirable? If you have multiple versions of Python
> installed, if each version has libpython linked statically, and comes
> with its own set of header files and .py files: there wouldn't be any
> older, incompatible versions of the library that you may need to
> defend against.

You shouldn't really be thinking "multiple versions of Python" here but 
"multiple products which are implemented with different versions of Python". 
Think of the sort of things RealBasic can do, allowing end-users with a little 
programming experience create fullblown applications they can distribute. 
That's where I would like Python on OSX to go. And iff we get the ObjC stuff 
revived and make it work together with Cocoa then those end users can use 
Interface Builder to create their UI, so they main piece of functionality that 
Python misses when compared to RealBasic, the guibuilder, is provided for us 
by Apple.

> You may wonder why I question that feature so much. To me, it is
> similar to building a shared libpython.so on Unix. I think this will
> cause many problems, and in no way it should be done just for a single
> platform (there were patches to do this for Linux, I believe, and such
> a patch to make it work specifically on UnixWare only even made it
> into 2.1). I think it is desirable that the feature "shared libpython"
> is implemented as uniformly as possible across systems. 
> 
> If Mac OS X wants to do it differently from everybody else, you need a
> very good reason.

This is probably the crux of your reasoning, and I think I disagree. For one, 
MacOSX (ok, OpenStep is where they got it from) is the first unix-based system 
that I'm aware of that seems to have a solution to the versioning problem. All 
unixes that I know simply stick a version number in the .so filename and leave 
it at that, and the only reason dll-hell hasn't hit unix as hard as Windows is 
because it's deployed a lot less, used by more knowledgeable people and 
Microsoft doesn't get to touch it. But the problem is essentially there. Now 
that MacOSX has a versioning system we should put it to good use. And maybe 
we'll like it so much that we can use it for all Pythons.

Also, while I would like Python to be a model MacOSX citizen I'm also trying 
hard to make it as good a Python community citizen as it can be, so I'm 
putting a 100% compatible unix Python installation into the framework. Not 
only will this make pythonistas with a unix background feel at home, but it 
should also mean that there's no extra code needed in distutils and all the 
other stuff that knows what a Python installation looks like.

> The only platform that already has a shared pythonXY.dll has a good
> reason: you cannot export symbols from the application to extension
> modules on Windows. Yet, this is also the only platform where the API
> version magic is meaningless, since extension modules break with every
> new Python version exactly because libpython is shared.

The same is true for MacPython, and also true for Python on MacOSX. But 
whereas with MacPython extension modules also broke every single release (even 
if only because the underlying C library was a moving target) with MacOSX I 
think we can do a bit better. The only thing to work is is exactly how to put 
version numbers to good use, because extension modules linked against Python 
will be interested primarily in the API version number but applications using 
Python will be interested primarily in the 2.2-style version number, and I'm 
not sure yet how to work that out. Maybe prepending the API version number (so 
the full version would becomes something like "1011.2.2) will work.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 




From aahz@rahul.net  Tue Aug 14 15:03:36 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Tue, 14 Aug 2001 07:03:36 -0700 (PDT)
Subject: [Python-Dev] Proposed PEP: Optimizing Global Variable and Attribute Access
In-Reply-To: <15224.48600.621401.101492@beluga.mojam.com> from "Skip Montanaro" at Aug 14, 2001 12:57:44 AM
Message-ID: <20010814140336.C55D0E8C1@waltz.rahul.net>

Skip Montanaro wrote:
> 
>     The number of tracked objects should be relatively small.  All active
>     frames of all active threads could conceivably be tracking objects, but
>     this seems small compared to the number of functions defined in a given
>     application.

Hmmm?  That doesn't seem quite right to me.  That's just a knee-jerk
reaction, I don't have any real evidence.

As a side note, I'm not sure whether some mention should be made that
TRACK_OBJECT specifically does not work any differently from current
Python when it comes to threads.  That is, if it's a shared variable of
any sort, you need a mutex if you're going to perform multiple
operations on it.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From guido@python.org  Tue Aug 14 15:13:28 2001
From: guido@python.org (Guido van Rossum)
Date: Tue, 14 Aug 2001 10:13:28 -0400
Subject: [Python-Dev] Direction of PyChecker
In-Reply-To: Your message of "Tue, 14 Aug 2001 08:11:13 EDT."
 <3B78DD21.31888.224FE356@localhost>
References: <200108131356.JAA17936@cj20424-a.reston1.va.home.com>
 <3B78DD21.31888.224FE356@localhost>
Message-ID: <200108141413.KAA20547@cj20424-a.reston1.va.home.com>

> > ....  We've got
> > pychecker and pydoc as to drive the requirements.  
> 
> Don't forget freeze/modulefinder.py.

And pyclbr.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com (Skip Montanaro)  Tue Aug 14 15:48:41 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Tue, 14 Aug 2001 09:48:41 -0500
Subject: [Python-Dev] Proposed PEP: Optimizing Global Variable and Attribute Access
In-Reply-To: <20010814140336.C55D0E8C1@waltz.rahul.net>
References: <15224.48600.621401.101492@beluga.mojam.com>
 <20010814140336.C55D0E8C1@waltz.rahul.net>
Message-ID: <15225.14921.675135.797646@beluga.mojam.com>

    Aahz> Skip Montanaro wrote:

    >> The number of tracked objects should be relatively small.  All active
    >> frames of all active threads could conceivably be tracking objects,
    >> but this seems small compared to the number of functions defined in a
    >> given application.

    Aahz> Hmmm?  That doesn't seem quite right to me.  That's just a
    Aahz> knee-jerk reaction, I don't have any real evidence.

Here's my rationale.  Perhaps I've goofed somewhere.  TRACK_OBJECT and
UNTRACK_OBJECT opcodes will only be inserted in the bodies of functions.
Functions are only considered active when there is an active (but possibly
suspended) eval_frame call on the C stack that is executing that function's
byte code.  Suppose my module is

    def a():
      print "a"

    def b():
      print "b"
      a()

    def c():
      print "c"

    b()

At the point where a's print statement is executed, functions a and b are
active.  Function c is not.

This reasoning applies on a per-thread basis.  So the number of active
functions by my definition is the number of times eval_frame appears on the
call stack for all active threads.  In most situations that will be far less
(barring deep recursion) than the number of functions available to the
application.  All I guess I'm trying to communicate is that object tracking
is a dynamic thing, not a static thing.

    Aahz> As a side note, I'm not sure whether some mention should be made
    Aahz> that TRACK_OBJECT specifically does not work any differently from
    Aahz> current Python when it comes to threads.  That is, if it's a
    Aahz> shared variable of any sort, you need a mutex if you're going to
    Aahz> perform multiple operations on it.

That's true.  If a global object is shared among multiple threads, access to
it must still be protected.  I haven't added anything to the mix in that
regard.  I'll add a short section on threads.

Hmm...  What about this (dumb) code?

    l = []
    lock = threading.Lock()
    ...
    def fill_l():
        for i in range(1000):
            lock.acquire()
            l.append(math.sin(i))
            lock.release()
    ...
    def consume_l():
        while 1:
            lock.acquire()
            if l:
                elt = l.pop()
            lock.release()
            fiddle(elt)

It's not clear from a static analysis of the code what the lock is
protecting.  (You can't tell at compile-time that threads are even involved
can you?)  Would or should it affect attempts to track "l.append" or
"math.sin" in the fill_l function?

If we annotate the code with mythical track_object and untrack_object
builtins (I'm not proposing such functions (they can't be implemented with
Python's call-by-value semantics), just illustrating where stuff would go in
the bytecode), we get

    l = []
    lock = threading.Lock()
    ...
    def fill_l():
        track_object("l.append", append)
        track_object("math.sin", sin)
        for i in range(1000):
            lock.acquire()
            append(sin(i))
            lock.release()
        untrack_object("math.sin", sin)
        untrack_object("l.append", append)
    ...
    def consume_l():
        while 1:
            lock.acquire()
            if l:
                elt = l.pop()
            lock.release()
            fiddle(elt)

Is that correct both with and without threads?

Skip



From akuchlin@mems-exchange.org  Tue Aug 14 15:52:53 2001
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Tue, 14 Aug 2001 10:52:53 -0400
Subject: [Python-Dev] 2.2a1 patch from PyChecker
In-Reply-To: <15224.37281.601490.99294@anthem.wooz.org>; from barry@zope.com on Mon, Aug 13, 2001 at 10:49:05PM -0400
References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108092120.f79LKNo13875@odiug.digicool.com> <20010809180903.A31343@ute.cnri.reston.va.us> <15224.37281.601490.99294@anthem.wooz.org>
Message-ID: <20010814105253.B9746@ute.cnri.reston.va.us>

On Mon, Aug 13, 2001 at 10:49:05PM -0400, Barry A. Warsaw wrote:
>Which reminds me: do you think you'll one day resurrect PEP 222?

I'll take another look at it.  It seems impossible to converge on a
list of advanced features to add (everyone wants something different),
but it would be nice to clean out some of the cruft that's accumulated
in cgi.py.

--amk



From neal@metaslash.com  Tue Aug 14 16:13:32 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Tue, 14 Aug 2001 11:13:32 -0400
Subject: [Python-Dev] Re: AST mining (was Re: Direction of PyChecker)
References: <B79E2503.1606C%dgoodger@bigfoot.com>
Message-ID: <3B79401C.82D51FBA@metaslash.com>

David Goodger wrote:
> 
> >From the PyChecker AST discussion, it seems we may have a common goal. For

It seems that way.

> the Docstring Processing System, I am looking into gleaning information from
> the abstract syntax tree. From the working notes, under "Docstring
> Extractor":
> 
>     We need code that scans a parsed Python module, and returns an
>     ordered tree containing the names, docstrings (including
>     additional docstrings), and additional info (in parentheses below)
>     of all of the following objects:

>     - methods (+ formal parameters)
>     - functions (+ formal parameters)

Would you also want default parameter values?

PyChecker needs all of that, plus additional info.  So there is a lot
of overlap.

> I'd be very interested in pooling efforts to make this easier. I know almost
> nothing about ASTs now, but that could change in a hurry :-).

Pooling efforts would be good.  I also don't know anything about 
the ASTs/compiler, but am willing to work on it.

Neal


From jeremy@zope.com  Tue Aug 14 16:29:01 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Tue, 14 Aug 2001 11:29:01 -0400 (EDT)
Subject: [Python-Dev] AST mining (was Re: Direction of PyChecker)
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEIPLFAA.tim.one@home.com>
References: <B79E2503.1606C%dgoodger@bigfoot.com>
 <LNBBLJKPBEHFEDALKOLCOEIPLFAA.tim.one@home.com>
Message-ID: <15225.17341.690478.601359@slothrop.digicool.com>

>>>>> "TP" == Tim Peters <tim.one@home.com> writes:

  TP> Let me suggest you don't really want an AST -- you want an
  TP> object model for Python source that answers the questions above
  TP> directly.

Agreed.

  TP> directly.  An AST may be an effective (under the covers)
  TP> implementation technique to get such info, but if you don't want
  TP> to wait for people to argue about "the right" AST and "the
  TP> right" tree-based query language to make it better than
  TP> completely useless <wink>, you can answer all the stuff above by
  TP> building on tokenize.py now.

I wouldn't wait for people to argue about the right AST either.  Use
the one Greg and Bill came up with for p2c.  It's in Tools/compiler in
the Python distribution.  It's much simpler than the parse tree
produced by the parser module.  And, as far as I know, no one is
advocating a different AST.

It doesn't have a query language, but neither does tokenize.py <wink>.

Jeremy



From guido@python.org  Tue Aug 14 16:43:03 2001
From: guido@python.org (Guido van Rossum)
Date: Tue, 14 Aug 2001 11:43:03 -0400
Subject: [Python-Dev] 2.2a1: classmethod() and class attributes
In-Reply-To: Your message of "Tue, 14 Aug 2001 10:05:01 +0200."
 <200108140805.f7E851j01484@mira.informatik.hu-berlin.de>
References: <200108140805.f7E851j01484@mira.informatik.hu-berlin.de>
Message-ID: <200108141543.f7EFh3X10221@odiug.digicool.com>

> > If you add __dynamic__ = 1 to the class statement, it works as
> > expected.
> 
> Is that documented anywhere yet? I searched all three documents (the
> PEPs, and the intro), and could not find any mentioning of
> __dynamic__.

Not yet, sorry.  Based on feedback, I may have to change the default
to __dynamic__ = 1.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@zope.com  Tue Aug 14 16:50:40 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Tue, 14 Aug 2001 11:50:40 -0400 (EDT)
Subject: [Python-Dev] Re: AST mining (was Re: Direction of PyChecker)
In-Reply-To: <3B79401C.82D51FBA@metaslash.com>
References: <B79E2503.1606C%dgoodger@bigfoot.com>
 <3B79401C.82D51FBA@metaslash.com>
Message-ID: <15225.18640.291156.850128@slothrop.digicool.com>

>>>>> "NN" == Neal Norwitz <neal@metaslash.com> writes:

  >> I'd be very interested in pooling efforts to make this easier. I
  >> know almost nothing about ASTs now, but that could change in a
  >> hurry :-).

  NN> Pooling efforts would be good.  I also don't know anything about
  NN> the ASTs/compiler, but am willing to work on it.

I'm on the hook for AST/compiler documentation, which I plan to work
on this week.  I will probably have more time for it later in the week
than I will today or tomorrow.

In the absence of documentation, here's a trivial example program that
extracts some information about methods and attributes from a class
and its methods.  I wasn't exhaustive here.  I'll get attributes
assigned to by "self.x" in a method body and I'll get method
definitions in the class body.  I don't deal with obvious things like
attributes defined at the class level.

Jeremy

from compiler import parseFile, walk, ast

class Class:

    def __init__(self, name, bases):
        self.name = name
        self.bases = bases
        self.methods = {}
        self.attributes = {}

    def addMethod(self, meth):
        self.methods[meth.name] = meth

    def addInstanceAttr(self, name):
        self.attributes[name] = name

    def getMethodNames(self):
        return self.methods.keys()

    def getAttrNames(self):
        return self.attributes.keys()

class Method:

    def __init__(self, name, args, defaults):
        self.name = name
        self.args = args
        self.defaults = defaults

    def getSelf(self):
        return self.args[0]

class ClassExtractor:

    classes = []

    def visitClass(self, node, klass=None, meth=None):
        c = Class(node.name, node.bases)
        self.visit(node.code, c)
        self.classes.append(c)

    def visitFunction(self, node, klass=None, meth=None):
        if klass is not None and meth is None:
            m = Method(node.name, node.argnames, node.defaults)
            klass.addMethod(m)
            self.visit(node.code, klass, m)
        else:
            self.visit(node.code)

    def visitAssAttr(self, node, klass=None, meth=None):
        if isinstance(node.expr, ast.Name) and meth is not None:
            if node.expr.name == meth.getSelf():
                klass.addInstanceAttr(node.attrname)
        else:
            self.visit(node.expr)

def main(py_files):
    extractor = ClassExtractor()
    for py in py_files:
        ast = parseFile(py)
        walk(ast, extractor)

    for klass in extractor.classes:
        print klass.name
        print klass.getMethodNames()
        print klass.getAttrNames()
        print

if __name__ == "__main__":
    import sys
    main(sys.argv[1:])




From tony@lsl.co.uk  Tue Aug 14 16:56:12 2001
From: tony@lsl.co.uk (Tony J Ibbs (Tibs))
Date: Tue, 14 Aug 2001 16:56:12 +0100
Subject: [Docstring-develop] RE: [Python-Dev] AST mining (was Re: Direction
 of        PyChecker)
In-Reply-To: <15225.17341.690478.601359@slothrop.digicool.com>
Message-ID: <005301c124d9$a47e5350$f05aa8c0@lslp7o.int.lsl.co.uk>

Jeremy Hylton wrote:
> I wouldn't wait for people to argue about the right AST either.  Use
> the one Greg and Bill came up with for p2c.  It's in Tools/compiler in
> the Python distribution.  It's much simpler than the parse tree
> produced by the parser module.  And, as far as I know, no one is
> advocating a different AST.

Indeed, it was the one I used for my stpy.py "testbed" earlier this year
(this had code that trivially found docstrings for a module and its
classes and functions/methods, and presented them (albeit not very
prettily) in HTML as such. The code to use compiler was *very* simple,
at this level.

It even works under 1.5.2 (with, I think, two lines commented out). As I
remember, it wasn't terribly fast, but who cares...

It also lacked documentation, but that just means some more staring at
code (and maybe the parser documentation) to work out what to expect.

> It doesn't have a query language, but neither does tokenize.py <wink>.

Hmm - for the purposes we're after, any "query" is purely theoretical, I
think (i.e., in the sense that *everything* done on a datastructure
might be considered a query). At least for the moment.

Tibs

--
Tony J Ibbs (Tibs)      http://www.tibsnjoan.co.uk/
"How fleeting are all human passions compared with the massive
continuity of ducks." - Dorothy L. Sayers, "Gaudy Night"
My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.)



From tim.one@home.com  Tue Aug 14 16:58:20 2001
From: tim.one@home.com (Tim Peters)
Date: Tue, 14 Aug 2001 11:58:20 -0400
Subject: [Python-Dev] AST mining (was Re: Direction of PyChecker)
In-Reply-To: <15225.17341.690478.601359@slothrop.digicool.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEKKLFAA.tim.one@home.com>

[Jeremy Hylton]
> ...
> I wouldn't wait for people to argue about the right AST either.  Use
> the one Greg and Bill came up with for p2c.  It's in Tools/compiler in
> the Python distribution.  It's much simpler than the parse tree
> produced by the parser module.  And, as far as I know, no one is
> advocating a different AST.
>
> It doesn't have a query language, but neither does tokenize.py <wink>.

The likely problem I see is documentation:  exactly what is this AST, and
how does one use the compiler package?  compiler's demo.py is exactly
on-target for what people want to do here, and is very simple code, but how
would someone coming into this cold come up with this code on their own?

The import tricks don't help.  I just tried to reverse-engineer demo.py and
got stuck on the

    ast = compiler.parse(buf)

line.  Granted I've running a fever, but I'm not sure I would have thought
to study __init__.py even if not.

The relative feebleness of tokenize.py is an advantage in this respect:  it
doesn't do much, has some docs, and is all in one function, so it's
straightforward to guess what it's doing.



From aahz@rahul.net  Tue Aug 14 17:16:10 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Tue, 14 Aug 2001 09:16:10 -0700 (PDT)
Subject: [Python-Dev] Proposed PEP: Optimizing Global Variable and Attribute Access
In-Reply-To: <15225.14921.675135.797646@beluga.mojam.com> from "Skip Montanaro" at Aug 14, 2001 09:48:41 AM
Message-ID: <20010814161610.21ADDE8C1@waltz.rahul.net>

[sorry, excessive quoting ahead]

Skip Montanaro wrote:
> 
> 
>     Aahz> Skip Montanaro wrote:
> 
>     >> The number of tracked objects should be relatively small.  All active
>     >> frames of all active threads could conceivably be tracking objects,
>     >> but this seems small compared to the number of functions defined in a
>     >> given application.
> 
>     Aahz> Hmmm?  That doesn't seem quite right to me.  That's just a
>     Aahz> knee-jerk reaction, I don't have any real evidence.
> 
> Here's my rationale.  Perhaps I've goofed somewhere.  TRACK_OBJECT and
> UNTRACK_OBJECT opcodes will only be inserted in the bodies of functions.
> Functions are only considered active when there is an active (but possibly
> suspended) eval_frame call on the C stack that is executing that function's
> byte code.  Suppose my module is
> 
>     def a():
>       print "a"
> 
>     def b():
>       print "b"
>       a()
> 
>     def c():
>       print "c"
> 
>     b()
> 
> At the point where a's print statement is executed, functions a and b are
> active.  Function c is not.
>
> This reasoning applies on a per-thread basis.  So the number of active
> functions by my definition is the number of times eval_frame appears on the
> call stack for all active threads.  In most situations that will be far less
> (barring deep recursion) than the number of functions available to the
> application.  All I guess I'm trying to communicate is that object tracking
> is a dynamic thing, not a static thing.

That's true.  My point was more that, in my knee-jerk opinion, if one
uses, say, math.sin() at all in a module, it will be active in a large
number of the callframes.  Therefore, the number of tracked objects
could easily get quite large.  It's a relative thing, of course, and I
still think your statistical analysis about cost/benefit is accurate.

>     Aahz> As a side note, I'm not sure whether some mention should be made
>     Aahz> that TRACK_OBJECT specifically does not work any differently from
>     Aahz> current Python when it comes to threads.  That is, if it's a
>     Aahz> shared variable of any sort, you need a mutex if you're going to
>     Aahz> perform multiple operations on it.
> 
> That's true.  If a global object is shared among multiple threads, access to
> it must still be protected.  I haven't added anything to the mix in that
> regard.  I'll add a short section on threads.
> 
> Hmm...  What about this (dumb) code?
> 
>     l = []
>     lock = threading.Lock()
>     ...
>     def fill_l():
>         for i in range(1000):
>             lock.acquire()
>             l.append(math.sin(i))
>             lock.release()
>     ...
>     def consume_l():
>         while 1:
>             lock.acquire()
>             if l:
>                 elt = l.pop()
>             lock.release()
>             fiddle(elt)
> 
> It's not clear from a static analysis of the code what the lock is
> protecting.  (You can't tell at compile-time that threads are even involved
> can you?)  Would or should it affect attempts to track "l.append" or
> "math.sin" in the fill_l function?

Oooooo.  Ouch.

No, on third thought, it still doesn't matter.  If someone is playing
games with module globals and not using locks, that code is broken in
current Python.  We don't need your track() and untrack() functions, but
that should be made clear in the PEP, I think.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From fdrake@acm.org  Tue Aug 14 17:28:47 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 14 Aug 2001 12:28:47 -0400 (EDT)
Subject: [Python-Dev] Re: AST mining (was Re: Direction of PyChecker)
In-Reply-To: <15225.18640.291156.850128@slothrop.digicool.com>
References: <B79E2503.1606C%dgoodger@bigfoot.com>
 <3B79401C.82D51FBA@metaslash.com>
 <15225.18640.291156.850128@slothrop.digicool.com>
Message-ID: <15225.20927.158390.711237@cj42289-a.reston1.va.home.com>

Jeremy Hylton writes:
 > I'm on the hook for AST/compiler documentation, which I plan to work
 > on this week.  I will probably have more time for it later in the week
 > than I will today or tomorrow.

  So between standing over your shoulder to make sure this gets done,
and watching over Barry's shoulder to make sure the smtpd module gets
documented, I'm going to get a fair bit of exercise this week... well,
I guess some people would consider that a good thing.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From barry@zope.com  Tue Aug 14 18:03:45 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Tue, 14 Aug 2001 13:03:45 -0400
Subject: [Python-Dev] Python 2.2 Release Schedule
Message-ID: <15225.23025.251404.251797@yyz.digicool.com>

Here is PEP 251, the Python 2.2 release schedule, revised after our
Pythonlabs meeting of yesterday.  Please note the following
highlights:

- Final release is now scheduled for 19-Dec-2001

- We plan the second 2.2 alpha for 22-Aug-2001

- We plan to release a third alpha.

As always, the PEP is available online at

    http://python.sourceforge.net/peps/pep-0251.html

Enjoy,
-Barry

-------------------- snip snip --------------------
PEP: 251
Title: Python 2.2 Release Schedule
Version: $Revision: 1.4 $
Author: guido@python.org (Guido van Rossum), barry@zope.com (Barry A. Warsaw)
Status: Incomplete
Type: Informational
Created: 17-Apr-2001
Python-Version: 2.2
Post-History: 14-Aug-2001

Abstract

    This document describes the post-Python 2.1 development and
    release schedule.  The schedule primarily concerns itself with
    PEP-sized items.  Small bug fixes and changes will occur up until
    the first beta release.

    NOTE: the schedule below and the list of features under
    consideration are all subject to change!  If the energy in the
    community changes or the feedback on a PEP is particularly
    positive or negative, this may affect decisions.


Release Schedule

    Tentative future release dates.  Note that we've slipped this
    compared to the schedule posted around the release of 2.2a1.

    19-Dec-2001: 2.2   (final release)
    12-Dec-2001: 2.2c1 (release candidate)
    14-Nov-2001: 2.2b2
    10-Oct-2001: 2.2b1
    19-Sep-2001: 2.2a3 (new! a third alpha)
    22-Aug-2001: 2.2a2
    18-Jul-2001: 2.2a1


Release Manager

    Barry Warsaw will take over as the release manager.  Guido and
    Barry will release 2.2a2 together, after that Barry will be
    responsible for releases.


Release Mechanics

    We'd like to experiment with a new mechanism for releases: a week
    before every alpha, beta or other release, we'll fork off a branch
    which will become the release; changes to the branch will have to
    be approved before they can be checked in.  This is how some other
    large projects (e.g. Mozilla) work, and we hope it will help
    reduce the number of bugs introduced in releases at the last
    minute.


Planned features for 2.2

    The following features are already checked in on the head revision
    (for a more detailed account, see Misc/NEWS):

    - iterators (PEP 234)
    - generators (PEP 255)
    - division (PEP 238)
    - unification of types and classes (PEP 252, PEP 253)

    Work on the class/type unification is still ongoing.

    The following features are under consideration:

    - unifying long ints and plain ints (PEP 237)

    There needs to be more discussion of each of these before we can
    decide.


Copyright

    This document has been placed in the public domain.



Local Variables:
mode: indented-text
indent-tabs-mode: nil
End:


From guido@python.org  Tue Aug 14 19:42:42 2001
From: guido@python.org (Guido van Rossum)
Date: Tue, 14 Aug 2001 14:42:42 -0400
Subject: [Python-Dev] Revised PEP 237 - Unifying Long Integers and Integers
Message-ID: <200108141842.f7EIggo10895@odiug.digicool.com>

Please comment on the completely revised PEP 237:

  http://python.sourceforge.net/peps/pep-0237.html

I'm proposing an implementation plan that keeps int and long as
separate types, but otherwise will totally remove the differences.

I'm also proposing a transition plan that does one easy thing in 2.2
(change ops that raise OverflowError to return a long result instead),
and puts off addressing the more severe incompatibilities to 2.3.

This is mostly because I don't see how we can make time to design and
implement the necessary future statements or other compatibility
measures in time.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@zope.com  Tue Aug 14 19:53:40 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Tue, 14 Aug 2001 14:53:40 -0400 (EDT)
Subject: [Python-Dev] AST mining (was Re: Direction of PyChecker)
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEKKLFAA.tim.one@home.com>
References: <15225.17341.690478.601359@slothrop.digicool.com>
 <LNBBLJKPBEHFEDALKOLCEEKKLFAA.tim.one@home.com>
Message-ID: <15225.29620.135166.848323@slothrop.digicool.com>

>>>>> "TP" == Tim Peters <tim.one@home.com> writes:

  TP> [Jeremy Hylton]
  >> ...  I wouldn't wait for people to argue about the right AST
  >> either.  Use the one Greg and Bill came up with for p2c.  It's in
  >> Tools/compiler in the Python distribution.  It's much simpler
  >> than the parse tree produced by the parser module.  And, as far
  >> as I know, no one is advocating a different AST.
  >>
  >> It doesn't have a query language, but neither does tokenize.py
  >> <wink>.

  TP> The likely problem I see is documentation: exactly what is this
  TP> AST, and how does one use the compiler package?

As I already noted in this thread, the documentation problem will not
survive the week.

Jeremy



From skip@pobox.com (Skip Montanaro)  Tue Aug 14 20:03:40 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Tue, 14 Aug 2001 14:03:40 -0500
Subject: [Python-Dev] Re: Revised PEP 237 - Unifying Long Integers and Integers
In-Reply-To: <mailman.997814820.4250.clpa-moderators@python.org>
References: <mailman.997814820.4250.clpa-moderators@python.org>
Message-ID: <15225.30220.37379.978290@beluga.mojam.com>

    Guido> I'm proposing an implementation plan that keeps int and long as
    Guido> separate types, but otherwise will totally remove the
    Guido> differences.

    ...

    Guido> Please mail comments to me or to python-dev@python.org.

Looks reasonable to me.  I don't generally program anywhere near the edges
of current short integer arithmentic except when writing little demo
functions like fib or fact.  I wonder though, if some programmers might like
the option of telling integer arithmentic raise OverflowError instead of
coercing results to long ints.  If you're expecting to be playing with
numbers near zero and you start silently getting longs instead, it might be
nice to see an exception instead of having your program wander off into the
weeds (probably somewhat slowly).

Just a thought.

Skip



From guido@python.org  Tue Aug 14 20:13:40 2001
From: guido@python.org (Guido van Rossum)
Date: Tue, 14 Aug 2001 15:13:40 -0400
Subject: [Python-Dev] Re: Revised PEP 237 - Unifying Long Integers and Integers
In-Reply-To: Your message of "Tue, 14 Aug 2001 14:03:40 CDT."
 <15225.30220.37379.978290@beluga.mojam.com>
References: <mailman.997814820.4250.clpa-moderators@python.org>
 <15225.30220.37379.978290@beluga.mojam.com>
Message-ID: <200108141913.f7EJDeq11097@odiug.digicool.com>

> Looks reasonable to me.  I don't generally program anywhere near the
> edges of current short integer arithmentic except when writing
> little demo functions like fib or fact.  I wonder though, if some
> programmers might like the option of telling integer arithmentic
> raise OverflowError instead of coercing results to long ints.

You can do this by enabling the warning-on-overflow command line
option (not yet specified) and then using -Werr to turn the warning
into an error.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@loewis.home.cs.tu-berlin.de  Tue Aug 14 19:54:03 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Tue, 14 Aug 2001 20:54:03 +0200
Subject: [Python-Dev] Any NeXT users out there?
In-Reply-To: <20010814124304.01511303181@snelboot.oratrix.nl> (message from
 Jack Jansen on Tue, 14 Aug 2001 14:43:03 +0200)
References: <20010814124304.01511303181@snelboot.oratrix.nl>
Message-ID: <200108141854.f7EIs3u02092@mira.informatik.hu-berlin.de>

> Sharing is the answer.
...
> This is probably the crux of your reasoning, and I think I
> disagree. For one, MacOSX (ok, OpenStep is where they got it from)
> is the first unix-based system that I'm aware of that seems to have
> a solution to the versioning problem.

I'm a bit sceptical to any claim that the software versioning problem
is "solved"; experience tells me it isn't. If you allow me to make
some half-serious predictions out of the blue, I'd say:

- The API will change more often than you'ld like, so the newer
  version cannot replace the older one in a compatible way.

- every potential application that could make use of the Python
  framework will come with its own copy of Python, just to reduce the
  hassles for the users of the application to get a separate copy of
  Python, and to allow them to completely uninstall everything by
  deleting a single directory. This, specifically, includes Zope,
  which by tradition always comes with its own Python installation
  (with extension modules and everything).

- whenever the framework really is shared across applications, people
  will find that the application that did not bring the framework will
  break in subtle ways; authors of the application will stop using the
  framework in their next release.

So the only application that really ever uses the framework will be
the interpreter proper.

That said, I see that your patch removes a lot of old next stuff, and
replaces it with something that at least functions as designed, so I'm
+0 on this patch. I may be wrong with my predictions, and the only way
to prove me wrong is to use this patch, so just go ahead.

Regards,
Martin


From mwh@python.net  Tue Aug 14 22:35:15 2001
From: mwh@python.net (Michael Hudson)
Date: 14 Aug 2001 17:35:15 -0400
Subject: [Python-Dev] PEP 264
In-Reply-To: Michael Hudson's message of "08 Aug 2001 17:54:50 -0400"
References: <LNBBLJKPBEHFEDALKOLCGEHCLEAA.tim.one@home.com> <2m8zgudxp1.fsf@starship.python.net>
Message-ID: <2mitfq8gvg.fsf_-_@starship.python.net>

I *think* I'm done coding for implementing PEP 264; patch at:

http://sourceforge.net/tracker/?func=detail&atid=305470&aid=449043&group_id=5470

I've also done a first draft of docs.

The reason for mentioning this is I'm off on holiday tomorrow, so if
you want changes before the proposed date for 2.2a2, you've got aobut
12 hours to tell me about it.

Some tests might get attached to the patch later this evening.

In related news, the material on nested scopes in the appendix of the
refman should be moved out of the appendix.  Someone who's not typing
down a modem can do that.

Cheers,
M.

-- 
  The ultimate laziness is not using Perl.  That saves you so much
  work you wouldn't believe it if you had never tried it.
                                        -- Erik Naggum, comp.lang.lisp


From James_Althoff@i2.com  Tue Aug 14 22:37:11 2001
From: James_Althoff@i2.com (James_Althoff@i2.com)
Date: Tue, 14 Aug 2001 14:37:11 -0700
Subject: [Python-Dev] __class__ assignment in 2.2a1
Message-ID: <OF6A350424.CF4885E7-ON88256AA8.00765F5E@i2.com>

The following seems like it could use a more helpful error message.
Presumably it fails because you can only set the value of __class__ to that
of another class object whose __dynamic__  attribute == 1?

C:\>python
Python 2.2a1 (#21, Jul 18 2001, 04:25:46) [MSC 32 bit (Intel)] on win32
Type "copyright", "credits" or "license" for more information.
>>> class C(object): pass
...
>>> class D: pass
...
>>> class E: pass
...
>>> d = D()
>>> d.__class__ = E
>>> d.__class__ = C
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: __class__ must be set to a class
>>>

Jim



From fdrake@acm.org  Tue Aug 14 22:52:28 2001
From: fdrake@acm.org (Fred L. Drake)
Date: Tue, 14 Aug 2001 17:52:28 -0400 (EDT)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20010814215228.696DA28845@cj42289-a.reston1.va.home.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

A wide range of mostly small fixes and updates.  Some new C API functions
are documented.  Examples are given for how to mark up tables in the
"Documenting Python" guide.  The language reference contains more
information about support // and true division.



From neal@metaslash.com  Tue Aug 14 23:15:11 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Tue, 14 Aug 2001 18:15:11 -0400
Subject: [Python-Dev] Module docstrings patches
Message-ID: <3B79A2EF.F5207F26@metaslash.com>

I submitted the following to the patch manager at SF.

450979 Add module docstring - imputil
450980 Add module docstring - sre*, re
450981 Add module docstring - xmlrpc

Should the pre module be marked as deprecated?
There's still a comment in re that pre "will be removed in 2.0 final."

Neal


From James_Althoff@i2.com  Tue Aug 14 23:23:22 2001
From: James_Althoff@i2.com (James_Althoff@i2.com)
Date: Tue, 14 Aug 2001 15:23:22 -0700
Subject: [Python-Dev] __dynamic__ attribute
Message-ID: <OFE7CD3DA3.C1A7F83A-ON88256AA8.00785883@i2.com>

Given that  __dynamic__ will be part of Python in 2.2 . . .

Is it worth considering the possibility of making it more granular (somehow
controlling separately the modification of attribute values versus the
addition or deletion of attributes, perhaps)?  Currently it controls the
changing of a class object in all ways (if set to 1 it makes the class
object immutable).  It seems to me there is a significant difference in
general usage between:

>>> class A(object):
...   __dynamic__ = 1
...   count = 0
...   def incr(myclass): myclass.count += 1
...   incr = classmethod(incr)
...
>>> A.count
0
>>> A.incr()
>>> A.count
1
>>>

which is a pretty common idiom in Java and C++ and other more dynamic
things like adding/deleting attributes (data and methods) on the fly.

Jim




From James_Althoff@i2.com  Tue Aug 14 23:36:52 2001
From: James_Althoff@i2.com (James_Althoff@i2.com)
Date: Tue, 14 Aug 2001 15:36:52 -0700
Subject: [Python-Dev] re: __class__ assignment in 2.2a1
Message-ID: <OF4ABDFF3B.9C4EA796-ON88256AA8.007BD455@i2.com>

I guess the rule is that you can only assign __class__ for classic class
instances and the new value has to be a classic class also (ignoring
C-written metaclasses, etc.).  So __dynamic__ doesn't come into play.
Still seems like the error message is a bit confusing.

>>>>>>>>>>>>>>>>>>>.

The following seems like it could use a more helpful error message.
Presumably it fails because you can only set the value of __class__ to that
of another class object whose __dynamic__  attribute == 1?

C:\>python
Python 2.2a1 (#21, Jul 18 2001, 04:25:46) [MSC 32 bit (Intel)] on win32
Type "copyright", "credits" or "license" for more information.
>>> class C(object): pass
...
>>> class D: pass
...
>>> class E: pass
...
>>> d = D()
>>> d.__class__ = E
>>> d.__class__ = C
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: __class__ must be set to a class
>>>

Jim



From jeremy@zope.com  Tue Aug 14 23:42:42 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Tue, 14 Aug 2001 18:42:42 -0400 (EDT)
Subject: [Python-Dev] Re: AST mining (was Re: Direction of PyChecker)
In-Reply-To: <15225.20927.158390.711237@cj42289-a.reston1.va.home.com>
References: <B79E2503.1606C%dgoodger@bigfoot.com>
 <3B79401C.82D51FBA@metaslash.com>
 <15225.18640.291156.850128@slothrop.digicool.com>
 <15225.20927.158390.711237@cj42289-a.reston1.va.home.com>
Message-ID: <15225.43362.623842.699456@slothrop.digicool.com>

A first and quite incomplete draft of docs for the compiler package
has been checked in Tools/compiler/doc.  The HTML version is currently
available from
    http://www.python.org/~jeremy/compiler/

Jeremy



From barry@zope.com  Tue Aug 14 23:48:57 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Tue, 14 Aug 2001 18:48:57 -0400
Subject: [Python-Dev] PEP comments, bugs, and patches
References: <200108080713.JAA21365@pandora.informatik.hu-berlin.de>
Message-ID: <15225.43737.762160.158737@anthem.wooz.org>

>>>>> "MvL" == Martin von Loewis <loewis@informatik.hu-berlin.de> writes:

    MvL> I just found that PEP 1 is confusing in its instructions of
    MvL> how to submit feedback on a PEP.

I've just added some clarification about this to PEP 1.

-Barry


From barry@zope.com  Wed Aug 15 01:08:56 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Tue, 14 Aug 2001 20:08:56 -0400
Subject: [Python-Dev] Proposed PEP: Optimizing Global Variable and Attribute Access
References: <15224.48600.621401.101492@beluga.mojam.com>
Message-ID: <15225.48536.32259.429582@anthem.wooz.org>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    SM> Here's a first stab at a PEP for optimizing global variable
    SM> and attribute access.  Barry, when you get a chance, could you
    SM> assign it a number and let me know about any PEP-related
    SM> formatting mistakes I made?

Done.  It's PEP 266, but I had to shorten the title.  It was too long
even with the new limit <44 wink> (give 'em an inch... :).  Other than
that you did a fine job for a virgin PEP author!  I made a few other
edits, but mostly so I could piss on the hydrant. :)
    
    SM> (It would be nice if PEP 1 referenced a piece of boilerplate
    SM> people could use.  While the PEP Style section lists the
    SM> headers, I wasn't sure what the Version and Last-Modified
    SM> headers were actually supposed to look like.  Also, it wasn't
    SM> clear what I was supposed to put in the Post-History: header.)

Both excellent suggestions.  I've added PEP 9, which can be used as a
template for both Informational and Standards Track PEPs.  I've often
found I wanted something I could just copy-and-edit to get started
quickly on a new PEP.

I've clarified a few things in PEP 1.

Thanks!
-Barry



From skip@pobox.com (Skip Montanaro)  Wed Aug 15 01:19:53 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Tue, 14 Aug 2001 19:19:53 -0500
Subject: [Python-Dev] Proposed PEP: Optimizing Global Variable and Attribute Access
In-Reply-To: <15225.48536.32259.429582@anthem.wooz.org>
References: <15224.48600.621401.101492@beluga.mojam.com>
 <15225.48536.32259.429582@anthem.wooz.org>
Message-ID: <15225.49193.119814.964280@beluga.mojam.com>

    SM> Barry, when you get a chance, could you assign it a number and let
    SM> me know about any PEP-related formatting mistakes I made?

    BAW> Done.  It's PEP 266, but I had to shorten the title.  It was too
    BAW> long even with the new limit <44 wink>

Hmmm...  Wasn't aware there was a limit.  I guess that's why you're the PEP
wizard Barry!

Thanks,

Skip


From gward@python.net  Wed Aug 15 01:48:55 2001
From: gward@python.net (Greg Ward)
Date: Tue, 14 Aug 2001 20:48:55 -0400
Subject: [Python-Dev] New regrtest.py gimmick
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEDGLFAA.tim.one@home.com>; from tim.one@home.com on Sat, Aug 11, 2001 at 09:24:15PM -0400
References: <LNBBLJKPBEHFEDALKOLCKEDGLFAA.tim.one@home.com>
Message-ID: <20010814204855.A3097@gerg.ca>

On 11 August 2001, Tim Peters said:
> For you it will print something like:
> 
> ...
> MMM tests OK.
> NN tests skipped: XXX YYY ...
> Ask someone to teach regrtest.py about which tests are
> expected to get skipped on [sys.platform].
> 
> 
> You do that by changing the new _ExpectedSkips.__init__ near the bottom of
> regrtest.py.
> 
> If sys.platform isn't enough to determine the set of skipped tests, then
> sys.platform isn't particularly useful <0.9 wink>.

Note that distutils.util.get_platform() is, for POSIX platforms at
least, sys.platform on steroids:

  def get_platform ():
      """Return a string that identifies the current platform.  This is used
      mainly to distinguish platform-specific build directories and
      platform-specific built distributions.  Typically includes the OS name
      and version and the architecture (as supplied by 'os.uname()'),
      although the exact information included depends on the OS; eg. for IRIX
      the architecture isn't particularly important (IRIX only runs on SGI
      hardware), but for Linux the kernel version isn't particularly
      important.

      Examples of returned values:
         linux-i586
         linux-alpha (?)
         solaris-2.6-sun4u
         irix-5.3
         irix64-6.2

      For non-POSIX platforms, currently just returns 'sys.platform'.
      """

But then, much of the Distutils is other parts of the standard library
on steroids.  (Sigh.)

        Greg
-- 
Greg Ward - Linux geek                                  gward@python.net
http://starship.python.net/~gward/
A man without religion is like a fish without a bicycle.


From gward@python.net  Wed Aug 15 01:56:18 2001
From: gward@python.net (Greg Ward)
Date: Tue, 14 Aug 2001 20:56:18 -0400
Subject: [Python-Dev] Parrot -- should life imitate satire?
In-Reply-To: <3B77143F.F9158ED1@ActiveState.com>; from paulp@ActiveState.com on Sun, Aug 12, 2001 at 04:41:51PM -0700
References: <Pine.LNX.4.21.0108121606420.1411-100000@tuatha.sidhe.org> <3B77143F.F9158ED1@ActiveState.com>
Message-ID: <20010814205618.B3097@gerg.ca>

On 12 August 2001, Paul Prescod said:
> In fact, there are many bits of the C library that Python people would
> like to replace or improve but we don't have the resources alone. Maybe
> if all of the scripting language groups got together we would be able to
> do it. I'm not talking about the larger project of sharing bytecodes and
> so forth, but I could see that we probably have very similar needs with
> respect to files, memory allocation, networking, event loops, process
> handling and maybe threads. And the other scripting languages are
> probably not too far off either.

Isn't a lot of that addressed by glib, the C library used by GTK+ (and
therefore Gnome) applications for all those things that C programs tend
to do that aren't GUI-related?

And speaking of I/O libraries, whatever happened to sfio?  I remember
struggling with it and PerlIO back around Perl 5.003/5.004 in a vain
attempt to get FastCGI working.  sfio seemed at the time like a good
idea just didn't catch on, probably because stdio is "good enough" for
most people.  (But then, "most people" don't implement Perl or Python.)

(I also remember that sfio's version number is the year of release -- so
much for "release early, release often".  It also had a rather
idiosyncratic build system -- autoconf? what's that?)

        Greg
-- 
Greg Ward - Unix bigot                                  gward@python.net
http://starship.python.net/~gward/
I'd rather have a bottle in front of me than have to have a frontal lobotomy.


From dgoodger@bigfoot.com  Wed Aug 15 02:48:44 2001
From: dgoodger@bigfoot.com (David Goodger)
Date: Tue, 14 Aug 2001 21:48:44 -0400
Subject: [Python-Dev] Re: AST mining (was Re: Direction of PyChecker)
In-Reply-To: <3B79401C.82D51FBA@metaslash.com>
Message-ID: <B79F4D3B.16174%dgoodger@bigfoot.com>

Neal Norwitz <neal@metaslash.com> wrote on 2001-08-14 11:13:
> Would you also want default parameter values?

Yes.

> PyChecker needs all of that, plus additional info.  So there is a lot
> of overlap.

I think the DPS will use a small subset of what PyChecker needs. The only
extra information the DPS may need is attribute and additional docstrings,
string literals in certain contexts that are not currently recognized as
docstrings (see PEP 258).

> Pooling efforts would be good.  I also don't know anything about
> the ASTs/compiler, but am willing to work on it.

I look forward to it, and to reading Jeremy's docs. (Go, doc writer!)

-- 
David Goodger    dgoodger@bigfoot.com    Open-source projects:
 - Python Docstring Processing System: http://docstring.sourceforge.net
 - reStructuredText: http://structuredtext.sourceforge.net
 - The Go Tools Project: http://gotools.sourceforge.net



From aahz@rahul.net  Wed Aug 15 03:15:18 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Tue, 14 Aug 2001 19:15:18 -0700 (PDT)
Subject: [Python-Dev] Re: AST mining (was Re: Direction of PyChecker)
In-Reply-To: <15225.43362.623842.699456@slothrop.digicool.com> from "Jeremy Hylton" at Aug 14, 2001 06:42:42 PM
Message-ID: <20010815021519.8371AE8C2@waltz.rahul.net>

Jeremy Hylton wrote:
> 
> A first and quite incomplete draft of docs for the compiler package
> has been checked in Tools/compiler/doc.  The HTML version is currently
> available from
>     http://www.python.org/~jeremy/compiler/

You interested in comments on this incomplete draft?  No?  Well, here
they are, anyway:

http://www.python.org/~jeremy/compiler/node4.html

    What do you mean by "visitor instance"?  Is that a fancy name for
    function (in which case, just say "function"), or is it something
    else?

    "Compile the file filename and generated filename.pyc" should be
    "generate" I think.

http://www.python.org/~jeremy/compiler/node8.html

    getChildNodes() says "This method is like getChildNodes", which
    should be getChildren

Hmmm...  I guess that wasn't very much, in the end.  I avoided making
comments on areas that clearly were going to get more work.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From aahz@rahul.net  Wed Aug 15 16:07:29 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Wed, 15 Aug 2001 08:07:29 -0700 (PDT)
Subject: [Python-Dev] Re: 2.2a1: classmethod() and class attributes
In-Reply-To: <no.id> from "aahz" at Aug 13, 2001 07:31:11 PM
Message-ID: <20010815150729.D2A2CE8CA@waltz.rahul.net>

Aahz:
> It's not particularly convenient for me to try out 2.2a1, so I'm just
> going by what's written.  One little hole that I don't see an answer to
> is what happens when you do this:
> 
> class C(object):
>     x = 0
>     def foo(cls):
>         cls.x += 1
>     foo = classmethod(foo)
> 
> C.foo()

Okay, after thinking about this a bit, I think that if the above code
requires __dynamic__=1 to work, then the default for __dynamic__ should
be changed.  I don't find the arguments about changing __class__ to be
particularly persuasive, but I think the above code *is* closely related
to standard Python idioms that should work by default.

+1 on changing __dynamic__ or at least enabling some kind of class
variable mutability by default.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From Samuele Pedroni <pedroni@inf.ethz.ch>  Wed Aug 15 16:17:58 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Wed, 15 Aug 2001 17:17:58 +0200 (MET DST)
Subject: [Python-Dev] Re: 2.2a1: classmethod() and class attributes
Message-ID: <200108151518.RAA15602@core.inf.ethz.ch>

> Aahz:
> > It's not particularly convenient for me to try out 2.2a1, so I'm just
> > going by what's written.  One little hole that I don't see an answer to
> > is what happens when you do this:
> > 
> > class C(object):
> >     x = 0
> >     def foo(cls):
> >         cls.x += 1
> >     foo = classmethod(foo)
> > 
> > C.foo()
> 
> Okay, after thinking about this a bit, I think that if the above code
> requires __dynamic__=1 to work, then the default for __dynamic__ should
> be changed.  I don't find the arguments about changing __class__ to be
> particularly persuasive, but I think the above code *is* closely related
> to standard Python idioms that should work by default.
> 
> +1 on changing __dynamic__ or at least enabling some kind of class
> variable mutability by default.
> -- 

My 2cts: I think this is also a matter of how much complex the language
is perceived:

Before 2.2: everything is dynamic and mutable (pretty general rule,
  a performance killer but that's another point ;) )
 
After:
 - this is mutable and that's not (a more complex picture)
 vs.
 
 - by default everything is mutable ... (advanced user)
  but you can tune that (for good reasons)
  
A marginal note: I agree with James Althoff that some more granularity
about dynamic behavior would be a good thing (but only if the newbie
is protected from the burden)

regards, Samuele Pedroni.



From guido@python.org  Wed Aug 15 16:25:53 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 15 Aug 2001 11:25:53 -0400
Subject: [Python-Dev] Re: 2.2a1: classmethod() and class attributes
In-Reply-To: Your message of "Wed, 15 Aug 2001 08:07:29 PDT."
 <20010815150729.D2A2CE8CA@waltz.rahul.net>
References: <20010815150729.D2A2CE8CA@waltz.rahul.net>
Message-ID: <200108151525.LAA27011@cj20424-a.reston1.va.home.com>

> +1 on changing __dynamic__ or at least enabling some kind of class
> variable mutability by default.

After converting the Tools/compiler package to the new class system, I
tend to agree (I already said so on c.l.py in my last post there).

Here's a problem, related to performance.

The dispatch for special operations (like __iter__) from C uses a C
function stored in a corresponding slot (e.g. tp_iter) in the type
object.  So when class X has an __iter__ method, there needs to be a
slot wrapper (a C function that calls the __iter__ object) in the in
the tp_iter slot in the type object representing X.

For dynamic classes, I must assume that at any time someone can add a
special method (like __iter__) to a class.  It's hard to set up things
so that the dispatch function is set in the type object at the moment
C.__iter__ is assigned: it would require a class to keep track of all
its subclasses, without keeping those subclasses alive.  While I know
I can do that using weak references, I don't like having to maintain
all that administration.  So at the moment, when a class is dynamic, I
just stick all dispatch functions in the type object -- the dispatch
functions will raise AttributeError when their corresponding method is
not found.  (This is the same approach used for classic classes, BTW.)

This is slower than it should be -- the fully dynamic Tools/compiler
package compiles itself about 25% slower this way.  If I tweak it to
use all static classes (not very hard), it runs at about the same
speed as it does with classic classes.  I imagine I could make it
faster by using __slots__, but I don't know enough about the internals
yet to be able to do that.

My goal (before I'm happy with making __dynamic__=1 the default) is
that dynamic classes should be at least as fast as classic classes.  I
haven't profiled it yet -- it's possible that there's a cheap hack
possible by making more conservative assumptions about __getattr__
alone -- classic classes special-case __getattr__ too.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@zope.com  Wed Aug 15 16:30:01 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Wed, 15 Aug 2001 11:30:01 -0400
Subject: [Python-Dev] PEP 9, Sample PEP Template
Message-ID: <15226.38265.263478.150525@anthem.wooz.org>

A number of people have suggested that there be a standard PEP
template that makes it easier to get started with writing your own
PEP.  This is a good thing too, because hopefully it'll make the
submissions I see more consistent with less initial editorial
requirements (and fewer auto-rejects due to bad form <wink>).

To that end, I've written PEP 9, Sample PEP Template, which is
available at the following URL, and attached below.

    http://python.sourceforge.net/peps/pep-0009.html

Please take a look and let me know if any of the instructions in it
are confusing.  PEP 9 isn't a substitute for reading PEP 1, PEP
Purpose and Guidelines, but it should be considered an adjunct.

feeling-peppy-ly y'rs,
-Barry 

-------------------- snip snip --------------------
PEP: 9
Title: Sample PEP Template
Version: $Revision: 1.2 $
Last-Modified: $Date: 2001/08/15 15:28:57 $
Author: barry@zope.com (Barry A. Warsaw)
Status: Active
Type: Informational
Created: 14-Aug-2001
Post-History:


Abstract

    This PEP provides a boilerplate or sample template for creating
    your own PEPs.  This should make it easy for you to conform your
    own PEPs to the style outlined in PEP 1, PEP Guidelines[1].


Rationale

    PEP submissions come in a wide variety of forms, not all adhering
    to the style guidelines set forth in PEP 1.  Use this template, in
    conjunction with the prose guidelines in PEP 1, to ensure that
    your PEP submission won't get automatically rejected because of
    form.

    To use this template you must first decide whether your PEP is
    going to be Informational or Standards Track.  Most PEPs are
    Standards Track because they propose a new feature for the Python
    language or standard library.  When in doubt, read PEP 1 for
    details.

    Once you've decided which type of PEP yours is going to be, follow
    the directions in the appropriate section below.


How to Use This Template for Standard Track PEPs

    - Make a copy of this file and perform the following edits.

    - replace the "PEP: 9" header with "PEP: XXX" since you don't yet
      have a PEP number assignment.

    - Change the Title: header to the title of your PEP.

    - Leave the Version: and Last-Modified: headers alone; we'll take
      care of those when we check your PEP into CVS.

    - Change the Author: header to include your email address and
      name.  Be sure to follow the format carefully: your email
      address must appear first, and it should not be contained in
      angle brackets.  Your full name should appear second and it
      should appear in parentheses.

    - If there is a mailing list for discussion of your new feature,
      add a Discussions-To: header right after the Author: header.
      You should not add a Discussions-To: header if the mailing list
      to be used is either python-list@python.org or
      python-dev@python.org, or if discussions should be sent to you
      directly.

    - Change the Status: header to "Draft".

    - Change the Type: header to "Standards Track".

    - If your feature depends on the acceptance of some other currently
      in-development PEP, add a Requires: header right after the Type:
      header.  The value should be the PEP number of the PEP yours
      depends on.  Don't add this header if your dependent feature is
      described in a Final PEP.

    - Change the Created: header to today's date.  Be sure to follow
      the format carefully: it must be in dd-mmm-yyyy format, where
      the mmm is the 3 letter month abbreviation.

    - After the Created: header, add a Python-Version: header and set
      the value to the next planned version of Python, i.e. the one
      your new feature will hopefully make its first appearance in.
      Do not use an alpha or beta release designation here.  Thus, if
      the last version of Python was 2.2 alpha 1 and you're hoping to
      get your new feature into Python 2.2, set the header to:

      Python-Version: 2.2

    - Leave Post-History: alone for now; you'll add dates to this
      header after your PEP has been assigned a number and you've
      posted your PEP to python-list@python.org or
      python-dev@python.org.

    - Add a Replaces: header if your PEP describes a feature that will
      replace a feature described in some Final PEP.  Usually you
      won't need to add this header.  It's also unlikely that you'll
      ever add a Replaced-By: header.

    - Now write your abstract, rationale, and other content for your
      PEP, replacing all this gobbledygook with your own text.  Be
      sure to adhere to the PEP 1 style guidelines, specifically on
      the prohibition of tab characters and the indentation
      requirements.

    - Update your References and Copyright section.  Usually you'll
      place your PEP into the public domain, in which case just leave
      the Copyright section alone.  Alternatively, you can use the
      Open Publication License[2], but public domain is still strongly
      preferred.

    - Leave the little Emacs turd at the end of this file alone,
      including the formfeed character (^L, or \f).

    - Send your PEP submission to the PEP editor, along with $100k in
      unmarked pennies.  (Just kidding, I wanted to see if you were
      still awake. :)


How to Use This Template for Informational PEPs

    - Make a copy of this file and perform the following edits.

    - replace the "PEP: 9" header with "PEP: XXX" since you don't yet
      have a PEP number assignment.

    - Change the Title: header to the title of your PEP.

    - Leave the Version: and Last-Modified: headers alone; we'll take
      care of those when we check your PEP into CVS.

    - Change the Author: header to include your email address and
      name.  Be sure to follow the format carefully: your email
      address must appear first, and it should not be contained in
      angle brackets.  Your full name should appear second and it
      should appear in parentheses.

    - If there is a mailing list for discussion of your new feature,
      add a Discussions-To: header right after the Author: header.
      You should not add a Discussions-To: header if the mailing list
      to be used is either python-list@python.org or
      python-dev@python.org, or if discussions should be sent to you
      directly.  Most Information PEPs don't have a Discussions-To:
      header.

    - Change the Status: header to "Active".

    - Change the Type: header to "Informational".

    - Change the Created: header to today's date.  Be sure to follow
      the format carefully: it must be in dd-mmm-yyyy format, where
      the mmm is the 3 letter month abbreviation.

    - Leave Post-History: alone for now; you'll add dates to this
      header after your PEP has been assigned a number and you've
      posted your PEP to python-list@python.org or
      python-dev@python.org.

    - Now write your abstract, rationale, and other content for your
      PEP, replacing all this gobbledygook with your own text.  Be
      sure to adhere to the PEP 1 style guidelines, specifically on
      the prohibition of tab characters and the indentation
      requirements.

    - Update your References and Copyright section.  Usually you'll
      place your PEP into the public domain, in which case just leave
      the Copyright section alone.  Alternatively, you can use the
      Open Publication License[2], but public domain is still strongly
      preferred.

    - Leave the little Emacs turd at the end of this file alone,
      including the formfeed character (^L, or \f).

    - Send your PEP submission to the PEP editor, along with $100k in
      unmarked pennies.  (Just kidding, I wanted to see if you were
      still awake. :)


References

    [1] PEP 1, PEP Purpose and Guidelines
        http://www.python.org/peps/pep-0001.html

    [2] http://www.opencontent.org/openpub/



Copyright

    This document has been placed in the public domain.



Local Variables:
mode: indented-text
indent-tabs-mode: nil
End:



From tony@lsl.co.uk  Wed Aug 15 16:31:29 2001
From: tony@lsl.co.uk (Tony J Ibbs (Tibs))
Date: Wed, 15 Aug 2001 16:31:29 +0100
Subject: [Docstring-develop] Re: [Python-Dev] Re: AST mining (was Re:
 Direction        of PyChecker)
In-Reply-To: <15225.18640.291156.850128@slothrop.digicool.com>
Message-ID: <006d01c1259f$5ade9950$f05aa8c0@lslp7o.int.lsl.co.uk>

Hmm. Is everyone on that list of recipients really interested?

Anyway, I've spent a happy time (when I should have been earning money -
oh well) expanding Jeremy's example, ast-mining.py, to do more things.
I've done all of the easy stuff first (makes sense to me). Anyway, it
now reports on a lot more stuff, and is probably already useful enough
to be "plugged in" to initial DPS/reST usages.

Rather than make this message even longer, I've put it at:

	http://www.tibsnjoan.co.uk/reST/ast-mining.py

I've been testing it by running it on itself.

Comments, as ever, welcome.

Tibs

--
Tony J Ibbs (Tibs)      http://www.tibsnjoan.co.uk/
"How fleeting are all human passions compared with the massive
continuity of ducks." - Dorothy L. Sayers, "Gaudy Night"
My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.)



From skip@pobox.com (Skip Montanaro)  Wed Aug 15 17:00:52 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Wed, 15 Aug 2001 11:00:52 -0500
Subject: [Python-Dev] PyObject_SetAttr - is it the only way types should set attributes?
Message-ID: <15226.40116.68738.183684@beluga.mojam.com>

In thinking about the implementation of TRACK_GLOBAL some more, I'm trying
to figure out where object updating is going to take place.  Is
PyObject_GenericSetAttr the only way C-based objects should set attributes?
If so, that would be a central place to provide this functionality.
Otherwise, I fear that a lot of C code will have to be twiddled to provide
this functionality.

I worry especially about third-party extension modules.  Accordingly, I took
a look at

    http://www.python.org/doc/ext/dnt-type-methods.html

Unfortunately, that section seems to be truncated (in the TeX source as
well).  I filed a bug report:

    http://sourceforge.net/tracker/index.php?func=detail&aid=451217&group_id=5470&atid=105470

Skip


From jeremy@zope.com  Wed Aug 15 17:06:25 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Wed, 15 Aug 2001 12:06:25 -0400 (EDT)
Subject: [Python-Dev] PyObject_SetAttr - is it the only way types should set attributes?
In-Reply-To: <15226.40116.68738.183684@beluga.mojam.com>
References: <15226.40116.68738.183684@beluga.mojam.com>
Message-ID: <15226.40449.521721.265985@slothrop.digicool.com>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

  SM> In thinking about the implementation of TRACK_GLOBAL some more,
  SM> I'm trying to figure out where object updating is going to take
  SM> place.  Is PyObject_GenericSetAttr the only way C-based objects
  SM> should set attributes?

C code can manipulate __dict__, too, right?  It's a pretty common
idiom to get a module dictionary and use PyDict_SetItem().  

I'm also curious about the new type-class unification stuff.  Is it
possible to modify a C object via something like a C struct and have
that change be visible as a modified attrbute at the Python level?

Jeremy



From akuchlin@mems-exchange.org  Wed Aug 15 17:49:42 2001
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Wed, 15 Aug 2001 12:49:42 -0400
Subject: [Python-Dev] PyObject_SetAttr - is it the only way types should set attributes?
In-Reply-To: <15226.40449.521721.265985@slothrop.digicool.com>; from jeremy@zope.com on Wed, Aug 15, 2001 at 12:06:25PM -0400
References: <15226.40116.68738.183684@beluga.mojam.com> <15226.40449.521721.265985@slothrop.digicool.com>
Message-ID: <20010815124942.D15765@ute.cnri.reston.va.us>

On Wed, Aug 15, 2001 at 12:06:25PM -0400, Jeremy Hylton wrote:
>C code can manipulate __dict__, too, right?  It's a pretty common
>idiom to get a module dictionary and use PyDict_SetItem().  

But most commonly that's only done in the C extension's init*()
function, right?  

--amk



From akuchlin@mems-exchange.org  Wed Aug 15 17:56:40 2001
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Wed, 15 Aug 2001 12:56:40 -0400
Subject: [Python-Dev] New regrtest.py gimmick
In-Reply-To: <20010814204855.A3097@gerg.ca>; from gward@python.net on Tue, Aug 14, 2001 at 08:48:55PM -0400
References: <LNBBLJKPBEHFEDALKOLCKEDGLFAA.tim.one@home.com> <20010814204855.A3097@gerg.ca>
Message-ID: <20010815125640.E15765@ute.cnri.reston.va.us>

On Tue, Aug 14, 2001 at 08:48:55PM -0400, Greg Ward wrote:
>But then, much of the Distutils is other parts of the standard library
>on steroids.  (Sigh.)

It would certainly be worth booting some bits of the Distutils up into
the standard library: text_file, some of the functions in util.py, and
perhaps fancy_getopt.py.

--amk



From skip@pobox.com (Skip Montanaro)  Wed Aug 15 18:02:00 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Wed, 15 Aug 2001 12:02:00 -0500
Subject: [Python-Dev] PyObject_SetAttr - is it the only way types should set attributes?
In-Reply-To: <15226.40449.521721.265985@slothrop.digicool.com>
References: <15226.40116.68738.183684@beluga.mojam.com>
 <15226.40449.521721.265985@slothrop.digicool.com>
Message-ID: <15226.43784.663087.346750@beluga.mojam.com>

    SM> In thinking about the implementation of TRACK_GLOBAL some more, I'm
    SM> trying to figure out where object updating is going to take place.
    SM> Is PyObject_GenericSetAttr the only way C-based objects should set
    SM> attributes?

    Jeremy> C code can manipulate __dict__, too, right?  It's a pretty
    Jeremy> common idiom to get a module dictionary and use
    Jeremy> PyDict_SetItem().

Yeah, I believe so.  In theory, I think any code that could modify what
LOAD_GLOBAL or LOAD_ATTR instructions might return would have to be changed
to signal an update.

    Jeremy> I'm also curious about the new type-class unification stuff.  Is
    Jeremy> it possible to modify a C object via something like a C struct
    Jeremy> and have that change be visible as a modified attrbute at the
    Jeremy> Python level?

If a C extension defines a function that changes an exposed attribute I
suspect it would have to be changed.  I'll try and make this notification as
easy as possible for the programmer (a single function call), but it would
definitely require extension writers to be pretty careful in updating to a
version of Python that contains this functionality.  In the stuff I've seen
so far this seems to be the most difficult problem to resolve.

Skip



From fdrake@acm.org  Wed Aug 15 17:59:54 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 15 Aug 2001 12:59:54 -0400 (EDT)
Subject: [Python-Dev] Re: AST mining (was Re: Direction of PyChecker)
In-Reply-To: <20010815021519.8371AE8C2@waltz.rahul.net>
References: <15225.43362.623842.699456@slothrop.digicool.com>
 <20010815021519.8371AE8C2@waltz.rahul.net>
Message-ID: <15226.43658.208451.723960@cj42289-a.reston1.va.home.com>

Aahz Maruch writes:
 >     What do you mean by "visitor instance"?  Is that a fancy name for
 >     function (in which case, just say "function"), or is it something
 >     else?

  I'll let Jeremy fix this.  ;-)

 >     "Compile the file filename and generated filename.pyc" should be
 >     "generate" I think.

  Fixed in CVS.

 > http://www.python.org/~jeremy/compiler/node8.html
 > 
 >     getChildNodes() says "This method is like getChildNodes", which
 >     should be getChildren

  Fixed in CVS.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From guido@python.org  Wed Aug 15 18:14:10 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 15 Aug 2001 13:14:10 -0400
Subject: [Python-Dev] PyObject_SetAttr - is it the only way types should set attributes?
In-Reply-To: Your message of "Wed, 15 Aug 2001 12:06:25 EDT."
 <15226.40449.521721.265985@slothrop.digicool.com>
References: <15226.40116.68738.183684@beluga.mojam.com>
 <15226.40449.521721.265985@slothrop.digicool.com>
Message-ID: <200108151714.NAA27351@cj20424-a.reston1.va.home.com>

> C code can manipulate __dict__, too, right?  It's a pretty common
> idiom to get a module dictionary and use PyDict_SetItem().  

Yes -- always using the PyModule_GetDict() API.

> I'm also curious about the new type-class unification stuff.  Is it
> possible to modify a C object via something like a C struct and have
> that change be visible as a modified attrbute at the Python level?

Can you clarify this question?  I feel I should be qualified to answer
this but I don't understand the question...  Maybe an example would
help?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com (Skip Montanaro)  Wed Aug 15 18:37:10 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Wed, 15 Aug 2001 12:37:10 -0500
Subject: [Python-Dev] PyObject_SetAttr - is it the only way types should set attributes?
In-Reply-To: <200108151714.NAA27351@cj20424-a.reston1.va.home.com>
References: <15226.40116.68738.183684@beluga.mojam.com>
 <15226.40449.521721.265985@slothrop.digicool.com>
 <200108151714.NAA27351@cj20424-a.reston1.va.home.com>
Message-ID: <15226.45894.43208.544121@beluga.mojam.com>

    >> I'm also curious about the new type-class unification stuff.  Is it
    >> possible to modify a C object via something like a C struct and have
    >> that change be visible as a modified attrbute at the Python level?

    Guido> Can you clarify this question?  I feel I should be qualified to
    Guido> answer this but I don't understand the question...  Maybe an
    Guido> example would help?

I'll try to channel Jeremy.  Even if I fail, I think the problem I describe
is worth noting for PEP 0266.

Suppose I have an extension module that declares a new type containing a
"foo" slot in the C struct for the type.  If I implement a method named
"bar", it's obviously free to modify any of the contents in the struct in
any way it sees fit.  Now, suppose I also have a getattrfunc that will
return the contents of the "foo" slot when asked to.  There needs not be any
setattrfunc or setattrofunc, yet the value in this slot has to be tracked if
it might be accessed by a programmer.

I suspect this is just one more place a C extension programmer can blow a
gaping hole in his foot, but I worry that there's no obvious way to alert
the programmer to such breakage if/when they upgrade to a version of Python
that tracks globals.  Seems like a source of very subtle bugs.

Skip



From guido@python.org  Wed Aug 15 19:10:58 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 15 Aug 2001 14:10:58 -0400
Subject: [Python-Dev] PyObject_SetAttr - is it the only way types should set attributes?
In-Reply-To: Your message of "Wed, 15 Aug 2001 12:37:10 CDT."
 <15226.45894.43208.544121@beluga.mojam.com>
References: <15226.40116.68738.183684@beluga.mojam.com> <15226.40449.521721.265985@slothrop.digicool.com> <200108151714.NAA27351@cj20424-a.reston1.va.home.com>
 <15226.45894.43208.544121@beluga.mojam.com>
Message-ID: <200108151810.OAA30795@cj20424-a.reston1.va.home.com>

[Jeremy]
>     >> I'm also curious about the new type-class unification stuff.  Is it
>     >> possible to modify a C object via something like a C struct and have
>     >> that change be visible as a modified attrbute at the Python level?
> 
>     Guido> Can you clarify this question?  I feel I should be qualified to
>     Guido> answer this but I don't understand the question...  Maybe an
>     Guido> example would help?

[Skip]
> I'll try to channel Jeremy.  Even if I fail, I think the problem I describe
> is worth noting for PEP 0266.
> 
> Suppose I have an extension module that declares a new type containing a
> "foo" slot in the C struct for the type.  If I implement a method named
> "bar", it's obviously free to modify any of the contents in the struct in
> any way it sees fit.  Now, suppose I also have a getattrfunc that will
> return the contents of the "foo" slot when asked to.  There needs not be any
> setattrfunc or setattrofunc, yet the value in this slot has to be tracked if
> it might be accessed by a programmer.
> 
> I suspect this is just one more place a C extension programmer can blow a
> gaping hole in his foot, but I worry that there's no obvious way to alert
> the programmer to such breakage if/when they upgrade to a version of Python
> that tracks globals.  Seems like a source of very subtle bugs.

Under the new system, the recommended way to make "foo" an attribute
of the object is to have an attribute descriptor named foo in the
type's __dict__.  This descriptor is most likely created when the type
was initialized from the tp_members or tp_getset fields in the type
object.  (I've checked in an example to Modules/xxsubtype.c -- do a
cvs update and look for "struct memberlist".)

I suppose the globals-tracking mechanism could have enough smarts to
look in the type's __dict__ for attribute descriptors; if it finds
certain types of descriptors it should assume that these can be
changed at any time.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From MusicMaster" <aria@t2u.com  Wed Aug 15 19:15:19 2001
From: MusicMaster" <aria@t2u.com (MusicMaster)
Date: Wed, 15 Aug 2001 14:15:19 -0400
Subject: [Python-Dev] Just Updated!
Message-ID: <E15X5CD-0007DJ-00@mail.python.org>

This newsletter is provided to you FREE! 

To unsubscribe, reply to this email with REMOVE in the Subject.

August, 15, 2001

Today's Topics:
(Full Stories, See Below)

-MADONNA Cancels NJ Concert
-Blues Traveler: New Album, New Tour
-U2 Plans more US Tour Dates
-311 To Headline Following Warped Tour
-CONCERT TICKETS Anyone?


MADONNA CANCELS NJ CONCERT:
Madonna's Friday night (8/3) concert at the Continental Airlines Arena in
East Rutherford, N.J., has been canceled, promoter Clear Channel
Entertainment announced. According to the artist's management, she is
suffering from laryngitis. 
The canceled show--which will not be rescheduled--was to have been Madonna's
seventh sold-out performance in the New York area. The decision not to
reschedule immediately drew an angry response from fans posting on her
official website <http://www.madonnamusic.com>. 
"I think it's absolutely ridiculous that she didn't re-schedule the concert,"
one fan wrote. "I think it's safe to say that she alienated a lot of her fans
for not being willing to move the show to a later date." 
All Ticketmaster phone and Internet orders will be automatically refunded and
credited to the card holder within two weeks, according to a press release.
Tickets purchased at Ticketmaster outlets and at the Continental Airlines
Arena box office will be refunded at the point of purchase. (Ticketmaster is
LiveDaily's parent company.) 
Madonna's tour is expected to resume in Boston on Aug. 7 and 8. 


BLUES TRAVELER: NEW ALBUM, NEW TOUR
Blues Traveler returns to the tour circuit to promote its new album,
"Bridge." The tour is set to begin August 18th.
Blues Traveler has revealed the next batch of dates for its summer tour,
which kicks off on Aug. 18 in Oneida, Wis. 
The band will perform in mostly secondary markets including Sedona, Ariz.,
Buffalo, N.Y., and Portsmouth, Va. The Pontiac, Mich., show on Aug. 31 is
part of the free Labor Day festival Arts, Beats and Eats.
On Aug. 25, Blues Traveler will perform on "Weekend Today" on NBC. 
More October tour dates will be announced soon, according to the band's
spokesperson. 
The second leg of Blues Traveler's summer jaunt coincides with the release of
"Back in the Day," the second single from its fifth studio album, "Bridge"
(A&M Records/Interscope). 
GO Here For More Info:
http://www.livedaily.citysearch.com/news/printable.html?id=3478

U2 Plans More US Tour Dates:
U2, which wraps up a tour of Europe later this month, plans to return to
North America later this year to play 25 additional concert dates, the Times
of London reported this week. 
Though there have been persistent rumors that more North American dates will
be added, no dates have been confirmed by the band or by concert organizers. 
The Times of London reported that U2's Elevation tour has grossed $142
million from worldwide ticket sales, and the that the new dates are expected
to gross an additional $33 million. According to Pollstar, U2's March-June
run through North America grossed $69 million. 
If estimates prove accurate--and if U2 does add the additional dates--U2's
Elevation Tour could rank as the third-highest grossing North American tour
of all time, behind 1994 outings by the Rolling Stones ($121 million) and
Pink Floyd ($103.5 million), according to Pollstar. 

311 To Headline Following Warped Tour
As its stint on the Warped Tour winds down, 311 has revealed the dates for a
late summer headlining tour. 
Warped ends on Sunday (8/12) in Pontiac, Mich., after which 311 heads to
Europe for a handful of shows. The band returns to the United States in late
August and begins its stateside tour on Sept. 8 in San Diego. The last date,
according to its official website <http://www.311music.com>, is a Halloween
show in Los Angeles. 
The group will then spend the early part of November in Japan. 
All of the dates are in support of 311's latest record, "From Chaos," 311's
first for Volcano/Zomba.

CONCERT TICKETS Anyone?
We found a great place to get your next pair of tickets to your favorite
event! We have dealt with them many times and they have really prooved a
great source for tickets to concerts, sports, theater; All Nationwide. Check
them out at www.tixx4u.com


That's about it for now, see ya next time!


From skip@pobox.com (Skip Montanaro)  Wed Aug 15 19:24:41 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Wed, 15 Aug 2001 13:24:41 -0500
Subject: [Python-Dev] Just Updated!
In-Reply-To: <E15X5CD-0007DJ-00@mail.python.org>
References: <E15X5CD-0007DJ-00@mail.python.org>
Message-ID: <15226.48745.552793.340591@beluga.mojam.com>

    >> This newsletter is provided to you FREE! 

    >> To unsubscribe, reply to this email with REMOVE in the Subject.

    >> August, 15, 2001

    >> Today's Topics:
    >> (Full Stories, See Below)

    >> -MADONNA Cancels NJ Concert
    >> -Blues Traveler: New Album, New Tour
    >> -U2 Plans more US Tour Dates
    >> -311 To Headline Following Warped Tour
    >> -CONCERT TICKETS Anyone?

I wonder how these people knew to contact the Mojam and Musi-Cal webmaster
at this address?  Pretty smart, I'd say... ;-)

-- 
Skip Montanaro (skip@pobox.com)
http://www.mojam.com/
http://www.musi-cal.com/


From aahz@rahul.net  Wed Aug 15 19:46:28 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Wed, 15 Aug 2001 11:46:28 -0700 (PDT)
Subject: [Python-Dev] Just Updated!
In-Reply-To: <E15X5CD-0007DJ-00@mail.python.org> from "MusicMaster" at Aug 15, 2001 02:15:19 PM
Message-ID: <20010815184628.9DE18E8C2@waltz.rahul.net>

Well, now that Just has been updated, who's next?
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From fdrake@acm.org  Wed Aug 15 19:47:38 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 15 Aug 2001 14:47:38 -0400 (EDT)
Subject: [Python-Dev] Just Updated!
In-Reply-To: <20010815184628.9DE18E8C2@waltz.rahul.net>
References: <E15X5CD-0007DJ-00@mail.python.org>
 <20010815184628.9DE18E8C2@waltz.rahul.net>
Message-ID: <15226.50122.780207.399172@cj42289-a.reston1.va.home.com>

Aahz Maruch writes:
 > Well, now that Just has been updated, who's next?

  Better check your physical security devices!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From fdrake@acm.org  Wed Aug 15 20:41:36 2001
From: fdrake@acm.org (Fred L. Drake)
Date: Wed, 15 Aug 2001 15:41:36 -0400 (EDT)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20010815194136.8BEE328846@beowolf.digicool.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

More material on defining new types in C in "Extending & Embedding the
Python Interpreter":

    http://python.sourceforge.net/devel-docs/ext/defining-new-types.html



From mwh@python.net  Wed Aug 15 21:55:46 2001
From: mwh@python.net (Michael Hudson)
Date: 15 Aug 2001 16:55:46 -0400
Subject: [Python-Dev] PyObject_SetAttr - is it the only way types should set attributes?
In-Reply-To: Skip Montanaro's message of "Wed, 15 Aug 2001 11:00:52 -0500"
References: <15226.40116.68738.183684@beluga.mojam.com>
Message-ID: <2m8zgl59gt.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

> I worry especially about third-party extension modules.
> Accordingly, I took a look at
> 
>     http://www.python.org/doc/ext/dnt-type-methods.html
> 
> Unfortunately, that section seems to be truncated (in the TeX source as
> well).

That would be because I ran out of steam at that point - a few months
ago there were no docs on this at all!  I'm glad someone's fleshed it
out a bit more - something else I haven't gotten round to this summer.

Cheers,
M.

-- 
  My hat is lined with tinfoil for protection in the unlikely event
  that the droid gets his PowerPoint presentation working.
                               -- Alan W. Frame, alt.sysadmin.recovery


From tim.one@home.com  Wed Aug 15 22:24:20 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 15 Aug 2001 17:24:20 -0400
Subject: [Python-Dev] PEP 264
In-Reply-To: <2mitfq8gvg.fsf_-_@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEANLGAA.tim.one@home.com>

[Michael Hudson]
> I *think* I'm done coding for implementing PEP 264; patch at:
>
> http://sourceforge.net/tracker/?func=detail&atid=305470&aid=449043
> &group_id=5470
>
> I've also done a first draft of docs.
>
> The reason for mentioning this is I'm off on holiday tomorrow, so if
> you want changes before the proposed date for 2.2a2, you've got aobut
> 12 hours to tell me about it.

Thanks, Michael!  I was out sick yesterday, but intend to get on this
tonight.  You should enjoy your vacation!  If it's a disaster, I'll probably
villify you on python-dev, but will fix it anyway <wink>.

> ...
> In related news, the material on nested scopes in the appendix of the
> refman should be moved out of the appendix.  Someone who's not typing
> down a modem can do that.

I'm trusting that somebody else understood what that part meant.



From jeremy@zope.com  Wed Aug 15 23:03:09 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Wed, 15 Aug 2001 18:03:09 -0400 (EDT)
Subject: [Python-Dev] PEP 264
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEANLGAA.tim.one@home.com>
References: <2mitfq8gvg.fsf_-_@starship.python.net>
 <LNBBLJKPBEHFEDALKOLCEEANLGAA.tim.one@home.com>
Message-ID: <15226.61853.662980.474359@slothrop.digicool.com>

>>>>> "TP" == Tim Peters <tim.one@home.com> writes:

  >> ...  In related news, the material on nested scopes in the
  >> appendix of the refman should be moved out of the appendix.
  >> Someone who's not typing down a modem can do that.

  TP> I'm trusting that somebody else understood what that part meant.

They did, so you can ignore this message.

I added the appendix, after first trying to revise the ref manual.
(This was before nested scopes became optional for 2.1.)  I'll try to
dig up my original effort, regardless I think I am on the hook to fix
the ref manual to explain scope rules clearly.

Jeremy





From paulp@ActiveState.com  Tue Aug 14 07:50:28 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Mon, 13 Aug 2001 23:50:28 -0700
Subject: [Python-Dev] Free threading
References: <LNBBLJKPBEHFEDALKOLCOEGCLFAA.tim.one@home.com>
Message-ID: <3B78CA34.9A4E1765@ActiveState.com>

Tim Peters wrote:
> 
>...
> 
> You can't get massive scalability using OS threads, unless the OS is an
> oddball specifically designed for that.  Worming around the bloat of Windows
> and Linux (etc) OS threads is Stackless Python's natural domain (simulated
> threads).  The catch, of course, is they that don't run in parallel!  Or
> maybe by "massive" you mean a few dozen?  I worked in the "massively
> parallel supercomputer" biz, so I'm thinking more on the order of a few
> thousands of dozens.  I bet you're not.

I am thinking about getting as many threads as there are processors to
run truly in parallel.

> > ...
> > Why do people use mod_python, fcgi, mod_snake, mod_php and all of those
> > other things instead of pure CGI?
> 
> Beats me why people bother with web programming at all <wink>.

Yeah, that describes alot of us in the Python world. I think that
explains the emergence of PHP. <0.1 wink> Even Zope is really an OO
database that made a few concessions to the fashionable object delivery
mechanisms of the day. 

>...
> IIRC, Greg's fabled free-threading version of Python took a speed hit of
> about a factor of 2 (for a program using only 1 thread, compared to that
> same program without the free-threading patches).

The Perl guys considered this unacceptable and I can kind of see their
point. You have two processors but you get roughly the same performance
as one?

>..
> Well, the idea that threads don't share information is foreign to every
> intense belief about the world *I've* ever been paid to adopt <wink>, and
> I'm not enough of a Windows geek to believe "threads are always the answer"
> even so.  A pool of worker processes using OS-specific IPC as needed works
> great in my real-life experience, and if information sharing is rare, works
> especially great because it's not *fighting* the OS and C libraries tooth
> and nail.

I really don't see this as fighting anything. There are thousands of
C-coded libraries out there that can be used in multiple C threads
without sharing information between them. Some of those libraries happen
to be interpreters (e.g. PHP, VBScript) that are much more popular in
embedded server applications than Python. Maybe just a coincidence but I
think that there are a host of reasons and this may be one of them.

> What of your hypothetical user earlier, who "isn't going to be happy with
> the price of ... using IPC for information sharing either"?  That is, in
> what sense do isolated threads leave her happy about her information sharing
> needs?  

You share information (if at all) through the threaded host. The host
handles serialization of access to shared information. E.g. Apache wants
to tell you about the cookies that go with a request. That shouldn't be
copied from one process to another. It should be "just available."
Sendmail wants to tell you about the message contents of the message you
are filtering. Your SQL database with Python stored procedures has huge
shared state but the Python procedures would never share state
themselves. Your video game has a set of objects available representing
physical objects. Python interpreters would never talk to each other
about them directly -- they would always move the objects around the
objectspace and the engine routes the information between the
interpreters. 

There is no "IPC" between Python programs. All communication is between
the host environment and the interpreter. If the host chooses to share
the information with other interpreters then it is the host's job to
manage serialization and it is the host's fault if performance lags.

Anyhow, these servers exist and use threads for parallel processing.
Either we play nicely with them as a good citizen or you don't. We bend
over backwards to be a good embedded citizen on many other issues. We
could argue that Apache 2 made a wrong turn in moving from processes
towards threads but I think that's a little like announcing the falling
of the tree in the wood.
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook




From paulp@ActiveState.com  Tue Aug 14 09:08:17 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Tue, 14 Aug 2001 01:08:17 -0700
Subject: [Python-Dev] More on Perl threads
Message-ID: <3B78DC71.C1683936@ActiveState.com>

I've learned a bit more about Perl Ithreads. There is a lot that isn't
documented yet so I had to go to the developers.

The idea is that there is a function called "share" that you use to
share a value. Once you do so, the Perl interpreter builds a transparent
proxy that ensures that it is handled in a thread-safe way. Thus you
incur a performance price for only the variables you share. On the other
hand, that price sounds much higher than the price of sharing things
using the global interpreter lock (both in performance and coding
complexity). Therefore it is not a good replacement for the GIL but
perhaps a good alternative for those who need multiprocessor
performance.

According to Arthur Bergman:


"One could say that we do "proxying" internally, but that is totally
transparent to the perl programmer."

sub foo {
    my $object = {};
    bless($object, "Class");
    share($object);
    return $object;
}
"

Here's an interesting late-night idea of mine...could shareability be
contagious? If you followed a reference from a shared object to an
"ordinary" object in a different thread, the interpreter could build the
proxy for you. Actually I think that this is how XPCOM proxies work. The
more you work with stuff that came from another thread, the more your
program slows down. But if you copy data when it makes sense, you get
back to indpendent interpreter speeds until you need to share data
again.
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook




From dan@sidhe.org  Wed Aug 15 23:52:47 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Wed, 15 Aug 2001 18:52:47 -0400
Subject: [Python-Dev] More on Perl threads
In-Reply-To: <3B78DC71.C1683936@ActiveState.com>
Message-ID: <5.1.0.14.0.20010815184654.02269300@tuatha.sidhe.org>

At 01:08 AM 8/14/2001 -0700, Paul Prescod wrote:
>The idea is that there is a function called "share" that you use to
>share a value. Once you do so, the Perl interpreter builds a transparent
>proxy that ensures that it is handled in a thread-safe way. Thus you
>incur a performance price for only the variables you share. On the other
>hand, that price sounds much higher than the price of sharing things
>using the global interpreter lock (both in performance and coding
>complexity). Therefore it is not a good replacement for the GIL but
>perhaps a good alternative for those who need multiprocessor
>performance.

If your interpreters are otherwise disconnected, the proxy lock performance 
will be better than a GIL, if for no other reason then you won't have to 
get locks nearly so often. Other than that there's not much overhead--at 
most a dereference and a mutex get and release. That's not bad. It 
definitely makes a difference on SMP systems to be able to run multiple 
threads simultaneously.

>Here's an interesting late-night idea of mine...could shareability be
>contagious? If you followed a reference from a shared object to an
>"ordinary" object in a different thread, the interpreter could build the
>proxy for you.

Yeah, but you then run the risk of the sharing contagion spreading pretty 
wildly. That risk is probably lower in Python than it is in Perl, though. 
(Store a reference to an interpreter's global symbol table into a shared 
variable and watch the sharing run rampant)

Been there, done that, backed slowly away with my hands in full view. :)


					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From dan@sidhe.org  Wed Aug 15 23:59:47 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Wed, 15 Aug 2001 18:59:47 -0400
Subject: [Python-Dev] Free threading
In-Reply-To: <3B78CA34.9A4E1765@ActiveState.com>
References: <LNBBLJKPBEHFEDALKOLCOEGCLFAA.tim.one@home.com>
Message-ID: <5.1.0.14.0.20010815185358.01ba1e68@tuatha.sidhe.org>

At 11:50 PM 8/13/2001 -0700, Paul Prescod wrote:
>Tim Peters wrote:
> >...
> > IIRC, Greg's fabled free-threading version of Python took a speed hit of
> > about a factor of 2 (for a program using only 1 thread, compared to that
> > same program without the free-threading patches).
>
>The Perl guys considered this unacceptable and I can kind of see their
>point. You have two processors but you get roughly the same performance
>as one?

The perl speed hit's as much because of half-baked code as anything else. 
Changing the two instances of pthread_get_specific to 
pthread_get_specific_unchecked_np (which just bypasses the validation of 
the TSD key on Tru64 and VMS) sped up threaded perl by around 3-5%

The original, pthread-like threading model in perl was never finished for a 
variety of reasons. Do use many of the things it exposed as object lessons, 
but speed isn't one of them.

Or maybe it is. FWIW, here's a data point. When I took perl and wrapped all 
variable access in a thread-safe way (basically we got a lock before 
accessing a variable and let go when we were done) we took a 40% speed hit. 
Now, this was on OpenVMS where the mutexes are about 3-5x as expensive as 
on Tru64 on the same box, but it was a significant cost.

I racked up a whole list of "Things to Not Do With Threads" when hacking 
the original perl thread model. (The first of which is "wedge them into an 
interpreter that wasn't written with threads in mind..." :) Battle scars 
are viewable on request.

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From skip@pobox.com (Skip Montanaro)  Thu Aug 16 01:16:28 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Wed, 15 Aug 2001 19:16:28 -0500
Subject: [Python-Dev] PEP 266: Catching code that is not safe for tracking
Message-ID: <15227.4316.969063.877269@beluga.mojam.com>

While I was out for my semi-occasion-once-every-other-wednesday run this
afternoon, I thought about the problem of identifying objects that are not
safe for the tracking proposal in PEP 266.  My worry is that an extension
module that works in Python 2.2 will break in subtle ways in Python 2.3 if
my tracking idea is enabled, because the module's code is not doing anything
to notify the rest of the world when an attribute binding changes.

I came up with the following possible solution.  Add a new slot to type
objects that the object's author must explicitly set to Py_TRACKING_SAFE
(1?).  It would default to Py_TRACKING_UNSAFE (0?).  This value could be
checked by _PyObject_TrackName and a warning raised if a program tries to
track an unsafe object.

Would that work?

Skip


From guido@python.org  Thu Aug 16 07:46:58 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 16 Aug 2001 02:46:58 -0400
Subject: [Python-Dev] PEP 266: Catching code that is not safe for tracking
In-Reply-To: Your message of "Wed, 15 Aug 2001 19:16:28 CDT."
 <15227.4316.969063.877269@beluga.mojam.com>
References: <15227.4316.969063.877269@beluga.mojam.com>
Message-ID: <200108160646.CAA08652@cj20424-a.reston1.va.home.com>

> While I was out for my semi-occasion-once-every-other-wednesday run this
> afternoon, I thought about the problem of identifying objects that are not
> safe for the tracking proposal in PEP 266.  My worry is that an extension
> module that works in Python 2.2 will break in subtle ways in Python 2.3 if
> my tracking idea is enabled, because the module's code is not doing anything
> to notify the rest of the world when an attribute binding changes.
> 
> I came up with the following possible solution.  Add a new slot to type
> objects that the object's author must explicitly set to Py_TRACKING_SAFE
> (1?).  It would default to Py_TRACKING_UNSAFE (0?).  This value could be
> checked by _PyObject_TrackName and a warning raised if a program tries to
> track an unsafe object.
> 
> Would that work?

I have yet to find the time to read PEP 266, but I expect so -- this
is how I solve many such concerns in 2.2 already (e.g. an extension
type is only subclassable if it has the Py_TPFLAGS_BASETYPE flag,
which is not included in Py_TPFLAGS_DEFAULT).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com (Skip Montanaro)  Thu Aug 16 08:05:54 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Thu, 16 Aug 2001 02:05:54 -0500
Subject: [Python-Dev] PEP 266: Catching code that is not safe for tracking
In-Reply-To: <200108160646.CAA08652@cj20424-a.reston1.va.home.com>
References: <15227.4316.969063.877269@beluga.mojam.com>
 <200108160646.CAA08652@cj20424-a.reston1.va.home.com>
Message-ID: <15227.28882.297175.813711@beluga.mojam.com>

    >> I came up with the following possible solution.  Add a new slot to
    >> type objects that the object's author must explicitly set to
    >> Py_TRACKING_SAFE (1?).  It would default to Py_TRACKING_UNSAFE (0?).
    >> This value could be checked by _PyObject_TrackName and a warning
    >> raised if a program tries to track an unsafe object.
    >> 
    >> Would that work?

    Guido> I have yet to find the time to read PEP 266, but I expect so -- this
    Guido> is how I solve many such concerns in 2.2 already (e.g. an extension
    Guido> type is only subclassable if it has the Py_TPFLAGS_BASETYPE flag,
    Guido> which is not included in Py_TPFLAGS_DEFAULT).

Looks I should just add another flag to | into the tp_flags slot instead of
allocating another int-sized slot.

I hope to have some code for people to look at in the next few days.

thank-god-ellen's-out-of-town-ly, y'rs,

Skip


From fredrik@pythonware.com  Thu Aug 16 14:35:39 2001
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Thu, 16 Aug 2001 15:35:39 +0200
Subject: [Python-Dev] non-standard HTML tags in 2.1.1 documentation
Message-ID: <00a101c12658$58131e30$0900a8c0@spiff>

just grabbed the HTML versions of the 2.1.1 documentation,
and found a couple of rather interesting HTML tags in there:

<character>
<cdata>
<command>
<tex2html_cross_ref_visible_mark>
<tex2html_ampersand_mark>

can/should this be fixed?

(to find the culprits, run tidy on the files, and look for
Error lines.  or use grep...)

</F>




From fdrake@acm.org  Thu Aug 16 14:52:07 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 16 Aug 2001 09:52:07 -0400 (EDT)
Subject: [Python-Dev] non-standard HTML tags in 2.1.1 documentation
In-Reply-To: <00a101c12658$58131e30$0900a8c0@spiff>
References: <00a101c12658$58131e30$0900a8c0@spiff>
Message-ID: <15227.53255.682873.653118@cj42289-a.reston1.va.home.com>

Fredrik Lundh writes:
 > just grabbed the HTML versions of the 2.1.1 documentation,
 > and found a couple of rather interesting HTML tags in there:
...
 > can/should this be fixed?

  This should be fixed.  Please file a documentation bug report on SF
and assign it to me.  Thanks!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From just@letterror.com  Thu Aug 16 18:07:31 2001
From: just@letterror.com (Just van Rossum)
Date: Thu, 16 Aug 2001 19:07:31 +0200
Subject: [Python-Dev] PyArg_ParseTuple format string: "h" vs. "H"
Message-ID: <20010816190734-r01010700-898f3935-0910-010c@10.0.0.23>

I was just bit by an older PyArg_ParseTuple format string behavior change: these
days (since 2.1?) "h" barfs when you feed it an unsigned short value (even if
it's within 16 bits). So ok, I updated my code to use "H" instead, only to find
out hours later that "H" didn't exist yet in 1.5.2. I really need to be
compatible with both so I'll have to work around it :-(

Was this change really neccesary?

Just


From guido@python.org  Thu Aug 16 18:55:52 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 16 Aug 2001 13:55:52 -0400
Subject: [Python-Dev] PyArg_ParseTuple format string: "h" vs. "H"
In-Reply-To: Your message of "Thu, 16 Aug 2001 19:07:31 +0200."
 <20010816190734-r01010700-898f3935-0910-010c@10.0.0.23>
References: <20010816190734-r01010700-898f3935-0910-010c@10.0.0.23>
Message-ID: <200108161755.f7GHtqB17086@odiug.digicool.com>

> I was just bit by an older PyArg_ParseTuple format string behavior
> change: these days (since 2.1?) "h" barfs when you feed it an
> unsigned short value (even if it's within 16 bits). So ok, I updated
> my code to use "H" instead, only to find out hours later that "H"
> didn't exist yet in 1.5.2. I really need to be compatible with both
> so I'll have to work around it :-(
> 
> Was this change really neccesary?

Yes, the old behavior was indefensible.

Jack complained about it too, but in the end agreed he'd update his
code.

If you really want compatibility without ifdefs and don't care about
the range checks, you can use an i format and an int variable, and
cast the int to (unsigned short) when using it.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From just@letterror.com  Thu Aug 16 19:27:48 2001
From: just@letterror.com (Just van Rossum)
Date: Thu, 16 Aug 2001 20:27:48 +0200
Subject: [Python-Dev] PyArg_ParseTuple format string: "h" vs. "H"
In-Reply-To: <200108161755.f7GHtqB17086@odiug.digicool.com>
Message-ID: <20010816202751-r01010700-9187cf60-0910-010c@10.0.0.23>

Guido van Rossum wrote:

> Jack complained about it too, but in the end agreed he'd update his
> code.

I don't mind updating my code at all, it's writing special cases to be
compatible with multiple Python versions that is, erm, yucky.

> If you really want compatibility without ifdefs and don't care about
> the range checks, you can use an i format and an int variable, and
> cast the int to (unsigned short) when using it.

That's how I did it.

Just


From jack@oratrix.nl  Thu Aug 16 23:10:54 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Fri, 17 Aug 2001 00:10:54 +0200
Subject: [Python-Dev] Python version numbers and mac OS X
In-Reply-To: Message by "Martin v. Loewis" <martin@loewis.home.cs.tu-berlin.de> ,
 Tue, 14 Aug 2001 20:54:03 +0200 , <200108141854.f7EIs3u02092@mira.informatik.hu-berlin.de>
Message-ID: <20010816221059.9F86F14B630@oratrix.oratrix.nl>

There's a question of general interest at the very bottom of this
message (that satrted out as a braindump to explain MaxOSX version
numbers).

Recently, "Martin v. Loewis" <martin@loewis.home.cs.tu-berlin.de> said:
> > Sharing is the answer.
> ...
> > This is probably the crux of your reasoning, and I think I
> > disagree. For one, MacOSX (ok, OpenStep is where they got it from)
> > is the first unix-based system that I'm aware of that seems to have
> > a solution to the versioning problem.
> 
> I'm a bit sceptical to any claim that the software versioning problem
> is "solved"; experience tells me it isn't.

Agreed. But they've taken a number of steps that make me think that
for Python they have the exact right solution. Here's how it works.

They have two version numbers, the major and the minor. I say "number"
here, but the major number is actually an arbitrary string and the
minor number is a set of dotted numbers.

The major number is expected to change everytime you make an
incompatible change to your API (remove a routine, add a parameter to
an existing routine, change a structure), exactly like the
PYTHON_API_VERSION number. This number is encoded in the pathname of the
dynamic library, which is something like
/Library/Frameworks/Python.framework/Versions/1011/library.

When you link you actually link against .../Versions/Current/library,
and Current is a symlink to 1011. What is recorded in the binary is
the real pathname, so at runtime a library with major incompatibility
simply won't be found. And if you (the user) want to keep older
versions around that's fine too.

The minor version number is encoded in the library itself, and should
change everytime a compatible change to the API is made (adding a
routine, that sort of thing). Each library has an "oldest minor vesion
number" and a "current minor version number". When you link the
current number of the library is recorded in your application. When
you run the number in your application should be in the range
oldest..current of the library and you're fine.

All that said there's still one thing that breaks for Python, and that
is that it really has two distinct version numbers, the
PYTHON_API_VERSION and the "real" version (2.2).

Plugins are really only interested in the API version, but scripts
that use Python (or programs that embed it) are really only interested
in the real version number.

Programs that both embed and extend Python, such as Zope, are really a
pain because they're interested in both. Although this isn't as hard
as it seems, because in increment of the API version by definition
means an increment in the real version.

I don't have a solution for this yet, mainly because the incrementing
of the API version number hasn't always been completely fair to the
last iota and tittle. For instance, the change of the list.append()
semantics should really have incremented the API version number, but I
don't think it did.

If we could be sure that a newer Python core shared library could work
together with an older Python Lib directory (as long as the API
versions matched) and would provide at least the functionality of that
older Python version we would be fine: use the API version number in
the pathname and use the real version number in the lib/python2.2 and
such. 

Hmm, count this as a request, I guess.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 


From fdrake@acm.org  Thu Aug 16 23:09:36 2001
From: fdrake@acm.org (Fred L. Drake)
Date: Thu, 16 Aug 2001 18:09:36 -0400 (EDT)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20010816220936.4E38428845@cj42289-a.reston1.va.home.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

Improved documentation for the os.spawn*() functions.

Fix some HTML generation problems.



From tim.one@home.com  Fri Aug 17 06:39:07 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 17 Aug 2001 01:39:07 -0400
Subject: [Python-Dev] Objections to PEP 264?
Message-ID: <LNBBLJKPBEHFEDALKOLCGEFDLGAA.tim.one@home.com>

If anyone has objections to Michael Hudson's PEP 264:

    Future statements in simulated shells
    http://python.sourceforge.net/peps/pep-0264.html

raise them now.  I like it (+1 -- it's an effective approach to two
future-statement issues still unresolved).  The implementation patch is
here:

http://sf.net/tracker/?func=detail&aid=449043&group_id=5470&atid=305470

Since Michael is on vaction, I'm going to fiddle it a bit before checking it
in -- provided, of course, no killer objections are raised by you.



From jack@oratrix.nl  Fri Aug 17 10:20:14 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Fri, 17 Aug 2001 11:20:14 +0200
Subject: [Python-Dev] PyArg_ParseTuple format string: "h" vs. "H"
In-Reply-To: Message by Guido van Rossum <guido@python.org> ,
 Thu, 16 Aug 2001 13:55:52 -0400 , <200108161755.f7GHtqB17086@odiug.digicool.com>
Message-ID: <20010817092014.EFA1B303181@snelboot.oratrix.nl>

> > I was just bit by an older PyArg_ParseTuple format string behavior
> > change: these days (since 2.1?) "h" barfs when you feed it an
> > unsigned short value (even if it's within 16 bits). So ok, I updated
> > my code to use "H" instead, only to find out hours later that "H"
> > didn't exist yet in 1.5.2. I really need to be compatible with both
> > so I'll have to work around it :-(
> > 
> > Was this change really neccesary?
> 
> Yes, the old behavior was indefensible.
> 
> Jack complained about it too, but in the end agreed he'd update his
> code.

Well, "agreed" is a rather gentle description here. "Jack was dragged to his 
machine kicking and screaming where he spent months (literally) finding these 
nasty changes, and he still mutters and whimpers occasionally". I think that 
about 99% of the uses of the "h" format specifier were in code that I felt 
responsible for, and each of these had to be carefully inspected. I still find 
them, occasionally (there was another one last month).

And don't even _dream_ of "fixing" the "l" format in the same way:-)
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 




From guido@python.org  Fri Aug 17 12:00:03 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 17 Aug 2001 07:00:03 -0400
Subject: [Python-Dev] Objections to PEP 264?
In-Reply-To: Your message of "Fri, 17 Aug 2001 01:39:07 EDT."
 <LNBBLJKPBEHFEDALKOLCGEFDLGAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCGEFDLGAA.tim.one@home.com>
Message-ID: <200108171100.HAA23046@cj20424-a.reston1.va.home.com>

> If anyone has objections to Michael Hudson's PEP 264:
> 
>     Future statements in simulated shells
>     http://python.sourceforge.net/peps/pep-0264.html
> 
> raise them now.  I like it (+1 -- it's an effective approach to two
> future-statement issues still unresolved).  The implementation patch is
> here:
> 
> http://sf.net/tracker/?func=detail&aid=449043&group_id=5470&atid=305470
> 
> Since Michael is on vaction, I'm going to fiddle it a bit before checking it
> in -- provided, of course, no killer objections are raised by you.

+1

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Aug 17 13:24:36 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 17 Aug 2001 08:24:36 -0400
Subject: [Python-Dev] Python version numbers and mac OS X
In-Reply-To: Your message of "Fri, 17 Aug 2001 00:10:54 +0200."
 <20010816221059.9F86F14B630@oratrix.oratrix.nl>
References: <20010816221059.9F86F14B630@oratrix.oratrix.nl>
Message-ID: <200108171224.IAA24865@cj20424-a.reston1.va.home.com>

I think the crux here lies in the definitions of compatible and
incompatible changes.  You give some examples of incompatible changes:

> (remove a routine, add a parameter to an existing routine, change a
> structure)

and some examples of compatible changes:

> (adding a routine, that sort of thing)

Unfortunately, there's a vast area of possible changes that aren't
really covered by these.  An example from recent c.l.py discussions:
in Python 2.2a1, changed dir().  I didn't change the signature (it
still returns a sorted list of strings), but I changed what it returns
in some cases: dir([]) used to return ['append', 'count', ...]; in
2.2a1 it returns [].  That was deemed incompatible.  Interesting
enough, making it return a *longer* list of strings is deemed
compatible!  I'm sure it's easy to find examples in C as well.

The moral of the story: a compatible version doesn't guarantee much
beyond compatible linkage -- whether the application still works
depends on a lot more than can be expressed by a minor version number.

> I don't have a solution for this yet, mainly because the incrementing
> of the API version number hasn't always been completely fair to the
> last iota and tittle. For instance, the change of the list.append()
> semantics should really have incremented the API version number, but I
> don't think it did.

Interesting point.  For me, the API version number only describe the
*C* API version, not the Python API.  In this case, at least the
combination (python version, C API version) did change.

> If we could be sure that a newer Python core shared library could work
> together with an older Python Lib directory (as long as the API
> versions matched) and would provide at least the functionality of that
> older Python version we would be fine: use the API version number in
> the pathname and use the real version number in the lib/python2.2 and
> such. 

While this could work, but it would be confusing: sys.version would be
the version of the core shared library, and applications would expect
to find the corresponding Lib modules.  IOW, I don't think it's a
useful feature to have.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jack@oratrix.nl  Fri Aug 17 14:01:04 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Fri, 17 Aug 2001 15:01:04 +0200
Subject: [Python-Dev] Python version numbers and mac OS X
In-Reply-To: Message by Guido van Rossum <guido@python.org> ,
 Fri, 17 Aug 2001 08:24:36 -0400 , <200108171224.IAA24865@cj20424-a.reston1.va.home.com>
Message-ID: <20010817130104.90E8C303181@snelboot.oratrix.nl>

> > If we could be sure that a newer Python core shared library could work
> > together with an older Python Lib directory (as long as the API
> > versions matched) and would provide at least the functionality of that
> > older Python version we would be fine: use the API version number in
> > the pathname and use the real version number in the lib/python2.2 and
> > such. 
> 
> While this could work, but it would be confusing: sys.version would be
> the version of the core shared library, and applications would expect
> to find the corresponding Lib modules.  IOW, I don't think it's a
> useful feature to have.

Ah, bah! Count on Guido to come with a showstopper for a nifty idea:-(

This means that we must consider the core shared lib and the lib/python2.2 
directory to be very closely coupled. Which in turn means that the only 
reasonable thing to use as the version number in the framework pathname is the 
"2.2" style version number. Which means that although all the infrastructure 
is there we can't have backward compatible plugin modules. Which means Martin 
was right with his original comment.

... unless there is a way in which we could record a different pathname for 
the shared library depending on whether we are linking an extension or an 
application: the former would check that the API version matches, the latter 
that the 2.2-style version matches. Hmm, MacOSX also has special magic for 
plugins, I guess it's time to start looking into that.

--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 




From tim.one@home.com  Fri Aug 17 20:17:51 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 17 Aug 2001 15:17:51 -0400
Subject: [Python-Dev] test_builtin failing for anyone else?
Message-ID: <LNBBLJKPBEHFEDALKOLCOEHELGAA.tim.one@home.com>

I'm seeing this after Martin's large Unicode patch.  However, I'm also
working on a different patch that mucks with builtins, so I'd just like to
know whether I'm the only one seeing this:

C:\Code\python\PCbuild>python ../lib/test/test_builtin.py
4. Built-in functions
test_b1
__import__
abs
apply
callable
chr
cmp
coerce
compile
complex
delattr
dir
divmod
eval
execfile
filter
float
Traceback (most recent call last):
  File "../lib/test/test_builtin.py", line 9, in ?
    import test_b1
  File "../lib/test\test_b1.py", line 259, in ?
    if float(unicode("  \u0663.\u0661\u0664  ")) != 3.14:
ValueError: invalid literal for float(): \u0663.\u0661\u0664

C:\Code\python\PCbuild>

Oops -- I'm also seeing a Unicode-related failure in test_format.py:

...
u'abc %\\\u3000' % 1 works? ... no
Unexpected  exceptions.ValueError : "unsupported format character '\\'
 (0x5c) at index 5"

That's it.  Anyone else?



From guido@python.org  Fri Aug 17 20:20:18 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 17 Aug 2001 15:20:18 -0400
Subject: [Python-Dev] 2.2a2 release next Wednesday
Message-ID: <200108171920.PAA10067@cj20424-a.reston1.va.home.com>

Barry & I are planning to release Python 2.2a2 next Wednesday (August
19), according to the (updated) schedule in PEP 251.

Tomorrow (Saturday, August 15) I plan to fork off a short-lived
"release branch", from which the release will be done.  Nobody but
Barry & I should be checking things in on the release branch.  We'll
selectively merges the trunk into the branch if needed.  We'll merge
the branch back to the trunk after the release.

Anybody who wants something to show up in 2.2a2 would be wise to check
it in today or before dawn tomorrow.  Please mail me if you need me to
wait for something specific.

PS Martin: I currently have two failing tests!

- test_b1 crashes (in the "float" subtest):

Traceback (most recent call last):
  File "../Lib/test/test_b1.py", line 259, in ?
    if float(unicode("  \u0663.\u0661\u0664  ")) != 3.14:
ValueError: invalid literal for float(): \u0663.\u0661\u0664  

- test_format.py fails:

u'abc %\\\u3000' % 1 works? ... no
Unexpected  exceptions.ValueError : "unsupported format character '\\' (0x5c) at index 5"

I expect these are casualties of the --disable-unicode checkins.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From esr@thyrsus.com  Fri Aug 17 20:46:29 2001
From: esr@thyrsus.com (Eric S. Raymond)
Date: Fri, 17 Aug 2001 15:46:29 -0400
Subject: [Python-Dev] 2.2a2 release next Wednesday
In-Reply-To: <200108171920.PAA10067@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Aug 17, 2001 at 03:20:18PM -0400
References: <200108171920.PAA10067@cj20424-a.reston1.va.home.com>
Message-ID: <20010817154629.D1304@thyrsus.com>

Guido van Rossum <guido@python.org>:
> Anybody who wants something to show up in 2.2a2 would be wise to check
> it in today or before dawn tomorrow.  Please mail me if you need me to
> wait for something specific.

OK.  Don't wait on this, but I'm going to try to find time to check in
my stuff that provides compilerlike framework support for scripts.

Code is tested, docs are written.  The issue is packaging; I was going
to make it a separate ccframe module, but I'm thinking Greg Ewing's
suggestion that it should live in fileinput is on balance a good one.
But that means I have to merge in the docs.

I'll probably get to this tonight.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

To make inexpensive guns impossible to get is to say that you're
putting a money test on getting a gun.  It's racism in its worst form.
        -- Roy Innis, president of the Congress of Racial Equality (CORE), 1988


From gstein@lyra.org  Fri Aug 17 21:30:56 2001
From: gstein@lyra.org (Greg Stein)
Date: Fri, 17 Aug 2001 13:30:56 -0700
Subject: [Python-Dev] Free threading
In-Reply-To: <5.1.0.14.0.20010815185358.01ba1e68@tuatha.sidhe.org>; from dan@sidhe.org on Wed, Aug 15, 2001 at 06:59:47PM -0400
References: <LNBBLJKPBEHFEDALKOLCOEGCLFAA.tim.one@home.com> <3B78CA34.9A4E1765@ActiveState.com> <5.1.0.14.0.20010815185358.01ba1e68@tuatha.sidhe.org>
Message-ID: <20010817133056.C1223@lyra.org>

On Wed, Aug 15, 2001 at 06:59:47PM -0400, Dan Sugalski wrote:
> At 11:50 PM 8/13/2001 -0700, Paul Prescod wrote:
> >Tim Peters wrote:
> > >...
> > > IIRC, Greg's fabled free-threading version of Python took a speed hit of
> > > about a factor of 2 (for a program using only 1 thread, compared to that
> > > same program without the free-threading patches).

Yah. That's about right. The largest problem was dealing with dictionaries.
Since you never knew whether a specific dictionary was shared (between
threads) or not, you always had to lock the access. And since all namespaces
use dictionaries...

> >The Perl guys considered this unacceptable and I can kind of see their
> >point. You have two processors but you get roughly the same performance
> >as one?

No. For definitional purposes, let's say that for normal Python on one
processor, you get 1 Python Speed Unit (PSU). With my free-threading
patches, that uniprocessor would get around 0.6 PSU.

Move to a multiprocessor with 2 CPUs, running a 2-thread program with no
synchronization (e.g. each is simply doing their thing rather than
potentially intefering with each other). Regular Python would get about 0.95
PSU because the GIL imposes some overhead (and you can't ever get more than
1 because of the GIL). With the free-threading, you would get about 1.2 PSU.

On a three processor system, regular Python still gets 0.95 PSU. The
free-threading goes up to maybe 1.6 PSU.

We observed non-linear scaling with the processors under free threading. 2
processors was fine, but 3 or 4 didn't buy you much more than 2. The problem
was lock contention. With that many things going, the contention around
Python's internal structures simply killed further scaling performance.

>...
> I racked up a whole list of "Things to Not Do With Threads" when hacking 
> the original perl thread model. (The first of which is "wedge them into an 
> interpreter that wasn't written with threads in mind..." :) Battle scars 
> are viewable on request.

I hear ya. Same here. But Python has since become ever worse re: free
threading capability (more globals to arbitrate access to).

Last time, I tried to optimize the memory associate with each list/dict.
That slowed some things down. Atomic incr/decr wasn't really available under
Linux (Win has InterlockedIncrement and friends), so the Linux incr/decr was
a bit slower than it should it have been.

There are a number of things that I'd do differently the next time around.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From guido@python.org  Fri Aug 17 21:36:21 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 17 Aug 2001 16:36:21 -0400
Subject: [Python-Dev] Free threading
In-Reply-To: Your message of "Fri, 17 Aug 2001 13:30:56 PDT."
 <20010817133056.C1223@lyra.org>
References: <LNBBLJKPBEHFEDALKOLCOEGCLFAA.tim.one@home.com> <3B78CA34.9A4E1765@ActiveState.com> <5.1.0.14.0.20010815185358.01ba1e68@tuatha.sidhe.org>
 <20010817133056.C1223@lyra.org>
Message-ID: <200108172036.QAA15090@cj20424-a.reston1.va.home.com>

> Last time, I tried to optimize the memory associate with each
> list/dict.  That slowed some things down. Atomic incr/decr wasn't
> really available under Linux (Win has InterlockedIncrement and
> friends), so the Linux incr/decr was a bit slower than it should it
> have been.

This would be an opportunity for some highly platform-specific
assembler, assuming we'll always have
configure --{enable,disable}-free-threading.

> There are a number of things that I'd do differently the next time around.

When will that be?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gstein@lyra.org  Fri Aug 17 21:43:49 2001
From: gstein@lyra.org (Greg Stein)
Date: Fri, 17 Aug 2001 13:43:49 -0700
Subject: [Python-Dev] Free threading
In-Reply-To: <20010809061327.24EC299C81@waltz.rahul.net>; from aahz@rahul.net on Wed, Aug 08, 2001 at 11:13:27PM -0700
References: <200108082230.SAA09334@cj20424-a.reston1.va.home.com> <20010809061327.24EC299C81@waltz.rahul.net>
Message-ID: <20010817134349.D1223@lyra.org>

On Wed, Aug 08, 2001 at 11:13:27PM -0700, Aahz Maruch wrote:
> Guido van Rossum wrote:
> > Aahz:
> >> We need thread-local storage, anyway.  When I'm a little farther into my
> >> work on Decimal, I'm going to take a hard look at this because context
> >> needs to be global on a per-thread basis...
> > 
> > In the C API, use PyThreadState_GetDict(): it returns a per-thread
> > dictionary.  The convention is that you pick a key unique to your
> > extension module (e.g. your full extension module name) and then store
> > your per-thread data under this key.

Chicken and egg. Note that in a free threading context, there is no such
thing as _PyThreadState_Current. Thus, you can't get to that dict. Instead,
you pick up your thread state pointer from TLS (or an equiv mechanism on
other platforms).

>...
> But what I was really referring to was the annoyance of having every
> method call pull in the thread-local information.  Seems to me that
> there ought to be some way of making it happen magically.

Not much you can do here, except to start passing around the thread state as
a parameter. (not the interp! that is derived from the thread state, but it
can't be done the other way around)


Note that I posted some of the steps (towards free threading) last year.
Each of these are low cost, but will help set a framework for further
threading work:

    http://mail.python.org/pipermail/python-dev/2000-April/003605.html


Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From loewis@informatik.hu-berlin.de  Fri Aug 17 23:14:22 2001
From: loewis@informatik.hu-berlin.de (Martin von Loewis)
Date: Sat, 18 Aug 2001 00:14:22 +0200 (MEST)
Subject: [Python-Dev] test_builtin failing for anyone else?
Message-ID: <200108172214.AAA05711@punica.informatik.hu-berlin.de>

> I'm seeing this after Martin's large Unicode patch. ...
> That's it.  Anyone else?

You might think I should have seen it too. I didn't, because I ran the
testsuite for --enable-unicode without the patches to the tests, and
re-ran the test suite then only for --disable-unicode (*) :-(

Anyway, I just committed a patch that fixes it for me...

Regards,
Martin

(*) That also produces a number of failures, but that is expected:
some tests still absolutely require Unicode support, and some of the
libraries do as well, mainly pickle.


From skip@pobox.com (Skip Montanaro)  Sat Aug 18 00:54:27 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Fri, 17 Aug 2001 18:54:27 -0500
Subject: [Python-Dev] tempnam/tmpnam_r
Message-ID: <15229.44723.470234.595607@beluga.mojam.com>

I get this from gcc:

    libpython2.2.a(posixmodule.o): In function `posix_tmpnam':
    /home/skip/src/python/head/dist/src/build/../Modules/posixmodule.c:4262: the use of `tmpnam_r' is dangerous, better use `mkstemp'
    libpython2.2.a(posixmodule.o): In function `posix_tempnam':
    /home/skip/src/python/head/dist/src/build/../Modules/posixmodule.c:4217: the use of `tempnam' is dangerous, better use `mkstemp'

I realize this has come up before.  Is it possible to preserve the semantics
of posix_tmpnam and posix_tempnam while switching the implementation over to
mkstemp where it's available?  Alternatively, perhaps Python should produce
a warning similar to what gcc produces the first time these functions are
called. 

Skip


From gstein@lyra.org  Sat Aug 18 01:20:44 2001
From: gstein@lyra.org (Greg Stein)
Date: Fri, 17 Aug 2001 17:20:44 -0700
Subject: [Python-Dev] Free threading
In-Reply-To: <200108172036.QAA15090@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Aug 17, 2001 at 04:36:21PM -0400
References: <LNBBLJKPBEHFEDALKOLCOEGCLFAA.tim.one@home.com> <3B78CA34.9A4E1765@ActiveState.com> <5.1.0.14.0.20010815185358.01ba1e68@tuatha.sidhe.org> <20010817133056.C1223@lyra.org> <200108172036.QAA15090@cj20424-a.reston1.va.home.com>
Message-ID: <20010817172044.G1223@lyra.org>

On Fri, Aug 17, 2001 at 04:36:21PM -0400, Guido van Rossum wrote:
> > Last time, I tried to optimize the memory associate with each
> > list/dict.  That slowed some things down. Atomic incr/decr wasn't
> > really available under Linux (Win has InterlockedIncrement and
> > friends), so the Linux incr/decr was a bit slower than it should it
> > have been.
> 
> This would be an opportunity for some highly platform-specific
> assembler,

Yah.

> assuming we'll always have
> configure --{enable,disable}-free-threading.

Good assumption :-) As its been pointed out recently on this list, and in
the past, free threading isn't the "end all" of threading. There are some
tradeoffs involved.

> > There are a number of things that I'd do differently the next time around.
> 
> When will that be?

I started my note with if/when :-)  Between the Python http-based modules,
ViewCVS, edna, Subversion, Apache, and mod_dav... I'm a bit booked up for a
while :-) and don't have an "itch" to get free threading completed.

The "baby steps" note that I posted last year was a hope to get others
involved, who may have more immediate needs for this kind of functionality.
Kind of a road map from my earlier experience.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From guido@python.org  Sat Aug 18 02:28:24 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 17 Aug 2001 21:28:24 -0400
Subject: [Python-Dev] Free threading
In-Reply-To: Your message of "Fri, 17 Aug 2001 17:20:44 PDT."
 <20010817172044.G1223@lyra.org>
References: <LNBBLJKPBEHFEDALKOLCOEGCLFAA.tim.one@home.com> <3B78CA34.9A4E1765@ActiveState.com> <5.1.0.14.0.20010815185358.01ba1e68@tuatha.sidhe.org> <20010817133056.C1223@lyra.org> <200108172036.QAA15090@cj20424-a.reston1.va.home.com>
 <20010817172044.G1223@lyra.org>
Message-ID: <200108180128.VAA19233@cj20424-a.reston1.va.home.com>

> I started my note with if/when :-)  Between the Python http-based modules,
> ViewCVS, edna, Subversion, Apache, and mod_dav... I'm a bit booked up for a
> while :-) and don't have an "itch" to get free threading completed.

Understood.

> The "baby steps" note that I posted last year was a hope to get others
> involved, who may have more immediate needs for this kind of functionality.
> Kind of a road map from my earlier experience.

Thanks for pointing to those -- they are a good place to start indeed!

Now, when *I* have some free time on my hands...  (chuckle).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com (Skip Montanaro)  Sat Aug 18 03:02:20 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Fri, 17 Aug 2001 21:02:20 -0500
Subject: [Python-Dev] Free threading
In-Reply-To: <20010817172044.G1223@lyra.org>
References: <LNBBLJKPBEHFEDALKOLCOEGCLFAA.tim.one@home.com>
 <3B78CA34.9A4E1765@ActiveState.com>
 <5.1.0.14.0.20010815185358.01ba1e68@tuatha.sidhe.org>
 <20010817133056.C1223@lyra.org>
 <200108172036.QAA15090@cj20424-a.reston1.va.home.com>
 <20010817172044.G1223@lyra.org>
Message-ID: <15229.52396.371840.484869@beluga.mojam.com>

    Greg> The "baby steps" note that I posted last year was a hope to get
    Greg> others involved, who may have more immediate needs for this kind
    Greg> of functionality.  Kind of a road map from my earlier experience.

Of course, I deleted that reference already, but I can dig it up if I need
it.  Not that I'm volunteering for this or anything, but perhaps there are
some things that authors should be alerted so their code in the meantime
doesn't make a future free threading Python that much more difficult to
implement.  This sort of information might profitably be added to some
documentation now (ext/embed or c api manuals?).

Skip



From skip@pobox.com (Skip Montanaro)  Sat Aug 18 03:16:14 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Fri, 17 Aug 2001 21:16:14 -0500
Subject: [Python-Dev] Py_FatalError...
Message-ID: <15229.53230.429726.683520@beluga.mojam.com>

Dumb question:  Why is it Py_FatalError and not PyErr_FatalError?

Skip


From martin@loewis.home.cs.tu-berlin.de  Sat Aug 18 08:09:20 2001
From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis)
Date: Sat, 18 Aug 2001 09:09:20 +0200
Subject: [Python-Dev] tempnam/tmpnam_r
Message-ID: <200108180709.f7I79KC01025@mira.informatik.hu-berlin.de>

> I realize this has come up before.  Is it possible to preserve the
> semantics of posix_tmpnam and posix_tempnam while switching the
> implementation over to mkstemp where it's available?

No. If it was possible, the C library would have taken such an
approach to fix it, instead of coming up with a way to complain.

In case you wonder what is dangerous about these functions: they both
return strings of filenames that where unique at the time this was
tested. Typically, the application will then pass that string to
open(). Now, a malicious application may wait for the moment when
tempnam returns, and create the file. Then, the Python application
will open the temporary file, which happens to be created already.
Thus, the malicious application will be able to find out and modify
the data that the Python application has put into the temporary file.

The only true solution is to find a temporary name and open the file
for exclusive access (O_EXCL); if that fails, pick another temporary
name. mkstemp does that and returns both the open file handle, and the
file name.

Regards,
Martin



From skip@pobox.com (Skip Montanaro)  Sat Aug 18 17:32:07 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Sat, 18 Aug 2001 11:32:07 -0500
Subject: [Python-Dev] tempnam/tmpnam_r
In-Reply-To: <200108180709.f7I79KC01025@mira.informatik.hu-berlin.de>
References: <200108180709.f7I79KC01025@mira.informatik.hu-berlin.de>
Message-ID: <15230.39047.773294.802231@beluga.mojam.com>

    >> I realize this has come up before.  Is it possible to preserve the
    >> semantics of posix_tmpnam and posix_tempnam while switching the
    >> implementation over to mkstemp where it's available?

    Martin> No. If it was possible, the C library would have taken such an
    Martin> approach to fix it, instead of coming up with a way to complain.

Understood.

    Martin> In case you wonder what is dangerous about these functions: they
    Martin> both return strings of filenames that where unique at the time
    Martin> this was tested. Typically, the application will then pass that
    Martin> string to open(). Now, a malicious application may wait for the
    Martin> moment when tempnam returns, and create the file. Then, the
    Martin> Python application will open the temporary file, which happens
    Martin> to be created already.  Thus, the malicious application will be
    Martin> able to find out and modify the data that the Python application
    Martin> has put into the temporary file.

Given that these two functions have a race condition and that it can be
exploited when used in C code, it seems to me that it would be even easier
to exploit when used in Python code, since Python programs tend to run so
much slower than C programs at the granularity of measurement we're talking
about.  If that's the case, then I think it makes a stronger argument for
deprecating the existing tmpnam and tempnam functions in favor of mkstemp.

I realize there are some issues to resolve.  First, mkstemp returns a file
descriptor and modifies the input filename template.  The return value to
Python code would have to be a tuple (filename, fd).  Second, if we want to
encourage people to use it in place of the builtin open function, it will
have to return a normal Python file object instead of an integer file
descriptor.  Third, is mkstemp pretty universally available?

Finally, I still think use of tmpnam or tempnam should emit warnings.

Skip


From guido@python.org  Sat Aug 18 18:08:15 2001
From: guido@python.org (Guido van Rossum)
Date: Sat, 18 Aug 2001 13:08:15 -0400
Subject: [Python-Dev] Py_FatalError...
In-Reply-To: Your message of "Fri, 17 Aug 2001 21:16:14 CDT."
 <15229.53230.429726.683520@beluga.mojam.com>
References: <15229.53230.429726.683520@beluga.mojam.com>
Message-ID: <200108181708.NAA20608@cj20424-a.reston1.va.home.com>

> Dumb question:  Why is it Py_FatalError and not PyErr_FatalError?

Because it has nothing to do with exceptions.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sat Aug 18 18:11:58 2001
From: guido@python.org (Guido van Rossum)
Date: Sat, 18 Aug 2001 13:11:58 -0400
Subject: [Python-Dev] tempnam/tmpnam_r
In-Reply-To: Your message of "Sat, 18 Aug 2001 11:32:07 CDT."
 <15230.39047.773294.802231@beluga.mojam.com>
References: <200108180709.f7I79KC01025@mira.informatik.hu-berlin.de>
 <15230.39047.773294.802231@beluga.mojam.com>
Message-ID: <200108181711.NAA20953@cj20424-a.reston1.va.home.com>

> Finally, I still think use of tmpnam or tempnam should emit warnings.

Me too.  Go ahead and check something in!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gstein@lyra.org  Sat Aug 18 18:36:39 2001
From: gstein@lyra.org (Greg Stein)
Date: Sat, 18 Aug 2001 10:36:39 -0700
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib socket.py,1.11,1.12
In-Reply-To: <E15Xupe-0004Mk-00@usw-pr-cvs1.sourceforge.net>; from gvanrossum@users.sourceforge.net on Fri, Aug 17, 2001 at 06:23:22PM -0700
References: <E15Xupe-0004Mk-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <20010818103639.B8246@lyra.org>

On Fri, Aug 17, 2001 at 06:23:22PM -0700, Guido van Rossum wrote:
> Update of /cvsroot/python/python/dist/src/Lib
> In directory usw-pr-cvs1:/tmp/cvs-serv16772
> 
> Modified Files:
> 	socket.py 
> Log Message:
> When the socket is closed, don't just assign 0 to self._sock.
> This breaks software that excepts a socket.error but not an
> AttributeError.
>...
> + class _closedsocket:
> + 
> +     def _bummer(self):
> +         raise error(9, 'Bad file descriptor')
> + 
> +     _s = "def %s(self, *args): self._bummer()\n\n"
> +     for _m in _socketmethods:
> +         exec _s % _m

Isn't this last one easier written has:

  for _m in _socketmethods:
      vars()[_m] = _bummer

And change _bummer to have a *args.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From skip@pobox.com (Skip Montanaro)  Sat Aug 18 19:53:00 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Sat, 18 Aug 2001 13:53:00 -0500
Subject: [Python-Dev] tempnam/tmpnam_r
In-Reply-To: <200108181711.NAA20953@cj20424-a.reston1.va.home.com>
References: <200108180709.f7I79KC01025@mira.informatik.hu-berlin.de>
 <15230.39047.773294.802231@beluga.mojam.com>
 <200108181711.NAA20953@cj20424-a.reston1.va.home.com>
Message-ID: <15230.47500.33766.834605@beluga.mojam.com>

    Skip> Finally, I still think use of tmpnam or tempnam should emit
    Skip> warnings. 

    Guido> Me too.  Go ahead and check something in!

Done.

Skip


From guido@python.org  Sat Aug 18 21:59:29 2001
From: guido@python.org (Guido van Rossum)
Date: Sat, 18 Aug 2001 16:59:29 -0400
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib socket.py,1.11,1.12
In-Reply-To: Your message of "Sat, 18 Aug 2001 10:36:39 PDT."
 <20010818103639.B8246@lyra.org>
References: <E15Xupe-0004Mk-00@usw-pr-cvs1.sourceforge.net>
 <20010818103639.B8246@lyra.org>
Message-ID: <200108182059.QAA23666@cj20424-a.reston1.va.home.com>

> Isn't this last one easier written has:
> 
>   for _m in _socketmethods:
>       vars()[_m] = _bummer
> 
> And change _bummer to have a *args.

Good idea.  For an even better idea, see my checkin. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sat Aug 18 22:30:53 2001
From: guido@python.org (Guido van Rossum)
Date: Sat, 18 Aug 2001 17:30:53 -0400
Subject: [Python-Dev] Python 2.2a2 - release branch created
Message-ID: <200108182130.RAA25913@cj20424-a.reston1.va.home.com>

I've created the Python 2.2a2 release branch.  The CVS tag is
'r22a2-branch'.  (There's also a tag 'r22a2-fork' -- that's the point
on the trunk where the branch forks off.)  So far the only change in
the branch is the version number.

Only the release managers (Barry and I) should make checkins to the
branch.  All others check in to the trunk.  Please notify the release
managers (a comment in the checkin message would suffice) if you think
your checkin should be merged into the branch.

The release is planned for Wednesday, August 19.  After the release is
done, the branch will be merged back into the trunk.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gball@cfa.harvard.edu  Sat Aug 18 23:09:29 2001
From: gball@cfa.harvard.edu (Greg Ball)
Date: Sat, 18 Aug 2001 18:09:29 -0400 (EDT)
Subject: [Python-Dev] Objections to PEP 264?
Message-ID: <Pine.OSF.4.10.10108181755520.32062-100000@cfata6.harvard.edu>

Tim's latest change to __future__.py has toasted the build process. Python
itself builds fine, then is run to build the extensions.

site imports distutils
distutils imports re
re imports sre
sre imports copy_reg
copy_reg imports types
types imports __future__
__future__ imports new, uh oh, new doesn't yet exist...

python then tries to keep going but with distutils non-functional,
it doesn't stand a chance.

If the new module is now considered mainstream, maybe it could be built
statically?

--Greg Ball






From guido@python.org  Sun Aug 19 06:30:11 2001
From: guido@python.org (Guido van Rossum)
Date: Sun, 19 Aug 2001 01:30:11 -0400
Subject: [Python-Dev] Objections to PEP 264?
In-Reply-To: Your message of "Sat, 18 Aug 2001 18:09:29 EDT."
 <Pine.OSF.4.10.10108181755520.32062-100000@cfata6.harvard.edu>
References: <Pine.OSF.4.10.10108181755520.32062-100000@cfata6.harvard.edu>
Message-ID: <200108190530.BAA00766@cj20424-a.reston1.va.home.com>

> Tim's latest change to __future__.py has toasted the build process. Python
> itself builds fine, then is run to build the extensions.
> 
> site imports distutils
> distutils imports re
> re imports sre
> sre imports copy_reg
> copy_reg imports types
> types imports __future__
> __future__ imports new, uh oh, new doesn't yet exist...
> 
> python then tries to keep going but with distutils non-functional,
> it doesn't stand a chance.
> 
> If the new module is now considered mainstream, maybe it could be built
> statically?

Good catch!  (This uncovered some problems in my setup -- I hadn't
picked up the latest versions of a few modules. :-( )

I solved it a little differently, by adding hackery to __future__ that
avoids the dependency.  But maybe you're right and it should be built
statically (it's in the main DLL on Windows).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Sun Aug 19 17:22:54 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sun, 19 Aug 2001 12:22:54 -0400 (EDT)
Subject: [Python-Dev] Free threading
In-Reply-To: <20010817172044.G1223@lyra.org>
References: <LNBBLJKPBEHFEDALKOLCOEGCLFAA.tim.one@home.com>
 <3B78CA34.9A4E1765@ActiveState.com>
 <5.1.0.14.0.20010815185358.01ba1e68@tuatha.sidhe.org>
 <20010817133056.C1223@lyra.org>
 <200108172036.QAA15090@cj20424-a.reston1.va.home.com>
 <20010817172044.G1223@lyra.org>
Message-ID: <15231.59358.703308.343183@cj42289-a.reston1.va.home.com>

Greg Stein writes:
 > I started my note with if/when :-)  Between the Python http-based modules,
 > ViewCVS, edna, Subversion, Apache, and mod_dav... I'm a bit booked up for a

  And don't forget the various build-process bugs & patches for Expat,
either!  ;-)  For the patches, if you can tell me yea/nay, I can deal
with the mechanics of committing them.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From skip@pobox.com (Skip Montanaro)  Sun Aug 19 18:45:50 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Sun, 19 Aug 2001 12:45:50 -0500
Subject: [Python-Dev] Instruction count frequency server
Message-ID: <15231.64334.71337.225588@beluga.mojam.com>

Last night during the midst of a lengthy reply to a post by Alex Martelli,
it occurred to me that it might be useful for people fiddling around with
the Python VM to have access to a large body of instruction count
information.  Marc-Andre Lemburg posted a note a year or so ago that
summarized dynamic instruction count frequencies gathered over 100 million
PyVM instruction cycles.  While looking at his table I noticed that he
reported a very high frequency of LOAD_NAME instructions.  I took a quick
peek at static instruction counts in the core library and found that
LOAD_NAME almost never appears in that particular chunk of Python bytecode.
So, although MAL collected data over a large number of instruction cycles,
it's not obvious that his programs were "typical".

This morning I did something about that.  I wrote a little XML-RPC server
that allows people to post or retrieve instruction count frequencies.  It's
at manatee.mojam.com, port 7304.  To get a quick synopsis of the methods the
server implements, try this:

    dxp = xmlrpclib.Server("http://manatee.mojam.com:7304")
    print dxp.synopsis()

More detail can be had with

    print dxp.usage()

The usage response contains details about the individual methods, run-length
encode/decode functions, and hints on how to send sys.getdxp() output to the
server automatically whenever a program exits normally.

Server response isn't terribly speedy, mostly because when sending or
receiving instruction count data a fair number of bytes stream one way or
the other, even though the data is run-length encoded and the server will
gzip encode output when it can.  Even when queried from the same machine as
the server it takes about 5 seconds to return non-trivial output.  Oh well,
XML-RPC isn't perfect for everything.  Hopefully the load won't be too
heavy.

Feedback is welcome.

Skip


From tim.one@home.com  Sun Aug 19 20:34:02 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 19 Aug 2001 15:34:02 -0400
Subject: [Python-Dev] Objections to PEP 264?
In-Reply-To: <Pine.OSF.4.10.10108181755520.32062-100000@cfata6.harvard.edu>
Message-ID: <LNBBLJKPBEHFEDALKOLCGELFLGAA.tim.one@home.com>

[Greg Ball]
> Tim's latest change to __future__.py has toasted the build process.

All right!  I've been trying for *years* to screw the Linux build, and can
finally retire -- payback is soooo sweet <wink>.

eventually-python-won't-compile-on-anything-ly y'rs  - tim



From jack@oratrix.nl  Sun Aug 19 21:36:30 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Sun, 19 Aug 2001 22:36:30 +0200
Subject: [Python-Dev] test_threaded_import hanging on MacOS and Irix
Message-ID: <20010819203635.80A65162E14@oratrix.oratrix.nl>

test_threaded_import occasionally hangs in MacPython from the current
CVS tree (about 30% of the time) and I was going to look into this
shortly, assuming it to be a Mac-specific error.

But today suddenly it also hung on my Irix 6.5 Python, also
uninterruptible with control-C.

Is anyone else seeing this? Could this be a race that is
machine-independent? I do happen to know that the SGI is rather
aggressive with thread switching, and so is my MacPython pthread
implementation...
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | ++++ see http://www.xs4all.nl/~tank/ ++++


From tim.one@home.com  Sun Aug 19 23:46:54 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 19 Aug 2001 18:46:54 -0400
Subject: [Python-Dev] FW: Case Sensitivity: Survey Results
Message-ID: <LNBBLJKPBEHFEDALKOLCOELNLGAA.tim.one@home.com>

Wow, Sheila really worked on this!  Check out her web site.

-----Original Message-----
From: python-list-admin@python.org
[mailto:python-list-admin@python.org]On Behalf Of Sheila King
Sent: Saturday, August 18, 2001 2:25 AM
To: python-list@python.org
Subject: Case Sensitivity: Survey Results


In response to this message:
http://mail.python.org/pipermail/python-list/2001-July/054788.html

which continued the discussion about case sensitivity in the Python
language, I announced my intention to conduct a survey of computer
science teachers on this issue, here:
http://mail.python.org/pipermail/python-list/2001-July/055020.html

The survey is now complete and results are available.
I received a total of 129 responses. 

For summaries and to read the actual responses, you can go here:
http://www.thinkspot.net/sheila/computers/case/casesurvey.html

My take on the whole thing, is this:

The overwhelming majority of teachers surveyed on this topic felt it was
a non-issue (well over 80% of them). They felt that students have either
little or no difficulty due to case-sensitivity.

Perhaps, if one were designing a language from scratch (which is no
longer the case with Python), it would be reasonable to adopt
case-insensitivity. It seems there is little benefit to be derived from
case-sensitivity (although some respondents felt that precision in
approach and thinking, richness in language and naming choices, plus
compiler efficiency made sensitivity preferable). So, in the interest of
generating fewer error messages, insensitivity may be preferable.

However, given that this was felt to be an extremely minor issue, I see
no reason to entertain the issue of changing an already existing
programming language either from case-sensitive to case-insensitive or
vice-versa. There would be little to be gained, and certainly it would
not justify backward compatibility issues with already existing code.
Time would be better spent developing better error messages, or more
friendly programming environments for the beginner.

As I said, that is my take on this issue, and I feel that it is
supported by the survey results.

More details on the results of the survey, including all responses to
the survey, are online here:
http://www.thinkspot.net/sheila/computers/case/casesurvey.html

--
Sheila King
http://www.thinkspot.net/sheila/
http://www.k12groups.org/


-- 
http://mail.python.org/mailman/listinfo/python-list


From tim.one@home.com  Mon Aug 20 00:11:21 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 19 Aug 2001 19:11:21 -0400
Subject: [Python-Dev] test_threaded_import hanging on MacOS and Irix
In-Reply-To: <20010819203635.80A65162E14@oratrix.oratrix.nl>
Message-ID: <LNBBLJKPBEHFEDALKOLCKELPLGAA.tim.one@home.com>

[Jack Jansen]
> test_threaded_import occasionally hangs in MacPython from the current
> CVS tree (about 30% of the time) and I was going to look into this
> shortly, assuming it to be a Mac-specific error.

How do you run it?
    Via
        regrtest in a clump
        regrtest by itself
        directly
    debug or release build?
    with or without purging all .pyc files first?
    with or without -v?

Does varying any of the above change the symptoms?

Does it do anything *other* than hang (for example, display any msg(s) to
the screen)?

> But today suddenly it also hung on my Irix 6.5 Python, also
> uninterruptible with control-C.

Nothing is interesting on Irix unless you compile Python without
optimization there first (virtually every bug ever reported on SGI boxes has
been cured by turning optimization off).

> Is anyone else seeing this?

Not me.

> Could this be a race that is machine-independent?

More like a race that's machine-dependent, yes?  Since the point of the
import lock is to avoid races, of course it's just as likely to cause them
as to cure them <wink>.

> I do happen to know that the SGI is rather aggressive with thread
> switching, and so is my MacPython pthread implementation...

Ditto Windows, and no problem there.



From esr@thyrsus.com  Mon Aug 20 00:54:49 2001
From: esr@thyrsus.com (Eric S. Raymond)
Date: Sun, 19 Aug 2001 19:54:49 -0400
Subject: [Python-Dev] Apparent bug in 2.2a1
Message-ID: <20010819195449.A10264@thyrsus.com>

# This trivial Python program fails.
#
# Python 2.2a1+ (#12, Aug 18 2001, 14:42:50) 
# [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-85)] on linux2
#
# To really confuse yourself, set xml to 0 and watch it run without error.

xml = 1
stem = "a"
if xml:
    stem += "b",
else:
    stem += "b"

# Here is the error I see:
#
# Traceback (most recent call last):
#   File "foo.py", line 11, in ?
#     stem += "b",
# TypeError: cannot add type "tuple" to string

-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

The world is filled with violence. Because criminals carry guns, we
decent law-abiding citizens should also have guns. Otherwise they will
win and the decent people will lose.
        -- James Earl Jones


From alex_c@MIT.EDU  Mon Aug 20 01:00:50 2001
From: alex_c@MIT.EDU (Alex Coventry)
Date: 19 Aug 2001 20:00:50 -0400
Subject: [Python-Dev] Apparent bug in 2.2a1
In-Reply-To: "Eric S. Raymond"'s message of "Sun, 19 Aug 2001 19:54:49 -0400"
References: <20010819195449.A10264@thyrsus.com>
Message-ID: <etd4rr3d2h9.fsf@lola-granola.mit.edu>

The problem may be the comma following the "b".  That makes the RHS a
tuple. 

Alex.


From esr@thyrsus.com  Mon Aug 20 01:11:26 2001
From: esr@thyrsus.com (Eric S. Raymond)
Date: Sun, 19 Aug 2001 20:11:26 -0400
Subject: [Python-Dev] Apparent bug in 2.2a1
In-Reply-To: <etd4rr3d2h9.fsf@lola-granola.mit.edu>; from alex_c@MIT.EDU on Sun, Aug 19, 2001 at 08:00:50PM -0400
References: <20010819195449.A10264@thyrsus.com> <etd4rr3d2h9.fsf@lola-granola.mit.edu>
Message-ID: <20010819201126.A10488@thyrsus.com>

Alex Coventry <alex_c@MIT.EDU>:
> The problem may be the comma following the "b".  That makes the RHS a
> tuple. 

My bad.  Damn that trailing comma is easy to miss.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

In every country and in every age, the priest has been hostile to
liberty. He is always in alliance with the despot, abetting his abuses
in return for protection to his own.
	-- Thomas Jefferson, 1814


From skip@pobox.com (Skip Montanaro)  Mon Aug 20 01:32:20 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Sun, 19 Aug 2001 19:32:20 -0500
Subject: [Python-Dev] Apparent bug in 2.2a1
In-Reply-To: <20010819201126.A10488@thyrsus.com>
References: <20010819195449.A10264@thyrsus.com>
 <etd4rr3d2h9.fsf@lola-granola.mit.edu>
 <20010819201126.A10488@thyrsus.com>
Message-ID: <15232.23188.12163.512361@beluga.mojam.com>

    Eric>  Damn that trailing comma is easy to miss.

Which is why my tuples always snuggle up to parens.  I almost wish 
paren-free tuple creation was deprecated.  (You think the integer division
stink was bad... ;-)

Skip



From pedroni@inf.ethz.ch  Mon Aug 20 01:49:35 2001
From: pedroni@inf.ethz.ch (Samuele Pedroni)
Date: Mon, 20 Aug 2001 02:49:35 +0200
Subject: [Python-Dev] Q: playing with 2.2a1 => strange behaviour, or misunderstanding
References: <200108171920.PAA10067@cj20424-a.reston1.va.home.com>
Message-ID: <000d01c12911$fca5ed40$8a73fea9@newmexico>

I have tried this with 2.2a1:
1. I have not checked the related C code
2. I have not checked the bug report list

>>> class C(object): pass
...
>>> c=C()
>>> c.a=3
>>> object.__getattr__(c,'a')
3
>>> object.__setattr__(c,'a',6)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: function takes exactly 2 arguments (3 given)

Is that meaningful or a problem?
I'm trying to understand type/class unification to consider possible jython
issues.

regards, Samuele Pedroni



From guido@python.org  Mon Aug 20 03:23:25 2001
From: guido@python.org (Guido van Rossum)
Date: Sun, 19 Aug 2001 22:23:25 -0400
Subject: [Python-Dev] Apparent bug in 2.2a1
In-Reply-To: Your message of "Sun, 19 Aug 2001 19:32:20 CDT."
 <15232.23188.12163.512361@beluga.mojam.com>
References: <20010819195449.A10264@thyrsus.com> <etd4rr3d2h9.fsf@lola-granola.mit.edu> <20010819201126.A10488@thyrsus.com>
 <15232.23188.12163.512361@beluga.mojam.com>
Message-ID: <200108200223.WAA10316@cj20424-a.reston1.va.home.com>

>     Eric>  Damn that trailing comma is easy to miss.
> 
> Which is why my tuples always snuggle up to parens.  I almost wish 
> paren-free tuple creation was deprecated.  (You think the integer division
> stink was bad... ;-)

Idea for PyChecker?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Aug 20 14:11:15 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 20 Aug 2001 09:11:15 -0400
Subject: [Python-Dev] Q: playing with 2.2a1 => strange behaviour, or misunderstanding
In-Reply-To: Your message of "Mon, 20 Aug 2001 02:49:35 +0200."
 <000d01c12911$fca5ed40$8a73fea9@newmexico>
References: <200108171920.PAA10067@cj20424-a.reston1.va.home.com>
 <000d01c12911$fca5ed40$8a73fea9@newmexico>
Message-ID: <200108201311.JAA17884@cj20424-a.reston1.va.home.com>

> I have tried this with 2.2a1:
> 1. I have not checked the related C code
> 2. I have not checked the bug report list
> 
> >>> class C(object): pass
> ...
> >>> c=C()
> >>> c.a=3
> >>> object.__getattr__(c,'a')
> 3
> >>> object.__setattr__(c,'a',6)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: function takes exactly 2 arguments (3 given)
> 
> Is that meaningful or a problem?
> I'm trying to understand type/class unification to consider possible jython
> issues.
> 
> regards, Samuele Pedroni

This was a 2.2a1 specific problem: in that version, the object class
didn't define a setattr dispatch slot, and object.__setattr__ ended up
returning a *bound* method of the object *class*.

In 2.2a2 this won't be a problem -- the object now defines
__setattr__.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@zope.com  Mon Aug 20 17:18:26 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Mon, 20 Aug 2001 12:18:26 -0400 (EDT)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib compilerlike.py,NONE,1.1
In-Reply-To: <E15Y2LQ-0005nv-00@usw-pr-cvs1.sourceforge.net>
References: <E15Y2LQ-0005nv-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <15233.14418.396030.108069@slothrop.digicool.com>

>>>>> "ESR" == Eric S Raymond <esr@users.sourceforge.net> writes:

  ESR> --- NEW FILE: compilerlike.py ---
  ESR> """ compilerlike -- framework code for building compiler-like
  ESR> programs.

  ESR> There is a common `compiler-like' pattern in Unix scripts which
  ESR> is useful for translation utilities of all sorts.  A program
  ESR> following this pattern behaves as a filter when no argument
  ESR> files are specified on the command line, but otherwise
  ESR> transforms each file individually into a corresponding output
  ESR> file.

I think the name of this module is confusing as best.  It doesn't have
anything to do with compilation -- and the pattern you describe
doesn't apply to compilers I'm familiar with (e.g. gcc).  This library
may be useful, but it needs a better name and short description to
help people understand its use.

I think there was a suggestion to integrate this code with fileinput,
which seems to make sense.  fileinput seems to describe what this
module does -- deal with files as input.

Jeremy



From pf@artcom-gmbh.de  Mon Aug 20 18:28:21 2001
From: pf@artcom-gmbh.de (Peter Funk)
Date: Mon, 20 Aug 2001 19:28:21 +0200 (CEST)
Subject: Choosing a new name for compilerlike.py (was: [Python-Dev] Re: [Python-checkins]
 CVS ...)
In-Reply-To: <15233.14418.396030.108069@slothrop.digicool.com> from Jeremy Hylton
 at "Aug 20, 2001 12:18:26 pm"
Message-ID: <m15Ysqb-001dMzC@artcom0.artcom-gmbh.de>

Hi,

> >>>>> "ESR" == Eric S Raymond <esr@users.sourceforge.net> writes:
> 
>   ESR> --- NEW FILE: compilerlike.py ---
>   ESR> """ compilerlike -- framework code for building compiler-like
>   ESR> programs.
> 
>   ESR> There is a common `compiler-like' pattern in Unix scripts which
>   ESR> is useful for translation utilities of all sorts.  A program
>   ESR> following this pattern behaves as a filter when no argument
>   ESR> files are specified on the command line, but otherwise
>   ESR> transforms each file individually into a corresponding output
>   ESR> file.

Jeremy Hylton wrote:
> I think the name of this module is confusing as best.  

I agree with Jeremy.

> It doesn't have
> anything to do with compilation -- and the pattern you describe
> doesn't apply to compilers I'm familiar with (e.g. gcc).  This library
> may be useful, but it needs a better name and short description to
> help people understand its use.
> 
> I think there was a suggestion to integrate this code with fileinput,
> which seems to make sense.  fileinput seems to describe what this
> module does -- deal with files as input.

Yes, this makes some sense.  On the other hand the main purpose of this
module seems to be to get Unix convention to use a hyphen as alias-name 
for 'stdin' and 'stdout' factored out into a standard module. 

What about calling this 'FilterFrameWork.py'?  

I had quick glance into this module and I think this needs some more
thoughts.  Using filecmp.cmp each time to find out, whether the
provided callback function has done something to the data looks costly 
and is not very generic.

Much of the framework functionality has to do with command line
argument processing, which is often coupled with option processing
(fancygetopt jumps into my mind here) and in many similar scripts
if have seen or written myself often these commands provide the
(additional) possibility to specify either input or output or both
using option arguments.
For example:
	foobar --input=baz.in --output=baz.out
    or 
        foobar -i baz.in -o baz.out

So IMO this module is not yet ready for the Python standard library.

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen, Germany)



From tim@zope.com  Mon Aug 20 20:50:39 2001
From: tim@zope.com (Tim Peters)
Date: Mon, 20 Aug 2001 15:50:39 -0400
Subject: [Python-Dev] RE: [Python-checkins] CVS: python/dist/src/Lib __future__.py,1.9.2.1,1.9.2.2
In-Reply-To: <E15YuRZ-0002pE-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <BIEJKCLHCIOIHAGOKOLHOEFCCFAA.tim@zope.com>

[Guido]
> ...
> I've discovered another reason not to *require* the new module: new is
> not allowed in restricted mode.  Without this patch, this means that
> any import from __future__ would fail, and in turn e.g. "import types"
> would fail.
>
> I'm beginning to wonder if sticking the CO_ constants in __future__
> was such a good idea...

Sure it was.  The problem has nothing to do with __future__, it has to do
with a failure of introspection, i.e. that the CO_xxx flags weren't exposed
anywhere.  That also causes by-hand duplication of the magic little integers
in pyassem.py.  We wouldn't have any problems if I had picked some module
other than "new" to host them (well, any other module that happens always to
be available, unlike "new"); but "new" is the module they fit best.  I could
move 'em to, e.g., sys.



From guido@python.org  Mon Aug 20 22:53:24 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 20 Aug 2001 17:53:24 -0400
Subject: [Python-Dev] RE: [Python-checkins] CVS: python/dist/src/Lib __future__.py,1.9.2.1,1.9.2.2
In-Reply-To: Your message of "Mon, 20 Aug 2001 15:50:39 EDT."
 <BIEJKCLHCIOIHAGOKOLHOEFCCFAA.tim@zope.com>
References: <BIEJKCLHCIOIHAGOKOLHOEFCCFAA.tim@zope.com>
Message-ID: <200108202153.RAA18839@cj20424-a.reston1.va.home.com>

> [Guido]
> > ...
> > I've discovered another reason not to *require* the new module: new is
> > not allowed in restricted mode.  Without this patch, this means that
> > any import from __future__ would fail, and in turn e.g. "import types"
> > would fail.
> >
> > I'm beginning to wonder if sticking the CO_ constants in __future__
> > was such a good idea...
> 
> Sure it was.  The problem has nothing to do with __future__, it has to do
> with a failure of introspection, i.e. that the CO_xxx flags weren't exposed
> anywhere.  That also causes by-hand duplication of the magic little integers
> in pyassem.py.  We wouldn't have any problems if I had picked some module
> other than "new" to host them (well, any other module that happens always to
> be available, unlike "new"); but "new" is the module they fit best.  I could
> move 'em to, e.g., sys.

Sys would make more sense, for sure.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip@pobox.com (Skip Montanaro)  Mon Aug 20 23:42:21 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Mon, 20 Aug 2001 17:42:21 -0500
Subject: [Python-Dev] Improving bsddb detection in setup.py
Message-ID: <15233.37453.222314.927103@beluga.mojam.com>

Folks,

Back about the time 2.1.1 was in the throes of final release, I submitted a
patch to setup.py.  Thomas didn't want to apply it that close to the
release.  It's still sitting there:

    http://sourceforge.net/tracker/?func=detail&aid=440725&group_id=5470&atid=105470

I'd like to see this or something like it get into the distribution.
Sleepycat keeps changing file formats, so there are now three incompatible
versions of libdb out there.  It's a pain in the tush making the switch.

I know there's another bsddb module floating around out there as well.  If
that turns out to be backward compatible with the existing bsddb module,
that would be okay too.  I think you're always going to have to do a fancy
dance in setup.py because of the proliferation of incompatible versions of
libdb. 

Skip


From fdrake@acm.org  Mon Aug 20 23:41:02 2001
From: fdrake@acm.org (Fred L. Drake)
Date: Mon, 20 Aug 2001 18:41:02 -0400 (EDT)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20010820224102.B942E28846@beowolf.digicool.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

Final documentation for Python 2.2 alpha 2.



From barry@zope.com  Mon Aug 20 23:54:28 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 20 Aug 2001 18:54:28 -0400
Subject: [Python-Dev] Improving bsddb detection in setup.py
References: <15233.37453.222314.927103@beluga.mojam.com>
Message-ID: <15233.38180.702486.900448@anthem.wooz.org>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    SM> Back about the time 2.1.1 was in the throes of final release,
    SM> I submitted a patch to setup.py.  Thomas didn't want to apply
    SM> it that close to the release.  It's still sitting there:

    SM>
    SM> http://sourceforge.net/tracker/?func=detail&aid=440725&group_id=5470&atid=105470

    SM> I'd like to see this or something like it get into the
    SM> distribution.  Sleepycat keeps changing file formats, so there
    SM> are now three incompatible versions of libdb out there.  It's
    SM> a pain in the tush making the switch.

    SM> I know there's another bsddb module floating around out there
    SM> as well.  If that turns out to be backward compatible with the
    SM> existing bsddb module, that would be okay too.  I think you're
    SM> always going to have to do a fancy dance in setup.py because
    SM> of the proliferation of incompatible versions of libdb.

Robin Dunn & Andrew Kuchling's PyBSDDB3

    pybsddb.sf.net

It claims to be a backwards compatible drop in for the default bsddb
module, supporting back to 1.85.  It's been On My List of things to
look at for a while now, but I just haven't had time.  I've used
PyBSDDB3 for some Zope/ZODB work and it seems quite stable.  I'd like
it to be the default for Python 2.2 if the backwards compatibility
works out acceptably.

-Barry


From gstein@lyra.org  Tue Aug 21 02:53:23 2001
From: gstein@lyra.org (Greg Stein)
Date: Mon, 20 Aug 2001 18:53:23 -0700
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0268.txt,NONE,1.1 pep-0000.txt,1.128,1.129 pep-0267.txt,1.1,1.2
In-Reply-To: <E15Yz00-0007SI-00@usw-pr-cvs1.sourceforge.net>; from bwarsaw@users.sourceforge.net on Mon, Aug 20, 2001 at 05:02:28PM -0700
References: <E15Yz00-0007SI-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <20010820185322.J11073@lyra.org>

I saw no 267....

A while back, when I asked about adding PEPs, the answer was, "yes, you can
add them yourself if you want; just pick off the next number." So I don't
want to hear about a "spank" from you :-)

And per my checkin message: the rest of the content arrives tonite...

Cheers,
-g

On Mon, Aug 20, 2001 at 05:02:28PM -0700, Barry Warsaw wrote:
> Update of /cvsroot/python/python/nondist/peps
> In directory usw-pr-cvs1:/tmp/cvs-serv28647
> 
> Modified Files:
> 	pep-0000.txt pep-0267.txt 
> Added Files:
> 	pep-0268.txt 
> Log Message:
> Greg Stein was a bad boy!  <268 spank>
> 
> Jeremy Hylton's "Optimized Access to Module Namespaces" PEP gets 267.
> 
> We'll let Greg slide this time, so his (albeit incomplete other than a
> Rationale) PEP gets 268.  It must get filled in soon though or it'll
> get moved to the "Empty" category. :(
>...

-- 
Greg Stein, http://www.lyra.org/


From guido@python.org  Tue Aug 21 03:35:19 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 20 Aug 2001 22:35:19 -0400
Subject: [Python-Dev] Improving bsddb detection in setup.py
In-Reply-To: Your message of "Mon, 20 Aug 2001 17:42:21 CDT."
 <15233.37453.222314.927103@beluga.mojam.com>
References: <15233.37453.222314.927103@beluga.mojam.com>
Message-ID: <200108210235.WAA19541@cj20424-a.reston1.va.home.com>

> Folks,
> 
> Back about the time 2.1.1 was in the throes of final release, I submitted a
> patch to setup.py.  Thomas didn't want to apply it that close to the
> release.  It's still sitting there:
> 
>     http://sourceforge.net/tracker/?func=detail&aid=440725&group_id=5470&atid=105470
> 
> I'd like to see this or something like it get into the distribution.
> Sleepycat keeps changing file formats, so there are now three incompatible
> versions of libdb out there.  It's a pain in the tush making the switch.
> 
> I know there's another bsddb module floating around out there as well.  If
> that turns out to be backward compatible with the existing bsddb module,
> that would be okay too.  I think you're always going to have to do a fancy
> dance in setup.py because of the proliferation of incompatible versions of
> libdb. 
> 
> Skip

Skip, now would be the perfect time to check it in.  We'll try it in
the alpha.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Aug 21 03:38:52 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 20 Aug 2001 22:38:52 -0400
Subject: [Python-Dev] Improving bsddb detection in setup.py
In-Reply-To: Your message of "Mon, 20 Aug 2001 18:54:28 EDT."
 <15233.38180.702486.900448@anthem.wooz.org>
References: <15233.37453.222314.927103@beluga.mojam.com>
 <15233.38180.702486.900448@anthem.wooz.org>
Message-ID: <200108210238.WAA19589@cj20424-a.reston1.va.home.com>

> Robin Dunn & Andrew Kuchling's PyBSDDB3
> 
>     pybsddb.sf.net
> 
> It claims to be a backwards compatible drop in for the default bsddb
> module, supporting back to 1.85.  It's been On My List of things to
> look at for a while now, but I just haven't had time.  I've used
> PyBSDDB3 for some Zope/ZODB work and it seems quite stable.  I'd like
> it to be the default for Python 2.2 if the backwards compatibility
> works out acceptably.

But that requires that you have BSDDB 3, right?  Because of its
licensing, not everybody has it.  So I think it's still a good idea to
have Skip's patch as well.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@zope.com  Tue Aug 21 03:44:25 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 20 Aug 2001 22:44:25 -0400
Subject: [Python-Dev] Improving bsddb detection in setup.py
References: <15233.37453.222314.927103@beluga.mojam.com>
 <15233.38180.702486.900448@anthem.wooz.org>
 <200108210238.WAA19589@cj20424-a.reston1.va.home.com>
Message-ID: <15233.51977.331659.112484@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> But that requires that you have BSDDB 3, right?  Because of
    GvR> its licensing, not everybody has it.  So I think it's still a
    GvR> good idea to have Skip's patch as well.

No, PyBSDDB claims to be BSD 1.85 compatible.  I haven't tried that
yet, but I think it will work even if you have the free, old version
around.

-Barry


From guido@python.org  Tue Aug 21 03:46:06 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 20 Aug 2001 22:46:06 -0400
Subject: [Python-Dev] Improving bsddb detection in setup.py
In-Reply-To: Your message of "Mon, 20 Aug 2001 22:44:25 EDT."
 <15233.51977.331659.112484@anthem.wooz.org>
References: <15233.37453.222314.927103@beluga.mojam.com> <15233.38180.702486.900448@anthem.wooz.org> <200108210238.WAA19589@cj20424-a.reston1.va.home.com>
 <15233.51977.331659.112484@anthem.wooz.org>
Message-ID: <200108210246.WAA19717@cj20424-a.reston1.va.home.com>

>     GvR> But that requires that you have BSDDB 3, right?  Because of
>     GvR> its licensing, not everybody has it.  So I think it's still a
>     GvR> good idea to have Skip's patch as well.
> 
> No, PyBSDDB claims to be BSD 1.85 compatible.  I haven't tried that
> yet, but I think it will work even if you have the free, old version
> around.

So it's *both* compatible with BSD 1.95 *and* with the old bsddb
module.  Then what does it buy you?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From grant.griffin@iowegian.com  Tue Aug 21 04:05:52 2001
From: grant.griffin@iowegian.com (Grant Griffin)
Date: Mon, 20 Aug 2001 22:05:52 -0500
Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value
Message-ID: <4.3.2.7.2.20010820215632.00cb69e0@pop.kcnet.com>

PEP: 265
Title: Sorting Dictionaries by Value
Version: $Revision: 0.1 $
Author: g2@iowegian.com (Grant Griffin)
Status: Draft
Type: Standards Track
Created: 8-Aug-2001
Python-Version: 2.2
Post-History:

Abstract

This PEP suggests a "sort by value" operation for dictionaries.  The
primary benefit would be in terms of "batteries included" support for a
common Python idiom which, in its current form, is both difficult for
beginners to understand and cumbersome for all to implement.

Motivation

A common use of dictionaries is to count occurrences by setting the
value of d[key] to "1" on its first occurrence, then increment the
value on each subsequent occurrence.  This can be done several
different ways, but the "get" method is the most succinct:

	d[key] = d.get(key, 0) + 1

Once all occurrences have been counted, a common use of the resulting
dictionary is to print the occurrences in occurrence-sorted order, often
with the largest value first.

This leads to a need to sort a dictionary's items by value.  The
canonical method of doing so in Python is to first use d.items() to get
a list of the dictionary's items, then invert the ordering of each
item's tuple from (key, value) into (value, key), then sort the list;
since Python sorts the list based on the first item of the tuple, the
list of (inverted) items is therefore sorted by value.  If desired, the
list can then be reversed, and the (value, key) tuples can be re-
inverted back to (key, value).  (However, in my experience, the
inverted tuple ordering is fine for most purposes, e.g. printing out
the list.)

For example, given an occurrence count of:

 >>> d = {'a':2, 'b':23, 'c':5, 'd':17, 'e':1}

we might do:

 >>> items = d.items()
 >>> items = [(v, k) for (k, v) in items]
 >>> items.sort()
 >>> items.reverse()		# so largest is first
 >>> items = [(k, v) for (v, k) in items]

resulting in:

 >>> items
[('b', 23), ('d', 17), ('c', 5), ('a', 2), ('e', 1)]

which shows the list in by-value order, largest first.  (In this case,
'b' was found to have the most occurrences.)

This works fine, but is "hard to use" in two aspects.  First, although
this idiom is known to veteran Pythoneers, it is not at all obvious to
newbies--either in terms of its algorithm (inverting the ordering of
item tuples) or its implementation (using list comprehensions--which
are difficult for beginners.)  Second, it requires having to repeatedly
type a lot of "gunge", resulting in both tedium and mistakes.

We therefore would rather Python provide a method of sorting
dictionaries by value which would be both easy for newbies to
understand (or, better yet, not to _have to_ understand) and easier for
all to use.

Rationale

As Tim Peters has pointed out, this sort of thing brings on the problem
of trying to be all things to all people.  Therefore, we will limit its
scope to try to hit "the sweet spot".  Unusual cases (e.g. sorting via
a custom comparison function) can, of course, be handled "manually"
using present methods.

Here are some simple possibilities:

The "items" method of dictionaries can be augmented with new parameters
having default values that provide for full backwards-compatibility:

	1) items(sort_by_values=0, reversed=0)

or maybe just:

        2) items(sort_by_values=0)

since reversing a list is easy enough.

Alternatively, items() could simply let us control the (key, value)
order:

	3) items(values_first=0)

Again, this is fully backwards-compatible.  It does less "work" than
the others, but it at least eases the most complicated/tricky part of
the sort-by-value problem: inverting the order of item tuples.  Using
this is very simple:

	items = d.items(1)
	items.sort()
	items.reverse()		# (if desired)

The primary drawback of the preceding three approaches is the
additional overhead for the parameter-less "items()" case, due to
having to process default parameters.  (However, if one assumes that
items() gets used primarily for creating sort-by-value lists, this is
not really a drawback in practice.)

Alternatively, we might add a new dictionary method which somehow
embodies "sorting".  This approach offers two advantages.  First, it
avoids adding overhead to the items() method.  Second, it is perhaps
more accessible to newbies: when they go looking for a method for
sorting dictionaries, they hopefully run into this one, and they will
not have to understand the finer points of tuple inversion and list
sorting to achieve sort-by-value.

To allow the four basic possibilities of sorting by key/value and in
forward/reverse order, we could add this method:

	4) sorted_items(by_value=0, reversed=0)

I believe the most common case would actually be "by_value=1,
reversed=1", but the defaults values given here might lead to fewer
surprises by users: sorted_items() would be the same as items()
followed by sort().

Finally (as a last resort), we could use:

	5) items_sorted_by_value(reversed=0)

Implementation

The proposed dictionary methods would necessarily be implemented in C.
Presumably, the implementation would be fairly simple since it involves
just adding a few calls to Python's existing machinery.

Concerns

Aside from the run-time overhead already addressed in possibilities 1-
3, concerns with this proposal probably will fall into the categories
of "feature bloat" and/or "code bloat".  However, I believe that
several of the suggestions made here will result in quite minimal
bloat, resulting in a good tradeoff between bloat and "value added".

Tim Peters has noted that implementing this in C might not be
significantly faster than implementing it in Python today.  However,
the major benefits intended here are "accessibility" and "ease of use",
not "speed".  Therefore, as long as it is not noticeably slower (in the
case of plain "items()", speed need not be a consideration.

References

A related thread called "counting occurrences" appeared on
comp.lang.python in August, 2001.  This included examples of approaches
to systematizing the sort-by-value problem by implementing it as
reusable Python functions and classes.

Copyright

This document has been placed in the public domain.

--------------------------------------------------------------------
Also...this just in from Tim Peters:

<Tim>
Here's one to add to the PEP as an alternative:

     dict.sort(sort_specifier)

where sort_specifier is one of these strings:

     'kv->kv'
     'kv->vk'
     'kv->k'
     'kv->v'
     'vk->kv'
     'vk->vk'
     'vk->k'
     'vk->v'

The first 4 sort by keys first, the second 4 by values first.  The string to
the right of '->' says whether you want a list of (key, value) pairs,
(value, key) pairs, just keys, or just values.

So to sort by values and return only the keys (corresponding to increasing
order of values),

     dict.sort('vk->k')

There are two other sort_specifier possibilities:

     'k'
     'v'

dict.sort('k') is the same as ksort(dict):

def ksort(dict):
     x=dict.keys()
     x.sort()
     return x

and similarly for dict.sort('v') returning dict.values() sorted.

Reversals are the user's problem.
</Tim>

Your comments and suggestions are welcome and appreciated.  I think the two 
big questions for the community here are:

1) Is this useful enough to justify the feature/code bloat?  (Tim has 
suggested that this might be a hard sell to Guido, so if you like it in 
whatever form, let's hear from you!)
2) Of all the many possibilities, which do you find the most useful and 
Pythonic?

thanks-much-ly y'rs,

=g2
--
_____________________________________________________________________

Grant R. Griffin                                       g2@dspguru.com
Publisher of dspGuru                           http://www.dspguru.com
Iowegian International Corporation            http://www.iowegian.com



From barry@zope.com  Tue Aug 21 04:58:20 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 20 Aug 2001 23:58:20 -0400
Subject: [Python-Dev] Improving bsddb detection in setup.py
References: <15233.37453.222314.927103@beluga.mojam.com>
 <15233.38180.702486.900448@anthem.wooz.org>
 <200108210238.WAA19589@cj20424-a.reston1.va.home.com>
 <15233.51977.331659.112484@anthem.wooz.org>
 <200108210246.WAA19717@cj20424-a.reston1.va.home.com>
Message-ID: <15233.56412.2158.771422@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> So it's *both* compatible with BSD 1.95 *and* with the old
    GvR> bsddb module.  Then what does it buy you?

It will link against the newer libraries too, if you have them.

-Barry


From skip@pobox.com (Skip Montanaro)  Tue Aug 21 05:27:36 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Mon, 20 Aug 2001 23:27:36 -0500
Subject: [Python-Dev] Improving bsddb detection in setup.py
In-Reply-To: <200108210235.WAA19541@cj20424-a.reston1.va.home.com>
References: <15233.37453.222314.927103@beluga.mojam.com>
 <200108210235.WAA19541@cj20424-a.reston1.va.home.com>
Message-ID: <15233.58168.610817.101621@beluga.mojam.com>

    >> Back about the time 2.1.1 was in the throes of final release, I
    >> submitted a patch to setup.py.  Thomas didn't want to apply it that
    >> close to the release.  It's still sitting there:
    >> 
    >> http://sourceforge.net/tracker/?func=detail&aid=440725&group_id=5470&atid=105470
    ...

    Guido> Skip, now would be the perfect time to check it in.  We'll try it
    Guido> in the alpha.

He takes a deep breath, closes his eyes *very* tight, and pushes the magic
pink button on the console that says "Push only in case of emergency"...
Strange, nothing bad happened.

At any rate, I just checked in setup.py 1.51 with the patch I had previously
submitted and assigned that patch to myself.  I'm fairly certain it will
need some tweaking, but is almost assuredly better than 1.50.

Skip



From skip@pobox.com (Skip Montanaro)  Tue Aug 21 05:35:45 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Mon, 20 Aug 2001 23:35:45 -0500
Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value
In-Reply-To: <4.3.2.7.2.20010820215632.00cb69e0@pop.kcnet.com>
References: <4.3.2.7.2.20010820215632.00cb69e0@pop.kcnet.com>
Message-ID: <15233.58657.726456.31064@beluga.mojam.com>

    Grant> ... since Python sorts the list based on the first item of the
    Grant> tuple, the list of (inverted) items is therefore sorted by value.

Well, to be picky, it uses all the elements of the tuple in the comparison,
it's just that the highest priority item is in the first slot. ;-)

Skip



From greg@electricrain.com  Tue Aug 21 05:36:16 2001
From: greg@electricrain.com (Gregory P. Smith)
Date: Mon, 20 Aug 2001 21:36:16 -0700
Subject: [pybsddb] Re: [Python-Dev] Improving bsddb detection in setup.py
In-Reply-To: <200108210246.WAA19717@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Aug 20, 2001 at 10:46:06PM -0400
References: <15233.37453.222314.927103@beluga.mojam.com> <15233.38180.702486.900448@anthem.wooz.org> <200108210238.WAA19589@cj20424-a.reston1.va.home.com> <15233.51977.331659.112484@anthem.wooz.org> <200108210246.WAA19717@cj20424-a.reston1.va.home.com>
Message-ID: <20010820213616.A25688@zot.electricrain.com>

On Mon, Aug 20, 2001 at 10:46:06PM -0400, Guido van Rossum wrote:
> >     GvR> But that requires that you have BSDDB 3, right?  Because of
> >     GvR> its licensing, not everybody has it.  So I think it's still a
> >     GvR> good idea to have Skip's patch as well.
> > 
> > No, PyBSDDB claims to be BSD 1.85 compatible.  I haven't tried that
> > yet, but I think it will work even if you have the free, old version
> > around.
> 
> So it's *both* compatible with BSD 1.95 *and* with the old bsddb
> module.  Then what does it buy you?

pybsddb only links with BerkeleyDB 3.x.  It supports a bsddb module
compatible interface to make upgrading apps that use the old bsddb
module smoother.

If you want to use the free old version you still need the old bsddb
module.  If you want to use the great new features (transactions,
threading, cursors, etc.) available in BerkeleyDB 3 you need pybsddb.

I'm not certain it makes sense to include pybsddb with python as it
generally needs updating for every 3.x release of BerkeleyDB since
seepycat keeps improving the library interface in minor ways.

Including a mention of pybsddb and a hyperlink to the sourceforge
project in the bsddb module documentation "would be a good thing" (tm).

-- 
Gregory P. Smith   gnupg/pgp: http://electricrain.com/greg/keys/
                   C379 1F92 3703 52C9 87C4  BE58 6CDA DB87 105D 9163


From barry@zope.com  Tue Aug 21 05:54:23 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Tue, 21 Aug 2001 00:54:23 -0400
Subject: [pybsddb] Re: [Python-Dev] Improving bsddb detection in setup.py
References: <15233.37453.222314.927103@beluga.mojam.com>
 <15233.38180.702486.900448@anthem.wooz.org>
 <200108210238.WAA19589@cj20424-a.reston1.va.home.com>
 <15233.51977.331659.112484@anthem.wooz.org>
 <200108210246.WAA19717@cj20424-a.reston1.va.home.com>
 <20010820213616.A25688@zot.electricrain.com>
Message-ID: <15233.59775.192458.503056@anthem.wooz.org>

>>>>> "GPS" == Gregory P Smith <greg@electricrain.com> writes:

    GPS> pybsddb only links with BerkeleyDB 3.x.  It supports a bsddb
    GPS> module compatible interface to make upgrading apps that use
    GPS> the old bsddb module smoother.

Okay, darn.  Forget that idea.

-Barry


From pf@artcom-gmbh.de  Tue Aug 21 07:43:00 2001
From: pf@artcom-gmbh.de (Peter Funk)
Date: Tue, 21 Aug 2001 08:43:00 +0200 (CEST)
Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value
In-Reply-To: <4.3.2.7.2.20010820215632.00cb69e0@pop.kcnet.com> from Grant Griffin
 at "Aug 20, 2001 10:05:52 pm"
Message-ID: <m15Z5Fc-0010t7C@artcom0.artcom-gmbh.de>

Grant Griffin:
> PEP: 265
> Title: Sorting Dictionaries by Value
[...]
> 1) Is this useful enough to justify the feature/code bloat?  (Tim has 
> suggested that this might be a hard sell to Guido, so if you like it in 
> whatever form, let's hear from you!)

I think, this is feature bloat.  IMHO using dictionaries as counters
for something is only one very special application of this generic 
basic data structure. 

A standard library module containing a class derived from
dictionary should be able to fullfill this 'batteries included' wish.
The documentation of the builtin type dictionary could contain
pointers to useful derived classes, so that newbies seeking such
a feature have a better chance to find it.

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen, Germany)



From content-management@high-tech-communcations.com  Tue Aug 21 08:11:06 2001
From: content-management@high-tech-communcations.com (Victor Black)
Date: Tue, 21 Aug 2001 00:11:06 -0700
Subject: [Python-Dev] New web utility
Message-ID: <200108210711.f7L7B6J02297@mail.high-tech-communications.com>

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD><TITLE></TITLE>
<META http-equiv=Content-Type content="text/html; charset=iso-8859-1">
<META content="MSHTML 5.50.4616.200" name=GENERATOR></HEAD>
<BODY>
<P><FONT size=2><FONT face=Arial>I noticed your email address on a list serve 
related to technology and web development.&nbsp; With your permission, 
we<BR>would like to send you information regarding new web tools and utilities 
based on your interests.&nbsp; Please click the<BR>following link and opt-in to 
our product updates and e-newsletter, click here: </FONT><A target=_blank 
href="http://216.133.228.90/cm/"><FONT 
face=Arial>http://216.133.228.90/cm/</FONT></A><BR><BR><FONT 
face=Arial>Cordially,<BR><BR>Victor 
Black<BR>High-Tech-Communications.com</FONT></FONT><FONT face=Arial> </FONT></P>
<P><FONT size=2><FONT face=Arial>If you would like to be removed from our 
database, please click here: </FONT><A 
href="http://216.133.228.90/cm/remove.cgi"><FONT 
face=Arial>http://216.133.228.90/cm/remove.cgi</FONT></A></FONT></P>
<P><FONT face=Arial size=2></FONT>&nbsp;</P></BODY></HTML>



From jack@oratrix.nl  Tue Aug 21 15:56:32 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Tue, 21 Aug 2001 16:56:32 +0200
Subject: [Python-Dev] Putting a builtin module into a package
Message-ID: <20010821145632.7CC15303181@snelboot.oratrix.nl>

Is there a way to put a builtin module into a package, or should I really not 
want to do this?

It seems that if in the module init() routine I call 
Py_InitModule("Carbon.Res")
and if I put the module into _PyImport_Inittab as "Carbon.Res" things sort-of 
work, at first glance.

Or should I rename the module to an underscore name and simply put a wrapper 
module in the package?
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | ++++ see http://www.xs4all.nl/~tank/ ++++




From skip@pobox.com (Skip Montanaro)  Tue Aug 21 16:58:34 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Tue, 21 Aug 2001 10:58:34 -0500
Subject: [Python-Dev] Putting a builtin module into a package
In-Reply-To: <20010821145632.7CC15303181@snelboot.oratrix.nl>
References: <20010821145632.7CC15303181@snelboot.oratrix.nl>
Message-ID: <15234.34090.225332.420431@beluga.mojam.com>

    Jack> Or should I rename the module to an underscore name and simply put
    Jack> a wrapper module in the package?

This seems the safest (and the sanest) to me.  I realize you're thinking
mostly about MacOS, but putting object files inside packages that are
otherwise Python code makes the whole package platform-dependent.  In
general, it would prevent system administrators from installing the
Lib/site-packages directory tree on a shared filesystem like NFS.  In your
package __init__.py file you might want to check for the existence of any
object modules and raise ImportError as early as possible.

Skip


From DavidA@ActiveState.com  Tue Aug 21 20:01:18 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Tue, 21 Aug 2001 12:01:18 -0700
Subject: [Python-Dev] copy, len and the like as 'object' methods?
Message-ID: <3B82AFFE.652A19F5@ActiveState.com>

I just came back from teaching Python on a cruise ship attended by
mostly Perl folks.  It was an interesting experience teaching Python
with Randal in the audience =).

One issue that came up is some of the lack of uniformity on what things
are statements, methods and built-in functions.   In the long-term
version of the type/class unification work, things like int() become
class constructors, which makes beautiful sense.  The fact that int()
calls __int__ methods fits nicely with type conversion mechanisms. 
However, there are a few things which still seem oddballish:

  copy.copy(), copy.deepcopy(), len()

These basically call magic methods of their arguments (whether tp_slots
or __methods__), and many languages implement them strictly as object
methods.  

str() and repr() are a little weird -- I'm not sure which one will gain
'class constructor' status when the type/class unification work is done
-- from the definition I'd say repr() should win, but the name is quite
unfortunate given its new role...  Guido, thoughts?

Summary: Should copy, deepcopy and len be added as object methods?  And
if yes, how?  Not all objects are copyable or measurable.  Interfaces
seem the right way to do this, but interfaces aren't in the plans so far
that I know...

What about a stringification method?

--david


From gstein@lyra.org  Tue Aug 21 21:40:36 2001
From: gstein@lyra.org (Greg Stein)
Date: Tue, 21 Aug 2001 13:40:36 -0700
Subject: [Python-Dev] PEP 268 - Extended HTTP functionality and WebDAV
Message-ID: <20010821134036.J13229@lyra.org>

I've completed my first draft of PEP 268 and am seeking feedback.

    http://python.sourceforge.net/peps/pep-0268.html


My next step is to begin implementing the 'httpx' module and rejiggering the
'davlib' module. This work will occur in the /nondist/sandbox/ area
(probably under a Lib subdir there).

This work is intended to go into Python 2.2. Possibly appearing in time for
2.2a3, but 2.2b1 at the latest.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From jack@oratrix.nl  Tue Aug 21 22:41:31 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Tue, 21 Aug 2001 23:41:31 +0200
Subject: [Python-Dev] Re: [Pythonmac-SIG] Java-Python Extension
In-Reply-To: Message by Jim Harrison <jhrsn@pitt.edu> ,
 Tue, 21 Aug 2001 16:27:00 -0400 , <B7A83C53.4981%jhrsn@pitt.edu>
Message-ID: <20010821214136.3C982162E06@oratrix.oratrix.nl>

Python-dev-ils: this just came in on the Pythonmac-SIG. Did anyone
have a look at this already?

Recently, Jim Harrison <jhrsn@pitt.edu> said:
> Here's something that might have implications for cross-platform GUI
> development in Python (something recently discussed on the list)...
> 
> "JPE is a seamless, complete, and efficient integration of Java and standard
> Python (C Python). JPE provides Java access to Python's native extension,
> and Java's Swing to Python developers."
> 
> http://sourceforge.net/projects/jpe/

The mind boggles... This person (at least: there seems to be only one
person in the developer group right now) has apparently hung a
complete Python interpreter inside a JVM and written all the glue
code.

I have no time to look into this further, but from a quick inspection
of the CVS tree it looks as though it it reasonably complete. It
definitely looks a lot better than many of the vaporware 1-person
projects you can find on sourceforge...

--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 


From barry@scottb.demon.co.uk  Wed Aug 22 00:21:53 2001
From: barry@scottb.demon.co.uk (Barry Scott)
Date: Wed, 22 Aug 2001 00:21:53 +0100
Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value
In-Reply-To: <4.3.2.7.2.20010820215632.00cb69e0@pop.kcnet.com>
Message-ID: <000201c12a98$0fe70b00$060210ac@private>

Seems to me that the problem is the sort method of lists is lacking
not that dictionaries need some special smarts.

I'd look to improve sort for some common cases where sort and lambda
are deemed to hard. (Is using lambda a definition of hard?)

If lambda is used you would solve the problem with:

>>> items = d.items()
>>> items.sort( lambda a, b: cmp( b[1], a[1] ) )

What if you interpret the parameter of sort:

* <missing> - do as today
* callable -  do as today
* int - sort by index into sequence
* string - sort by this field of object

If you had the improved sort you could:

>>> items = d.items()
>>> items.sort( 1 )
>>> items.reverse()

		BArry

> For example, given an occurrence count of:
> 
>  >>> d = {'a':2, 'b':23, 'c':5, 'd':17, 'e':1}
> 
> we might do:
> 
>  >>> items = d.items()
>  >>> items = [(v, k) for (k, v) in items]
>  >>> items.sort()
>  >>> items.reverse()		# so largest is first
>  >>> items = [(k, v) for (v, k) in items]
> 
> resulting in:
> 
>  >>> items
> [('b', 23), ('d', 17), ('c', 5), ('a', 2), ('e', 1)]
> 



From guido@python.org  Wed Aug 22 01:43:33 2001
From: guido@python.org (Guido van Rossum)
Date: Tue, 21 Aug 2001 20:43:33 -0400
Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value
In-Reply-To: Your message of "Wed, 22 Aug 2001 00:21:53 BST."
 <000201c12a98$0fe70b00$060210ac@private>
References: <000201c12a98$0fe70b00$060210ac@private>
Message-ID: <200108220043.UAA24429@cj20424-a.reston1.va.home.com>

> Seems to me that the problem is the sort method of lists is lacking
> not that dictionaries need some special smarts.

Hear, hear.

> I'd look to improve sort for some common cases where sort and lambda
> are deemed to hard. (Is using lambda a definition of hard?)
> 
> If lambda is used you would solve the problem with:
> 
> >>> items = d.items()
> >>> items.sort( lambda a, b: cmp( b[1], a[1] ) )
> 
> What if you interpret the parameter of sort:
> 
> * <missing> - do as today
> * callable -  do as today
> * int - sort by index into sequence
> * string - sort by this field of object

Or better still, list.sort() wouldn't need to be changed at all if
there was a standard library module to create compison routines, e.g.

from __future__ import nested_scopes

def cmp_by_index(i):
    return lambda a, b: cmp((a[i], a), (b[i], b))

# etc.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Wed Aug 22 01:48:15 2001
From: guido@python.org (Guido van Rossum)
Date: Tue, 21 Aug 2001 20:48:15 -0400
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: Your message of "Tue, 21 Aug 2001 12:01:18 PDT."
 <3B82AFFE.652A19F5@ActiveState.com>
References: <3B82AFFE.652A19F5@ActiveState.com>
Message-ID: <200108220048.UAA24452@cj20424-a.reston1.va.home.com>

> I just came back from teaching Python on a cruise ship attended by
> mostly Perl folks.  It was an interesting experience teaching Python
> with Randal in the audience =).

Tell us more...

> One issue that came up is some of the lack of uniformity on what things
> are statements, methods and built-in functions.   In the long-term
> version of the type/class unification work, things like int() become
> class constructors, which makes beautiful sense.  The fact that int()
> calls __int__ methods fits nicely with type conversion mechanisms. 

Except that __int__ is poorly defined -- sometimes it is used as a
precision-losing number cast (e.g. int(3.5) returns 3), sometimes it
is an exact conversion (e.g. int("3")).

> However, there are a few things which still seem oddballish:
> 
>   copy.copy(), copy.deepcopy(), len()
> 
> These basically call magic methods of their arguments (whether tp_slots
> or __methods__), and many languages implement them strictly as object
> methods.  

Yes, it's an old Python choice.  OO zealots don't like it much, but it
has the advantage that you don't have to introduce methods right
away.  Too late to change.

> str() and repr() are a little weird -- I'm not sure which one will gain
> 'class constructor' status when the type/class unification work is done
> -- from the definition I'd say repr() should win, but the name is quite
> unfortunate given its new role...  Guido, thoughts?

str() has won -- in part because of its name, in part because it is
the "copy constructor", returning a string input unchanged.

> Summary: Should copy, deepcopy and len be added as object methods?  And
> if yes, how?  Not all objects are copyable or measurable.  Interfaces
> seem the right way to do this, but interfaces aren't in the plans so far
> that I know...
> 
> What about a stringification method?

I'd say leave it alone (we're getting enough complaints about
"gratuitous" language changes as it is, and the changes we've
committed to have very good reasons).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From grant.griffin@iowegian.com  Wed Aug 22 14:32:57 2001
From: grant.griffin@iowegian.com (Grant Griffin)
Date: Wed, 22 Aug 2001 08:32:57 -0500
Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value
In-Reply-To: <m15Z5Fc-0010t7C@artcom0.artcom-gmbh.de>
References: <4.3.2.7.2.20010820215632.00cb69e0@pop.kcnet.com>
Message-ID: <4.3.2.7.2.20010822081617.00dcab00@pop.kcnet.com>

At 08:43 AM 8/21/2001 +0200, Peter Funk wrote:
>Grant Griffin:
> > PEP: 265
> > Title: Sorting Dictionaries by Value
>[...]
> > 1) Is this useful enough to justify the feature/code bloat?  (Tim has
> > suggested that this might be a hard sell to Guido, so if you like it in
> > whatever form, let's hear from you!)
>
>I think, this is feature bloat.  IMHO using dictionaries as counters
>for something is only one very special application of this generic
>basic data structure.
>
>A standard library module containing a class derived from
>dictionary should be able to fullfill this 'batteries included' wish.
>The documentation of the builtin type dictionary could contain
>pointers to useful derived classes, so that newbies seeking such
>a feature have a better chance to find it.

That's not a bad idea, and if that's how this turns out, I will think it 
had turned out well.

However, the problem I see with a class-based approach is that this is such 
a small thing that I'm not even sure how to write a class for it that isn't 
more grandiose than it deserves.  (Tim's suggestion might help here by 
expanding the scope of the idea <wink>.)  In comparison, the simplest of my 
proposals--the one I'm favoring at the moment--is just to add a boolean 
parameter to items:

         items(values_first=0)

This basically avoids the need for the tuple swap:

         >>> items = [(v, k) for (k, v) in items]

and it's somewhat faster in the sort-by-values case, because the tuples are 
built in the right order rather than being built in the wrong order, then 
swapped.

So, in terms of feature bloat, we have one new parameter.  And in terms of 
code bloat, we have what I estimate to be 4-15 lines of C (depending on 
whether you optimize on speed or size.)

I guess my sense is that the bloat is small enough to be worth it.

thanks,

=g2




From guido@python.org  Wed Aug 22 15:15:37 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 22 Aug 2001 10:15:37 -0400
Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value
In-Reply-To: Your message of "Wed, 22 Aug 2001 08:32:57 CDT."
 <4.3.2.7.2.20010822081617.00dcab00@pop.kcnet.com>
References: <4.3.2.7.2.20010820215632.00cb69e0@pop.kcnet.com>
 <4.3.2.7.2.20010822081617.00dcab00@pop.kcnet.com>
Message-ID: <200108221415.KAA27226@cj20424-a.reston1.va.home.com>

> However, the problem I see with a class-based approach is that this
> is such a small thing that I'm not even sure how to write a class
> for it that isn't more grandiose than it deserves.

Believe me, adding a new method to a heavily-used built-in object is
much more grandiose than adding a library module.  In addition to "15
lines of code" we need to update the documentation, add a few lines to
the test suite, add code to Jython mimicking the feature; 30 books
will be a little more out of date, and so on.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From DavidA@ActiveState.com  Wed Aug 22 17:27:09 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Wed, 22 Aug 2001 09:27:09 -0700
Subject: [Python-Dev] copy, len and the like as 'object' methods?
References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com>
Message-ID: <3B83DD5D.3260BC9C@ActiveState.com>

Guido van Rossum wrote:
> 
> > I just came back from teaching Python on a cruise ship attended by
> > mostly Perl folks.  It was an interesting experience teaching Python
> > with Randal in the audience =).
> 
> Tell us more...

Not much to tell.  Randal was polite in my class and didn't interrupt
too much =).  There were some interesting discussions between Paul
Prescod, Cameron Laird, Damian Conway, Mark-Jason Dominus, Randal and
myself about Perl 6, which is looking more and more like Python from
what I can tell.  The most interesting bit of knowledge I learned is
that the $, % and @ characters have a name -- they are sigils.  And
their use is going to change in Perl 6 to be consistent across contexts,
as they'll be bound with the variable.  Soon I may be able to say that
"If Python hadn't been around, Perl 6 would be my language of choice".
<snicker/>

> Except that __int__ is poorly defined -- sometimes it is used as a
> precision-losing number cast (e.g. int(3.5) returns 3), sometimes it
> is an exact conversion (e.g. int("3")).

Actually, that doesn't bother me -- the definition which works well is
"convert yourself to an integer".  The fact that the float object
decided to consider that as a truncation is possibly a bug in the float
object's decision, but __int__ is still 'convert to int, however you
(the type) see fit'.

> Yes, it's an old Python choice.  OO zealots don't like it much, but it
> has the advantage that you don't have to introduce methods right
> away.  Too late to change.

I wasn't suggesting that we change it.  Only that we add methods to the
objects, much like we added string methods but kept the string module.

> str() has won -- in part because of its name, in part because it is
> the "copy constructor", returning a string input unchanged.

Ah, subtle point I had missed.  

> I'd say leave it alone (we're getting enough complaints about
> "gratuitous" language changes as it is, and the changes we've
> committed to have very good reasons).

Again, I wasn't considering getting rid of old builtins.  We're going to
be adding new methods to all of the types anyway (since they'll derive
from object), so I was just suggesting that len or length be an alias
for __len__.

A minor issue at most.

Cheers,

--david


From guido@python.org  Wed Aug 22 17:40:45 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 22 Aug 2001 12:40:45 -0400
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: Your message of "Wed, 22 Aug 2001 09:27:09 PDT."
 <3B83DD5D.3260BC9C@ActiveState.com>
References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com>
 <3B83DD5D.3260BC9C@ActiveState.com>
Message-ID: <200108221640.f7MGejt29123@odiug.digicool.com>

> Again, I wasn't considering getting rid of old builtins.  We're going to
> be adding new methods to all of the types anyway (since they'll derive
> from object), so I was just suggesting that len or length be an alias
> for __len__.

Hm, but this would violate TOOWTDI.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From Samuele Pedroni <pedroni@inf.ethz.ch>  Wed Aug 22 17:46:52 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Wed, 22 Aug 2001 18:46:52 +0200 (MET DST)
Subject: [Python-Dev] PEP 252, PEP 253 and jython issues
Message-ID: <200108221647.SAA28110@core.inf.ethz.ch>

FYI I'm up to finish with the whole picture tour of PEP 252/PEP 253
wrt to pontential jython issues.

My head has not exploded but I'm a bit worried:
- on one hand for the amount of work that seems necessary on Jython
  side (but that's our problem)
- on the other hand by some design tangle: how to mostly preserve the codebase,
 how to make the life for advanced users easy and natural, how to follow the 
spec.

I hope both that I'm just not clever enough and I have overlooked
things wrongly ... 

Here is an outline:

high-probably non-issues:
 descr stuff up to backward compatibility with non doc/
 supported stuff
 
 top elements of the type/metatype hiearchies.

potential issues (design tangles):

  "layout/inheritance scheme for types/subclassed types":

   - subclassing of types (in java, in Jython)
   - multiple inheritance in the context of type subclassing
   - mro
   - potential mismatch between  
   concrete subclassing (how Jython is coded and
   probably users hope to do subclassing) and   "delegation"
    (how CPython is   coded)
   - interdipendence between object behaviours in CPython, and Jython
     codebase and typical idioms
  
   - how super/dynamic binded behaviour should be invoked? 
   from C/Python/Java
  
tomorrow will follow a filled-in version.

regards, Samuele Pedroni.



From DavidA@ActiveState.com  Wed Aug 22 17:55:54 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Wed, 22 Aug 2001 09:55:54 -0700
Subject: [Python-Dev] copy, len and the like as 'object' methods?
References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com>
 <3B83DD5D.3260BC9C@ActiveState.com> <200108221640.f7MGejt29123@odiug.digicool.com>
Message-ID: <3B83E41A.DC0DE5D@ActiveState.com>

Guido van Rossum wrote:
> 
> > Again, I wasn't considering getting rid of old builtins.  We're going to
> > be adding new methods to all of the types anyway (since they'll derive
> > from object), so I was just suggesting that len or length be an alias
> > for __len__.
> 
> Hm, but this would violate TOOWTDI.

So do string methods =).

--da


From paulp@ActiveState.com  Wed Aug 22 18:39:39 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Wed, 22 Aug 2001 09:39:39 -0800
Subject: [Python-Dev] copy, len and the like as 'object' methods?
References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com>
Message-ID: <3B83EE5B.CEB38C48@ActiveState.com>

David Ascher wrote:
> 
>...
> 
> Not much to tell.  Randal was polite in my class and didn't interrupt
> too much =).  

Randal was polite in general but there was a point where he interjected:
"And you say *Perl* has a lot of special cases? Python seems to have
just as many!" 

These "function syntaxed methods" are one example but there are others.
Some of the others are convenient (e.g. special casing of non-tuples
after %) and some are backwards compatibility issues.

Also, every time we'd point to a feature of Python (OO syntax, exception
handling, generators, *) that was clearly better than Perl, Randal
claimed it was already slated to be fixed for Perl 6. We suggested he
print t-shirts with that refrain.

> ... There were some interesting discussions between Paul
> Prescod, Cameron Laird, Damian Conway, Mark-Jason Dominus, Randal and
> myself about Perl 6, which is looking more and more like Python from
> what I can tell. 

My sense was "Second System Syndrome" but Damian Conway is confident
that they don't have that problem.

* from my point of view it seemed that we agreed that Python was better
than Perl at almost everything but then I didn't sit in on Damian's
"Data Munging" talk.
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From jeremy@zope.com  Wed Aug 22 18:52:06 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Wed, 22 Aug 2001 13:52:06 -0400 (EDT)
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: <3B83EE5B.CEB38C48@ActiveState.com>
References: <3B82AFFE.652A19F5@ActiveState.com>
 <200108220048.UAA24452@cj20424-a.reston1.va.home.com>
 <3B83DD5D.3260BC9C@ActiveState.com>
 <3B83EE5B.CEB38C48@ActiveState.com>
Message-ID: <15235.61766.827878.699521@slothrop.digicool.com>

>>>>> "PP" == Paul Prescod <paulp@ActiveState.com> writes:

  PP> My sense was "Second System Syndrome" but Damian Conway is
  PP> confident that they don't have that problem.

Isn't that one of the signs of the second system effect? <0.5 wink>

Did he offer any reason for his confidence?

Jeremy



From James_Althoff@i2.com  Wed Aug 22 19:11:51 2001
From: James_Althoff@i2.com (James_Althoff@i2.com)
Date: Wed, 22 Aug 2001 11:11:51 -0700
Subject: [Python-Dev] copy, len and the like as 'object' methods?
Message-ID: <OF40A80954.E157C713-ON88256AB0.00634CFE@i2.com>

David Ascher wrote:
>Again, I wasn't considering getting rid of old builtins.  We're going to
>be adding new methods to all of the types anyway (since they'll derive
>from object), so I was just suggesting that len or length be an alias
>for __len__.

(While admittedly being an OO zealot ;-) I too would be pleased to see a
len/length method.  And I really enjoy using the new string methods.
Thanks for adding them!  :-)

Jim



From paulp@ActiveState.com  Wed Aug 22 19:35:23 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Wed, 22 Aug 2001 10:35:23 -0800
Subject: [Python-Dev] copy, len and the like as 'object' methods?
References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com>
 <3B83DD5D.3260BC9C@ActiveState.com> <200108221640.f7MGejt29123@odiug.digicool.com>
Message-ID: <3B83FB6B.5AF81238@ActiveState.com>

Guido van Rossum wrote:
> 
> > Again, I wasn't considering getting rid of old builtins.  We're going to
> > be adding new methods to all of the types anyway (since they'll derive
> > from object), so I was just suggesting that len or length be an alias
> > for __len__.
> 
> Hm, but this would violate TOOWTDI.

I am not an OO zealot but I am a consistency zealot. Calling a function
that just turns around and calls a method is somewhat confusing and
increasingly out of step with all of the other languages that support
OO.

I would appreciate if you could put this on the long-term fixes list and
then you could choose an appropriate time when the howls of pain would
be subdued. 

It really isn't in the same class as the division change because it is
easy to interpret old code in the old way. We could support the
"preferred syntax" and the "legacy syntax" indefinitely and there would
still only one RIGHT way to do it. (just like you could still throw
strings as exceptions but you aren't supposed to...)
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From DavidA@ActiveState.com  Wed Aug 22 19:44:36 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Wed, 22 Aug 2001 11:44:36 -0700
Subject: [Python-Dev] copy, len and the like as 'object' methods?
References: <3B82AFFE.652A19F5@ActiveState.com>
 <200108220048.UAA24452@cj20424-a.reston1.va.home.com>
 <3B83DD5D.3260BC9C@ActiveState.com>
 <3B83EE5B.CEB38C48@ActiveState.com> <15235.61766.827878.699521@slothrop.digicool.com>
Message-ID: <3B83FD94.515D871@ActiveState.com>

Jeremy Hylton wrote:
> 
> >>>>> "PP" == Paul Prescod <paulp@ActiveState.com> writes:
> 
>   PP> My sense was "Second System Syndrome" but Damian Conway is
>   PP> confident that they don't have that problem.
> 
> Isn't that one of the signs of the second system effect? <0.5 wink>
> 
> Did he offer any reason for his confidence?

I think the fundamental source of their confidence lay in Randal's
answer when I challenged the claim that they would write translators
which would automatically translate 95% of Perl5 code to Perl6:  "If
we're smart enough to write Perl5, we can do this".  I think it glosses
over a lot of important issues, but this isn't the forum to discuss it.

Perl6 is being defined pretty much as the kitchen sink -- any cool
feature from any cool language is going to be supported.  Paul and I are
skeptical about the ability to make that a coherent whole, but then
again Perl doesn't really claim coherence, from what I can tell.

Perl6 will have generators, coroutines but not continuations,
higher-order functions, etc. etc.  At the same time, they're trying to
fix some of the issues which made Perl4/5 hard to learn, such as sigil
mutability, etc.  It's designed by committees (30 or so?), on the
assumption that integration is a smaller issue than designing each
'feature' well.  It's an interesting project to watch from the outside. 
There is also apparently a real split between people who are involved in
Perl6 and those who aren't at all.

I did have a very interesting chat w/ Damian Conway -- he said that
Perl6 is already succeeding because it's revitalized the Perl5
community, and that there are lots of high-quality modules added to CPAN
inspired by the Perl6 discussions.  It also makes me a little jealous --
Perl manages to be a place where significant progress is made in the
libraries as distinct from the core.  In Pythonia, there is too much
emphasis among the elite (us =) on adding features to the core as
opposed to library modules, IMO.  Related is the fact that much
'cutting-edge' work in Perl is written in Perl, while most of the
cutting edge stuff in Pythonia is done in C.  Some of that relates to
the almost infinite mutability of the Perl engine by Perl itself, which
is not a design goal of Python's -- we don't want 'just anyone' adding
new keywords, for example.  Still, there is room for exploration and
prototyping there which we don't have such easy access to.

--david


From barry@zope.com  Wed Aug 22 19:55:49 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Wed, 22 Aug 2001 14:55:49 -0400
Subject: [Python-Dev] RELEASED: Python 2.2a2 is out!
Message-ID: <15236.53.399240.678090@yyz.digicool.com>

We've released Python 2.2a2, the second alpha for Python 2.2, for your
enjoyment, education, and edification.  Download it from:

  http://www.python.org/2.2/

Give it a good try, and report what breaks to the bug tracker:

  http://sourceforge.net/bugs/?group_id=5470

New features in this release include:

- Tim Peters developed a brand new Windows installer using Wise 8.1,
  generously donated to us by Wise Solutions.

- The floor division operator // has been added as outlined in PEP
  238. The / operator still provides classic division (and will until
  Python 3.0) unless "from __future__ import division" is included, in
  which case the / operator will provide true division.

- The type/class unification (PEP 252-253) was integrated into the
  trunk and is not so tentative any more (the exact specification of
  some features is still tentative).  A lot of work has done on fixing
  bugs and adding robustness and features (performance still has to
  come a long way).

- Lots of bug fixes, contributed patches, and other stuff.  See the
  Misc/NEWS file in the distribution, or see the release notes on
  SourceForge:

  http://sourceforge.net/project/shownotes.php?release_id=49302

As usual, Andrew Kuchling is writing a gentle introduction to the most
important changes (currently excluding type/class unification), titled
"What's New in Python 2.2":

  http://www.amk.ca/python/2.2/

There is an introduction to the type/class unification at:

  http://www.python.org/2.2/descrintro.html

Thanks to everybody who contributed to this release, including all the
2.2 alpha 1 testers!

Enjoy!
-Barry & Guido


From paulp@ActiveState.com  Wed Aug 22 19:56:33 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Wed, 22 Aug 2001 10:56:33 -0800
Subject: [Python-Dev] copy, len and the like as 'object' methods?
References: <3B82AFFE.652A19F5@ActiveState.com>
 <200108220048.UAA24452@cj20424-a.reston1.va.home.com>
 <3B83DD5D.3260BC9C@ActiveState.com>
 <3B83EE5B.CEB38C48@ActiveState.com> <15235.61766.827878.699521@slothrop.digicool.com>
Message-ID: <3B840061.A55BA194@ActiveState.com>

Jeremy Hylton wrote:
> 
> >>>>> "PP" == Paul Prescod <paulp@ActiveState.com> writes:
> 
>   PP> My sense was "Second System Syndrome" but Damian Conway is
>   PP> confident that they don't have that problem.
> 
> Isn't that one of the signs of the second system effect? <0.5 wink>
> 
> Did he offer any reason for his confidence?

Nothing directly to that question (we ran out of time). But overall of
the Perl guys said things like:

 a) we're got a bunch smart people working on it (Larry, Damian, Dan,
etc.)

 b) everything we're doing has been done somewhere before so it isn't
research (e.g. generators, proper exception handling, decent OO syntax,
etc.)

 c) Perl 5 is a pretty complex system and people of similar caliber
built that so why should Perl 6 be any different 

 d) Perl 6's implementation will be so much cleaner than Perl 5's that
integration of parts will be a lot easier.

 e) we've set up all of these working groups to parallelize development
and that will help alot too.
http://www.perl.com/pub/a/2000/09/perl6mail.html

I think that they are way too optimistic, but there were a few things I
learned that at least hinted in directions of sanity: Perl 5 support may
be provided through some sort of dual interpreter model rather than
trying to write a single interpreter that supports both. Dual
interpreters are pretty well understood (pyperl, jpe, Perl.NET etc.) So
"full backwards compatibility" may be possible using that trick. And
Damian is trying to talk Larry out of a strict requirement that the Perl
compiler be written in Perl.

Also, as David just said, you have to remember that Python people have a
much higher bar of coherence to jump over than the Perl guys. If they
put a bunch of random stuff in the language in a way that didn't really
feel right together -- then it would be just like Perl 5 only with more
features. Whereas we spend a week debating how the 'yield' keyword
should interact with the 'def' keyword.

Damian also said that Perl 6 has to be faster than Perl 5 or nobody will
use it.

Neil Bauman asked him why Perl wasn't being written in Perl (Neil's
opinion is that Perl is literally the greatest language ever created).
Damian responded with a great quote: "Performance. If Performance wasn't
an issue, I'd probably write Perl 6 in Python or something like it with
wonderfully beautiful OO abstractions and it would run like a dog."

-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From akuchlin@mems-exchange.org  Wed Aug 22 20:07:46 2001
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Wed, 22 Aug 2001 15:07:46 -0400
Subject: [Python-Dev] Library development
In-Reply-To: <3B83FD94.515D871@ActiveState.com>; from DavidA@ActiveState.com on Wed, Aug 22, 2001 at 11:44:36AM -0700
References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> <15235.61766.827878.699521@slothrop.digicool.com> <3B83FD94.515D871@ActiveState.com>
Message-ID: <20010822150746.A21476@ute.mems-exchange.org>

On Wed, Aug 22, 2001 at 11:44:36AM -0700, David Ascher wrote:
>which would automatically translate 95% of Perl5 code to Perl6:  "If
>we're smart enough to write Perl5, we can do this".  I think it glosses

<snicker> "We let our code base deteroriate into an incomprehensible
mess of code, and damn it, we can do it again!"

>libraries as distinct from the core.  In Pythonia, there is too much
>emphasis among the elite (us =) on adding features to the core as
>opposed to library modules, IMO.  Related is the fact that much
>'cutting-edge' work in Perl is written in Perl, while most of the
>cutting edge stuff in Pythonia is done in C.  Some of that relates to

*Yes*.  That's mostly why I've been drifting away from python-dev,
too; the core is becoming more and more dull, the new features are
more and more esoteric, and the more interesting action is in
Python-based applications and libraries.  As a bonus, if I work on a
standalone package, I don't need to worry if a change is going to
screw up the X thousands of Python users; only the much smaller pool
of users of the package is at risk, and it's easier to decide to break
things or provide backwards compatibility.

>new keywords, for example.  Still, there is room for exploration and
>prototyping there which we don't have such easy access to.

Tools/compiler lets you try out language modifications, though, so
it's equally possible to do that with Python (witness PTL).  We just
don't do that very much, that's all.

--amk


From barry@scottb.demon.co.uk  Wed Aug 22 20:19:40 2001
From: barry@scottb.demon.co.uk (Barry Scott)
Date: Wed, 22 Aug 2001 20:19:40 +0100
Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value
In-Reply-To: <200108221415.KAA27226@cj20424-a.reston1.va.home.com>
Message-ID: <000701c12b3f$642ab8e0$060210ac@private>

Of course you'd still have to update the docs to point the folks to
the useful comparison functions to use with sort.

		Barry


> -----Original Message-----
> From: python-dev-admin@python.org [mailto:python-dev-admin@python.org]On
> Behalf Of Guido van Rossum
> Sent: 22 August 2001 15:16
> To: Grant Griffin
> Cc: Peter Funk; python-dev@python.org
> Subject: Re: [Python-Dev] PEP 265 - Sorting Dictionaries by Value
> 
> 
> > However, the problem I see with a class-based approach is that this
> > is such a small thing that I'm not even sure how to write a class
> > for it that isn't more grandiose than it deserves.
> 
> Believe me, adding a new method to a heavily-used built-in object is
> much more grandiose than adding a library module.  In addition to "15
> lines of code" we need to update the documentation, add a few lines to
> the test suite, add code to Jython mimicking the feature; 30 books
> will be a little more out of date, and so on.
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> 


From barry@scottb.demon.co.uk  Wed Aug 22 20:25:05 2001
From: barry@scottb.demon.co.uk (Barry Scott)
Date: Wed, 22 Aug 2001 20:25:05 +0100
Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value
In-Reply-To: <200108220043.UAA24429@cj20424-a.reston1.va.home.com>
Message-ID: <000801c12b40$25ae3640$060210ac@private>

Oh I like that!

> Or better still, list.sort() wouldn't need to be changed at all if
> there was a standard library module to create compison routines, e.g.
> 
> from __future__ import nested_scopes
> 
> def cmp_by_index(i):
>     return lambda a, b: cmp((a[i], a), (b[i], b))

How should all the variations be handled?

* uses a naming convention and create the functions one by one?

cmp_by_index_accending - sort low to high at index n
cmp_by_index_decending - sort high to low at index n
cmp_by_field_accending - sort low to high at field n
cmp_by_field_decending - sort high to low at field n
... case blind, case sensitive versions...

* have a make_cmp function that takes a description and build
  the cmp function on demand?

  Would the description be a string with keywords in it?
  "index=4,accending field=5,case-blind,decending"
  sort by index 4 accending then by field=5 case blind comparison.

		Barry



From guido@python.org  Wed Aug 22 20:38:25 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 22 Aug 2001 15:38:25 -0400
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: Your message of "Wed, 22 Aug 2001 09:39:39 -0800."
 <3B83EE5B.CEB38C48@ActiveState.com>
References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com>
 <3B83EE5B.CEB38C48@ActiveState.com>
Message-ID: <200108221938.f7MJcPj29382@odiug.digicool.com>

> My sense was "Second System Syndrome" but Damian Conway is confident
> that they don't have that problem.

Yeah, right.  It has second system syndrome written all over it. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@zope.com  Wed Aug 22 20:38:33 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Wed, 22 Aug 2001 15:38:33 -0400
Subject: [Python-Dev] copy, len and the like as 'object' methods?
References: <OF40A80954.E157C713-ON88256AB0.00634CFE@i2.com>
Message-ID: <15236.2617.169057.464118@anthem.wooz.org>

>>>>> "JA" == James Althoff <James_Althoff@i2.com> writes:

    JA> (While admittedly being an OO zealot ;-) I too would be
    JA> pleased to see a len/length method.

I have to admit I miss them too sometimes. ;)



From Greg.Wilson@baltimore.com  Wed Aug 22 20:36:21 2001
From: Greg.Wilson@baltimore.com (Greg Wilson)
Date: Wed, 22 Aug 2001 15:36:21 -0400
Subject: [Python-Dev] Re: copy, len and the like as 'object' methods?
Message-ID: <930BBCA4CEBBD411BE6500508BB3328F3A8ECC@nsamcanms1.ca.baltimore.com>

FWIW, David's 'len()' methods would definitely make Python easier
to teach --- just about every beginner types 'x.len()' instead of
'len(x)' over and over during their first two or three exercises.

Greg


-----------------------------------------------------------------------------------------------------------------
The information contained in this message is confidential and is intended 
for the addressee(s) only.  If you have received this message in error or 
there are any problems please notify the originator immediately.  The 
unauthorized use, disclosure, copying or alteration of this message is 
strictly forbidden. Baltimore Technologies plc will not be liable for direct, 
special, indirect or consequential damages arising from alteration of the 
contents of this message by a third party or as a result of any virus being 
passed on.

In addition, certain Marketing collateral may be added from time to time to 
promote Baltimore Technologies products, services, Global e-Security or 
appearance at trade shows and conferences.
 
This footnote confirms that this email message has been swept by 
Baltimore MIMEsweeper for Content Security threats, including
computer viruses.


From guido@python.org  Wed Aug 22 20:46:01 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 22 Aug 2001 15:46:01 -0400
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: Your message of "Wed, 22 Aug 2001 11:11:51 PDT."
 <OF40A80954.E157C713-ON88256AB0.00634CFE@i2.com>
References: <OF40A80954.E157C713-ON88256AB0.00634CFE@i2.com>
Message-ID: <200108221946.f7MJk1j29412@odiug.digicool.com>

> David Ascher wrote:
> >Again, I wasn't considering getting rid of old builtins.  We're going to
> >be adding new methods to all of the types anyway (since they'll derive
> >from object), so I was just suggesting that len or length be an alias
> >for __len__.

James Althoff:
> (While admittedly being an OO zealot ;-) I too would be pleased to see a
> len/length method.  And I really enjoy using the new string methods.
> Thanks for adding them!  :-)

While it will admittedly take a while, the intention of string methods
was to eventually phase out the string module.  So TOOWTDI is only
violated during the transition.

If we decide to add a len() method, we'll also have to decide to
(eventually) get rid of len(), or we'd violate TOOWTDI *permanently*.

Personally, I don't think that the "problem" this change is addressing
is worth asking everybody to change their code -- so we'd have live
with len(x) and x.len() as viable ways to ask for x's length
*forever*, violating TOOWTDI permanently.  I don't like that.

(You may disagree, but at least I have thought about it and I can
explain my motivation. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From dan@sidhe.org  Wed Aug 22 20:57:34 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Wed, 22 Aug 2001 15:57:34 -0400
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: <200108221938.f7MJcPj29382@odiug.digicool.com>
References: <Your message of "Wed, 22 Aug 2001 09:39:39 -0800." <3B83EE5B.CEB38C48@ActiveState.com>
 <3B82AFFE.652A19F5@ActiveState.com>
 <200108220048.UAA24452@cj20424-a.reston1.va.home.com>
 <3B83DD5D.3260BC9C@ActiveState.com>
 <3B83EE5B.CEB38C48@ActiveState.com>
Message-ID: <5.1.0.14.0.20010822155714.08b95330@tuatha.sidhe.org>

At 03:38 PM 8/22/2001 -0400, Guido van Rossum wrote:
> > My sense was "Second System Syndrome" but Damian Conway is confident
> > that they don't have that problem.
>
>Yeah, right.  It has second system syndrome written all over it. :-)

You underestimate the amount of black-hearted cynicism we can muster. ;-P

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From dan@sidhe.org  Wed Aug 22 20:57:01 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Wed, 22 Aug 2001 15:57:01 -0400
Subject: [Python-Dev] Library development
In-Reply-To: <20010822150746.A21476@ute.mems-exchange.org>
References: <3B83FD94.515D871@ActiveState.com>
 <3B82AFFE.652A19F5@ActiveState.com>
 <200108220048.UAA24452@cj20424-a.reston1.va.home.com>
 <3B83DD5D.3260BC9C@ActiveState.com>
 <3B83EE5B.CEB38C48@ActiveState.com>
 <15235.61766.827878.699521@slothrop.digicool.com>
 <3B83FD94.515D871@ActiveState.com>
Message-ID: <5.1.0.14.0.20010822155219.08c3a4b0@tuatha.sidhe.org>

At 03:07 PM 8/22/2001 -0400, Andrew Kuchling wrote:
>On Wed, Aug 22, 2001 at 11:44:36AM -0700, David Ascher wrote:
> >which would automatically translate 95% of Perl5 code to Perl6:  "If
> >we're smart enough to write Perl5, we can do this".  I think it glosses
>
><snicker> "We let our code base deteroriate into an incomprehensible
>mess of code, and damn it, we can do it again!"

Now, now, let's not throw stones. :)

>*Yes*.  That's mostly why I've been drifting away from python-dev,
>too; the core is becoming more and more dull, the new features are
>more and more esoteric, and the more interesting action is in
>Python-based applications and libraries.  As a bonus, if I work on a
>standalone package, I don't need to worry if a change is going to
>screw up the X thousands of Python users; only the much smaller pool
>of users of the package is at risk, and it's easier to decide to break
>things or provide backwards compatibility.

Is this a bad thing, though? C's not changed much in the past decade or 
two, but look at how much more canned functionality's available now than 
in, say, 1980. (Heck, add Perl, Python, and Ruby to the list of C's canned 
functionality if you like)


					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From dan@sidhe.org  Wed Aug 22 20:51:59 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Wed, 22 Aug 2001 15:51:59 -0400
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: <3B83FD94.515D871@ActiveState.com>
References: <3B82AFFE.652A19F5@ActiveState.com>
 <200108220048.UAA24452@cj20424-a.reston1.va.home.com>
 <3B83DD5D.3260BC9C@ActiveState.com>
 <3B83EE5B.CEB38C48@ActiveState.com>
 <15235.61766.827878.699521@slothrop.digicool.com>
Message-ID: <5.1.0.14.0.20010822153946.08c02de8@tuatha.sidhe.org>

At 11:44 AM 8/22/2001 -0700, David Ascher wrote:
>Jeremy Hylton wrote:
> >
> > >>>>> "PP" == Paul Prescod <paulp@ActiveState.com> writes:
> >
> >   PP> My sense was "Second System Syndrome" but Damian Conway is
> >   PP> confident that they don't have that problem.
> >
> > Isn't that one of the signs of the second system effect? <0.5 wink>
> >
> > Did he offer any reason for his confidence?
>
>I think the fundamental source of their confidence lay in Randal's
>answer when I challenged the claim that they would write translators
>which would automatically translate 95% of Perl5 code to Perl6:  "If
>we're smart enough to write Perl5, we can do this".  I think it glosses
>over a lot of important issues, but this isn't the forum to discuss it.

Randal's confidence is based in some part on the fact that he's watching 
other people do it, and it's a lot easier to be confident that way. :) I, 
on the other hand, am nervous as all hell, honestly. (OTOH I feel the same 
way about raising my kids)

Perl 5 to perl 6 isn't that tough though (we do chunks of Python and Ruby 
to perl 5, after all), and we do have a full-featured perl 5 parser handy. 
It's as much a task of either reimplementing the perl 5 C-based parser in 
perl as part of the perl 6 parser front-end (Non-trivial, but not rocket 
science either) or build a perl 5 bytecode->perl 6 bytecode translator, 
which is significantly easier. Or just teach the perl 5 parser to spit out 
a syntax tree, which it pretty much does already.

>Perl6 is being defined pretty much as the kitchen sink -- any cool
>feature from any cool language is going to be supported.
>
>It's designed by committees (30 or so?), on the
>assumption that integration is a smaller issue than designing each
>'feature' well.

Forgive the snippage, but the above two quotes are directly related.

Perl 6 the language is *not* being designed by committees, though parts of 
it look that way from the outside. There's exactly one person responsible 
for the design of perl 6--Larry. Damian's feeding a lot of interesting 
ideas to him, but Larry's ultimately responsible for putting together a 
coherent whole. The committee setup at the beginning was designed to let 
the people who had an interest in a particular topic get together and flesh 
out a proposal or four for Larry, but we made no promises that the 
proposals would see the light of day unchanged in the final language design.

The internals design is being handled by a small group of folks, about six, 
each of us with a different area of competence. We're working things out so 
we have some good walls between components so they can be implemented 
semi-separately and in parallel.

Part of me would like to have the "design by committee" feel stand until we 
pull the rabbit out of the hat later, but that might convince people that 
committee design can actually work in large projects and, well, we have 
enough of that nonsense around as it is.

>I did have a very interesting chat w/ Damian Conway -- he said that
>Perl6 is already succeeding because it's revitalized the Perl5
>community,

FWIW, the success does spread beyond perl. If nothing else, there's a fair 
amount of "Perl can do that? Well, then, so can *my* language!" feeling 
amongst folks who feel strongly about languages and such. (Can't imagine 
who that'd be... :) Perl's doing more, which is prompting folks to do more 
in other languages, as a tit for tat thing if nothing else. And most any 
time you add functionality, everybody wins.


					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From skip@pobox.com (Skip Montanaro)  Wed Aug 22 21:01:30 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Wed, 22 Aug 2001 15:01:30 -0500
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: <3B83FD94.515D871@ActiveState.com>
References: <3B82AFFE.652A19F5@ActiveState.com>
 <200108220048.UAA24452@cj20424-a.reston1.va.home.com>
 <3B83DD5D.3260BC9C@ActiveState.com>
 <3B83EE5B.CEB38C48@ActiveState.com>
 <15235.61766.827878.699521@slothrop.digicool.com>
 <3B83FD94.515D871@ActiveState.com>
Message-ID: <15236.3994.507401.287832@beluga.mojam.com>

    David> It also makes me a little jealous -- Perl manages to be a place
    David> where significant progress is made in the libraries as distinct
    David> from the core.  In Pythonia, there is too much emphasis among the
    David> elite (us =) on adding features to the core as opposed to library
    David> modules, IMO.  Related is the fact that much 'cutting-edge' work
    David> in Perl is written in Perl, while most of the cutting edge stuff
    David> in Pythonia is done in C.

I think there is a lot of emphasis in the Python community on getting the
platform "right".  That leads to long debates about the correct way to do
division, for instance.  Also, Python is more platform-independent than
Perl, so more work is necessary in the core to make and keep it that way.
In addition, check out "man perlguts".  The first statement begins

    This document attempts to describe how to use the Perl API, ...

No hedging is necessary in either the Python C API manual or the Python
Extending and Embedding manual.  In fact, in the Extending and Embedding
manual begins

    It is quite easy to add new built-in modules to Python, if you know how
    to program in C.

So the problem you describe is is clearly Guido fault.  He made Python's
internals too clean.  If he would have obfuscated the internals more, more
people would have been drawn to the libraries (or repelled by the
internals). ;-)

On the flip side, I also think there is a lot of cool stuff going on in the
library sector.  You don't hear about it as much in part because Python
doesn't have an equivalent to CPAN.  I'm not talking just about the
cataloging part (Vaults of Parnassus), but the build part (distutils) and
automatic mirroring/dependency checking/download part (for which Python has,
as yet, nothing remotely like it).  That makes the barrier to use of
libraries (and thus use by a critical mass) not in the core that much
higher.  There are how many Python Server Pages like things?  They probably
all have some very nifty features, but tend to be used by rather small
pockets of people, at least relative to the overall segment of the Python
community doing webish stuff.  I'm not entirely sure why that's the case,
but I suspect it has to do in part with the barrier to discovering,
downloading, and installing these packages.

Skip


From paulp@ActiveState.com  Wed Aug 22 21:45:57 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Wed, 22 Aug 2001 13:45:57 -0700
Subject: [Python-Dev] More feedback from script scape
Message-ID: <3B841A05.F292DD5B@ActiveState.com>

Something else I heard at script scape is that for simple extensions
(wrappers around C code) Perl has not only caught up to Python but
surpassed it thanks to inline::C, inline::Python etc. As soon as you
move to doing anything complicated with the interpreter internals,
Python wins again, but it still annoys me that Perl has leapfrogged us
in wrapping simplicity!
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From paulp@ActiveState.com  Wed Aug 22 22:03:42 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Wed, 22 Aug 2001 14:03:42 -0700
Subject: [Python-Dev] Perl5->Perl6
References: <3B82AFFE.652A19F5@ActiveState.com>
 <200108220048.UAA24452@cj20424-a.reston1.va.home.com>
 <3B83DD5D.3260BC9C@ActiveState.com>
 <3B83EE5B.CEB38C48@ActiveState.com>
 <15235.61766.827878.699521@slothrop.digicool.com> <5.1.0.14.0.20010822153946.08c02de8@tuatha.sidhe.org>
Message-ID: <3B841E2E.C3B462C8@ActiveState.com>

Dan Sugalski wrote:
> 
>...
> 
> Perl 5 to perl 6 isn't that tough though (we do chunks of Python and Ruby
> to perl 5, after all), and we do have a full-featured perl 5 parser handy.

You guys are really focused on the syntax, but to me, the difficult
things to handle are subtle *semantic* changes. That's why I said in
another message that a two-interpreter system might be the easiest
route.
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From dan@sidhe.org  Wed Aug 22 22:17:12 2001
From: dan@sidhe.org (Dan Sugalski)
Date: Wed, 22 Aug 2001 17:17:12 -0400
Subject: [Python-Dev] Perl5->Perl6
In-Reply-To: <3B841E2E.C3B462C8@ActiveState.com>
References: <3B82AFFE.652A19F5@ActiveState.com>
 <200108220048.UAA24452@cj20424-a.reston1.va.home.com>
 <3B83DD5D.3260BC9C@ActiveState.com>
 <3B83EE5B.CEB38C48@ActiveState.com>
 <15235.61766.827878.699521@slothrop.digicool.com>
 <5.1.0.14.0.20010822153946.08c02de8@tuatha.sidhe.org>
Message-ID: <5.1.0.14.0.20010822170534.08c50170@tuatha.sidhe.org>

At 02:03 PM 8/22/2001 -0700, Paul Prescod wrote:
>Dan Sugalski wrote:
> >
> >...
> >
> > Perl 5 to perl 6 isn't that tough though (we do chunks of Python and Ruby
> > to perl 5, after all), and we do have a full-featured perl 5 parser handy.
>
>You guys are really focused on the syntax, but to me, the difficult
>things to handle are subtle *semantic* changes. That's why I said in
>another message that a two-interpreter system might be the easiest
>route.

Well, as I've had some Python (and Java) folks happily point out, perl 
doesn't have any well-defined semantics anyway. :)

Seriously, we've a reasonably comprehensive test suite now with perl 5, and 
a very large code base with expected behavior to draw on to test the 
conversion. Much of the perl 5->perl 6 transition is strictly syntactic. 
It's possible that some of perl 5's undefined edge behaviour won't 
translate, but that'll only be because Larry's decided not to make it 
translate. Anything else is a bug, and one to be squashed. (We're not 
losing turing-completeness--perfect execution of past behaviour isn't 
unattainable)

This isn't, after all, the first, or second, or even third time we've done 
this. The version number's 6 for a reason...

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                      teddy bears get drunk



From gstein@lyra.org  Wed Aug 22 23:29:42 2001
From: gstein@lyra.org (Greg Stein)
Date: Wed, 22 Aug 2001 15:29:42 -0700
Subject: huge sig (was: Re: [Python-Dev] Re: copy, len and the like as 'object' methods?)
In-Reply-To: <930BBCA4CEBBD411BE6500508BB3328F3A8ECC@nsamcanms1.ca.baltimore.com>; from Greg.Wilson@baltimore.com on Wed, Aug 22, 2001 at 03:36:21PM -0400
References: <930BBCA4CEBBD411BE6500508BB3328F3A8ECC@nsamcanms1.ca.baltimore.com>
Message-ID: <20010822152941.E15670@lyra.org>

Euh... that .sig is *way* bigger than your message. And it doesn't really
apply to a public forum, now does it? :-)

Cheers,
-g

On Wed, Aug 22, 2001 at 03:36:21PM -0400, Greg Wilson wrote:
> FWIW, David's 'len()' methods would definitely make Python easier
> to teach --- just about every beginner types 'x.len()' instead of
> 'len(x)' over and over during their first two or three exercises.
> 
> Greg
> 
> 
> -----------------------------------------------------------------------------------------------------------------
> The information contained in this message is confidential and is intended 
> for the addressee(s) only.  If you have received this message in error or 
> there are any problems please notify the originator immediately.  The 
> unauthorized use, disclosure, copying or alteration of this message is 
> strictly forbidden. Baltimore Technologies plc will not be liable for direct, 
> special, indirect or consequential damages arising from alteration of the 
> contents of this message by a third party or as a result of any virus being 
> passed on.
> 
> In addition, certain Marketing collateral may be added from time to time to 
> promote Baltimore Technologies products, services, Global e-Security or 
> appearance at trade shows and conferences.
>  
> This footnote confirms that this email message has been swept by 
> Baltimore MIMEsweeper for Content Security threats, including
> computer viruses.
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev

-- 
Greg Stein, http://www.lyra.org/


From aahz@rahul.net  Wed Aug 22 23:48:54 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Wed, 22 Aug 2001 15:48:54 -0700 (PDT)
Subject: [Python-Dev] Deprecation of the string module
In-Reply-To: <200108221946.f7MJk1j29412@odiug.digicool.com> from "Guido van Rossum" at Aug 22, 2001 03:46:01 PM
Message-ID: <20010822224855.31B8FE8C7@waltz.rahul.net>

Guido van Rossum wrote:
> 
> While it will admittedly take a while, the intention of string methods
> was to eventually phase out the string module.  So TOOWTDI is only
> violated during the transition.

I'm glad to see that you're using the past tense with regard to your
intentions for the string module.

(I'm phrasing this as a joke, but I'm utterly serious.)
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.


From barry@zope.com  Wed Aug 22 23:53:05 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Wed, 22 Aug 2001 18:53:05 -0400
Subject: [Python-Dev] More feedback from script scape
References: <3B841A05.F292DD5B@ActiveState.com>
Message-ID: <15236.14289.311782.621168@anthem.wooz.org>

>>>>> "PP" == Paul Prescod <paulp@ActiveState.com> writes:

    PP> Something else I heard at script scape is that for simple
    PP> extensions (wrappers around C code) Perl has not only caught
    PP> up to Python but surpassed it thanks to inline::C,
    PP> inline::Python etc. As soon as you move to doing anything
    PP> complicated with the interpreter internals, Python wins again,
    PP> but it still annoys me that Perl has leapfrogged us in
    PP> wrapping simplicity!

How about a PEP, Paul?!

I've no clue what inline::C or inline::Python do, and no time to learn
about it, but if you have a good grasp of them, try to think about how
those ideas might translate to Python.  A PEP on simplifying extension
writing would be welcome.

-Barry


From paulp@ActiveState.com  Wed Aug 22 23:53:53 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Wed, 22 Aug 2001 15:53:53 -0700
Subject: [Python-Dev] copy, len and the like as 'object' methods?
References: <OF40A80954.E157C713-ON88256AB0.00634CFE@i2.com> <200108221946.f7MJk1j29412@odiug.digicool.com>
Message-ID: <3B843801.BA1D8541@ActiveState.com>

Guido van Rossum wrote:
> 
>...
> 
> While it will admittedly take a while, the intention of string methods
> was to eventually phase out the string module.  So TOOWTDI is only
> violated during the transition.

Why wouldn't a similar strategy work for length and string attributes?

>...
> Personally, I don't think that the "problem" this change is addressing
> is worth asking everybody to change their code -- so we'd have live
> with len(x) and x.len() as viable ways to ask for x's length
> *forever*, violating TOOWTDI permanently.  I don't like that.

How was the string methods problem more serious than this one?

I'm not trying to be difficult: I really don't perceive a difference.
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From ping@lfw.org  Thu Aug 23 00:05:23 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Wed, 22 Aug 2001 16:05:23 -0700 (PDT)
Subject: [Python-Dev] Inline code in other languages
In-Reply-To: <15236.14289.311782.621168@anthem.wooz.org>
Message-ID: <Pine.LNX.4.32.0108221557590.5627-100000@ziggy.localdomain.fake>

On Wed, 22 Aug 2001, Barry A. Warsaw wrote:
> I've no clue what inline::C or inline::Python do,

I went and checked this out after reading Paul's message.

It seems the basic idea is that the Inline module will take a string
containing C code, parse out the function declarations, generate the
appropriate wrapper functions, compile an extension module to produce
a .so file in a temporary directory, and dynamically load it on the spot.
Functions with convertible argument and return types become subroutines
callable from Perl.

Quite slick.

The end result is that you can say:

    use Inline C => 'void greet() { printf("Hello world!\n"); }';
    greet;

...and have this work.  This takes a long time the first time you
do it, but the text of the inline code is MD5-hashed and the .so file
is left lying around, so it's fast the next time around.

It should be pretty feasible to do this for Python, i think, provided
the sys module provides enough details about how Python was compiled
and installed.


-- ?!ng



From paulp@ActiveState.com  Thu Aug 23 00:10:58 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Wed, 22 Aug 2001 16:10:58 -0700
Subject: [Python-Dev] More feedback from script scape
References: <3B841A05.F292DD5B@ActiveState.com> <15236.14289.311782.621168@anthem.wooz.org>
Message-ID: <3B843C02.6EFEC933@ActiveState.com>

"Barry A. Warsaw" wrote:
> 
>...
> 
> How about a PEP, Paul?!
> 
> I've no clue what inline::C or inline::Python do, and no time to learn
> about it, but if you have a good grasp of them, try to think about how
> those ideas might translate to Python.  A PEP on simplifying extension
> writing would be welcome.

I have only a superficial understanding which I can convey in this
email. If you think my superficial understanding is worth PEP-ing, then
I can do that.

http://inline.perl.org/

Okay, remember in the old days when you did inline assembly in C code?
You did that because assembly was a speedup over C and there were a few
things you could do in assembly but not in C (e.g. talk directly to
devices and registers by memory address). Fast forward to the present.
Why do you use C extensions? 1. Speedup 2. Direct access to stuff that
has a C API. So why not inline C in Python (or Perl)?

use Inline C => <<'END_C';
    void greet() {
        printf("Hello, world\n");
    }
    END_C    

greet();

This does what you would expect. But what's going on under the hood?

 1. Inline pseudo-parses the C code and looks for function and parameter
declarations.

 2. Inline generates the wrapper code that Python programmers usually
either write by hand or get SWIG to do.

 3. Inline generates a dynamic library in a special "cache" directory
using MakeMaker, the system C compiler, etc.

 4. Inline loads the library it just created as a Perl extension module.

You (the user) invoke all of this stuff just by running the program.
Inline doesn't recompile over and over again because it uses MD5 to
recognize that the thing has already been compiled. Inline does have a
strategy for dealing with modules that is supposed to be distributed
instead of just used locally. It essentially compiles the code once and
then you ask an external program to package it as a source or binary
distribution.

I suggest you skim the examples in this article:

http://www.perl.com/pub/a/2001/02/inline.html

You'll see: 

 * calling win32 functions without using distutils or MakeMaker
explicitly

 * writing a C program that behaves as if it were a script (no build
step)

 * embedding Python in Perl

and other weird stuff that will make you feel like your "head has just
been wrapped around a brick". (quote from the article)

-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From paulp@ActiveState.com  Thu Aug 23 00:19:19 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Wed, 22 Aug 2001 16:19:19 -0700
Subject: [Python-Dev] Inline code in other languages
References: <Pine.LNX.4.32.0108221557590.5627-100000@ziggy.localdomain.fake>
Message-ID: <3B843DF7.31198022@ActiveState.com>

Ka-Ping Yee wrote:
> 
>...
> 
> It should be pretty feasible to do this for Python, i think, provided
> the sys module provides enough details about how Python was compiled
> and installed.

I agree!

The fundamental advantage of inline is that you don't have to worry
about refcounts and PyObjects but you also don't have to learn to use
SWIG. (SWIG might be helpful under the covers as the wrapper generator
technology...) It also removes the whole build step from projects that
want to wrap C code. This makes C extension wrapping much more Pythonic.
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From guido@python.org  Thu Aug 23 00:17:33 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 22 Aug 2001 19:17:33 -0400
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: Your message of "Wed, 22 Aug 2001 15:53:53 PDT."
 <3B843801.BA1D8541@ActiveState.com>
References: <OF40A80954.E157C713-ON88256AB0.00634CFE@i2.com> <200108221946.f7MJk1j29412@odiug.digicool.com>
 <3B843801.BA1D8541@ActiveState.com>
Message-ID: <200108222317.TAA30578@cj20424-a.reston1.va.home.com>

> > While it will admittedly take a while, the intention of string methods
> > was to eventually phase out the string module.  So TOOWTDI is only
> > violated during the transition.
> 
> Why wouldn't a similar strategy work for length and string attributes?

It hardly worked for string methods (see Aahz's response).

> >...
> > Personally, I don't think that the "problem" this change is addressing
> > is worth asking everybody to change their code -- so we'd have live
> > with len(x) and x.len() as viable ways to ask for x's length
> > *forever*, violating TOOWTDI permanently.  I don't like that.
> 
> How was the string methods problem more serious than this one?

It was motivated by the advent of Unicode, which required us to think
more OO.

AFAICT, the len() method proposal is just an attempt to be more OO,
and to avoid being made fun of by a bunch of Perlers.  I don't care
about #1, and it's naive to think that solving one issue will avoid
#2.

Any change like this has a *huge* cost to the community (just reread
the division thread).

> I'm not trying to be difficult: I really don't perceive a difference.

I do -- see above.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From James_Althoff@i2.com  Thu Aug 23 00:52:40 2001
From: James_Althoff@i2.com (James_Althoff@i2.com)
Date: Wed, 22 Aug 2001 16:52:40 -0700
Subject: [Python-Dev] copy, len and the like as 'object' methods?
Message-ID: <OFA337644B.1FCBCD08-ON88256AB0.00826E15@i2.com>

Guido wrote:
>AFAICT, the len() method proposal is just an attempt to be more OO,
>and to avoid being made fun of by a bunch of Perlers.  I don't care
>about #1, and it's naive to think that solving one issue will avoid
>#2.

I agree that it isn't so important to be swayed by #1 and #2 above.  But
there is a #3.  I've introduced over fifty programmers (coming from Java)
to Jython and a high percentage trip over this initially (momentarily, at
least ;-).  It doesn't take much to get past it, obviously, but it's a
little nicer not to stumble to begin with.  (my final 2 cents worth :-)

Jim



From paulp@ActiveState.com  Thu Aug 23 00:54:21 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Wed, 22 Aug 2001 16:54:21 -0700
Subject: [Python-Dev] copy, len and the like as 'object' methods?
References: <OF40A80954.E157C713-ON88256AB0.00634CFE@i2.com> <200108221946.f7MJk1j29412@odiug.digicool.com>
 <3B843801.BA1D8541@ActiveState.com> <200108222317.TAA30578@cj20424-a.reston1.va.home.com>
Message-ID: <3B84462D.230FFF3F@ActiveState.com>

Guido van Rossum wrote:
> 
>...
>
> AFAICT, the len() method proposal is just an attempt to be more OO,
> and to avoid being made fun of by a bunch of Perlers.  I don't care
> about #1, and it's naive to think that solving one issue will avoid
> #2.

OO is merely accidental. The real goal is to be more consistent. Python
is a language that uses methods for polymorphism so consistency pushes
towards methods. If Python were Scheme and had an OO appendage in the
middle of its standard library I'd dislike that too. 

The only reason Perlers come into the picture is because they are
pointing out that we claim to be consistent but in this case are not.
And the only reason any of us care about consistency is because
inconsistency has a cost in education and memorization...and according
to Greg Wilson he's seen it first-hand in this case. (I personally had
more experience with confusion over string functions instead of string
methods)

> Any change like this has a *huge* cost to the community (just reread
> the division thread).

Division is a special case. There is a big difference between
re-purposing existing syntax and adding new methods to existing objects.

If one is clearly the "new, better" way and the other is historical,
then the books document the new way and most users eventually forget
that the old way even exists. e.g. string exceptions, string.functions,
UserDict (after type-class), regex, etc.
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From ping@lfw.org  Thu Aug 23 01:04:23 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Wed, 22 Aug 2001 17:04:23 -0700 (PDT)
Subject: [Python-Dev] Re: copy, len and the like as 'object' methods?
In-Reply-To: <3B84462D.230FFF3F@ActiveState.com>
Message-ID: <Pine.LNX.4.32.0108221657520.5627-100000@ziggy.localdomain.fake>

On Wed, 22 Aug 2001, Paul Prescod wrote:
> OO is merely accidental. The real goal is to be more consistent. Python
> is a language that uses methods for polymorphism so consistency pushes
> towards methods.

Hmm.  And what of, say, abs(-3)?  Should that be -3.abs()?

Would you also argue for [1, 2, 3].repr() and "abc".hash()?

Where do you draw the line?  Maybe my habits are too ingrained
at this point, but i kind of like the boundary between __methods__
and ordinary methods... it would somehow bug me to have all these
methods like abs, len, hash crowding in on my ordinary-method
namespace.


-- ?!ng



From barry@zope.com  Thu Aug 23 01:52:25 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Wed, 22 Aug 2001 20:52:25 -0400
Subject: [Python-Dev] Re: Inline code in other languages
References: <15236.14289.311782.621168@anthem.wooz.org>
 <Pine.LNX.4.32.0108221557590.5627-100000@ziggy.localdomain.fake>
Message-ID: <15236.21449.231509.725435@anthem.wooz.org>

>>>>> "KY" == Ka-Ping Yee <ping@lfw.org> writes:

    KY> ...and have this work.  This takes a long time the first time
    KY> you do it, but the text of the inline code is MD5-hashed and
    KY> the .so file is left lying around, so it's fast the next time
    KY> around.

    KY> It should be pretty feasible to do this for Python, i think,
    KY> provided the sys module provides enough details about how
    KY> Python was compiled and installed.

Slick indeed!  I don't see much that's beyond Python's grasp.  The
syntax of course would be more Pythonic, but you could probably put
all this in a module, hook it up with distutils, and hack out
something rough in a day or two.  Tim will port it to Windows along
with a frozen cygwin gcc compiler in a huge frozen binary.  Viola!

-Barry


From barry@zope.com  Thu Aug 23 01:55:25 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Wed, 22 Aug 2001 20:55:25 -0400
Subject: [Python-Dev] More feedback from script scape
References: <3B841A05.F292DD5B@ActiveState.com>
 <15236.14289.311782.621168@anthem.wooz.org>
 <3B843C02.6EFEC933@ActiveState.com>
Message-ID: <15236.21629.394538.234568@anthem.wooz.org>

>>>>> "PP" == Paul Prescod <paulp@ActiveState.com> writes:

    PP> I have only a superficial understanding which I can convey in
    PP> this email. If you think my superficial understanding is worth
    PP> PEP-ing, then I can do that.

>From what you and Ping described, I doubt it needs to be PEP'd.  It
could all be done in a module, without any impact on the language I'd
imagine.

-Barry


From ping@lfw.org  Thu Aug 23 02:02:46 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Wed, 22 Aug 2001 18:02:46 -0700 (PDT)
Subject: [Python-Dev] Re: Inline code in other languages
In-Reply-To: <15236.21449.231509.725435@anthem.wooz.org>
Message-ID: <Pine.LNX.4.32.0108221756480.5627-100000@ziggy.localdomain.fake>

> Slick indeed!  I don't see much that's beyond Python's grasp.  The
> syntax of course would be more Pythonic,

Perhaps:

    import inline
    inline.C('''

    void greet(char* str)
        {
        printf("%s, %s, %s, lovely %s\n", str, str, str, str);
        }

    ''')

    from inline.C import greet
    greet('spam')

...?


-- ?!ng



From paulp@ActiveState.com  Thu Aug 23 02:08:40 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Wed, 22 Aug 2001 18:08:40 -0700
Subject: [Python-Dev] Re: copy, len and the like as 'object' methods?
References: <Pine.LNX.4.32.0108221657520.5627-100000@ziggy.localdomain.fake>
Message-ID: <3B845798.7505DC91@ActiveState.com>

Ka-Ping Yee wrote:
> 
>...
> 
> Hmm.  And what of, say, abs(-3)?  Should that be -3.abs()?

The former is more in-line with traditional mathematical notation. I
think that counts for something. And anyhow, abs is not fully
polymorphic so it is a little bit of a different case.

> Would you also argue for [1, 2, 3].repr() and "abc".hash()?

Yes. But hashing and repring are done much less often than taking the
length. So I wouldn't worry about them...in this round anyhow. Plus repr
has a first-class syntax so the function is kind of redundant already!

> Where do you draw the line? 

If a function takes a single argument that is a Python object and
immediately dispatches to a magic method, and it is "important" enough
to be a builtin, then it should probably just be a method. What is the
point of hiding what is really going on -- a method call!

>  Maybe my habits are too ingrained
> at this point, but i kind of like the boundary between __methods__
> and ordinary methods... it would somehow bug me to have all these
> methods like abs, len, hash crowding in on my ordinary-method
> namespace.

I can appreciate that your tastes are different from mine...in fact I
said something similar to David when he first suggested it to me but on
reflection it seemed such a subtle thing when compared against the
weirdness of the definition of len:

def len(obj):
   return obj.__length__()

It does a little bit more but not much! If you put that in a module you
wrote you would consider it a little bit bizarre, useless and confusing.
Plus it is another indirection which saps a tiny bit of performance for
nothing.
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From skip@pobox.com (Skip Montanaro)  Thu Aug 23 02:13:43 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Wed, 22 Aug 2001 20:13:43 -0500
Subject: [Python-Dev] More feedback from script scape
In-Reply-To: <15236.21629.394538.234568@anthem.wooz.org>
References: <3B841A05.F292DD5B@ActiveState.com>
 <15236.14289.311782.621168@anthem.wooz.org>
 <3B843C02.6EFEC933@ActiveState.com>
 <15236.21629.394538.234568@anthem.wooz.org>
Message-ID: <15236.22727.471154.470699@beluga.mojam.com>

    BAW> From what you and Ping described, I doubt it needs to be PEP'd.  It
    BAW> could all be done in a module, without any impact on the language
    BAW> I'd imagine.

Agreed.  I believe it's the same with Perl's Inline module.  In fact, I
first encountered it about a year ago when trying to figure out how to write
Mason components in Python.  I seem to recall that the author(s) wrote
Inline as a lark.

Skip



From barry@zope.com  Thu Aug 23 02:20:51 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Wed, 22 Aug 2001 21:20:51 -0400
Subject: [Python-Dev] Re: Inline code in other languages
References: <15236.21449.231509.725435@anthem.wooz.org>
 <Pine.LNX.4.32.0108221756480.5627-100000@ziggy.localdomain.fake>
Message-ID: <15236.23155.857942.461104@anthem.wooz.org>

>>>>> "KY" == Ka-Ping Yee <ping@lfw.org> writes:

    >> Slick indeed!  I don't see much that's beyond Python's grasp.
    >> The syntax of course would be more Pythonic,

    KY> Perhaps:

    |     import inline
    |     inline.C('''

    |     void greet(char* str)
    |         {
    |         printf("%s, %s, %s, lovely %s\n", str, str, str, str);
    |         }

    |     ''')

    |     from inline.C import greet
    |     greet('spam')

    KY> ...?

Works for me!
-Barry


From DavidA@ActiveState.com  Thu Aug 23 02:28:57 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Wed, 22 Aug 2001 18:28:57 -0700
Subject: [Python-Dev] copy, len and the like as 'object' methods?
References: <OF40A80954.E157C713-ON88256AB0.00634CFE@i2.com> <200108221946.f7MJk1j29412@odiug.digicool.com>
 <3B843801.BA1D8541@ActiveState.com> <200108222317.TAA30578@cj20424-a.reston1.va.home.com>
Message-ID: <3B845C59.5DF822A4@ActiveState.com>

Guido van Rossum wrote:

> AFAICT, the len() method proposal is just an attempt to be more OO,
> and to avoid being made fun of by a bunch of Perlers.  I don't care
> about #1, and it's naive to think that solving one issue will avoid
> #2.

Actually, I couldn't care less about #1 or #2.  Paul and I hold our own
against Perl folks every day =), and I'm not a fan of pure-OO'ness.  I
do try to make _learning_ Python as easy as possible, and
inconsistencies such as this one do get in the way.

People who learn the language ask things like "how do I make a copy of
something?" and the answer to that is currently quite complicated.  It
just gets in the way, and it wastes synapses.

My only point is that if the language were to be designed today, my
guess is that those things would be methods (or interfaces :-).

--david


From paulp@ActiveState.com  Thu Aug 23 02:43:21 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Wed, 22 Aug 2001 18:43:21 -0700
Subject: [Python-Dev] PyInline
References: <3B841A05.F292DD5B@ActiveState.com>
 <15236.14289.311782.621168@anthem.wooz.org>
 <3B843C02.6EFEC933@ActiveState.com>
 <15236.21629.394538.234568@anthem.wooz.org> <15236.22727.471154.470699@beluga.mojam.com>
Message-ID: <3B845FB9.2C165BD2@ActiveState.com>

Skip Montanaro wrote:
> 
>     BAW> From what you and Ping described, I doubt it needs to be PEP'd.  It
>     BAW> could all be done in a module, without any impact on the language
>     BAW> I'd imagine.
> 
> Agreed.  I believe it's the same with Perl's Inline module.  In fact, I
> first encountered it about a year ago when trying to figure out how to write
> Mason components in Python.  I seem to recall that the author(s) wrote
> Inline as a lark.

Yes but Inline has grown into an all-consuming obsession. He left his
job at ActiveState to pursue it "full time". There is even an Inline
t-shirt. Inline won an award for best new module at the Perl conference.
Outside of Perl 6, it is probably the biggest new thing in the Perl
world.

It isn't as trivial as you think. It is essentially a superset of SWIG's
Python-related functionality. Don't forget that an important part of it
is trying to autodetect types. It quickly gets into figuring out structs
and then classes and then templates and then madness ensues. Obviously
you have to decide how much you are going to support and just require
"compliant" wrappers for the rest. Using SWIG might allow you to offload
a bunch of that effort for PyInline::C.

Plus, there is Inline::C, Inline::Python, Inline::Fortran, Inline::ASM,
Inline::Java, etc. 

http://search.cpan.org/search?mode=module&query=Inline%3a%3a

Finally, you have to think hard about how the thing interacts with
binary distributions for people without compilers. That includes a hefty
chunk of Python's user base. This is much more important than the other
two points.

I don't want to discourage anyone thinking about tackling it but I also
want to give you a reasonable sense of what needs to be implemented.
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From beazley@cs.uchicago.edu  Thu Aug 23 03:16:02 2001
From: beazley@cs.uchicago.edu (David Beazley)
Date: Wed, 22 Aug 2001 21:16:02 -0500 (CDT)
Subject: [Python-Dev] Inline and SWIG
In-Reply-To: <E15ZjYK-0006WI-00@mail.python.org>
References: <E15ZjYK-0006WI-00@mail.python.org>
Message-ID: <15236.26466.84309.855339@gargoyle.cs.uchicago.edu>

 > 
 > It isn't as trivial as you think. It is essentially a superset of SWIG's
 > Python-related functionality. Don't forget that an important part of it
 > is trying to autodetect types. It quickly gets into figuring out structs
 > and then classes and then templates and then madness ensues. Obviously
 > you have to decide how much you are going to support and just require
 > "compliant" wrappers for the rest. Using SWIG might allow you to offload
 > a bunch of that effort for PyInline::C.

I've looked briefly at Inline and agree that it's a pretty nifty
approach.  However, usability aside, it's got a number of pretty major
limitations once you start getting into structure wrapping, C++
classes, and other advanced types of extension wrapping. I think a
common misconception about wrapper generation tools in general is that
it's just a matter of hacking up a quick parser and throwing something
together in a couple of days. The *real* problem IMHO is actually
trying to figure out how to properly handle all of the facets of the
C/C++ type system (pointers, references, qualifiers, classes,
templates, typedef, inheritance, multiple inheritance, member
pointers, pass by value, etc.) and how that's supposed to hook into
Python.  What's worse, if you get *ANY* part of the type system wrong,
everything becomes a hellish nightmare later (and I can speak from 
personal experience on this :-).

We've been working on an upcoming major new SWIG release for quite
some time now and about 80% of that work has been related to
redesigning the internal type system.  Maybe this sort of thing is
obvious to everyone else and I'm missing some trivial point, but I
sure haven't found the C++ type system to be this totally trivial 
thing I could just hack up in a few days and stick in a code generator!  

In any case, I think the coolest thing about Inline is the ability to
add small little bits of C code on-the-fly without having to restart
the interpreter.  That's a pretty neat trick that would probably be
useful in the context of working with larger C extensions.  I'm not
sure I'd want to build a huge 400 function C extension module with
that kind of approach, but something like that would definitely be
useful for prototyping and adding code on the fly.  I almost wonder if
something like SWIG could be used behind the scenes to do code
generation for something like that.

Cheers,

Dave

















From ping@lfw.org  Thu Aug 23 03:25:49 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Wed, 22 Aug 2001 19:25:49 -0700 (PDT)
Subject: [Python-Dev] Re: Inline and SWIG
In-Reply-To: <15236.26466.84309.855339@gargoyle.cs.uchicago.edu>
Message-ID: <Pine.LNX.4.32.0108221923330.5627-100000@ziggy.localdomain.fake>

On Wed, 22 Aug 2001, David Beazley wrote:
> I've looked briefly at Inline and agree that it's a pretty nifty
> approach.  However, usability aside, it's got a number of pretty major
> limitations once you start getting into structure wrapping, C++
> classes, and other advanced types of extension wrapping.

I think the audiences are different.  SWIG is for taking an existing
library of C or C++ code and making it available in Python.  It seems
to me that Inline is about making multiple languages available to a
programmer while the programmer is writing new code.  I agree with you
that the first task is much, much harder.  However, there is also a
useful payoff from the second.


-- ?!ng



From beazley@cs.uchicago.edu  Thu Aug 23 03:31:00 2001
From: beazley@cs.uchicago.edu (David Beazley)
Date: Wed, 22 Aug 2001 21:31:00 -0500 (CDT)
Subject: [Python-Dev] Re: Inline and SWIG
In-Reply-To: <Pine.LNX.4.32.0108221923330.5627-100000@ziggy.localdomain.fake>
References: <15236.26466.84309.855339@gargoyle.cs.uchicago.edu>
 <Pine.LNX.4.32.0108221923330.5627-100000@ziggy.localdomain.fake>
Message-ID: <15236.27364.304751.836868@gargoyle.cs.uchicago.edu>

Ka-Ping Yee writes:

 > I think the audiences are different.  SWIG is for taking an existing
 > library of C or C++ code and making it available in Python.  It seems
 > to me that Inline is about making multiple languages available to a
 > programmer while the programmer is writing new code.  I agree with you
 > that the first task is much, much harder.  However, there is also a
 > useful payoff from the second.

Definitely!  I saw a demo of Inline when I visited Activestate awhile
back and was blown away ("You mean you can do that?!?").  Pretty neat
stuff.  

Cheers,

Dave



From guido@python.org  Thu Aug 23 03:29:35 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 22 Aug 2001 22:29:35 -0400
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: Your message of "Wed, 22 Aug 2001 16:54:21 PDT."
 <3B84462D.230FFF3F@ActiveState.com>
References: <OF40A80954.E157C713-ON88256AB0.00634CFE@i2.com> <200108221946.f7MJk1j29412@odiug.digicool.com> <3B843801.BA1D8541@ActiveState.com> <200108222317.TAA30578@cj20424-a.reston1.va.home.com>
 <3B84462D.230FFF3F@ActiveState.com>
Message-ID: <200108230229.WAA31882@cj20424-a.reston1.va.home.com>

> OO is merely accidental. The real goal is to be more consistent. Python
> is a language that uses methods for polymorphism so consistency pushes
> towards methods. If Python were Scheme and had an OO appendage in the
> middle of its standard library I'd dislike that too. 
> 
> The only reason Perlers come into the picture is because they are
> pointing out that we claim to be consistent but in this case are not.
> And the only reason any of us care about consistency is because
> inconsistency has a cost in education and memorization...and according
> to Greg Wilson he's seen it first-hand in this case. (I personally had
> more experience with confusion over string functions instead of string
> methods)
> 
> > Any change like this has a *huge* cost to the community (just reread
> > the division thread).
> 
> Division is a special case. There is a big difference between
> re-purposing existing syntax and adding new methods to existing objects.
> 
> If one is clearly the "new, better" way and the other is historical,
> then the books document the new way and most users eventually forget
> that the old way even exists. e.g. string exceptions, string.functions,
> UserDict (after type-class), regex, etc.

I see nothing that you didn't say before.  Nothing to sway me to set
the wheels in motion to make this particular change.  My position
remains "it ain't broken, so don't fix it."

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Aug 23 03:33:02 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 22 Aug 2001 22:33:02 -0400
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: Your message of "Wed, 22 Aug 2001 18:28:57 PDT."
 <3B845C59.5DF822A4@ActiveState.com>
References: <OF40A80954.E157C713-ON88256AB0.00634CFE@i2.com> <200108221946.f7MJk1j29412@odiug.digicool.com> <3B843801.BA1D8541@ActiveState.com> <200108222317.TAA30578@cj20424-a.reston1.va.home.com>
 <3B845C59.5DF822A4@ActiveState.com>
Message-ID: <200108230233.WAA31918@cj20424-a.reston1.va.home.com>

> Actually, I couldn't care less about #1 or #2.  Paul and I hold our own
> against Perl folks every day =), and I'm not a fan of pure-OO'ness.  I
> do try to make _learning_ Python as easy as possible, and
> inconsistencies such as this one do get in the way.

This one seems to be something that you stumble over once, and then
get used to.

> People who learn the language ask things like "how do I make a copy of
> something?" and the answer to that is currently quite complicated.  It
> just gets in the way, and it wastes synapses.

Maybe they are asking the wrong question.  I think that copying isn't
all that important in Python -- that's why it's in a module.

> My only point is that if the language were to be designed today, my
> guess is that those things would be methods (or interfaces :-).

Sure.  But they're not so broken as to warrant a redesign.  There are
more important battles.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From grant.griffin@iowegian.com  Thu Aug 23 03:47:56 2001
From: grant.griffin@iowegian.com (Grant Griffin)
Date: Wed, 22 Aug 2001 21:47:56 -0500
Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value
In-Reply-To: <200108221415.KAA27226@cj20424-a.reston1.va.home.com>
References: <Your message of "Wed, 22 Aug 2001 08:32:57 CDT." <4.3.2.7.2.20010822081617.00dcab00@pop.kcnet.com>
 <4.3.2.7.2.20010820215632.00cb69e0@pop.kcnet.com>
 <4.3.2.7.2.20010822081617.00dcab00@pop.kcnet.com>
Message-ID: <4.3.2.7.2.20010822212254.00cbd220@pop.kcnet.com>

At 10:15 AM 8/22/2001 -0400, Guido van Rossum wrote:

> > However, the problem I see with a class-based approach is that this
> > is such a small thing that I'm not even sure how to write a class
> > for it that isn't more grandiose than it deserves.
>
>Believe me, adding a new method to a heavily-used built-in object is
>much more grandiose than adding a library module.  In addition to "15
>lines of code" we need to update the documentation, add a few lines to
>the test suite, add code to Jython mimicking the feature; 30 books
>will be a little more out of date, and so on.

I guess I don't see how this one is different from any other feature change 
to Python in that regard: they all require code, tests, and docs.

The only real logistical advantage I can see for implementing it as module 
is that the port to Jython would be virtually free. Aside from that, it 
seems like adding a new module (in any form) would cause much _more_ bloat 
overall in terms of SLOCs, tests and documentation.  For example, a new 
library module would get a doc page of its own, but a single new Boolean 
parameter added to items() could be documented by a footnote on the 
"Mapping Types" page, e.g., "If the 'values_first' parameter is set to 
true, the item tuples returned are in (value, key) order rather than the 
default (key, value) order."

The test suite for the new parameter could be just a line or two which 
would compare the values-first items of a test dictionary to the expected 
list.

So, in total, my preferred suggestion, "items(values_first=0)", (which 
actually isn't a _new_ method!) would cause bloat on the order of:

         15 lines of C (and Java)
         1 sentence of documentation
         3 lines of test code

and best of all!:

         0 __future__ statements

And as for making 30 books a little more out of date...well...if Tim 
O'Reilly doesn't give me a big wet kiss for that, I'm gonna demand he 
refund the $54.95 I just paid for the Second Edition of "Programming 
Python" <wink>.

=g2



From greg@cosc.canterbury.ac.nz  Thu Aug 23 03:57:37 2001
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 23 Aug 2001 14:57:37 +1200 (NZST)
Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value
In-Reply-To: <4.3.2.7.2.20010822081617.00dcab00@pop.kcnet.com>
Message-ID: <200108230257.OAA04095@s454.cosc.canterbury.ac.nz>

Grant Griffin <grant.griffin@iowegian.com>:

> items(values_first=0)

Or perhaps a separate method called smeti() :-)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From greg@cosc.canterbury.ac.nz  Thu Aug 23 04:01:49 2001
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 23 Aug 2001 15:01:49 +1200 (NZST)
Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value
In-Reply-To: <000801c12b40$25ae3640$060210ac@private>
Message-ID: <200108230301.PAA04098@s454.cosc.canterbury.ac.nz>

Barry Scott <barry@scottb.demon.co.uk>:

> How should all the variations be handled?

  def order(index = None, field = None, descending = 0):
    ...

Examples:

  from sorting import order

  x.sort(order(index = 3))
  x.sort(order(field = "customer_name", descending = 1))

The next step is to allow sort() itself to take the same
keyword parameters directly:

  x.sort(index = 3)
  x.sort(field = "customer_name", descending = 1)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From DavidA@ActiveState.com  Thu Aug 23 04:31:55 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Wed, 22 Aug 2001 20:31:55 -0700
Subject: [Python-Dev] Re: Inline and SWIG
References: <15236.26466.84309.855339@gargoyle.cs.uchicago.edu>
 <Pine.LNX.4.32.0108221923330.5627-100000@ziggy.localdomain.fake> <15236.27364.304751.836868@gargoyle.cs.uchicago.edu>
Message-ID: <3B84792B.7DDF7C1B@ActiveState.com>

Note that I know of at least two if not three different inline-like
projects for Python.  

One is the compiler module in scipy (Eric Jones & Travis Oliphant):

http://scipy.net/cgi-bin/viewcvs.cgi/~checkout~/scipy/compiler/doc/readme?rev=1.2&content-type=text/plain

One is the stuff that Pat Miller from LLNL presented at OSCON in San
Diego, which sounded similar in spirit (alas, I missed the talk, and
don't know anything about its availability -- Pat?), and there was a
third one which I heard mentioned but don't know any details about. 
Eric, do you remember what I'm talking about?

I'd recommend that people track down the authors of those modules, as I
do think there is a great opportunity there.  (I take credit for
inspiring Eric Jones by mentioning Inline:: to him in Long Beach =).

--david ascher


From tim.one@home.com  Thu Aug 23 04:32:50 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 22 Aug 2001 23:32:50 -0400
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: <200108230233.WAA31918@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEGBLHAA.tim.one@home.com>

[David Ascher]
> My only point is that if the language were to be designed today, my
> guess is that those things would be methods (or interfaces :-).

[Guido]
> Sure.

Say what?!  What kind of stupid-ass language spells the len() function as if
it were a method?  That's carrying consistency to an irrational extreme --
trust your decade-old intuition on this one (which was challenged no later
than 9 years ago on the same ground).

> But they're not so broken as to warrant a redesign.  There are
> more important battles.

Proof:  every thread bloats in inverse proportion to its true importance.
So there <wink>.

incisively y'rs  - tim



From DavidA@ActiveState.com  Thu Aug 23 04:45:07 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Wed, 22 Aug 2001 20:45:07 -0700
Subject: [Python-Dev] copy, len and the like as 'object' methods?
References: <OF40A80954.E157C713-ON88256AB0.00634CFE@i2.com> <200108221946.f7MJk1j29412@odiug.digicool.com> <3B843801.BA1D8541@ActiveState.com> <200108222317.TAA30578@cj20424-a.reston1.va.home.com>
 <3B845C59.5DF822A4@ActiveState.com> <200108230233.WAA31918@cj20424-a.reston1.va.home.com>
Message-ID: <3B847C43.2082509E@ActiveState.com>

Guido van Rossum wrote:
> 
> > Actually, I couldn't care less about #1 or #2.  Paul and I hold our own
> > against Perl folks every day =), and I'm not a fan of pure-OO'ness.  I
> > do try to make _learning_ Python as easy as possible, and
> > inconsistencies such as this one do get in the way.
> 
> This one seems to be something that you stumble over once, and then
> get used to.

I don't disagree, but people have said the same thing about
case-sensitivity, integer division, and Perl. =)

> > People who learn the language ask things like "how do I make a copy of
> > something?" and the answer to that is currently quite complicated.  It
> > just gets in the way, and it wastes synapses.
> 
> Maybe they are asking the wrong question.  I think that copying isn't
> all that important in Python -- that's why it's in a module.

I'm tired of this argument, so I'll quit.  I'll just say that IMO
students can't ask the wrong questions.  The body of knowledge that they
bring to the table is perfectly valid.  They're trying to 'grok' a large
complicated system, and the questions they ask reflect steps in
understanding.  FWIW, I only talk about copying after I've discussed
references in depth, since that makes the argument you mention more
convincing.  I only half buy the argument myself, however, since the
elegance of the presentation is always marred by having to explain such
oddities, no matter how smoothly I finagle it.

I see much of Python's recent evolution as getting rid of wrinkles in
the system.  I just pointed out a wrinkle, that's all.  Note that I
started with a question, and I'll accept that you don't want to change
it.

Next topic.

--david


From tim.one@home.com  Thu Aug 23 05:06:28 2001
From: tim.one@home.com (Tim Peters)
Date: Thu, 23 Aug 2001 00:06:28 -0400
Subject: [Python-Dev] Cuddly constants seek loving, stable home
Message-ID: <LNBBLJKPBEHFEDALKOLCEEGELHAA.tim.one@home.com>

__future__.py now contains this abomination:

try:
    import new as _new  # for CO_xxx symbols
except ImportError: # May happen during build
    class _new:
        CO_NESTED = 0x0010
        CO_GENERATOR_ALLOWED = 0x1000
        CO_FUTURE_DIVISION = 0x2000

"new" turned out to be a bad module to hold these constants, despite that
that's where they belong (i.e., along with the function that creates new
code objects), first because "new" wasn't always built "in time" on Unixy
boxes, and second because "new" isn't available at all in restricted mode.

So they can't live in "new", but duplicating their values by hand is also
unbearable.  "sys" is the only other module I see where they *might* fit,
but they don't really belong there (indeed, trying to make a case that they
do is a lot like arguing "len" should be a method <wink>).

So I propose a new builtin module, _sysconsts.  The underscore is to
announce its obscure internal nature.  I propose only to expose the CO_xxx
#defines at the start, but of course others could be added.

Any objections?

ain't-worth-a-pep-unless-there-are-ly y'rs  - tim



From guido@python.org  Thu Aug 23 05:07:22 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 23 Aug 2001 00:07:22 -0400
Subject: [Python-Dev] Cuddly constants seek loving, stable home
In-Reply-To: Your message of "Thu, 23 Aug 2001 00:06:28 EDT."
 <LNBBLJKPBEHFEDALKOLCEEGELHAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCEEGELHAA.tim.one@home.com>
Message-ID: <200108230407.AAA00339@cj20424-a.reston1.va.home.com>

> So I propose a new builtin module, _sysconsts.  The underscore is to
> announce its obscure internal nature.  I propose only to expose the CO_xxx
> #defines at the start, but of course others could be added.

+1 on the concept, +0 on the name.  Maybe _sysconst (singular)?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@home.com  Thu Aug 23 05:17:45 2001
From: tim.one@home.com (Tim Peters)
Date: Thu, 23 Aug 2001 00:17:45 -0400
Subject: [Python-Dev] Cuddly constants seek loving, stable home
In-Reply-To: <200108230407.AAA00339@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEGFLHAA.tim.one@home.com>

[Tim, proposes a new _sysconst module]

[Guido]
> +1 on the concept, +0 on the name.  Maybe _sysconst (singular)?

Ya, that's what I said -- you must have suffered an email error <wink>.

the-singular-is-certainly-easier-to-pronounce-ly y'rs  - tim



From esr@snark.thyrsus.com  Thu Aug 23 05:12:56 2001
From: esr@snark.thyrsus.com (Eric S. Raymond)
Date: Thu, 23 Aug 2001 00:12:56 -0400
Subject: [Python-Dev] Cuddly constants seek loving, stable home
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEGELHAA.tim.one@home.com>; from tim.one@home.com on Thu, Aug 23, 2001 at 12:06:28AM -0400
References: <LNBBLJKPBEHFEDALKOLCEEGELHAA.tim.one@home.com>
Message-ID: <20010823001256.A3031@thyrsus.com>

Tim Peters <tim.one@home.com>:
> So I propose a new builtin module, _sysconsts.  The underscore is to
> announce its obscure internal nature.  I propose only to expose the CO_xxx
> #defines at the start, but of course others could be added.
> 
> Any objections?

+0.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

I don't like the idea that the police department seems bent on keeping
a pool of unarmed victims available for the predations of the criminal
class.
         -- David Mohler, 1989, on being denied a carry permit in NYC


From guido@python.org  Thu Aug 23 05:40:39 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 23 Aug 2001 00:40:39 -0400
Subject: [Python-Dev] Near Final PEP 237 - Unifying Long Integers and Integers
Message-ID: <200108230440.AAA00660@cj20424-a.reston1.va.home.com>

Please comment on a new version of PEP 237:

  http://python.sourceforge.net/peps/pep-0237.html

I've updated a few parts of the PEP for more accuracy and to reflect
more implementation details.

I've had very few negative responses to this PEP, and quite a few
positive, so I'm assuming this is not controversial and we can follow
the transition plan described in the PEP.  Only phase A of the
transition plan will be implemented in Python 2.2.

I've implemented phase A completely, and tentatively checked it into
CVS; unless a major showstopper is brought up I will release it with
Python 2.2a3 (it didn't make it into 2.2a2).

Please use the SF bug tracker to report bugs in the implementation.

Please mail comments on the PEP to python-dev@python.org or to me.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@home.com  Thu Aug 23 06:47:45 2001
From: tim.one@home.com (Tim Peters)
Date: Thu, 23 Aug 2001 01:47:45 -0400
Subject: [Python-Dev] Near Final PEP 237 - Unifying Long Integers and Integers
In-Reply-To: <200108230440.AAA00660@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEGJLHAA.tim.one@home.com>

[Guido]
> ...
> I've had very few negative responses to this PEP, and quite a few
> positive, so I'm assuming this is not controversial and we can follow
> the transition plan described in the PEP.  Only phase A of the
> transition plan will be implemented in Python 2.2.

FYI, there have been no negatives on c.l.py either, and at least two
positives -- but it's barely been discussed at all yet.  I changed the
subject line of the one thread that drifted into it to contain "PEP 237".

+1 from me.



From greg@cosc.canterbury.ac.nz  Thu Aug 23 07:01:28 2001
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu, 23 Aug 2001 18:01:28 +1200 (NZST)
Subject: [Python-Dev] Inline and SWIG
In-Reply-To: <15236.26466.84309.855339@gargoyle.cs.uchicago.edu>
Message-ID: <200108230601.SAA04304@s454.cosc.canterbury.ac.nz>

David Beazley <beazley@cs.uchicago.edu>:

> I've looked briefly at Inline and agree that it's a pretty nifty
> approach.  However, usability aside, it's got a number of pretty major
> limitations once you start getting into structure wrapping, C++
> classes, and other advanced types of extension wrapping.

Indeed, you've put your finger on what seems like the
biggest problem with the current crop of semi-automatic
extension generators.

I don't think there is ever going to be a completely
automatic solution to this, because, once you get beyond
the basic types like ints, floats and strings, there is
no unique mapping between Python types and C types.

While the tool could provide some sort of default
translation for arbitrary types, it's unlikely to be
what you need to interface directly with some existing
C library you're trying to wrap. So you're faced with
writing a lot of tedious and error-prone code for messing 
with PyObjects and reference counts.

I've had some ideas floating around in my head for a
while concerning what to do about this. The key
observation is that such code usually spends a lot
more lines manipulating Python data than it does C
data, which leads me to the conclusion that C is the
wrong language to write it in. 

What's the right language? One that's good at manipulating
Python objects. Which language is the best at doing
that? Why, Python, of course. But not just Python,
because we need to be able to manipulate C data
structures as well. So we need a language that looks
like Python at one end and C at the other...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+



From ping@lfw.org  Thu Aug 23 09:37:51 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Thu, 23 Aug 2001 01:37:51 -0700 (PDT)
Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEGELHAA.tim.one@home.com>
Message-ID: <Pine.LNX.4.32.0108230128510.5627-100000@ziggy.localdomain.fake>

On Thu, 23 Aug 2001, Tim Peters wrote:
> So I propose a new builtin module, _sysconsts.  The underscore is to
> announce its obscure internal nature.  I propose only to expose the CO_xxx
> #defines at the start, but of course others could be added.

Just a suggestion, but how about sys.consts?  Like os.path, as in
sys.consts.CO_NESTED, sys.consts.CO_FUTURE_DIVISION, etc.  So there's
only one new object in sys, and it can contain the weird internal stuff. :)

(Errr... the names are rather weird.  CO_NESTED_SCOPES, CO_TRUE_DIVISION,
and CO_GENERATORS would be better.  If i had my druthers it would be

    CO_* constant       FUTURE_* constant       __future__ symbol

    CO_NESTED_SCOPES    FUTURE_NESTED_SCOPES    nested_scopes
    CO_GENERATORS       FUTURE_GENERATORS       generators
    CO_TRUE_DIVISION    FUTURE_TRUE_DIVISION    true_division

instead.  If CO_NESTED, why not CO_TRUE?  If CO_GENERATORS_ALLOWED,
why not CO_NESTED_SCOPES_ALLOWED?)

Anyway.

Hmm, as a side thought, perhaps keywords and tokens should go here too
(sys.keywords, sys.tokens) instead of having to keep separate .py files
in sync with the interpreter core.


-- ?!ng



From fredrik@pythonware.com  Thu Aug 23 09:06:56 2001
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Thu, 23 Aug 2001 10:06:56 +0200
Subject: [Python-Dev] PEP 268 - Extended HTTP functionality and WebDAV
References: <20010821134036.J13229@lyra.org>
Message-ID: <00bf01c12bae$c03cd2c0$0900a8c0@spiff>

greg wrote:


> I've completed my first draft of PEP 268 and am seeking feedback.
>
>     http://python.sourceforge.net/peps/pep-0268.html

looks good to me.

> My next step is to begin implementing the 'httpx' module and rejiggering the
> 'davlib' module. This work will occur in the /nondist/sandbox/ area
> (probably under a Lib subdir there).

excellent!

</F>




From fredrik@pythonware.com  Thu Aug 23 09:17:51 2001
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Thu, 23 Aug 2001 10:17:51 +0200
Subject: [Python-Dev] Re: copy, len and the like as 'object' methods?
References: <930BBCA4CEBBD411BE6500508BB3328F3A8ECC@nsamcanms1.ca.baltimore.com>
Message-ID: <00c001c12bae$c053b620$0900a8c0@spiff>

greg wilson wrote:


> FWIW, David's 'len()' methods would definitely make Python easier
> to teach --- just about every beginner types 'x.len()' instead of
> 'len(x)' over and over during their first two or three exercises.

my students don't -- so it's probably a bug in your training
material, not in Python ;-)

cheers /F




From larsga@garshol.priv.no  Thu Aug 23 09:41:14 2001
From: larsga@garshol.priv.no (Lars Marius Garshol)
Date: 23 Aug 2001 10:41:14 +0200
Subject: [Python-Dev] Re: Inline code in other languages
In-Reply-To: <15236.21449.231509.725435@anthem.wooz.org>
References: <15236.14289.311782.621168@anthem.wooz.org> 	<Pine.LNX.4.32.0108221557590.5627-100000@ziggy.localdomain.fake> <15236.21449.231509.725435@anthem.wooz.org>
Message-ID: <m34rqzyxqt.fsf@lambda.garshol.priv.no>

* Barry A. Warsaw
| 
| Slick indeed!  I don't see much that's beyond Python's grasp.  The
| syntax of course would be more Pythonic, [...]

Why not make it import C files, instead of putting them inline? I
guess you very often want to reuse the C functions in several
different places anyway, and you'd mostly want to have includes and
stuff as well.

This way you can also get rid of the MD5-ing and just use ordinary
modification times to figure out what needs to be recompiled.

You can't inline pieces smaller than functions anyway, and to me the
mix between Python and C code looks very ugly.

--Lars M.



From Paul.Moore@atosorigin.com  Thu Aug 23 10:12:45 2001
From: Paul.Moore@atosorigin.com (Moore, Paul)
Date: Thu, 23 Aug 2001 10:12:45 +0100
Subject: [Python-Dev] Re: copy, len and the like as 'object' methods?
Message-ID: <714DFA46B9BBD0119CD000805FC1F53B01B5AFA3@UKRUX002.rundc.uk.origin-it.com>

> I can appreciate that your tastes are different from
> mine...in fact I said something similar to David when he
> first suggested it to me but on reflection it seemed such
> a subtle thing when compared against the weirdness of the
> definition of len:
>
>    def len(obj):
>        return obj.__length__()
>
> It does a little bit more but not much!

A strange thought, but with the advent of type/class unification and the
'object' base class, is it possible to inject new methods into the 'object'
type at runtime, allowing:

    def len(self):
        return self.__length__()
    object.len = len

This would seem quite slick (or is that "sick" :-), but it makes the
language seem more mutable than might be healthy... (Or is 'object' not this
mutable...? Must try 2.2a2 and see...)

Paul.


From simon@netthink.co.uk  Thu Aug 23 10:52:41 2001
From: simon@netthink.co.uk (Simon Cozens)
Date: Thu, 23 Aug 2001 10:52:41 +0100
Subject: [Python-Dev] Perl5->Perl6
In-Reply-To: <5.1.0.14.0.20010822170534.08c50170@tuatha.sidhe.org>
References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> <15235.61766.827878.699521@slothrop.digicool.com> <5.1.0.14.0.20010822153946.08c02de8@tuatha.sidhe.org> <5.1.0.14.0.20010822170534.08c50170@tuatha.sidhe.org>
Message-ID: <20010823105241.B8531@netthink.co.uk>

On Wed, Aug 22, 2001 at 05:17:12PM -0400, Dan Sugalski wrote:
> Seriously, we've a reasonably comprehensive test suite now with perl 5, and 
> a very large code base with expected behavior to draw on to test the 
> conversion. Much of the perl 5->perl 6 transition is strictly syntactic. 

Also don't forget that we have a prototype of Perl 6 which implements pretty
much everything that Larry has specified in Apocalypses 1 and 2.

It is a patch of nearly 200 lines to the current Perl 5 interpreter.

We're not changing *that* drastically, folks.



From gmcm@hypernet.com  Thu Aug 23 11:37:04 2001
From: gmcm@hypernet.com (Gordon McMillan)
Date: Thu, 23 Aug 2001 06:37:04 -0400
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: <3B847C43.2082509E@ActiveState.com>
Message-ID: <3B84A490.15494.5052EA06@localhost>

[Guido]
> > Maybe they are asking the wrong question.  I think that copying
> > isn't all that important in Python -- that's why it's in a
> > module.
[David] 
> I'm tired of this argument, so I'll quit.  I'll just say that IMO
> students can't ask the wrong questions.  

Not to prolong it, but Guido's comment made me realize that I 
haven't imported copy in years. Haven't written a clone() 
method in years, either, despite the fact that coming from Java 
and C++ that was one of my favorite methods.

get-yourself-a-big-stick-and-teach-'em-Zen-ly y'rs

- Gordon


From skip@pobox.com (Skip Montanaro)  Thu Aug 23 14:11:42 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Thu, 23 Aug 2001 08:11:42 -0500
Subject: [Python-Dev] Inline caveat
Message-ID: <15237.270.649981.730739@beluga.mojam.com>

There is, of course, a definite downside to Inline.  Your code will get
harder to read and maintain because you and whoever else has to maintain
your code has to be familiar with all the languages you are inlining.

Skip


From thomas.heller@ion-tof.com  Thu Aug 23 14:27:05 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Thu, 23 Aug 2001 15:27:05 +0200
Subject: [Python-Dev] Inline caveat
References: <15237.270.649981.730739@beluga.mojam.com>
Message-ID: <034801c12bd7$4dabc3f0$e000a8c0@thomasnotebook>

> There is, of course, a definite downside to Inline.  Your code will get
> harder to read and maintain because you and whoever else has to maintain
> your code has to be familiar with all the languages you are inlining.
> 
> Skip
This is more an argument against mixed language programming
than against Inline.
Inline makes it easier (a no-brainer) to locate the code which
has to be changed, to determine what has to be recompiled,
where (and how) it has to be installed, and so on.

I've played a little bit with the idea and I certainly
like it.

Thomas



From guido@python.org  Thu Aug 23 14:24:52 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 23 Aug 2001 09:24:52 -0400
Subject: [Python-Dev] Re: copy, len and the like as 'object' methods?
In-Reply-To: Your message of "Thu, 23 Aug 2001 10:12:45 BST."
 <714DFA46B9BBD0119CD000805FC1F53B01B5AFA3@UKRUX002.rundc.uk.origin-it.com>
References: <714DFA46B9BBD0119CD000805FC1F53B01B5AFA3@UKRUX002.rundc.uk.origin-it.com>
Message-ID: <200108231324.JAA01440@cj20424-a.reston1.va.home.com>

> A strange thought, but with the advent of type/class unification and the
> 'object' base class, is it possible to inject new methods into the 'object'
> type at runtime, allowing:
> 
>     def len(self):
>         return self.__length__()
>     object.len = len

No you can't.  Built-in types are immutable.  Two reasons: they're
shared between multiple interpreters, and it would encourage modules
to modify a built-in type to suit their own needs, which it might
break other modules.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gstein@lyra.org  Thu Aug 23 14:59:07 2001
From: gstein@lyra.org (Greg Stein)
Date: Thu, 23 Aug 2001 06:59:07 -0700
Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value
In-Reply-To: <200108230301.PAA04098@s454.cosc.canterbury.ac.nz>; from greg@cosc.canterbury.ac.nz on Thu, Aug 23, 2001 at 03:01:49PM +1200
References: <000801c12b40$25ae3640$060210ac@private> <200108230301.PAA04098@s454.cosc.canterbury.ac.nz>
Message-ID: <20010823065907.W16727@lyra.org>

On Thu, Aug 23, 2001 at 03:01:49PM +1200, Greg Ewing wrote:
> Barry Scott <barry@scottb.demon.co.uk>:
> 
> > How should all the variations be handled?
> 
>   def order(index = None, field = None, descending = 0):

Lose the last arg... call them ascending() and descending(). Sorting already
implies an "order" so that name is redundant

>...
>   from sorting import order
> 
>   x.sort(order(index = 3))
>   x.sort(order(field = "customer_name", descending = 1))

  from sorting import ascending, descending
  
  x.sort(ascending(index = 3))
  x.sort(descending(field = 'customer_name'))

> The next step is to allow sort() itself to take the same
> keyword parameters directly:
> 
>   x.sort(index = 3)
>   x.sort(field = "customer_name", descending = 1)

eek... no :-)

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From gstein@lyra.org  Thu Aug 23 15:03:00 2001
From: gstein@lyra.org (Greg Stein)
Date: Thu, 23 Aug 2001 07:03:00 -0700
Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home
In-Reply-To: <Pine.LNX.4.32.0108230128510.5627-100000@ziggy.localdomain.fake>; from ping@lfw.org on Thu, Aug 23, 2001 at 01:37:51AM -0700
References: <LNBBLJKPBEHFEDALKOLCEEGELHAA.tim.one@home.com> <Pine.LNX.4.32.0108230128510.5627-100000@ziggy.localdomain.fake>
Message-ID: <20010823070259.X16727@lyra.org>

On Thu, Aug 23, 2001 at 01:37:51AM -0700, Ka-Ping Yee wrote:
> On Thu, 23 Aug 2001, Tim Peters wrote:
> > So I propose a new builtin module, _sysconsts.  The underscore is to
> > announce its obscure internal nature.  I propose only to expose the CO_xxx
> > #defines at the start, but of course others could be added.
> 
> Just a suggestion, but how about sys.consts?  Like os.path, as in
> sys.consts.CO_NESTED, sys.consts.CO_FUTURE_DIVISION, etc.  So there's
> only one new object in sys, and it can contain the weird internal stuff. :)

+1

-0 on a new module. Just make sys.consts some kind of attributed object.

>...
> Hmm, as a side thought, perhaps keywords and tokens should go here too
> (sys.keywords, sys.tokens) instead of having to keep separate .py files
> in sync with the interpreter core.

+1 ... Good call. That would toss some of the synchronization problems with
those files today.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From skip@pobox.com (Skip Montanaro)  Thu Aug 23 15:19:20 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Thu, 23 Aug 2001 09:19:20 -0500
Subject: [Python-Dev] Inline caveat
In-Reply-To: <034801c12bd7$4dabc3f0$e000a8c0@thomasnotebook>
References: <15237.270.649981.730739@beluga.mojam.com>
 <034801c12bd7$4dabc3f0$e000a8c0@thomasnotebook>
Message-ID: <15237.4328.899938.564006@beluga.mojam.com>

    >> There is, of course, a definite downside to Inline.  Your code will
    >> get harder to read and maintain because you and whoever else has to
    >> maintain your code has to be familiar with all the languages you are
    >> inlining.

    Thomas> This is more an argument against mixed language programming
    Thomas> than against Inline.

Certainly.  Today, you have to keep the various bits of code written in
different languages separate.  Inline certainly lowers that barrier and
makes it feasible to do some spot optimizations quickly.  For example,
several years ago I needed to be able to compute the distance between two
lat/longs.  I wrote:

    def distance(ll1, ll2):
        dlat = ll1[0] - ll2[0]
        longdist = (ll1[1] - ll2[1]) * math.fabs(math.cos(DEGTORAD*dlat))
        return math.sqrt(MILESPERDEGLAT**2 * (dlat*dlat+longdist*longdist))

This turned out to be too slow and I wound up writing a C extension that
exposed just that one function.  An Inline module would have been the
perfect solution to this problem (assuming it could figure out the shared
library needed to be linked with -lm).

    Thomas> Inline makes it easier (a no-brainer) to locate the code which
    Thomas> has to be changed, to determine what has to be recompiled, where
    Thomas> (and how) it has to be installed, and so on.

Yes, but my original assertion still stands.  With C and Python in separate
modules, I can tell people "pay no attention to the man behind the curtain"
(that being the C code).  With the C code right there in the midst of your
Python code it's pretty hard to ignore the fact that the man is not behind
the curtain and he's flashing you. ;-)

Skip



From barry@zope.com  Thu Aug 23 15:29:53 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 23 Aug 2001 10:29:53 -0400
Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home
References: <LNBBLJKPBEHFEDALKOLCEEGELHAA.tim.one@home.com>
 <Pine.LNX.4.32.0108230128510.5627-100000@ziggy.localdomain.fake>
 <20010823070259.X16727@lyra.org>
Message-ID: <15237.4961.597225.931722@anthem.wooz.org>

>>>>> "KY" == Ka-Ping Yee <ping@lfw.org> writes:

    KY> Just a suggestion, but how about sys.consts?  Like os.path, as
    KY> in sys.consts.CO_NESTED, sys.consts.CO_FUTURE_DIVISION, etc.
    KY> So there's only one new object in sys, and it can contain the
    KY> weird internal stuff. :)

+1.  It's much better to qualify them in a subpackage or object than
put them in sys's top level namespace.

-Barry


From guido@python.org  Thu Aug 23 15:32:20 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 23 Aug 2001 10:32:20 -0400
Subject: [Python-Dev] Inline caveat
In-Reply-To: Your message of "Thu, 23 Aug 2001 15:27:05 +0200."
 <034801c12bd7$4dabc3f0$e000a8c0@thomasnotebook>
References: <15237.270.649981.730739@beluga.mojam.com>
 <034801c12bd7$4dabc3f0$e000a8c0@thomasnotebook>
Message-ID: <200108231432.f7NEWKr05785@odiug.digicool.com>

> > There is, of course, a definite downside to Inline.  Your code will get
> > harder to read and maintain because you and whoever else has to maintain
> > your code has to be familiar with all the languages you are inlining.
> > 
> > Skip
> This is more an argument against mixed language programming
> than against Inline.
> Inline makes it easier (a no-brainer) to locate the code which
> has to be changed, to determine what has to be recompiled,
> where (and how) it has to be installed, and so on.
> 
> I've played a little bit with the idea and I certainly
> like it.
> 
> Thomas

If the alternative is writing an extension module, the inline approach
sounds viable to me.

I could also imagine auto-wrapping existing C functions based on
parseable function signatures.  I believe VB supports this.
Dangerous, but powerful, and very useful in the right hands.

It seems to me that the important idea here is to break away from
building and distributing a separate extension module, which gets you
in the edit-compile-link-test-run loop that Python tries to avoid.
Rather than requiring an expert who knows how to download, build,
install and use SWIG or how to write Python extensions, all the
expertise is automated.

Getting this to work on Windows for the average Windows user would be
a big hassle, limiting the portability and therefore the usefulness.

Of course, Jython already has all this, thanks to Java's introspection
capabilities.  I wonder if the new C++ standard makes enough run-time
type information able to be able to pull of the same trick?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Thu Aug 23 15:31:57 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 23 Aug 2001 10:31:57 -0400 (EDT)
Subject: [Python-Dev] Inline caveat
In-Reply-To: <15237.270.649981.730739@beluga.mojam.com>
References: <15237.270.649981.730739@beluga.mojam.com>
Message-ID: <15237.5085.898843.495797@cj42289-a.reston1.va.home.com>

Skip Montanaro writes:
 > There is, of course, a definite downside to Inline.  Your code will get
 > harder to read and maintain because you and whoever else has to maintain
 > your code has to be familiar with all the languages you are inlining.

  Inline::K, anyone?  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From barry@zope.com  Thu Aug 23 15:36:54 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 23 Aug 2001 10:36:54 -0400
Subject: [Python-Dev] Inline caveat
References: <15237.270.649981.730739@beluga.mojam.com>
Message-ID: <15237.5382.431639.149435@anthem.wooz.org>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    SM> There is, of course, a definite downside to Inline.  Your code
    SM> will get harder to read and maintain because you and whoever
    SM> else has to maintain your code has to be familiar with all the
    SM> languages you are inlining.

It's also harder to maintain because it's harder to edit and debug.

Maybe all them fancy IDEs you youngin's use can easily support editing
mutiple different languages in the same file, but if so, let me know!
The tools/editors/debuggers I'm familiar with don't deal with any of
that very well, if at all.

You have a similar problem trying to knit HTML and Python together, as
is proven by the proliferation of such approaches (e.g. DTML, PTL,
HTMLgen, etc., etc.).  It's bearable for one-liners and small bits of
intermingling, but anything large than a line or two gets pretty
unweildy pretty fast.

-Barry


From guido@python.org  Thu Aug 23 15:43:37 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 23 Aug 2001 10:43:37 -0400
Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home
In-Reply-To: Your message of "Thu, 23 Aug 2001 10:29:53 EDT."
 <15237.4961.597225.931722@anthem.wooz.org>
References: <LNBBLJKPBEHFEDALKOLCEEGELHAA.tim.one@home.com> <Pine.LNX.4.32.0108230128510.5627-100000@ziggy.localdomain.fake> <20010823070259.X16727@lyra.org>
 <15237.4961.597225.931722@anthem.wooz.org>
Message-ID: <200108231443.f7NEhb505947@odiug.digicool.com>

> >>>>> "KY" == Ka-Ping Yee <ping@lfw.org> writes:
> 
>     KY> Just a suggestion, but how about sys.consts?  Like os.path, as
>     KY> in sys.consts.CO_NESTED, sys.consts.CO_FUTURE_DIVISION, etc.
>     KY> So there's only one new object in sys, and it can contain the
>     KY> weird internal stuff. :)
> 
> +1.  It's much better to qualify them in a subpackage or object than
> put them in sys's top level namespace.

+0.  It's the better solution but I bet it's more work to code it than
_sysconst.  I'm not volunteering, and I want Tim to have time to help
with the type/class unification, which is behind schedule.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Thu Aug 23 15:44:16 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 23 Aug 2001 10:44:16 -0400 (EDT)
Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home
In-Reply-To: <20010823070259.X16727@lyra.org>
References: <LNBBLJKPBEHFEDALKOLCEEGELHAA.tim.one@home.com>
 <Pine.LNX.4.32.0108230128510.5627-100000@ziggy.localdomain.fake>
 <20010823070259.X16727@lyra.org>
Message-ID: <15237.5824.599253.83454@cj42289-a.reston1.va.home.com>

Greg Stein writes:
 > +1
 > 
 > -0 on a new module. Just make sys.consts some kind of attributed object.

  Agreed.  pyexpat just creates new module objects and fills them in
for things like error codes.

 > +1 ... Good call. That would toss some of the synchronization problems with
 > those files today.

  Yep.  The existing modules could import * from those new objects so
that backward compatibility is retained.
  The advantage of the current approach, however, is that we don't end
up with a bunch of initialization code that ends up sticking around as
part of the process.  Not a real problem for Unix or Windows, but it
might be on embedded platforms like the Palm.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From fdrake@acm.org  Thu Aug 23 16:03:20 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 23 Aug 2001 11:03:20 -0400 (EDT)
Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home
In-Reply-To: <200108231443.f7NEhb505947@odiug.digicool.com>
References: <LNBBLJKPBEHFEDALKOLCEEGELHAA.tim.one@home.com>
 <Pine.LNX.4.32.0108230128510.5627-100000@ziggy.localdomain.fake>
 <20010823070259.X16727@lyra.org>
 <15237.4961.597225.931722@anthem.wooz.org>
 <200108231443.f7NEhb505947@odiug.digicool.com>
Message-ID: <15237.6968.594854.181777@cj42289-a.reston1.va.home.com>

Guido van Rossum writes:
 > +0.  It's the better solution but I bet it's more work to code it than
 > _sysconst.  I'm not volunteering, and I want Tim to have time to help
 > with the type/class unification, which is behind schedule.

  One interesting thing about these modules is that the executable C
code is no longer needed once the modules has been initialized
(similar to the code object for a module; it creates things and then
goes away).  Perhaps there should be a C equivalent.
  When the shared-object importer loads a .so/.dll/.pyd, it looks for
init<module> and calls that, and then keeps the object loaded.
Perhaps a second name could be checked for, build<module>, which
indicates that the initialized module has no dependencies on the C
code, which which case the .so/.dll/.pyd gets unloaded after the
build<module> returns.
  Another possibility is to simply regenerate token.py and symbol.py
whenever token.h and graminit.h change.  That can be driven by make so
they don't fall out of sync with the rest of the grammar.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From skip@pobox.com (Skip Montanaro)  Thu Aug 23 16:41:49 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Thu, 23 Aug 2001 10:41:49 -0500
Subject: [Python-Dev] Inline caveat
In-Reply-To: <15237.5085.898843.495797@cj42289-a.reston1.va.home.com>
References: <15237.270.649981.730739@beluga.mojam.com>
 <15237.5085.898843.495797@cj42289-a.reston1.va.home.com>
Message-ID: <15237.9277.750760.986482@beluga.mojam.com>

    >> There is, of course, a definite downside to Inline.  Your code will
    >> get harder to read and maintain because you and whoever else has to
    >> maintain your code has to be familiar with all the languages you are
    >> inlining.

    Fred>   Inline::K, anyone?  ;-)

I was actually thinking more about Inline::Intercal. ;-)

Skip



From thomas.heller@ion-tof.com  Thu Aug 23 16:58:08 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Thu, 23 Aug 2001 17:58:08 +0200
Subject: [Python-Dev] Inline caveat
References: <15237.270.649981.730739@beluga.mojam.com>              <034801c12bd7$4dabc3f0$e000a8c0@thomasnotebook>  <200108231432.f7NEWKr05785@odiug.digicool.com>
Message-ID: <052201c12bec$675bf300$e000a8c0@thomasnotebook>

[Guido, about Inline]
> If the alternative is writing an extension module, the inline approach
> sounds viable to me.
> 
> I could also imagine auto-wrapping existing C functions based on
> parseable function signatures.  I believe VB supports this.
> Dangerous, but powerful, and very useful in the right hands.

Isn't this the calldll approach?

> It seems to me that the important idea here is to break away from
> building and distributing a separate extension module, which gets you
> in the edit-compile-link-test-run loop that Python tries to avoid.
> Rather than requiring an expert who knows how to download, build,
> install and use SWIG or how to write Python extensions, all the
> expertise is automated.
> 
> Getting this to work on Windows for the average Windows user would be
> a big hassle, limiting the portability and therefore the usefulness.

My prototype writes a xxx.c file, and uses distutils to build an
extension module of it. The only requirement is a working VC++ compiler.

Thomas



From cgw@alum.mit.edu  Thu Aug 23 17:27:31 2001
From: cgw@alum.mit.edu (Charles G Waldman)
Date: Thu, 23 Aug 2001 11:27:31 -0500
Subject: [Python-Dev] Re: Inline::caveat
In-Reply-To: <E15Zwv0-0004Vc-00@mail.python.org>
References: <E15Zwv0-0004Vc-00@mail.python.org>
Message-ID: <15237.12019.841104.61849@nyx.dyndns.org>


 > It's also harder to maintain because it's harder to edit and debug.
 > 
 > Maybe all them fancy IDEs you youngin's use can easily support editing
 > mutiple different languages in the same file, but if so, let me know!
 > The tools/editors/debuggers I'm familiar with don't deal with any of
 > that very well, if at all.

I agree that debugging any Inline code is going to be difficult (it's
already tricky enough debugging C modules generated by SWIG), but as
far as editor support goes - the "fancy IDE" of choice (XEmacs, of
course) actually can support such mixed-language editing, once you've
loaded up the appropriate elisp and done some customization...

Have you seen the "MMM" package?  MMM stands for "Multiple Major
Modes" which allows you to have, just like the name suggests, multiple
major modes within one buffer.  I have used this successfully for
editing HTML with embedded JavaScript (although I hate to admit that I
have written such junk!).  You can get it at:

http://members.tripod.com/gchen2/xemacs/

This package seems to be little-known.  Maybe it should be added to
the XEmacs package database?  (I'll bring this up on xemacs-beta)

FWIW,
    -C



From barry@zope.com  Thu Aug 23 18:33:29 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 23 Aug 2001 13:33:29 -0400
Subject: [Python-Dev] Re: Inline::caveat
References: <E15Zwv0-0004Vc-00@mail.python.org>
 <15237.12019.841104.61849@nyx.dyndns.org>
Message-ID: <15237.15977.879565.852317@anthem.wooz.org>

>>>>> "CGW" == Charles G Waldman <cgw@alum.mit.edu> writes:

    CGW> Have you seen the "MMM" package?  MMM stands for "Multiple
    CGW> Major Modes" which allows you to have, just like the name
    CGW> suggests, multiple major modes within one buffer.  I have
    CGW> used this successfully for editing HTML with embedded
    CGW> JavaScript (although I hate to admit that I have written such
    CGW> junk!).  You can get it at:

    CGW> http://members.tripod.com/gchen2/xemacs/

Except that trying to download the actual mmm.el.gz file it puts me
into a Tripod infloop.  I can't actually get at the file.  Can you
send me a copy?

    CGW> This package seems to be little-known.  Maybe it should be
    CGW> added to the XEmacs package database?  (I'll bring this up on
    CGW> xemacs-beta)

Indeed, it sounds cool.
-Barry


From thomas.heller@ion-tof.com  Thu Aug 23 18:49:40 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Thu, 23 Aug 2001 19:49:40 +0200
Subject: [Python-Dev] Inline and SWIG
References: <200108230601.SAA04304@s454.cosc.canterbury.ac.nz>
Message-ID: <05c601c12bfb$fcd46c50$e000a8c0@thomasnotebook>

From: "Greg Ewing" <greg@cosc.canterbury.ac.nz>
> David Beazley <beazley@cs.uchicago.edu>:
> 
> > I've looked briefly at Inline and agree that it's a pretty nifty
> > approach.  However, usability aside, it's got a number of pretty major
> > limitations once you start getting into structure wrapping, C++
> > classes, and other advanced types of extension wrapping.
> 
> Indeed, you've put your finger on what seems like the
> biggest problem with the current crop of semi-automatic
> extension generators.
> 
> I don't think there is ever going to be a completely
> automatic solution to this, because, once you get beyond
> the basic types like ints, floats and strings, there is
> no unique mapping between Python types and C types.
> 
> While the tool could provide some sort of default
> translation for arbitrary types, it's unlikely to be
> what you need to interface directly with some existing
> C library you're trying to wrap. So you're faced with
> writing a lot of tedious and error-prone code for messing 
> with PyObjects and reference counts.
> 
> I've had some ideas floating around in my head for a
> while concerning what to do about this. The key
> observation is that such code usually spends a lot
> more lines manipulating Python data than it does C
> data, which leads me to the conclusion that C is the
> wrong language to write it in. 
> 
> What's the right language? One that's good at manipulating
> Python objects. Which language is the best at doing
> that? Why, Python, of course. But not just Python,
> because we need to be able to manipulate C data
> structures as well. So we need a language that looks
> like Python at one end and C at the other...

I've written (well, it's not finished and maybe never will)
an extension allowing to create or wrap and manipulate C structures
and unions from Python. The basics are similar to an
OO-ified struct module, but the format codes are extensible.

It even handles the Microsoft TypeInfo structures, which
are full of recursions.

Makes a pretty good pair with a calldll-like extension module.

Thomas



From paulp@ActiveState.com  Thu Aug 23 19:31:29 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Thu, 23 Aug 2001 11:31:29 -0700
Subject: [Python-Dev] Inline and SWIG
References: <200108230601.SAA04304@s454.cosc.canterbury.ac.nz>
Message-ID: <3B854C01.A20AE630@ActiveState.com>

Greg Ewing wrote:
> 
>...
> 
> What's the right language? One that's good at manipulating
> Python objects. Which language is the best at doing
> that? Why, Python, of course. But not just Python,
> because we need to be able to manipulate C data
> structures as well. So we need a language that looks
> like Python at one end and C at the other...

You are describing inline.

-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From paulp@ActiveState.com  Thu Aug 23 19:36:11 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Thu, 23 Aug 2001 11:36:11 -0700
Subject: [Python-Dev] Re: Inline code in other languages
References: <15236.14289.311782.621168@anthem.wooz.org> 	<Pine.LNX.4.32.0108221557590.5627-100000@ziggy.localdomain.fake> <15236.21449.231509.725435@anthem.wooz.org> <m34rqzyxqt.fsf@lambda.garshol.priv.no>
Message-ID: <3B854D1B.1DA8191C@ActiveState.com>

Lars Marius Garshol wrote:
> 
>...
> 
> Why not make it import C files, instead of putting them inline? I
> guess you very often want to reuse the C functions in several
> different places anyway, and you'd mostly want to have includes and
> stuff as well.

If you want to reuse a C function then you put it in a seperate file and
use #include to include it. Allowing inline code is a choice...the
programmer can choose to modularize if it helps him or her. I think
there is a reason PHP is so much more popular than DTML-like solutions
(duck!).

> This way you can also get rid of the MD5-ing and just use ordinary
> modification times to figure out what needs to be recompiled.

MD5-ing is more reliable anyhow. Some build systems (e.g. CONS, SCONS)
are moving from mod times to build times anyhow.

> You can't inline pieces smaller than functions anyway, and to me the
> mix between Python and C code looks very ugly.

Beauty is in the eye of the beholder....I think that forcing stuff into
separate files for aesthetic or philosophical reasons is severe.
Ultimately there is an important usability benefit in allowing inlining
(and I'll refer again to the amazing popularity of PHP).
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From thomas.heller@ion-tof.com  Thu Aug 23 19:48:36 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Thu, 23 Aug 2001 20:48:36 +0200
Subject: [Python-Dev] Customizing the binding of attributes
Message-ID: <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook>

In Python 2.2, is it possible to customize the binding
of class attributes, to emulate the normal behaviour
of methods (function -> unbound method -> bound method)?

I think I have read something about this, but I can't
find it anymore.

Thomas



From guido@python.org  Thu Aug 23 20:23:42 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 23 Aug 2001 15:23:42 -0400
Subject: [Python-Dev] Customizing the binding of attributes
In-Reply-To: Your message of "Thu, 23 Aug 2001 20:48:36 +0200."
 <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook>
References: <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook>
Message-ID: <200108231923.f7NJNg307631@odiug.digicool.com>

> In Python 2.2, is it possible to customize the binding
> of class attributes, to emulate the normal behaviour
> of methods (function -> unbound method -> bound method)?
> 
> I think I have read something about this, but I can't
> find it anymore.

I'm not sure what you are asking about, but let me give an example.
You can write an auxiliary class that "describes" an instance
attribute.  Instances of this "descriptor" class are stored in a class
dict.  The descriptor class implements a method __get__ which is
called in two situations:

(a) when a class attribute is retrieved, <attr>.__get__(None, <class>)
    is called; this can return an unbound method

(b) when an instance attribute is retrieved, <attr>.__get__(<inst>, <class)
    is called; this can return a bound method

Does this help?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Thu Aug 23 20:27:01 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 23 Aug 2001 15:27:01 -0400
Subject: [Python-Dev] Inline and SWIG
In-Reply-To: <05c601c12bfb$fcd46c50$e000a8c0@thomasnotebook>
References: <200108230601.SAA04304@s454.cosc.canterbury.ac.nz>
 <05c601c12bfb$fcd46c50$e000a8c0@thomasnotebook>
Message-ID: <15237.22789.951079.581327@grendel.digicool.com>

Thomas Heller writes:
 > Makes a pretty good pair with a calldll-like extension module.

  I guess calldll is very different from the dl module, which has
documentation.  Is anyone up to writing docs for calldll?
  I've CC'd to the Doc-SIG since there may be volunteers there ;-) --
please remove python-dev from doc-related followups.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From gward@python.net  Thu Aug 23 21:17:45 2001
From: gward@python.net (Greg Ward)
Date: Thu, 23 Aug 2001 16:17:45 -0400
Subject: [Python-Dev] Putting a builtin module into a package
In-Reply-To: <15234.34090.225332.420431@beluga.mojam.com>; from skip@pobox.com on Tue, Aug 21, 2001 at 10:58:34AM -0500
References: <20010821145632.7CC15303181@snelboot.oratrix.nl> <15234.34090.225332.420431@beluga.mojam.com>
Message-ID: <20010823161745.A2375@gerg.ca>

On 21 August 2001, Skip Montanaro said:
> 
>     Jack> Or should I rename the module to an underscore name and simply put
>     Jack> a wrapper module in the package?
> 
> This seems the safest (and the sanest) to me.  I realize you're thinking
> mostly about MacOS, but putting object files inside packages that are
> otherwise Python code makes the whole package platform-dependent.

I think Jack was talking about *built-in* extensions, not .so (or .pyd)
files.  Dynamically loaded extensions are indeed easy -- you just put
'em in the package directory, and it Just Works.  Yes, this means that
people maintaining a multi-architecture installation have to have prefix
!= external-prefix, but they already do: /usr/bin/python is
platform-specific!

        Greg
-- 
Greg Ward - Unix bigot                                  gward@python.net
http://starship.python.net/~gward/
I just read that 50% of the population has below median IQ!


From paulp@ActiveState.com  Thu Aug 23 22:12:32 2001
From: paulp@ActiveState.com (Paul Prescod)
Date: Thu, 23 Aug 2001 14:12:32 -0700
Subject: [Python-Dev] Perl5->Perl6
References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> <15235.61766.827878.699521@slothrop.digicool.com> <5.1.0.14.0.20010822153946.08c02de8@tuatha.sidhe.org> <5.1.0.14.0.20010822170534.08c50170@tuatha.sidhe.org> <20010823105241.B8531@netthink.co.uk>
Message-ID: <3B8571C0.4ABCEDC5@ActiveState.com>

Simon Cozens wrote:
> 
>...
> 
> Also don't forget that we have a prototype of Perl 6 which implements pretty
> much everything that Larry has specified in Apocalypses 1 and 2.
> 
> It is a patch of nearly 200 lines to the current Perl 5 interpreter.
> 
> We're not changing *that* drastically, folks.

According to Damian there will be generators and a first-class class
syntax and method getters/setters and metaclasses and optional static
type declarations and ... Larry just hasn't got to the heavy part of the
Camel book. :-)
-- 
Take a recipe. Leave a recipe.  
Python Cookbook!  http://www.ActiveState.com/pythoncookbook


From Samuele Pedroni <pedroni@inf.ethz.ch>  Thu Aug 23 22:56:42 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Thu, 23 Aug 2001 23:56:42 +0200 (MET DST)
Subject: [Python-Dev] PEP 252, PEP 253 and jython issues
Message-ID: <200108232156.XAA09481@core.inf.ethz.ch>

Hi.

OK, here is my major concern

Given  the following three types subclassed in Pyhon:
class A(CSpecialList): ...

class B(list): ...

class C(A,B): ...

where CSpecialList is a subtype of list written in C.

Now the old mro would be:

C A CSpecialList list object B list object

that translate to the new mro

C A CSpecialList B list object

The question is it possible for CSpecialList for the code that define
for example sq_ass_slice to use sq_slice code in a safe manner?
(AFAIK listobject does something similar) 

And how can safely CSpecialList invoke a  "super" behaviour, it is safe
for it to refer to the behaviour offered by list. In principle
given the mro e.g. a __getslice__ redefined in B should shadow
such a behaviour?

A natural way to design PEP252, and PEP253 for Jython would
be start to start from the equation:

object = org.python.core.PyObject

then it is necessary to build a class that truly correspond to the type
metatype, actually jython fakes using the metatype used for Java Classes for 
that.

It would make sense to use Java subclassing to implement 
type subclassing, at least at layout level this does not clash with
the multiple inheritance rule in best_base.

And actually is also how Jython now works: for example
PyList extends PySequence that extends PyObject.

But than there is the issue of method resolution order: from the viewpoint
of Python code we can implement anything, not that easy ...

But the at the Java level, where we construct types, the codebase uses the 
normal single inheritance of java and the codebase is full of super.foo 
invocations and of methods that cross-call each other (potentially overriden) 
versions, and this happen also for behaviour that correspond to the slots/ 
__foo__ special methods of CPython.

That's why the two questions are important?

regards.



From neilh@scintilla.org  Thu Aug 23 23:15:53 2001
From: neilh@scintilla.org (Neil Hodgson)
Date: Fri, 24 Aug 2001 08:15:53 +1000
Subject: [Python-Dev] Re: Inline::caveat
References: <E15Zwv0-0004Vc-00@mail.python.org> <15237.12019.841104.61849@nyx.dyndns.org>
Message-ID: <012b01c12c21$2e332050$0acc8490@neil>

Charles G Waldman:

> ... but as
> far as editor support goes - the "fancy IDE" of choice (XEmacs, of
> course) actually can support such mixed-language editing, once you've
> loaded up the appropriate elisp and done some customization...

   How do you find the language boundaries to enable mixed language editing
or debugging? If Python gets an inline module, I'd hope to see enough
lexical clues so that an editor can work out where the C++ is so it can be
treated as C++.

   Neil



From greg@cosc.canterbury.ac.nz  Thu Aug 23 23:55:00 2001
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Fri, 24 Aug 2001 10:55:00 +1200 (NZST)
Subject: [Python-Dev] Inline and SWIG
In-Reply-To: <3B854C01.A20AE630@ActiveState.com>
Message-ID: <200108232255.KAA04393@s454.cosc.canterbury.ac.nz>

Paul Prescod <paulp@ActiveState.com>:

> You are describing inline.

No, I'm not describing Inline, that's the point. I
have some fairly detailed ideas on what I am describing,
but this message is too small to contain them...

As a brief example, the Inline::C example
posted earlier would look something like this:

  cdef printf(char *,...): extern

  def hello(s):
    printf("Hello, world, %s\n", s)

But that's way too simple an example to really
convey anything. More later...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

    


From gstein@lyra.org  Fri Aug 24 03:30:00 2001
From: gstein@lyra.org (Greg Stein)
Date: Thu, 23 Aug 2001 19:30:00 -0700
Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home
In-Reply-To: <200108231443.f7NEhb505947@odiug.digicool.com>; from guido@python.org on Thu, Aug 23, 2001 at 10:43:37AM -0400
References: <LNBBLJKPBEHFEDALKOLCEEGELHAA.tim.one@home.com> <Pine.LNX.4.32.0108230128510.5627-100000@ziggy.localdomain.fake> <20010823070259.X16727@lyra.org> <15237.4961.597225.931722@anthem.wooz.org> <200108231443.f7NEhb505947@odiug.digicool.com>
Message-ID: <20010823193000.N26054@lyra.org>

On Thu, Aug 23, 2001 at 10:43:37AM -0400, Guido van Rossum wrote:
> > >>>>> "KY" == Ka-Ping Yee <ping@lfw.org> writes:
> >     KY> Just a suggestion, but how about sys.consts?  Like os.path, as
> >     KY> in sys.consts.CO_NESTED, sys.consts.CO_FUTURE_DIVISION, etc.
> >     KY> So there's only one new object in sys, and it can contain the
> >     KY> weird internal stuff. :)
>...
> +0.  It's the better solution but I bet it's more work to code it than
> _sysconst.  I'm not volunteering, and I want Tim to have time to help
> with the type/class unification, which is behind schedule.

Hmm... that sounds a lot like trading off expediency for Rightness. If it is
the better solution, then that should be done.

If you don't want Tim to work on it, then that is a different problem. It
could be done later, or a volunteer could step up to do it. It isn't exactly
a time critical change, now is it?

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From beazley@cs.uchicago.edu  Fri Aug 24 03:26:09 2001
From: beazley@cs.uchicago.edu (David Beazley)
Date: Thu, 23 Aug 2001 21:26:09 -0500 (CDT)
Subject: [Python-Dev] Near Final PEP 237 - Unifying Long Integers and Integers
In-Reply-To: <mailman.998610878.10688.clpa-moderators@python.org>
References: <mailman.998610878.10688.clpa-moderators@python.org>
Message-ID: <15237.47937.50866.571629@gargoyle.cs.uchicago.edu>

Guido van Rossum writes:
 > Please comment on a new version of PEP 237:
 > 
 > Please mail comments on the PEP to python-dev@python.org or to me.
 > 

The PEP looks good to me.   Can I make a very small C API feature 
request? (assuming this hasn't been implemented already).

Since "integers" can now have arbitrary precision and can represent
large unsigned values, can you add three new format characters to
PyArg_ParseTuple() and Py_BuildValue() for the C datatypes "unsigned int", 
"unsigned long", and "unsigned long long"?  

The "u" and "l" namespace is a little crowded (and I don't think you
would want to break that).  However, here's one idea:

'I'     -  unsigned int  (consistent with H and B)
'p'     -  unsigned long ('p' is for positive)
'P'     -  unsigned long long

Cheers,

Dave
  


From jeremy@zope.com  Fri Aug 24 03:59:30 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 23 Aug 2001 22:59:30 -0400 (EDT)
Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home
In-Reply-To: <20010823193000.N26054@lyra.org>
References: <LNBBLJKPBEHFEDALKOLCEEGELHAA.tim.one@home.com>
 <Pine.LNX.4.32.0108230128510.5627-100000@ziggy.localdomain.fake>
 <20010823070259.X16727@lyra.org>
 <15237.4961.597225.931722@anthem.wooz.org>
 <200108231443.f7NEhb505947@odiug.digicool.com>
 <20010823193000.N26054@lyra.org>
Message-ID: <15237.49938.566295.179733@slothrop.digicool.com>

>>>>> "GS" == Greg Stein <gstein@lyra.org> writes:

  GS> It isn't exactly a time critical change, now is it?

Not any more time critical than documenting the new httplib API <duck>.

speaking-as-an-expert-on-putting-off-documentation-ly y'rs,
Jeremy








From tim.one@home.com  Fri Aug 24 04:31:57 2001
From: tim.one@home.com (Tim Peters)
Date: Thu, 23 Aug 2001 23:31:57 -0400
Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home
In-Reply-To: <15237.49938.566295.179733@slothrop.digicool.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEKNLHAA.tim.one@home.com>

Something that might change your views about Elegance here:  calling these
particular constants "cuddly" was intended to be ironic.  If *any* program
uses one of these, I can pretty much guarantee it will eventually break.
They're not part of a public API.  The two current uses (__future__ and
pyassem) are part of the implementation of the core, and just happen to be
coded in Python.

For that reason I'm -1 on putting these particular constants in a location
with an inviting name, or making their own names prettier, etc etc -- I'm
even -1 on documenting them (outside of comments in the internal C header
file where they're #define'd).

Rather than argue about it, I'm going to revert the changes to new and
__future__, backing off to defining them by hand in __future__.py.

doesn't-stop-anyone-else-from-doing-the-right-thing-later-just-think-
    twice-about-how-right-it-is-ly y'rs  - tim



From eric@scipy.org  Fri Aug 24 04:05:57 2001
From: eric@scipy.org (eric jones)
Date: Thu, 23 Aug 2001 23:05:57 -0400
Subject: [Python-Dev] Re: Inline and SWIG
References: <15236.26466.84309.855339@gargoyle.cs.uchicago.edu>	<Pine.LNX.4.32.0108221923330.5627-100000@ziggy.localdomain.fake> <15236.27364.304751.836868@gargoyle.cs.uchicago.edu> <3B84792B.7DDF7C1B@ActiveState.com>
Message-ID: <003701c12c49$b33bf3d0$c27ba8c0@ericlaptop>

The scipy.compiler modules (www.scipy.org) is aimed at Numeric
arrays and is slightly different than inlining C code in that it translates
a Python Numeric expression to C++(blitz++ arrays), compiles the
result to a function, and calls it automatically.  It figures out the array
types  the local name space, and keeps up with the compiled module
after the first compile so that they don't have to be compiled again.  The
entire process was greatly eased by Jeremy AST stuff, distutils, and
blitz++.  The most up-to-date description of it is here:

http://www.scipy.org/site_content/tutorials/compiler_notes

Much of this machinery is identical to what is needed for inline C,
and I've tinkered with it a little.  It looks like it won't be extremely
hard (famous last words...) to handle inlining of numeric looping code. The
limited focus on numerical stuff gets around many of the hairy issues
involved
with handling lists of arbitrary types, and got a usable tool out the door
quickly.
Luckily it also attacks the most expensive part of scientific codes and
provides
2-10 speed-up over Numeric array expressions.  I'd like to circle out
from the numeric-only approach and tackle some of the more difficult typing
issues after the numeric stuff all works.  Any input on this thing is
welcome.

Patrick Miller's PyCOD is another compiler that takes a different approach
to compiling (processing bytecodes).  It's a little more ambitious than
scipy.compiler because it handles more than just numeric expressions.  I
don't
know what its release status is either.

The other compiler project I know about is Psyco discussed here:

    http://aspn.activestate.com/ASPN/Mail/Message/609552

It sounds extremely cool and is the most ambitious of the three.  Not sure
how it affects inline C though.  Seems more like this aims to make inline
C obsolete before it even exists.

> I'd recommend that people track down the authors of those modules, as I
> do think there is a great opportunity there.  (I take credit for
> inspiring Eric Jones by mentioning Inline:: to him in Long Beach =).

Heck, if you'll sign up to add support for array broadcasting, I'll name it
scipy.da_compiler...

eric


----- Original Message -----
From: "David Ascher" <DavidA@ActiveState.com>
To: "David Beazley" <beazley@cs.uchicago.edu>
Cc: "Ka-Ping Yee" <ping@lfw.org>; <python-dev@python.org>; <eric@scipy.org>;
<dubois@users.sourceforge.net>; <miller35@llnl.gov>
Sent: Wednesday, August 22, 2001 11:31 PM
Subject: Re: [Python-Dev] Re: Inline and SWIG


> Note that I know of at least two if not three different inline-like
> projects for Python.
>
> One is the compiler module in scipy (Eric Jones & Travis Oliphant):
>
>
http://scipy.net/cgi-bin/viewcvs.cgi/~checkout~/scipy/compiler/doc/readme?re
v=1.2&content-type=text/plain
>
> One is the stuff that Pat Miller from LLNL presented at OSCON in San
> Diego, which sounded similar in spirit (alas, I missed the talk, and
> don't know anything about its availability -- Pat?), and there was a
> third one which I heard mentioned but don't know any details about.
> Eric, do you remember what I'm talking about?
>
> I'd recommend that people track down the authors of those modules, as I
> do think there is a great opportunity there.  (I take credit for
> inspiring Eric Jones by mentioning Inline:: to him in Long Beach =).
>
> --david ascher



From thomas.heller@ion-tof.com  Fri Aug 24 08:00:31 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Fri, 24 Aug 2001 09:00:31 +0200
Subject: [Python-Dev] Customizing the binding of attributes
References: <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook>  <200108231923.f7NJNg307631@odiug.digicool.com>
Message-ID: <09f401c12c6a$7717ab80$e000a8c0@thomasnotebook>

From: "Guido van Rossum" <guido@python.org>
> I'm not sure what you are asking about, but let me give an example.
> You can write an auxiliary class that "describes" an instance
> attribute.  Instances of this "descriptor" class are stored in a class
> dict.  The descriptor class implements a method __get__ which is
> called in two situations:
> 
> (a) when a class attribute is retrieved, <attr>.__get__(None, <class>)
>     is called; this can return an unbound method
> 
> (b) when an instance attribute is retrieved, <attr>.__get__(<inst>, <class)
>     is called; this can return a bound method
> 
> Does this help?
> 

Yes, thanks. Exactly what I meant. You only forgot to mention that
the descriptor class must derive from object: is this intended?

I needed this to write

class X:
    method = myinstancemethod(func)

instead of

class X:
    pass
X.method = new.instancemethod(func, None, X)

where myinstancemethod is the descriptor class, and func
is any callable object (maybe a function implemented in C).

Two additional comments:

- Shouldn't (my)instancemethod be a builtin? Similar to
staticmethod and classmethod?

- The following seems to be a bug (remember, I'm on Windows):

C:\>c:\python22\python.exe
Python 2.2a2 (#22, Aug 22 2001, 01:24:03) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> class myinstancemethod(object):
...     def __init__(self, func):
...         self.func = func
...     def __get__(self, inst, klass=None):
...         import new
...         return new.instancemethod(self.func, inst, klass)
...
>>> class X:
...     meth = myinstancemethod(lambda *x: x)
...
>>> X().meth
<bound method X.<lambda> of <__main__.X instance at 007C7F04>>
>>> X().meth()
(<__main__.X instance at 007E80E4>,)
>>> X.meth
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
SystemError: NULL object passed to Py_BuildValue

Thomas



From guido@python.org  Fri Aug 24 10:08:48 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 24 Aug 2001 05:08:48 -0400
Subject: [Python-Dev] Near Final PEP 237 - Unifying Long Integers and Integers
In-Reply-To: Your message of "Thu, 23 Aug 2001 21:26:09 CDT."
 <15237.47937.50866.571629@gargoyle.cs.uchicago.edu>
References: <mailman.998610878.10688.clpa-moderators@python.org>
 <15237.47937.50866.571629@gargoyle.cs.uchicago.edu>
Message-ID: <200108240908.FAA03478@cj20424-a.reston1.va.home.com>

> Since "integers" can now have arbitrary precision and can represent
> large unsigned values, can you add three new format characters to
> PyArg_ParseTuple() and Py_BuildValue() for the C datatypes "unsigned int", 
> "unsigned long", and "unsigned long long"?  
> 
> The "u" and "l" namespace is a little crowded (and I don't think you
> would want to break that).  However, here's one idea:
> 
> 'I'     -  unsigned int  (consistent with H and B)
> 'p'     -  unsigned long ('p' is for positive)
> 'P'     -  unsigned long long

Good idea.  I've added this to the SF bug tracker as a feature
request.  Hopefully someone will submit a patch too!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Aug 24 11:03:34 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 24 Aug 2001 06:03:34 -0400
Subject: [Python-Dev] Customizing the binding of attributes
In-Reply-To: Your message of "Fri, 24 Aug 2001 09:00:31 +0200."
 <09f401c12c6a$7717ab80$e000a8c0@thomasnotebook>
References: <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook> <200108231923.f7NJNg307631@odiug.digicool.com>
 <09f401c12c6a$7717ab80$e000a8c0@thomasnotebook>
Message-ID: <200108241003.GAA04383@cj20424-a.reston1.va.home.com>

> From: "Guido van Rossum" <guido@python.org>
> > I'm not sure what you are asking about, but let me give an example.
> > You can write an auxiliary class that "describes" an instance
> > attribute.  Instances of this "descriptor" class are stored in a class
> > dict.  The descriptor class implements a method __get__ which is
> > called in two situations:
> > 
> > (a) when a class attribute is retrieved, <attr>.__get__(None, <class>)
> >     is called; this can return an unbound method
> > 
> > (b) when an instance attribute is retrieved, <attr>.__get__(<inst>, <class)
> >     is called; this can return a bound method
> > 
> > Does this help?

From: "Thomas Heller" <thomas.heller@ion-tof.com>
> Yes, thanks. Exactly what I meant. You only forgot to mention that
> the descriptor class must derive from object: is this intended?

Yes, almost all new features require you to switch to new-style
classes; the classic class implementation is left unchanged (as much
as possible) for compatibility.  In particular, classic classes can't
define __get__ to the effect above.

These descriptors semi-work as class attributes of classic classes,
but the corresponding __set__ feature does not, because classic
classes *always* store in __dict__ upon assignment; new-style classes
first look for a descriptor implementing __set__ (really the C
dispatch function tp_descr_set) and use that to override the default
assignment.

> I needed this to write
> 
> class X:
>     method = myinstancemethod(func)
> 
> instead of
> 
> class X:
>     pass
> X.method = new.instancemethod(func, None, X)
> 
> where myinstancemethod is the descriptor class, and func
> is any callable object (maybe a function implemented in C).
> 
> Two additional comments:
> 
> - Shouldn't (my)instancemethod be a builtin? Similar to
> staticmethod and classmethod?

Yes.  I just checked this in.  The new name is 'getset' unless you
have a better idea.

> - The following seems to be a bug (remember, I'm on Windows):
> 
> C:\>c:\python22\python.exe
> Python 2.2a2 (#22, Aug 22 2001, 01:24:03) [MSC 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> class myinstancemethod(object):
> ...     def __init__(self, func):
> ...         self.func = func
> ...     def __get__(self, inst, klass=None):
> ...         import new
> ...         return new.instancemethod(self.func, inst, klass)
> ...
> >>> class X:
> ...     meth = myinstancemethod(lambda *x: x)
> ...
> >>> X().meth
> <bound method X.<lambda> of <__main__.X instance at 007C7F04>>
> >>> X().meth()
> (<__main__.X instance at 007E80E4>,)
> >>> X.meth
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> SystemError: NULL object passed to Py_BuildValue

Yup.  Thanks!  Try this patch:

*** typeobject.c	2001/08/22 19:24:42	2.43
--- typeobject.c	2001/08/24 10:05:25
***************
*** 2806,2811 ****
--- 2806,2815 ----
  		Py_INCREF(self);
  		return self;
  	}
+ 	if (obj == NULL)
+ 		obj = Py_None;
+ 	if (type == NULL)
+ 		type = Py_None;
  	return PyObject_CallFunction(get, "OOO", self, obj, type);
  }
  
--Guido van Rossum (home page: http://www.python.org/~guido/)


From Samuele Pedroni <pedroni@inf.ethz.ch>  Fri Aug 24 11:16:28 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Fri, 24 Aug 2001 12:16:28 +0200 (MET DST)
Subject: [Python-Dev] jython issues and PEP 252, PEP 253
Message-ID: <200108241016.MAA07882@core.inf.ethz.ch>

Hi.

I'm feeling a bit annoying and I don't like that.
I'm not in hurry for a answer for my questions on PEP253 and PEP252

but were them at least parseable, did they make sense?

or are too vague, and underdetailed
should I bring more elements in the picture?

regards, Samuele Pedroni.



From jack@oratrix.nl  Fri Aug 24 11:48:02 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Fri, 24 Aug 2001 12:48:02 +0200
Subject: [Python-Dev] Inline and SWIG
In-Reply-To: Message by David Beazley <beazley@cs.uchicago.edu> ,
 Wed, 22 Aug 2001 21:16:02 -0500 (CDT) , <15236.26466.84309.855339@gargoyle.cs.uchicago.edu>
Message-ID: <20010824104802.B28D0303181@snelboot.oratrix.nl>

> The *real* problem IMHO is actually
> trying to figure out how to properly handle all of the facets of the
> C/C++ type system (pointers, references, qualifiers, classes,
> templates, typedef, inheritance, multiple inheritance, member
> pointers, pass by value, etc.) and how that's supposed to hook into
> Python.

And at this time I customarily put in my plug for bgen. Bgen is far ahead of 
SWIG in this respect, and completely customizable in Python.

It will definitely not do all the things you specify, but it'll do a lot more 
than SWIG,
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 




From guido@python.org  Fri Aug 24 13:54:10 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 24 Aug 2001 08:54:10 -0400
Subject: [Python-Dev] PEP 252, PEP 253 and jython issues
In-Reply-To: Your message of "Thu, 23 Aug 2001 23:56:42 +0200."
 <200108232156.XAA09481@core.inf.ethz.ch>
References: <200108232156.XAA09481@core.inf.ethz.ch>
Message-ID: <200108241254.IAA05377@cj20424-a.reston1.va.home.com>

> OK, here is my major concern
> 
> Given  the following three types subclassed in Pyhon:
> class A(CSpecialList): ...
> 
> class B(list): ...
> 
> class C(A,B): ...
> 
> where CSpecialList is a subtype of list written in C.
> 
> Now the old mro would be:
> 
> C A CSpecialList list object B list object
> 
> that translate to the new mro
> 
> C A CSpecialList B list object

Correct.

> The question is it possible for CSpecialList for the code that define
> for example sq_ass_slice to use sq_slice code in a safe manner?
> (AFAIK listobject does something similar) 

I'm not 100% sure I understand this question, but CSpecialList can
certainly override sq_ass_slice.  Then, if it wants to call
self.__slice__ (in Python terms) it could call
obj->ob_type->tp_as_sequence->sq_slice -- this would call C's
obj->ob_type->tp_as_sequence->__slice__ if it had one.

> And how can safely CSpecialList invoke a  "super" behaviour, it is safe
> for it to refer to the behaviour offered by list. In principle
> given the mro e.g. a __getslice__ redefined in B should shadow
> such a behaviour?

I don't know if cooperative super calls are possible at the C level.
I certainly didn't define an API for this.  I didn't define a Python
level API either, but I know how to o it and it can even be coded in
Python.  Hm, I guess that makes it a theoretical possibility at the C
level.

I don't expect this to be a common use pattern though (famous last
words :-).

> A natural way to design PEP252, and PEP253 for Jython would
> be start to start from the equation:
> 
> object = org.python.core.PyObject

Yes.

> then it is necessary to build a class that truly correspond to the type
> metatype, actually jython fakes using the metatype used for Java Classes for 
> that.
> 
> It would make sense to use Java subclassing to implement 
> type subclassing, at least at layout level this does not clash with
> the multiple inheritance rule in best_base.

So you would rule out multiple inheritance of types?  Fine with me,
but you may regret this a few yearsfrom now (when we all code multiple
inheritance all the time :-).

> And actually is also how Jython now works: for example
> PyList extends PySequence that extends PyObject.

And that's how it should be.

> But than there is the issue of method resolution order: from the viewpoint
> of Python code we can implement anything, not that easy ...
> 
> But the at the Java level, where we construct types, the codebase uses the 
> normal single inheritance of java and the codebase is full of super.foo 
> invocations and of methods that cross-call each other (potentially overriden) 
> versions, and this happen also for behaviour that correspond to the slots/ 
> __foo__ special methods of CPython.
> 
> That's why the two questions are important?

I see.  CPython doesn't have this problem (yet) because it doesn't use
inheritance at the C level.  I propose that you just try to live with
this.  A safe rule would be to require that all Python classes in a
given inheritance graph should derive from the same C/Java class.  I
don't know if we should enforce this or just warn about it in the
documentation.  I don't want to force you to call a cooperative super
method at the Java level -- it would be very slow, I suspect...

--Guido van Rossum (home page: http://www.python.org/~guido/)



From nas@python.ca  Fri Aug 24 14:32:22 2001
From: nas@python.ca (Neil Schemenauer)
Date: Fri, 24 Aug 2001 06:32:22 -0700
Subject: [Python-Dev] Customizing the binding of attributes
In-Reply-To: <200108241003.GAA04383@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Aug 24, 2001 at 06:03:34AM -0400
References: <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook> <200108231923.f7NJNg307631@odiug.digicool.com> <09f401c12c6a$7717ab80$e000a8c0@thomasnotebook> <200108241003.GAA04383@cj20424-a.reston1.va.home.com>
Message-ID: <20010824063222.A12847@glacier.fnational.com>

Guido van Rossum wrote:
> In particular, classic classes can't define __get__ to the effect
> above.

Why __get__() and __set__() and not get() and set()?

  Neil


From Samuele Pedroni <pedroni@inf.ethz.ch>  Fri Aug 24 14:57:25 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Fri, 24 Aug 2001 15:57:25 +0200 (MET DST)
Subject: [Python-Dev] Comments: PEP 252, PEP 253 and jython issues
Message-ID: <200108241357.PAA16863@core.inf.ethz.ch>

Thanks for the answers, I'm happy the questions were parseable,
Here are some comments, I think they may be meanigful also for CPython

> > The question is it possible for CSpecialList for the code that define
> > for example sq_ass_slice to use sq_slice code in a safe manner?
> > (AFAIK listobject does something similar) 
> 
> I'm not 100% sure I understand this question, but CSpecialList can
> certainly override sq_ass_slice.  Then, if it wants to call
> self.__slice__ (in Python terms) it could call
> obj->ob_type->tp_as_sequence->sq_slice -- this would call C's
> obj->ob_type->tp_as_sequence->__slice__ if it had one.
I overlooked this, I think we can implement that also in Jython.

An important point: should that be the only way to use a Python level
exposed and overridable method from C code, is that left
to the programmer ?

The problem is that with the new mro possibly independently C defined 
things  get interleaved with Python level code, 
so I imagine that the effect should be a bit disciplined ...

I saw that list_ass_slice calls list_slice directly, is that OK?

> > And how can safely CSpecialList invoke a  "super" behaviour, it is saf
> > for it to refer to the behaviour offered by list. In principle
> > given the mro e.g. a __getslice__ redefined in B should shadow
> > such a behaviour?
> 
> I don't know if cooperative super calls are possible at the C level.
> I certainly didn't define an API for this.  I didn't define a Python
> level API either, but I know how to o it and it can even be coded in
> Python.  Hm, I guess that makes it a theoretical possibility at the C
> level.
> 
> I don't expect this to be a common use pattern though (famous last
> words :-).

But it is certainely a pattern that a writer of Jython types would expect
to use. The problem is still that C/Java stuff with the new mro
get interleaved with Python level behaviour.

IMHO you would better discipline this and also the above point <wink>.
I don't know if there is some GUI toolkit that implement inheritance
using C and not some language enforcing OO like C++, etc.

Other OO languages like Smalltalk, Self, or Java allow only to define
primitives or single methods to add lowel-level functionality,
they don't try to allow subtyping at that level, the previous
scenario in Python where you could define just opaque types,
and in particular your C code never/normally got interleaved with Python
 behaviour was on that line.

At worst you could say that you allow subtyping but not code sharing,
but then even a C method for an object should be very careful using
another overridable method of the very same object.

OK you could say: that's C level, everybody should care for herself,
but with Java we cannot do the same.

> > It would make sense to use Java subclassing to implement 
> > type subclassing, at least at layout level this does not clash with
> > the multiple inheritance rule in best_base.
> 
> So you would rule out multiple inheritance of types?  Fine with me,
> but you may regret this a few yearsfrom now (when we all code multiple
> inheritance all the time :-).
No it is ruled out at Java level, not Jython level.
But it is not also ruled out in CPython at C level?

> > But than there is the issue of method resolution order: from the viewpoint
> > of Python code we can implement anything, not that easy ...
> > 
> > But the at the Java level, where we construct types, the codebase uses the 
> > normal single inheritance of java and the codebase is full of super.foo 
> > invocations and of methods that cross-call each other (potentially overriden) 
> > versions, and this happen also for behaviour that correspond to the slots/ 
> > __foo__ special methods of CPython.
> > 
> > That's why the two questions are important?
> 
> I see.  CPython doesn't have this problem (yet) because it doesn't use
> inheritance at the C level.
see above.

>  I propose that you just try to live with
> this.
Related e.g. to Zope porting projects there have been a lot of pressure
on jython-dev regarding how to code new types and metaclasses,
A Jython type writer expect to be able to use Java naturally (i.e. use inheritance),
for that, so the situation isn't that easy,
is not even easy with the current codebase, because given the new mro one can even
put some Python behaviour between:

PyList PythonLevelCode and PyObject :-( (*)

>  A safe rule would be to require that all Python classes in a
> given inheritance graph should derive from the same C/Java class.  I
That would mean that all C/Java behaviour is always only at the top
of the mro, right?

I had the same idea because this would make my life easier.

A problem: some of the Zope porters reported that there are classes
that inherits from more than a single ExtensionClass in Zope :-(
don't know if that is true. (**)

> don't know if we should enforce this or just warn about it in the
> documentation.
It would make our (jython-dev) life easier and avoid some impredictable
 problem even with CPython and code sharing,
  but see previous note (**)

>  I don't want to force you to call a cooperative super
> method at the Java level -- it would be very slow, I suspect...
Yes ... very slow and we would have to use that also at the very core
of the hierarchy, see (*)
but the problem is more complicated, in Java given three classes

C extends B extends A
and a method m
C.m overrides B.m overrides A.m

there is not direct way (even using reflection) 
to apply B.m behaviour to a C object, unless
the programmer has left some hook in C using super
or in B ( a method B_m e.g.).

... complicated :(

regards.



From guido@python.org  Fri Aug 24 15:04:43 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 24 Aug 2001 10:04:43 -0400
Subject: [Python-Dev] Customizing the binding of attributes
In-Reply-To: Your message of "Fri, 24 Aug 2001 06:32:22 PDT."
 <20010824063222.A12847@glacier.fnational.com>
References: <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook> <200108231923.f7NJNg307631@odiug.digicool.com> <09f401c12c6a$7717ab80$e000a8c0@thomasnotebook> <200108241003.GAA04383@cj20424-a.reston1.va.home.com>
 <20010824063222.A12847@glacier.fnational.com>
Message-ID: <200108241404.KAA05644@cj20424-a.reston1.va.home.com>

> Guido van Rossum wrote:
> > In particular, classic classes can't define __get__ to the effect
> > above.
> 
> Why __get__() and __set__() and not get() and set()?

Because they're mapped to the dispatch slots tp_descr_get and
tp_descr_set.  (Note that PEP 252 has get() and set() in some places,
like the C API section.  This is a mistake.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Samuele Pedroni <pedroni@inf.ethz.ch>  Fri Aug 24 15:21:14 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Fri, 24 Aug 2001 16:21:14 +0200 (MET DST)
Subject: [Python-Dev] Afterthoughts: PEP 252, PEP 253 and jython issues
Message-ID: <200108241421.QAA18057@core.inf.ethz.ch>

Yes, I'm thinking furiously.

GvR >  A safe rule would be to require that all Python classes in a
GvR > given inheritance graph should derive from the same C/Java class.  I

> That would mean that all C/Java behaviour is always only at the top
> of the mro, right?
> 
> I had the same idea because this would make my life easier.
> 
> A problem: some of the Zope porters reported that there are classes
> that inherits from more than a single ExtensionClass in Zope :-(
> don't know if that is true. (**)
Oops, this is already a problem with the actual best_base rule,
I don't know if my reinterpretation is OK, but doesn't it 
basically states that all C types subclassed by a new C level class
should be in a  subtyping relationship that means:

with
A isa list isa object
B isa special_list isa list isa object
C isa object

class D(A,B,C): ... is OK

but with

Z isa Cbadboy isa object

then class D(Z,A) isn't allowed.

but if that is already *not* *a* *problem* then I think the rule could
be not that bad ...

regards.





From guido@python.org  Fri Aug 24 15:35:45 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 24 Aug 2001 10:35:45 -0400
Subject: [Python-Dev] Comments: PEP 252, PEP 253 and jython issues
In-Reply-To: Your message of "Fri, 24 Aug 2001 15:57:25 +0200."
 <200108241357.PAA16863@core.inf.ethz.ch>
References: <200108241357.PAA16863@core.inf.ethz.ch>
Message-ID: <200108241435.KAA05784@cj20424-a.reston1.va.home.com>

> Thanks for the answers, I'm happy the questions were parseable,
> Here are some comments, I think they may be meanigful also for CPython
> 
> > > The question is it possible for CSpecialList for the code that define
> > > for example sq_ass_slice to use sq_slice code in a safe manner?
> > > (AFAIK listobject does something similar) 
> > 
> > I'm not 100% sure I understand this question, but CSpecialList can
> > certainly override sq_ass_slice.  Then, if it wants to call
> > self.__slice__ (in Python terms) it could call
> > obj->ob_type->tp_as_sequence->sq_slice -- this would call C's
> > obj->ob_type->tp_as_sequence->__slice__ if it had one.

> I overlooked this, I think we can implement that also in Jython.
> 
> An important point: should that be the only way to use a Python level
> exposed and overridable method from C code, is that left
> to the programmer ?

I'm not sure I understand this question.  Can you give an example?  I
don't expect that calling into Python from C will be very common.
There are ways to call any Python object from C if you need to.

> The problem is that with the new mro possibly independently C defined 
> things  get interleaved with Python level code, 
> so I imagine that the effect should be a bit disciplined ...

I don't expect this to happen regularly.  if you count on this I think
you're on really thin ice.

> I saw that list_ass_slice calls list_slice directly, is that OK?

It's certainly safe -- any subclass of list will share the same
structure lay-out (this is enforced by the subclassing machinery) so
the list_slice call will find the object properly inititialized.

There are lots of places where the C code doesn't bother to look for
overridden methods -- the PyDict_* API is a common example.  I have no
intention of fixing this.

> > > And how can safely CSpecialList invoke a  "super" behaviour, it is saf
> > > for it to refer to the behaviour offered by list. In principle
> > > given the mro e.g. a __getslice__ redefined in B should shadow
> > > such a behaviour?
> > 
> > I don't know if cooperative super calls are possible at the C level.
> > I certainly didn't define an API for this.  I didn't define a Python
> > level API either, but I know how to o it and it can even be coded in
> > Python.  Hm, I guess that makes it a theoretical possibility at the C
> > level.
> > 
> > I don't expect this to be a common use pattern though (famous last
> > words :-).
> 
> But it is certainely a pattern that a writer of Jython types would expect
> to use. The problem is still that C/Java stuff with the new mro
> get interleaved with Python level behaviour.

If you think this will be a problem in Jython, you may have to provide
a fast efficient Java-level "cooperative super" operation.

> IMHO you would better discipline this and also the above point <wink>.

What do you mean by "discipline"?

> I don't know if there is some GUI toolkit that implement inheritance
> using C and not some language enforcing OO like C++, etc.

Sorry, I don't understand the relevance of this?

> Other OO languages like Smalltalk, Self, or Java allow only to define
> primitives or single methods to add lowel-level functionality,
> they don't try to allow subtyping at that level, the previous
> scenario in Python where you could define just opaque types,
> and in particular your C code never/normally got interleaved with Python
>  behaviour was on that line.

Well, if you like that better, maybe I should remove the instructions
for subclassing at the C level? :-)

My intention is to make the maximum number of different paradigms
usable in Python.  You can do cooperative multiple inheritance, but
only of all your classes are designed with cooperation in mind.  If
some class uses C-level (or Java-level) inheritance and is not written
cooperatively, you have to be a little careful using it as a base
class in a cooperatively written class.  I don't want to enforce
cooperative coding at the Python level either: the
"BaseClass.method(self, args)" approach is still valid, but you have
to beware of the consequences.

> At worst you could say that you allow subtyping but not code sharing,
> but then even a C method for an object should be very careful using
> another overridable method of the very same object.

I definitely have to support code sharing.  I think subclassing at the
C level is useful to create variations of built-in objects with
additional properties, e.g. numbers with additional operations or
formatting parameters, dictionaries with certain restrictions on the
keys, etc.  I don't think such subclasses should be used in complex
multiple inheritance lattices.

> OK you could say: that's C level, everybody should care for herself,
> but with Java we cannot do the same.

Why not?

> > > It would make sense to use Java subclassing to implement 
> > > type subclassing, at least at layout level this does not clash with
> > > the multiple inheritance rule in best_base.
> > 
> > So you would rule out multiple inheritance of types?  Fine with me,
> > but you may regret this a few yearsfrom now (when we all code multiple
> > inheritance all the time :-).

> No it is ruled out at Java level, not Jython level.
> But it is not also ruled out in CPython at C level?

Mostly because I don't provide a way to create multiple base classes.
That would not be hard to add though -- if there's demand.

> > > But than there is the issue of method resolution order: from the
> > > viewpoint of Python code we can implement anything, not that
> > > easy ...
> > > 
> > > But the at the Java level, where we construct types, the
> > > codebase uses the normal single inheritance of java and the
> > > codebase is full of super.foo invocations and of methods that
> > > cross-call each other (potentially overriden) versions, and this
> > > happen also for behaviour that correspond to the slots/ __foo__
> > > special methods of CPython.
> > > 
> > > That's why the two questions are important?
> > 
> > I see.  CPython doesn't have this problem (yet) because it doesn't use
> > inheritance at the C level.

> see above.
> 
> >  I propose that you just try to live with this.

> Related e.g. to Zope porting projects there have been a lot of
> pressure on jython-dev regarding how to code new types and
> metaclasses, A Jython type writer expect to be able to use Java
> naturally (i.e. use inheritance), for that, so the situation isn't
> that easy, is not even easy with the current codebase, because given
> the new mro one can even put some Python behaviour between:
> 
> PyList PythonLevelCode and PyObject :-( (*)

Just sa no. :-)

> >  A safe rule would be to require that all Python classes in a
> > given inheritance graph should derive from the same C/Java class.  I

> That would mean that all C/Java behaviour is always only at the top
> of the mro, right?

If you mean at the end, yes.

> I had the same idea because this would make my life easier.

Sounds good to me.

> A problem: some of the Zope porters reported that there are classes
> that inherits from more than a single ExtensionClass in Zope :-(
> don't know if that is true. (**)

Me neither.  But the problem is probably shallow -- Zope mostly uses
mixins of various sorts, so these classes probably don't override
stuff -- they just define new methods.  (I haven't looked at what it
would take to replace ExtensionClass with the new metaclasses, but
that's definitely on the agenda.)

> > don't know if we should enforce this or just warn about it in the
> > documentation.

> It would make our (jython-dev) life easier and avoid some impredictable
>  problem even with CPython and code sharing,
>   but see previous note (**)

So enforce it and see how it turns out in practice.

> >  I don't want to force you to call a cooperative super
> > method at the Java level -- it would be very slow, I suspect...

> Yes ... very slow and we would have to use that also at the very core
> of the hierarchy, see (*)
> but the problem is more complicated, in Java given three classes
> 
> C extends B extends A
> and a method m
> C.m overrides B.m overrides A.m
> 
> there is not direct way (even using reflection) 
> to apply B.m behaviour to a C object, unless
> the programmer has left some hook in C using super
> or in B ( a method B_m e.g.).

I guess C.m has to call super.m to invoke B.m, right?

Does this mean that the following can't be made to work?

class C(list):
    def add_spam(self):
        return list.append(self, "spam")

It's legal Python (although in general questionable style of course).

> ... complicated :(

:-(

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Aug 24 15:52:28 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 24 Aug 2001 10:52:28 -0400
Subject: [Python-Dev] Afterthoughts: PEP 252, PEP 253 and jython issues
In-Reply-To: Your message of "Fri, 24 Aug 2001 16:21:14 +0200."
 <200108241421.QAA18057@core.inf.ethz.ch>
References: <200108241421.QAA18057@core.inf.ethz.ch>
Message-ID: <200108241452.KAA05961@cj20424-a.reston1.va.home.com>

> Yes, I'm thinking furiously.

See if I can keep up with you. :-)

> GvR >  A safe rule would be to require that all Python classes in a
> GvR > given inheritance graph should derive from the same C/Java class.  I
> 
> > That would mean that all C/Java behaviour is always only at the top
> > of the mro, right?
> > 
> > I had the same idea because this would make my life easier.
> > 
> > A problem: some of the Zope porters reported that there are classes
> > that inherits from more than a single ExtensionClass in Zope :-(
> > don't know if that is true. (**)

> Oops, this is already a problem with the actual best_base rule,
> I don't know if my reinterpretation is OK, but doesn't it 
> basically states that all C types subclassed by a new C level class
> should be in a  subtyping relationship that means:
> 
> with
> A isa list isa object
> B isa special_list isa list isa object
> C isa object
> 
> class D(A,B,C): ... is OK
> 
> but with
> 
> Z isa Cbadboy isa object
> 
> then class D(Z,A) isn't allowed.

Correct.  You can't multiply inherit from two types that have a
different structure layout.  The C code must be able to safely cast a
PyObject* to a PyListObject* or a Cbadboy* and if those don't have the
same structure it's not safe.  The implementation restricts this
pretty effectively I believe.  (You can't multiply inherit from list
and dictionary, but you can from list and xxsubtype.spamlist.)

> but if that is already *not* *a* *problem* then I think the rule could
> be not that bad ...

What do you mean by "already *not* *a* *problem*"?  I hope "it is
already forbidden".  And "the rule" refers back to my proposal quoted
above with "GvR >", right?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Samuele Pedroni <pedroni@inf.ethz.ch>  Fri Aug 24 16:50:05 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Fri, 24 Aug 2001 17:50:05 +0200 (MET DST)
Subject: [Python-Dev] Comments2: PEP 252, PEP 253 and jython issues
Message-ID: <200108241550.RAA21561@core.inf.ethz.ch>

[GvR]
> 
> > The problem is that with the new mro possibly independently C defined 
> > things  get interleaved with Python level code, 
> > so I imagine that the effect should be a bit disciplined ...
> 
> I don't expect this to happen regularly.  if you count on this I think
> you're on really thin ice.

No I'm not counting on that, I'm worried about this scenario

A subclasses object and redefines __getattr__
B subclasses list

then class C(B,A): ...

will have the following mro:

C B list A object

In Jython codebase hypothetically but possibly list could contains calls like:

  __getattr__(...)
  and super.__getattr__
  
now A is in between list and object: what is the right thing here?
I don't expect or invite users to do that, but your rules allow A to
end up in that position in the mro, your philosophy seems to be
this is thin ice and so let it be ... 


> > I saw that list_ass_slice calls list_slice directly, is that OK?
> 
> It's certainly safe -- any subclass of list will share the same
> structure lay-out (this is enforced by the subclassing machinery) so
> the list_slice call will find the object properly inititialized.
> 
> There are lots of places where the C code doesn't bother to look for
> overridden methods -- the PyDict_* API is a common example.  I have no
> intention of fixing this.
Fine with that, but ...

> 
> > Other OO languages like Smalltalk, Self, or Java allow only to define
> > primitives or single methods to add lowel-level functionality,
> > they don't try to allow subtyping at that level, the previous
> > scenario in Python where you could define just opaque types,
> > and in particular your C code never/normally got interleaved with Python
> >  behaviour was on that line.
> 
> Well, if you like that better, maybe I should remove the instructions
> for subclassing at the C level? :-)
No, I'm just saying that you are moving from a world were you had docume
ntable C black-boxes and clear Python semantics, to a world where
(but you say that would be rare) you can have Python code whose behaviour
depends on very fine-grained decision inside C code.
On one side your rules make it very easy to interleave Python
and C code in the mro, on the other hand I agree that
will be rarely done by the wary user because it's just 
surprise-land, don't know about the unwary ...

> 
> My intention is to make the maximum number of different paradigms
> usable in Python.  You can do cooperative multiple inheritance, but
> only of all your classes are designed with cooperation in mind.  If
> some class uses C-level (or Java-level) inheritance and is not written
> cooperatively, you have to be a little careful using it as a base
> class in a cooperatively written class.  I don't want to enforce
> cooperative coding at the Python level either: the
> "BaseClass.method(self, args)" approach is still valid, but you have
> to beware of the consequences.
Do you feel is that a good thing, especially if new-style classes become
the default?
It is not a rethoric question, Python is becoming a little single dispatch
CLOS with a MOP, descriptors etc
but CLOS has only next-method and strange beasts like method combinators
and don't allow to interleave low-level coded behaviour in the class mro.

> 
> > At worst you could say that you allow subtyping but not code sharing,
> > but then even a C method for an object should be very careful using
> > another overridable method of the very same object.
> 
> I definitely have to support code sharing.  I think subclassing at the
> C level is useful to create variations of built-in objects with
> additional properties, e.g. numbers with additional operations or
> formatting parameters, dictionaries with certain restrictions on the
> keys, etc.  I don't think such subclasses should be used in complex
> multiple inheritance lattices.
Better to enforce that?

> 
> > OK you could say: that's C level, everybody should care for herself,
> > but with Java we cannot do the same.
> 
> Why not?
Because users would like writing Jython types to become easier, and be
possible using Java OOP model, without surprises, or undocumented
subtlities, any deviation is an hard sell because Java has already
an OOP model.


>> > > > It would make sense to use Java subclassing to implement 
> > > > type subclassing, at least at layout level this does not clash with
> > > > the multiple inheritance rule in best_base.
> > > 
> > > So you would rule out multiple inheritance of types?  Fine with me,
> > > but you may regret this a few yearsfrom now (when we all code multiple
> > > inheritance all the time :-).
> 
> > No it is ruled out at Java level, not Jython level.
> > But it is not also ruled out in CPython at C level?
> 
> Mostly because I don't provide a way to create multiple base classes.
> That would not be hard to add though -- if there's demand.
But this is a path that Jython could not follow.

 
> > see above.
> > 
> > >  I propose that you just try to live with this.
> 
> > Related e.g. to Zope porting projects there have been a lot of
> > pressure on jython-dev regarding how to code new types and
> > metaclasses, A Jython type writer expect to be able to use Java
> > naturally (i.e. use inheritance), for that, so the situation isn't
> > that easy, is not even easy with the current codebase, because given
> > the new mro one can even put some Python behaviour between:
> > 
> > PyList PythonLevelCode and PyObject :-( (*)
> 
> Just sa no. :-)
See above.

> 
> > >  A safe rule would be to require that all Python classes in a
> > > given inheritance graph should derive from the same C/Java class.  I
> 
> > That would mean that all C/Java behaviour is always only at the top
> > of the mro, right?
> 
> If you mean at the end, yes.
Of course.

> 
> > I had the same idea because this would make my life easier.
> 
> Sounds good to me.
> 
> > A problem: some of the Zope porters reported that there are classes
> > that inherits from more than a single ExtensionClass in Zope :-(
> > don't know if that is true. (**)
> 
> Me neither.  But the problem is probably shallow -- Zope mostly uses
> mixins of various sorts, so these classes probably don't override
> stuff -- they just define new methods.  (I haven't looked at what it
> would take to replace ExtensionClass with the new metaclasses, but
> that's definitely on the agenda.)
> 
> > > don't know if we should enforce this or just warn about it in the
> > > documentation.
> 
> > It would make our (jython-dev) life easier and avoid some impredictable
> >  problem even with CPython and code sharing,
> >   but see previous note (**)
> 
> So enforce it and see how it turns out in practice.
It's probably the best solution, at least for Jython,
and could even make sense for CPython (see above points)

 
> > >  I don't want to force you to call a cooperative super
> > > method at the Java level -- it would be very slow, I suspect...
> 
> > Yes ... very slow and we would have to use that also at the very core
> > of the hierarchy, see (*)
> > but the problem is more complicated, in Java given three classes
> > 
> > C extends B extends A
> > and a method m
> > C.m overrides B.m overrides A.m
> > 
> > there is not direct way (even using reflection) 
> > to apply B.m behaviour to a C object, unless
> > the programmer has left some hook in C using super
> > or in B ( a method B_m e.g.).
> 
> I guess C.m has to call super.m to invoke B.m, right?
Yup.

> 
> Does this mean that the following can't be made to work?
> 
> class C(list):
>     def add_spam(self):
>         return list.append(self, "spam")
> 
> It's legal Python (although in general questionable style of course).
Yes, it can be made to work (it already works for Java subclassing and
type subclassing would be some flavor of that), but the point is that
your Java stuff should be at the top of the mro. Because then you can insert
some hooks using super.

regards.



From guido@python.org  Fri Aug 24 18:09:37 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 24 Aug 2001 13:09:37 -0400
Subject: [Python-Dev] super()
In-Reply-To: Your message of "Fri, 24 Aug 2001 18:55:38 +0200."
 <021001c12cbd$9a182800$e000a8c0@thomasnotebook>
References: <E15aK6p-0005bp-00@usw-pr-cvs1.sourceforge.net>
 <021001c12cbd$9a182800$e000a8c0@thomasnotebook>
Message-ID: <200108241709.NAA13060@cj20424-a.reston1.va.home.com>

Thomas Heller wrote (in private email, responding to a checkin message):

> From: "Guido van Rossum" <gvanrossum@users.sourceforge.net>
> > Add 'super', another new object type with magical properties.
> > 
> > super(type) -> unbound super object
> > super(type, obj) -> bound super object; requires isinstance(obj, type)
> > 
> > Typical use to call a cooperative superclass method:
> > 
> > class C(B):
> >     def meth(self, arg):
> >         super(C, self).meth(arg);
> 
> Shouldn't that be
> class C(B):
>     def meth(self, arg):
>         super(B, self).meth(arg)
>               ^
> to call B.meth(self, arg)?
> 
> Thomas

No.  Good question, though!  You have to pass your *own* class so that
this finds the right super-method when multiple inheritance involving
a "diamond" diagram.  You may want to read the section on Method
resolution order in PEP 253 first.

Look at this example:

    class A(object):
	def meth(self, a):
	    return "A.meth(%r)" % a

    print A().meth(1)

    class B(A):
	def __init__(self):
	    self.__super = super(B, self)
	def meth(self, a):
	    return "B.meth(%r) -> " % a + self.__super.meth(a)

    print B().meth(2)

    class C(A):
	__dynamic__ = 1
	def meth(self, a):
	    return "C.meth(%r) -> " % a + self.__super.meth(a)
    C._C__super = super(C)

    print C().meth(3)

    class D(C, B):
	def meth(self, a):
	    return "D.meth(%r) -> " % a + super(D, self).meth(a)

    print D().meth(4) # D.meth(4) -> C.meth(4) -> B.meth(4) -> A.meth(4)

D has the following inheritance graph:


                   A
                  ^ ^
                 /   \
                /     \
               /       \
              /         \
              C          B
              ^         ^
               \       /
                \     /
                 \   /
                  \ /
                   D

When you have a C or B instance, C.meth's super references A.
But when you have a D instance, C.meth's super references B!

In other words, D.meth calls C.meth calls B.meth calls A.meth.  This
is why you need to pass self to super() -- it needs the __mro__
sequence of the instance.  Remember that D.__mro__ == (D,C,B,A,object).
C.meth calls super(C, D()).meth.  This looks for C in D.__mro__, and
then starts searching for meth at the next class -- finding B.meth.

If you wonder *why* the MRO is designed this way, imagine that meth()
does some sort of saving to disk.  Each subclass saves its own stuff
and then calls the save method of its base class.  Without the
behavior described above, D.save() would have to call C.save() and
B.save(), but this would call A.save() twice!

This is part of a pattern called cooperative class design, described
in the book Putting Metaclasses to Work (for a reference, see PEP
253).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Aug 24 18:36:59 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 24 Aug 2001 13:36:59 -0400
Subject: [Python-Dev] Comments2: PEP 252, PEP 253 and jython issues
In-Reply-To: Your message of "Fri, 24 Aug 2001 17:50:05 +0200."
 <200108241550.RAA21561@core.inf.ethz.ch>
References: <200108241550.RAA21561@core.inf.ethz.ch>
Message-ID: <200108241736.NAA14289@cj20424-a.reston1.va.home.com>

[SP]
> > > The problem is that with the new mro possibly independently C
> > > defined things get interleaved with Python level code, so I
> > > imagine that the effect should be a bit disciplined ...
> > 
> [GvR]
> > 
> > I don't expect this to happen regularly.  if you count on this I
> > think you're on really thin ice.

[SP]
> No I'm not counting on that, I'm worried about this scenario
> 
> A subclasses object and redefines __getattr__
> B subclasses list
> 
> then class C(B,A): ...
> 
> will have the following mro:
> 
> C B list A object
> 
> In Jython codebase hypothetically but possibly list could contains
> calls like:
> 
>   __getattr__(...)
>   and super.__getattr__

This is Java code, right?

> now A is in between list and object: what is the right thing here?

Just say that this is what happens.  The MRO is inconsistent in this
case.

> I don't expect or invite users to do that, but your rules allow A to
> end up in that position in the mro, your philosophy seems to be
> this is thin ice and so let it be ... 

Exactly.

> > > I saw that list_ass_slice calls list_slice directly, is that OK?
> > 
> > It's certainly safe -- any subclass of list will share the same
> > structure lay-out (this is enforced by the subclassing machinery) so
> > the list_slice call will find the object properly inititialized.
> > 
> > There are lots of places where the C code doesn't bother to look
> > for overridden methods -- the PyDict_* API is a common example.  I
> > have no intention of fixing this.

> Fine with that, but ...
> 
> > > Other OO languages like Smalltalk, Self, or Java allow only to
> > > define primitives or single methods to add lowel-level
> > > functionality, they don't try to allow subtyping at that level,
> > > the previous scenario in Python where you could define just
> > > opaque types, and in particular your C code never/normally got
> > > interleaved with Python
> > >  behaviour was on that line.
> > 
> > Well, if you like that better, maybe I should remove the
> > instructions for subclassing at the C level? :-)

> No, I'm just saying that you are moving from a world were you had
> documentable C black-boxes and clear Python semantics, to a world
> where (but you say that would be rare) you can have Python code
> whose behaviour depends on very fine-grained decision inside C code.

I would recommend that C programmers not indulge in this.  They
should continue to write C types that can be treated as black boxes.
The C-level single inheritance is best seen as code sharing.

> On one side your rules make it very easy to interleave Python and C
> code in the mro, on the other hand I agree that will be rarely done
> by the wary user because it's just surprise-land, don't know about
> the unwary ...

On the third hand, if a C programmer *wants* to write a class with the
proper cooperative super calling, they can do so, it's just a bit
clumsy.  Should I provide a C API to make this easier?  I doubt that
it will be used much.  If there's demand, it can be added later.

> > My intention is to make the maximum number of different paradigms
> > usable in Python.  You can do cooperative multiple inheritance,
> > but only of all your classes are designed with cooperation in
> > mind.  If some class uses C-level (or Java-level) inheritance and
> > is not written cooperatively, you have to be a little careful
> > using it as a base class in a cooperatively written class.  I
> > don't want to enforce cooperative coding at the Python level
> > either: the "BaseClass.method(self, args)" approach is still
> > valid, but you have to beware of the consequences.

> Do you feel is that a good thing, especially if new-style classes become
> the default?

Time will tell.  I expect that some people will design vast
cooperative class hierarchies.  I expect that other people will stick
to single inheritance or very simple-minded uses of multiple
inheritance (for example, they could use the mix-in pattern that was
about the only sensible use of multiple inheritance with classic
classes).

Anyway, I expect new classes won't become the default until Python
3.0.

> It is not a rethoric question, Python is becoming a little single
> dispatch CLOS with a MOP, descriptors etc but CLOS has only
> next-method and strange beasts like method combinators and don't
> allow to interleave low-level coded behaviour in the class mro.

Nothing new: classic classes already allow you to cheat or mess up in
various ways by not calling your superclass method.

> > > At worst you could say that you allow subtyping but not code
> > > sharing, but then even a C method for an object should be very
> > > careful using another overridable method of the very same
> > > object.
> > 
> > I definitely have to support code sharing.  I think subclassing at
> > the C level is useful to create variations of built-in objects
> > with additional properties, e.g. numbers with additional
> > operations or formatting parameters, dictionaries with certain
> > restrictions on the keys, etc.  I don't think such subclasses
> > should be used in complex multiple inheritance lattices.

> Better to enforce that?

Why?  I can think of situations where you have a MRO like this:

  C list B object

but where B doesn't override anything that list might use.  In fact,
this is the typical mix-in situation.  I don't want to forbid this.

> > > OK you could say: that's C level, everybody should care for
> > > herself, but with Java we cannot do the same.
> > 
> > Why not?

> Because users would like writing Jython types to become easier,

Do you mean writing Jython types in Java, or in Python?

> and be possible using Java OOP model, without surprises, or
> undocumented subtlities, any deviation is an hard sell because Java
> has already an OOP model.

Recommend strongly that they stick to single inheritance for types
implemented in Java -- that should do the trick.

> > > > > It would make sense to use Java subclassing to implement
> > > > > type subclassing, at least at layout level this does not
> > > > > clash with the multiple inheritance rule in best_base.
> > > > 
> > > > So you would rule out multiple inheritance of types?  Fine
> > > > with me, but you may regret this a few yearsfrom now (when we
> > > > all code multiple inheritance all the time :-).
> > 
> > > No it is ruled out at Java level, not Jython level.
> > > But it is not also ruled out in CPython at C level?
> > 
> > Mostly because I don't provide a way to create multiple base
> > classes.  That would not be hard to add though -- if there's
> > demand.

> But this is a path that Jython could not follow.

Correct -- another reason why it's not a good idea to encourage this.
What you could do though, even in Jython, is simulate everything that
goes on in a class statement: build your own dict, populate it with
function objects, and then call type.__new__(name, bases, dict).
Voila, instant class. :-)

> > > see above.
> > > 
> > > >  I propose that you just try to live with this.
> > 
> > > Related e.g. to Zope porting projects there have been a lot of
> > > pressure on jython-dev regarding how to code new types and
> > > metaclasses, A Jython type writer expect to be able to use Java
> > > naturally (i.e. use inheritance), for that, so the situation
> > > isn't that easy, is not even easy with the current codebase,
> > > because given the new mro one can even put some Python behaviour
> > > between:
> > > 
> > > PyList PythonLevelCode and PyObject :-( (*)
> > 
> > Just sa no. :-)

> See above.

Sorry, my context stack just overflowed.  If this is still an issue,
please start over. :-)

[snip]

> > > I had the same idea because this would make my life easier.
> > 
> > Sounds good to me.
> > 
> > > A problem: some of the Zope porters reported that there are
> > > classes that inherits from more than a single ExtensionClass in
> > > Zope :-( don't know if that is true. (**)
> > 
> > Me neither.  But the problem is probably shallow -- Zope mostly
> > uses mixins of various sorts, so these classes probably don't
> > override stuff -- they just define new methods.  (I haven't looked
> > at what it would take to replace ExtensionClass with the new
> > metaclasses, but that's definitely on the agenda.)
> > 
> > > > don't know if we should enforce this or just warn about it in
> > > > the documentation.
> > 
> > > It would make our (jython-dev) life easier and avoid some
> > > impredictable problem even with CPython and code sharing, but
> > > see previous note (**)
> > 
> > So enforce it and see how it turns out in practice.

> It's probably the best solution, at least for Jython,
> and could even make sense for CPython (see above points)

Good.  We seem to agree.

> > > > I don't want to force you to call a cooperative super method
> > > > at the Java level -- it would be very slow, I suspect...
> > 
> > > Yes ... very slow and we would have to use that also at the very
> > > core of the hierarchy, see (*) but the problem is more
> > > complicated, in Java given three classes
> > > 
> > > C extends B extends A
> > > and a method m
> > > C.m overrides B.m overrides A.m
> > > 
> > > there is not direct way (even using reflection) 
> > > to apply B.m behaviour to a C object, unless
> > > the programmer has left some hook in C using super
> > > or in B ( a method B_m e.g.).
> > 
> > I guess C.m has to call super.m to invoke B.m, right?

> Yup.

> > Does this mean that the following can't be made to work?
> > 
> > class C(list):
> >     def add_spam(self):
> >         return list.append(self, "spam")
> > 
> > It's legal Python (although in general questionable style of course).

> Yes, it can be made to work (it already works for Java subclassing
> and type subclassing would be some flavor of that), but the point is
> that your Java stuff should be at the top of the mro. Because then
> you can insert some hooks using super.
> 
> regards.

I hope this discussion so far has been helpful -- I have to pack for
my trip and will soon be off to San Francisco until next Wednesday.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh@python.net  Fri Aug 24 19:09:53 2001
From: mwh@python.net (Michael Hudson)
Date: 24 Aug 2001 14:09:53 -0400
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: Paul Prescod's message of "Wed, 22 Aug 2001 10:56:33 -0800"
References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> <15235.61766.827878.699521@slothrop.digicool.com> <3B840061.A55BA194@ActiveState.com>
Message-ID: <2m7kvtcosu.fsf@starship.python.net>

Paul Prescod <paulp@ActiveState.com> writes:

> Neil Bauman asked him why Perl wasn't being written in Perl (Neil's
> opinion is that Perl is literally the greatest language ever
> created).  Damian responded with a great quote: "Performance. If
> Performance wasn't an issue, I'd probably write Perl 6 in Python or
> something like it with wonderfully beautiful OO abstractions and it
> would run like a dog."

<chortle>
Tell them to use ocaml.
</chortle>

Cheers,
M.

-- 
  You have run into the classic Dmachine problem: your machine has
  become occupied by a malevolent spirit.  Replacing hardware or
  software will not fix this - you need an exorcist. 
                                       -- Tim Bradshaw, comp.lang.lisp


From mwh@python.net  Fri Aug 24 19:11:53 2001
From: mwh@python.net (Michael Hudson)
Date: 24 Aug 2001 14:11:53 -0400
Subject: [Python-Dev] Re: copy, len and the like as 'object' methods?
In-Reply-To: Paul Prescod's message of "Wed, 22 Aug 2001 18:08:40 -0700"
References: <Pine.LNX.4.32.0108221657520.5627-100000@ziggy.localdomain.fake> <3B845798.7505DC91@ActiveState.com>
Message-ID: <2m4rqxcopi.fsf@starship.python.net>

Paul Prescod <paulp@ActiveState.com> writes:

> > Would you also argue for [1, 2, 3].repr() and "abc".hash()?
> 
> Yes. But hashing and repring are done much less often than taking the
> length.

Note that this is an argument for keeping len(o) over o.len() - at
least you bump into it early.

Reminds me of what Laura Creighton said about the results of her
case-sensitivity survey.

Cheers,
M.

-- 
  First of all, email me your AOL password as a security measure. You
  may find that won't be able to connect to the 'net for a while. This
  is normal. The next thing to do is turn your computer upside down
  and shake it to reboot it.                     -- Darren Tucker, asr


From mwh@python.net  Fri Aug 24 19:17:58 2001
From: mwh@python.net (Michael Hudson)
Date: 24 Aug 2001 14:17:58 -0400
Subject: [Python-Dev] PEP 264
In-Reply-To: "Tim Peters"'s message of "Wed, 15 Aug 2001 17:24:20 -0400"
References: <LNBBLJKPBEHFEDALKOLCEEANLGAA.tim.one@home.com>
Message-ID: <2m1ym1cofd.fsf@starship.python.net>

"Tim Peters" <tim.one@home.com> writes:

> [Michael Hudson]
> > I *think* I'm done coding for implementing PEP 264; patch at:
> >
> > http://sourceforge.net/tracker/?func=detail&atid=305470&aid=449043
> > &group_id=5470
> >
> > I've also done a first draft of docs.
> >
> > The reason for mentioning this is I'm off on holiday tomorrow, so if
> > you want changes before the proposed date for 2.2a2, you've got aobut
> > 12 hours to tell me about it.
> 
> Thanks, Michael!  I was out sick yesterday, but intend to get on this
> tonight.  You should enjoy your vacation!  If it's a disaster, I'll probably
> villify you on python-dev, but will fix it anyway <wink>.

Thank you in turn for fixing all my cock-ups!

I'd just like to note in passing that I considered most of the places
the CO_* constants ended up in the last week, and rejected them as too
gross.  My idea was that 

__future__.generators.compiler_flag

was to be the *only* (Python) name for CO_GENERATORS_ALLOWED (for
example).  Are there problems with this (other than the fact that you
have to make sure two sets of magic numbers match up - but you have to
edit __future__.py when you add a new future feature anyway...).

I should probably add some words to the PEP on this topic, anyway.

Cheers,
M.

-- 
  ... when all the programmes on all the channels actually were made
  by actors with cleft pallettes speaking lines by dyslexic writers
  filmed by blind cameramen instead of merely seeming like that, it
  somehow made the whole thing more worthwhile.   -- HHGTG, Episode 11


From mwh@python.net  Fri Aug 24 19:23:23 2001
From: mwh@python.net (Michael Hudson)
Date: 24 Aug 2001 14:23:23 -0400
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Python bltinmodule.c,2.227,2.228
In-Reply-To: Tim Peters's message of "Fri, 17 Aug 2001 15:11:30 -0700"
References: <E15Xrpy-0003z6-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <2my9o9b9lw.fsf@starship.python.net>

Tim Peters <tim_one@users.sourceforge.net> writes:

> ! 	/* XXX Warn if (supplied_flags & PyCF_MASK_OBSOLETE) != 0? */

I think not.

If, as I *think*, all code objects compiled in 2.2 have

   ob.co_flags & __future__.nested_scopes.compiler_flag

non-zero, then, eg., idle will compile all code typed interactively
(after the first line, anyway) with
__future__.nested_scopes.compiler_flag in the supplied_flags argument
to compile().  This could be wormed around, obviously, but I don't see
the point.

Cheers,
M.


From Samuele Pedroni <pedroni@inf.ethz.ch>  Fri Aug 24 19:28:50 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Fri, 24 Aug 2001 20:28:50 +0200 (MET DST)
Subject: [Python-Dev] nightmare: PEP 252, PEP 253 and jython issues
Message-ID: <200108241829.UAA26901@core.inf.ethz.ch>

[GvR]
>
> > now A is in between list and object: what is the right thing here?
> 
> Just say that this is what happens.  The MRO is inconsistent in this
> case.
> 
> > I don't expect or invite users to do that, but your rules allow A to
> > end up in that position in the mro, your philosophy seems to be
> > this is thin ice and so let it be ... 
> 
> Exactly.
I wish I would not feel uneasy with that <wink>

> 
> On the third hand, if a C programmer *wants* to write a class with the
> proper cooperative super calling, they can do so, it's just a bit
> clumsy.  Should I provide a C API to make this easier?  I doubt that
> it will be used much.  If there's demand, it can be added later.
I hope there will not be.


GvR> I don't think such subclasses
GvR> should be used in complex multiple inheritance lattices.
> 
> > Better to enforce that?
> 
> Why?  I can think of situations where you have a MRO like this:
> 
>   C list B object
> 
> but where B doesn't override anything that list might use.  In fact,
> this is the typical mix-in situation.  I don't want to forbid this.

OK, I see. (*)


GVR> So enforce it and see how it turns out in practice.
> 
> > It's probably the best solution, at least for Jython,
> > and could even make sense for CPython (see above points)
> 
> Good.  We seem to agree.
Yup but I see your point (*) so probably I can't.

[me] *bad-guy-java*
> > > > Yes ... very slow and we would have to use that also at the very
> > > > core of the hierarchy, see (*) but the problem is more
> > > > complicated, in Java given three classes
> > > > 
> > > > C extends B extends A
> > > > and a method m
> > > > C.m overrides B.m overrides A.m
> > > > 
> > > > there is not direct way (even using reflection) 
> > > > to apply B.m behaviour to a C object, unless
> > > > the programmer has left some hook in C using super
> > > > or in B ( a method B_m e.g.).

> 
> I hope this discussion so far has been helpful -- I have to pack for
> my trip and will soon be off to San Francisco until next Wednesday.
> 
Yes it was, thanks but
I overlooked a central point, namely *bad-guy-java*

I fear, because of it, we have to code all the built-in types this way,


class SpamType {

... SpamTypes__foo__ { ...
}

... __foo__ { }

}

Or find other slower ways, otherwise no descrs for them 

A nightmare :-(

regards.



From guido@python.org  Fri Aug 24 19:44:05 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 24 Aug 2001 14:44:05 -0400
Subject: [Python-Dev] nightmare: PEP 252, PEP 253 and jython issues
In-Reply-To: Your message of "Fri, 24 Aug 2001 20:28:50 +0200."
 <200108241829.UAA26901@core.inf.ethz.ch>
References: <200108241829.UAA26901@core.inf.ethz.ch>
Message-ID: <200108241844.OAA14685@cj20424-a.reston1.va.home.com>

[snip]
[SP]
> *bad-guy-java*
> > > > > Yes ... very slow and we would have to use that also at the very
> > > > > core of the hierarchy, see (*) but the problem is more
> > > > > complicated, in Java given three classes
> > > > > 
> > > > > C extends B extends A
> > > > > and a method m
> > > > > C.m overrides B.m overrides A.m
> > > > > 
> > > > > there is not direct way (even using reflection) 
> > > > > to apply B.m behaviour to a C object, unless
> > > > > the programmer has left some hook in C using super
> > > > > or in B ( a method B_m e.g.).
> 
[snip]
> I overlooked a central point, namely *bad-guy-java*
> 
> I fear, because of it, we have to code all the built-in types this way,
> 
> 
> class SpamType {
> 
> ... SpamTypes__foo__ { ...
> }
> 
> ... __foo__ { }
> 
> }
> 
> Or find other slower ways, otherwise no descrs for them 

As long as the slowness only applies when someone invokes
SpamType.__foo__(x), that's acceptable.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From thomas.heller@ion-tof.com  Fri Aug 24 19:57:04 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Fri, 24 Aug 2001 20:57:04 +0200
Subject: [Python-Dev] Customizing the binding of attributes
References: <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook> <200108231923.f7NJNg307631@odiug.digicool.com>              <09f401c12c6a$7717ab80$e000a8c0@thomasnotebook>  <200108241003.GAA04383@cj20424-a.reston1.va.home.com>
Message-ID: <03cf01c12cce$90822d20$e000a8c0@thomasnotebook>

[I wrote]
> > - Shouldn't (my)instancemethod be a builtin? Similar to
> > staticmethod and classmethod?
> 
> Yes.  I just checked this in.  The new name is 'getset' unless you
> have a better idea.

I wasn't talking about your getset object (which is already mentioned
in the PEP), I was thinking of an 'instancemethod' object,
which would accept any callable (for example a *function*
implemented in an extension module), and convert it into an unbound
or bound method when retrieved from a class or an instance.
Something like

class instancemethod(object):
    def __init__(self, callable):
        self.callable = callable

    def __get__(self, inst, klass=None):
        import new
        return new.instancemethod(self.callable, inst, klass)

only faster ;-)

Thomas        



From guido@python.org  Fri Aug 24 20:49:42 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 24 Aug 2001 15:49:42 -0400
Subject: [Python-Dev] Customizing the binding of attributes
In-Reply-To: Your message of "Fri, 24 Aug 2001 20:57:04 +0200."
 <03cf01c12cce$90822d20$e000a8c0@thomasnotebook>
References: <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook> <200108231923.f7NJNg307631@odiug.digicool.com> <09f401c12c6a$7717ab80$e000a8c0@thomasnotebook> <200108241003.GAA04383@cj20424-a.reston1.va.home.com>
 <03cf01c12cce$90822d20$e000a8c0@thomasnotebook>
Message-ID: <200108241949.PAA16386@cj20424-a.reston1.va.home.com>

> I wasn't talking about your getset object (which is already mentioned
> in the PEP), I was thinking of an 'instancemethod' object,
> which would accept any callable (for example a *function*
> implemented in an extension module), and convert it into an unbound
> or bound method when retrieved from a class or an instance.
> Something like
> 
> class instancemethod(object):
>     def __init__(self, callable):
>         self.callable = callable
> 
>     def __get__(self, inst, klass=None):
>         import new
>         return new.instancemethod(self.callable, inst, klass)
> 
> only faster ;-)

I believe the (unbound) instance method type itself has this
functionality, except it has no working constructor.  I will fix that
(not today :-).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Samuele Pedroni <pedroni@inf.ethz.ch>  Fri Aug 24 20:54:07 2001
From: Samuele Pedroni <pedroni@inf.ethz.ch> (Samuele Pedroni)
Date: Fri, 24 Aug 2001 21:54:07 +0200 (MET DST)
Subject: [Python-Dev] nightmare: PEP 252, PEP 253 and jython issues
Message-ID: <200108241954.VAA29770@core.inf.ethz.ch>

[GvR]
> > [SP]
> > *bad-guy-java*
> > > > > > Yes ... very slow and we would have to use that also at the very
> > > > > > core of the hierarchy, see (*) but the problem is more
> > > > > > complicated, in Java given three classes
> > > > > > 
> > > > > > C extends B extends A
> > > > > > and a method m
> > > > > > C.m overrides B.m overrides A.m
> > > > > > 
> > > > > > there is not direct way (even using reflection) 
> > > > > > to apply B.m behaviour to a C object, unless
> > > > > > the programmer has left some hook in C using super
> > > > > > or in B ( a method B_m e.g.).
> > 
> [snip]
> > I overlooked a central point, namely *bad-guy-java*
> > 
> > I fear, because of it, we have to code all the built-in types this way,
> > 
> > 
> > class SpamType {
> > 
> > ... SpamTypes__foo__ { ...
> > }
> > 
> > ... __foo__ { }
> > 
> > }
> > 
> > Or find other slower ways, otherwise no descrs for them 
> 
> As long as the slowness only applies when someone invokes
> SpamType.__foo__(x), that's acceptable.
Not that simple, *bad-guy-java* is really nasty given your design
for PEP 252 and PEP 253, you should put some hook to overridable behaviour
or completely give up with Java OOP ...

but it's our problem, your design
up to my mro concerns is clearly is sound.

But Jython 2.2 will be a major challenge.

Samuele.



From tim.one@home.com  Fri Aug 24 23:26:13 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 24 Aug 2001 18:26:13 -0400
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Python bltinmodule.c,2.227,2.228
In-Reply-To: <2my9o9b9lw.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEOBLHAA.tim.one@home.com>

> ! 	/* XXX Warn if (supplied_flags & PyCF_MASK_OBSOLETE) != 0? */

[Michael Hudson]
> I think not.
>
> If, as I *think*, all code objects compiled in 2.2 have
>
>    ob.co_flags & __future__.nested_scopes.compiler_flag
>
> non-zero, then, eg., idle will compile all code typed interactively
> (after the first line, anyway) with
> __future__.nested_scopes.compiler_flag in the supplied_flags argument
> to compile().  This could be wormed around, obviously, but I don't see
> the point.

Your belief was true of 2.2a1 but not of 2.2a2.  For 2.2a1, Guido simply
turned on nested_scopes by default (and so every code object had CO_NESTED
set).  For 2.2a2, Jeremy returned from paternity leave and took away the
*possibility* of turning nested scopes off.  As a result, the *only*
reference to CO_NESTED now in the C part of Python is in the #define of
PyCF_MASK_OBSOLETE, and Python itself never sets CO_NESTED.  That's part of
what I meant when I said (in a different recent thread) that any piece of
code using these flags is eventually going to get surprised:  these
implementation details are as internal and transient as they get.

This isn't critical now, but I can't help but notice that we've got 8 bit
positions assigned in the co_flags int.  One is meaningless in 2.2a2
(CO_NESTED), and two more will eventually become meaningless
(CO_GENERATOR_ALLOWED and CO_FUTURE_DIVISION).  I'm not worried about
running out of "real" co_flags bits, but so long as we're stuck *also* using
co_flags to communicate-- from caller to dynamically-compiled callee --which
still-optional future-features are in effect, we burn another bit position
every time we dream up a future-feature with semantic content.

So, but later rather than sooner, I expect we're going to have to recycle
these temporary future-feature co_flags bits.  When that happens, it may be
prudent to *warn* about obsolete flag values for a release cycle (before
recycling them), for the benefit of insane people who can't restrain
themselves from playing with undocumented bits <wink>.



From tim.one@home.com  Fri Aug 24 23:41:05 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 24 Aug 2001 18:41:05 -0400
Subject: [Python-Dev] PEP 264
In-Reply-To: <2m1ym1cofd.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEOCLHAA.tim.one@home.com>

[Michael Hudson]
> Thank you in turn for fixing all my cock-ups!

There were few -- it turned out to take much longer than "it should have"
for reasons I don't want to bother recounting now, but rest assured there
was plenty of blame to go around.  Not nearly so much blame as glory, though
<wink>.

> I'd just like to note in passing that I considered most of the places
> the CO_* constants ended up in the last week, and rejected them as too
> gross.  My idea was that
>
> __future__.generators.compiler_flag
>
> was to be the *only* (Python) name for CO_GENERATORS_ALLOWED (for
> example).  Are there problems with this (other than the fact that you
> have to make sure two sets of magic numbers match up - but you have to
> edit __future__.py when you add a new future feature anyway...).

The problem is that nobody is going to *remember* the subtleties.  IIRC,
almost every bug I tracked down was due to *someone* neglecting to fiddle
all the places that needed to be fiddled when changing future-flags.  So The
Rule I implemented was two-fold:

1. If possible, remove the need to remember at all (e.g., that was
   done for PyEval_MergeCompilerFlags, which hadn't been updated to
   know about CO_FUTURE_DIVISION (a bug), but was still uselessly
   merging CO_NESTED (a waste)).

2. When not possible, guarantee that a dumb-ass global editor search
   will find all "conceptual" uses of a CO_xxx flag name.

#2 is why __future__.py contains the names "CO_NESTED" etc.  Since it's not
possible to hide the need entirely, it's imperative to make the dumbest
approach imaginable likely to succeed for the next time, when the novelty
has warn off and nobody quite remembers anymore exactly what needs to be
done.  pyassem.py (in the Compiler package) does the same thing.

> I should probably add some words to the PEP on this topic, anyway.

I don't think it's needed; at this point the bugs have all been shamed into
good behavior by the relentless application of simple good taste <wink>.



From skip@pobox.com (Skip Montanaro)  Fri Aug 24 23:52:08 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Fri, 24 Aug 2001 17:52:08 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Python bltinmodule.c,2.227,2.228
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEOBLHAA.tim.one@home.com>
References: <2my9o9b9lw.fsf@starship.python.net>
 <LNBBLJKPBEHFEDALKOLCKEOBLHAA.tim.one@home.com>
Message-ID: <15238.55960.530563.607085@cb40507-c.elmhst1.il.home.com>

    Tim> This isn't critical now, but I can't help but notice that we've got
    Tim> 8 bit positions assigned in the co_flags int....

    Tim> So, but later rather than sooner, I expect we're going to have to
    Tim> recycle these temporary future-feature co_flags bits.

I asked about reserving three bits to specify the virtual machine the
codestring is meant to be executed on. ;-)

Skip


From nas@python.ca  Sat Aug 25 02:05:58 2001
From: nas@python.ca (Neil Schemenauer)
Date: Fri, 24 Aug 2001 18:05:58 -0700
Subject: [Python-Dev] shutting up tmpnam, tempnam, link warnings
Message-ID: <20010824180558.A14251@glacier.fnational.com>

If you're using a recent version of GNU libc you know what I mean.  They
are annoying as hell and there seems to be no way to disable them
cleanly.  Here's a hack:

  cd /lib
  perl -pe 's/gnu\.warning\.(tempnam|tmpnam|tmpnam_r)/gnu.wanting.\1/g' \
    < libc-2.2.3.so > /tmp/libc-2.2.3.so
  cp -b /tmp/libc-2.2.3.so libc-2.2.3.so

No warranty implied. :-)

  Neil


From esr@thyrsus.com  Sat Aug 25 00:51:56 2001
From: esr@thyrsus.com (Eric S. Raymond)
Date: Fri, 24 Aug 2001 19:51:56 -0400
Subject: [Python-Dev] Negative index in insert()
Message-ID: <20010824195156.A1787@thyrsus.com>

Given 

>>> a = [1, 2, 3, 4]

because

>>> a[-2]
-3

I expected that a.insert(-2, 0) would yield [1, 2, 0, 3, 4].  It was a 
rude shock to discover that 

>>> a
[0, 1, 2, 3, 4]

In fact I think this may be the nastiest surprise Python has handed me since
I started using it.  

The reference manual says "same as s[i:i] = [x]  if i >= 0" which of course
doesn't cover the i < 0 case.  David Beasley's reference says "Inserts x 
at index i" which sounds like the behavior I was expecting but didn't get.

Is this a deliberate design choice, an oversight, or a plain bug?  If it's
a choice, it's damn poorly documented -- this deserves at least a footnote
in the list methods table.  If it's an oversight or bug, I volunteer to fix it.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

"America is at that awkward stage.  It's too late to work within the system,
but too early to shoot the bastards."
	-- Claire Wolfe


From Greg.Wilson@baltimore.com  Sat Aug 25 16:05:00 2001
From: Greg.Wilson@baltimore.com (Greg Wilson)
Date: Sat, 25 Aug 2001 11:05:00 -0400
Subject: [Python-Dev] %b formatting
Message-ID: <930BBCA4CEBBD411BE6500508BB3328F3A8F0F@nsamcanms1.ca.baltimore.com>

Patch 455076 adds "%b" (binary) formatting for integers and
long integers to both strings and unicodes.  As this is my
first time hacking on the Python source, I'd be grateful for
comments.

Thanks,

Greg

p.s. apologies for the length of the signature --- it's another
joy of corporate life :-(


-----------------------------------------------------------------------------------------------------------------
The information contained in this message is confidential and is intended 
for the addressee(s) only.  If you have received this message in error or 
there are any problems please notify the originator immediately.  The 
unauthorized use, disclosure, copying or alteration of this message is 
strictly forbidden. Baltimore Technologies plc will not be liable for direct, 
special, indirect or consequential damages arising from alteration of the 
contents of this message by a third party or as a result of any virus being 
passed on.

In addition, certain Marketing collateral may be added from time to time to 
promote Baltimore Technologies products, services, Global e-Security or 
appearance at trade shows and conferences.
 
This footnote confirms that this email message has been swept by 
Baltimore MIMEsweeper for Content Security threats, including
computer viruses.


From mwh@python.net  Sat Aug 25 17:42:12 2001
From: mwh@python.net (Michael Hudson)
Date: 25 Aug 2001 12:42:12 -0400
Subject: [Python-Dev] Re: copy, len and the like as 'object' methods?
In-Reply-To: Michael Hudson's message of "24 Aug 2001 14:11:53 -0400"
References: <Pine.LNX.4.32.0108221657520.5627-100000@ziggy.localdomain.fake> <3B845798.7505DC91@ActiveState.com> <2m4rqxcopi.fsf@starship.python.net>
Message-ID: <2mr8u03xcr.fsf@starship.python.net>

Michael Hudson <mwh@python.net> writes:

> Reminds me of what Laura Creighton said about the results of her
> case-sensitivity survey.

As Aahz has reminded me, it was Sheila King.

-- 
  languages shape the way we think, or don't.
                                        -- Erik Naggum, comp.lang.lisp


From tim.one@home.com  Sun Aug 26 05:08:18 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 26 Aug 2001 00:08:18 -0400
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Python bltinmodule.c,2.227,2.228
In-Reply-To: <15238.55960.530563.607085@cb40507-c.elmhst1.il.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEBILIAA.tim.one@home.com>

[Skip Montanaro]
> I asked about reserving three bits to specify the virtual machine the
> codestring is meant to be executed on. ;-)

Not a problem -- CO_GENERATOR is the only "real" new co_flag added in years,
in 10 years we've only defined 5 real co_flags total, and there are no PEPs
on the table that would need another one.  So you may as well ask for 27 VM
bits <wink>.



From tim.one@home.com  Sun Aug 26 07:14:15 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 26 Aug 2001 02:14:15 -0400
Subject: [Python-Dev] Negative index in insert()
In-Reply-To: <20010824195156.A1787@thyrsus.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEBMLIAA.tim.one@home.com>

[Eric S. Raymond]
> Given
>
> >>> a = [1, 2, 3, 4]
>
> because
>
> >>> a[-2]
> -3
>
> I expected that a.insert(-2, 0) would yield [1, 2, 0, 3, 4].  It was a
> rude shock to discover that
>
> >>> a
> [0, 1, 2, 3, 4]
>
> In fact I think this may be the nastiest surprise Python has
> handed me since I started using it.

I never noticed it, but agree it's strange.  Did you open a feature request
on SourceForge?  Note that array objects (created by the array module) share
this behavior.

> The reference manual says "same as s[i:i] = [x]  if i >= 0" which
> of course doesn't cover the i < 0 case.

Which formally means its behavior is undefined.

> David Beasley's reference says "Inserts x at index i" which sounds like
> the behavior I was expecting but didn't get.

Ya, so file a bug report with David <wink>.

> Is this a deliberate design choice, an oversight, or a plain bug?  If
> it's a choice, it's damn poorly documented -- this deserves at least a
> footnote in the list methods table.  If it's an oversight or bug, I
> volunteer to fix it.

Only Guido *may* be able to say for sure.  Plausible:  the behavior for i<0
was deliberate at the start, and when Python *grew* negative indices (it
didn't always accept them), fear of incompatibility prevented changing
.insert behavior (in the other cases where negative indices "grew a
meaning", the old behavior was to raise an exception; but .insert accepted
them silently).

Digging thru CVS, the current behavior for i<0 has always been that way
(since listobject.c rev 1.1); and the "if i >= 0" qualifier was added to
libtypes.tex a bit over 6 years ago, with checkin comment

    correct description of list.insert()

Hard call!  I agree it should change, but if it breaks code the "but it was
formally undefined" comeback doesn't really warm peoples' hearts.

first-one-to-suggest-a-future-stmt-dies-ly y'rs  - tim



From gstein@lyra.org  Sun Aug 26 07:17:35 2001
From: gstein@lyra.org (Greg Stein)
Date: Sat, 25 Aug 2001 23:17:35 -0700
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects stringobject.c,2.124,2.125
In-Reply-To: <E15aLkW-0003Zp-00@usw-pr-cvs1.sourceforge.net>; from bwarsaw@users.sourceforge.net on Fri, Aug 24, 2001 at 11:32:08AM -0700
References: <E15aLkW-0003Zp-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <20010825231735.B2111@lyra.org>

On Fri, Aug 24, 2001 at 11:32:08AM -0700, Barry Warsaw wrote:
>...
> + PyObject *
> + PyString_FromFormat(const char *format, ...) 
> + {
> + 	va_list vargs;
> + 
> + #ifdef HAVE_STDARG_PROTOTYPES
> + 	va_start(vargs, format);
> + #else
> + 	va_start(vargs);
> + #endif
> + 	return PyString_FromFormatV(format, vargs);
> + }

For safety/propriety, you should call va_end() on the vargs variable.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From gstein@lyra.org  Sun Aug 26 07:21:47 2001
From: gstein@lyra.org (Greg Stein)
Date: Sat, 25 Aug 2001 23:21:47 -0700
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects stringobject.c,2.124,2.125
In-Reply-To: <E15aLkW-0003Zp-00@usw-pr-cvs1.sourceforge.net>; from bwarsaw@users.sourceforge.net on Fri, Aug 24, 2001 at 11:32:08AM -0700
References: <E15aLkW-0003Zp-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <20010825232147.C2111@lyra.org>

On Fri, Aug 24, 2001 at 11:32:08AM -0700, Barry Warsaw wrote:
> Update of /cvsroot/python/python/dist/src/Objects
> In directory usw-pr-cvs1:/tmp/cvs-serv13719/Objects
> 
> Modified Files:
> 	stringobject.c
>...
> +  end:
> + 	_PyString_Resize(&string, s - PyString_AsString(string));

You could use: s - PyString_AS_STRING(string)

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/


From esr@thyrsus.com  Sun Aug 26 07:30:40 2001
From: esr@thyrsus.com (Eric S. Raymond)
Date: Sun, 26 Aug 2001 02:30:40 -0400
Subject: [Python-Dev] Negative index in insert()
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEBMLIAA.tim.one@home.com>; from tim.one@home.com on Sun, Aug 26, 2001 at 02:14:15AM -0400
References: <20010824195156.A1787@thyrsus.com> <LNBBLJKPBEHFEDALKOLCOEBMLIAA.tim.one@home.com>
Message-ID: <20010826023040.A32589@thyrsus.com>

Tim Peters <tim.one@home.com>:
> I never noticed it, but agree it's strange.  Did you open a feature request
> on SourceForge? 

Feature request?  Heck, if Guido green-lights this I'll fix it myself.
I haven't looked at the code yet but I'll be deeply surprised if it
takes me longer than half an hour to nail this bug.

> Hard call!  I agree it should change, but if it breaks code the "but it was
> formally undefined" comeback doesn't really warm peoples' hearts.

If we're going to "fix" division, we sure ought to fix this.

I've grown so used to Python being a smooth, consistent tool that fits
comfortably in my hand that I found this really quite shocking.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

"Both oligarch and tyrant mistrust the people, 
and therefore deprive them of arms."
	--Aristotle


From esr@snark.thyrsus.com  Sun Aug 26 10:11:46 2001
From: esr@snark.thyrsus.com (Eric S. Raymond)
Date: Sun, 26 Aug 2001 05:11:46 -0400
Subject: [Python-Dev] Generalizing "in" to pairs of sequences
Message-ID: <200108260911.f7Q9BkD30038@snark.thyrsus.com>

This seems to be my week to ask simple, stupid questions.

Is there any good semantic or philosophical reason that these aren't legal?

>>> "ab" in "cabcd"
1
>>> "xy" in "cabcd"
0
>>> (1, 2) in (0, 1, 2, 3)
1
>>> (9, 8) in (0, 1, 2, 3)
0

Proposed -- the following definition of "in" as a binary operation on
sequence types: a is "in" b if a occurs as a subsequence of b.

I'd like to see Python support this mainly to get rid of the following
common but ugly idiom:

if b.find(a) > -1:
    # Do one thing
else:
    # Do another

It's really easy to forget and miscode the guard "if b.find(a):". 
By contrast, see how much clearer this is:

if a in b:
    # Do one thing
else:
    # Do another

Thoughts?
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

I don't like the idea that the police department seems bent on keeping
a pool of unarmed victims available for the predations of the criminal
class.
         -- David Mohler, 1989, on being denied a carry permit in NYC


From ping@lfw.org  Sun Aug 26 10:25:31 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Sun, 26 Aug 2001 02:25:31 -0700 (PDT)
Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences
In-Reply-To: <200108260911.f7Q9BkD30038@snark.thyrsus.com>
Message-ID: <Pine.LNX.4.32.0108260222150.8123-100000@ziggy.localdomain.fake>

On Sun, 26 Aug 2001, Eric S. Raymond wrote:
> This seems to be my week to ask simple, stupid questions.
>
> Is there any good semantic or philosophical reason that these aren't legal?

Yes.

> >>> "ab" in "cabcd"
> 1
> >>> "xy" in "cabcd"
> 0
> >>> (1, 2) in (0, 1, 2, 3)
> 1
> >>> (9, 8) in (0, 1, 2, 3)
> 0

The current meaning of "in" is:

    Given a sequence b, a is "in" b if a is an element of b.

Your "subsequence" interpretation would conflict with this meaning.
Here's current behaviour:

    >>> "ab" in ("ab", "cd")
    1
    >>> "ab" in ("a", "b", "c", "d")
    0
    >>> (1, 2) in ((0, 1), (1, 2), (2, 3))
    1
    >>> (1, 2) in (0, 1, 2, 3)
    0

"in" cannot have both meanings.


-- ?!ng

"Computers are useless.  They can only give you answers."
    -- Pablo Picasso


From ping@lfw.org  Sun Aug 26 10:39:55 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Sun, 26 Aug 2001 02:39:55 -0700 (PDT)
Subject: [Python-Dev] Re: Negative index in insert()
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEBMLIAA.tim.one@home.com>
Message-ID: <Pine.LNX.4.32.0108260226480.8123-100000@ziggy.localdomain.fake>

Eric S. Raymond wrote:
> >>> a = [1, 2, 3, 4]
[...]
> I expected that a.insert(-2, 0) would yield [1, 2, 0, 3, 4].  It was a
> rude shock to discover that
>
> >>> a
> [0, 1, 2, 3, 4]

If this is fixed, i think we should do a complete job of it and
make sure exceptions are raised when the index is out of range.

Notice (in current Python):

    >>> a = [1, 2, 3, 4]
    >>> a.insert(999, 0)
    >>> a
    [1, 2, 3, 4, 0]
    >>> a.insert(-999, 0)
    >>> a
    [0, 1, 2, 3, 4, 0]

So what's happening is that Python is silently clipping the
index to [0..n].  To bring this in line with indexing behaviour,
it should not only interpret values in the range [-n..n], but also
raise IndexError when the value is outside of [-n..n].  (Compare
to pop() -- the behaviour i'm suggesting is exactly what pop()
does, except that the range for pop() is [-n..n-1].)

This has the potential for breaking code, and so the transition
should probably be done with a minor-version's worth of warning,
but i think if we're going to change it we should make it right.

When this is done, i believe we'll be down to exactly two ways of
handling sequence indices --

    (a) x and y are silently clipped to [-n..n-1]
        *only* in an [x:y]-style slice

    (b) in all other cases, an IndexError is raised if the
        index is out of range

and these two cases are easy to separate and explain, and all
will be consistent in its interpretation of negative numbers.
Such simplicity would be nice to have.


-- ?!ng

"Computers are useless.  They can only give you answers."
    -- Pablo Picasso


From ping@lfw.org  Sun Aug 26 11:43:06 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Sun, 26 Aug 2001 03:43:06 -0700 (PDT)
Subject: [Python-Dev] Re: Negative index in insert()
Message-ID: <Pine.LNX.4.32.0108260342560.8123-100000@ziggy.localdomain.fake>

On Sun, 26 Aug 2001, Ka-Ping Yee wrote:
>     (a) x and y are silently clipped to [-n..n-1]
>         *only* in an [x:y]-style slice

Sorry -- of course i meant [-n..n] here.


-- ?!ng




From esr@thyrsus.com  Sun Aug 26 11:54:37 2001
From: esr@thyrsus.com (Eric S. Raymond)
Date: Sun, 26 Aug 2001 06:54:37 -0400
Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences
In-Reply-To: <Pine.LNX.4.32.0108260222150.8123-100000@ziggy.localdomain.fake>; from ping@lfw.org on Sun, Aug 26, 2001 at 02:25:31AM -0700
References: <200108260911.f7Q9BkD30038@snark.thyrsus.com> <Pine.LNX.4.32.0108260222150.8123-100000@ziggy.localdomain.fake>
Message-ID: <20010826065437.A8269@thyrsus.com>

Ka-Ping Yee <ping@lfw.org>:
> Your "subsequence" interpretation would conflict with this meaning.
> Here's current behaviour:
> 
>     >>> "ab" in ("ab", "cd")
>     1
>     >>> "ab" in ("a", "b", "c", "d")
>     0
>     >>> (1, 2) in ((0, 1), (1, 2), (2, 3))
>     1
>     >>> (1, 2) in (0, 1, 2, 3)
>     0
> 
> "in" cannot have both meanings.

You're right.  But since a string can't be an element of a string, the
case I'm really interested in would still work.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

"Both oligarch and tyrant mistrust the people, 
and therefore deprive them of arms."
	--Aristotle


From ping@lfw.org  Sun Aug 26 12:01:01 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Sun, 26 Aug 2001 04:01:01 -0700 (PDT)
Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences
In-Reply-To: <20010826065437.A8269@thyrsus.com>
Message-ID: <Pine.LNX.4.32.0108260355580.8123-100000@ziggy.localdomain.fake>

On Sun, 26 Aug 2001, Eric S. Raymond wrote:
> You're right.  But since a string can't be an element of a string, the
> case I'm really interested in would still work.

It's arguable.  A character can indeed be an element of a string, and
in Python characters are one-character strings.  So this violates

    x in s   <=>  there exists i such that s[i] == x

You could argue that strings should have special behaviour for "in"
to support the common case of finding a substring; then you would have
to argue on grounds of "practicality beats purity".

I might support the argument on grounds of practicality beating purity,
but i'd have to think carefully about violating the above definition.

I can see that "substring in string" would be very convenient, but then

    if char in '0123456789': ...

would suddenly have a very different meaning -- it would succeed for
char = '12'.


-- ?!ng



From skip@pobox.com (Skip Montanaro)  Sun Aug 26 17:36:52 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Sun, 26 Aug 2001 11:36:52 -0500
Subject: [Python-Dev] can anonymous bug reports/patches be squelched?
Message-ID: <15241.9636.961594.597780@beluga.mojam.com>

I noticed a bsddbmodule report today and checked it out.  An anonymous
submitter had reported a bug with bsddbmodule.c.  Martin responded with "try
--without-libdb" and asked the person to report whether that helped.  Two
weeks later, after no response from the submitter (how would he have seen
it?), Jeremy closed the bug.  

That was the right thing to do under the circumstances, but I would if
there's some way to not allow anonymous bug reports and patches.  It seems
almost impossible to actually get a response back to the submitter in such
cases, so I don't see how you can have a useful dialog about the
report/patch. 

Skip



From fdrake@acm.org  Sun Aug 26 17:43:23 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sun, 26 Aug 2001 12:43:23 -0400
Subject: [Python-Dev] can anonymous bug reports/patches be squelched?
In-Reply-To: <15241.9636.961594.597780@beluga.mojam.com>
References: <15241.9636.961594.597780@beluga.mojam.com>
Message-ID: <15241.10027.588676.942551@grendel.digicool.com>

Skip Montanaro writes:
 > That was the right thing to do under the circumstances, but I would if
 > there's some way to not allow anonymous bug reports and patches.  It seems
 > almost impossible to actually get a response back to the submitter in such
 > cases, so I don't see how you can have a useful dialog about the
 > report/patch. 

  Anonymous means the submitter was not logged in, but appearantly
they have an option to leave an email address (which is not shown,
which is fine).  I don't think there's a requirement for an email
address, though, and I don't see any way for us to determine if they
left one.
  If there were a way to *require* an email address, or at least
determine if there was one, I'd be satisfied.  Not allowing anonymous
reports at all would not be very good; many of them have proved
useful (it doesn't matter who reports a typo in the docs, just that it
gets assigned to me).


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From mwh@python.net  Sun Aug 26 19:41:39 2001
From: mwh@python.net (Michael Hudson)
Date: 26 Aug 2001 14:41:39 -0400
Subject: [Python-Dev] PEP 264
In-Reply-To: "Tim Peters"'s message of "Fri, 24 Aug 2001 18:41:05 -0400"
References: <LNBBLJKPBEHFEDALKOLCOEOCLHAA.tim.one@home.com>
Message-ID: <2m4rqu4qak.fsf@starship.python.net>

"Tim Peters" <tim.one@home.com> writes:

> [Michael Hudson]
> > I'd just like to note in passing that I considered most of the places
> > the CO_* constants ended up in the last week, and rejected them as too
> > gross.  My idea was that
> >
> > __future__.generators.compiler_flag
> >
> > was to be the *only* (Python) name for CO_GENERATORS_ALLOWED (for
> > example).  Are there problems with this (other than the fact that you
> > have to make sure two sets of magic numbers match up - but you have to
> > edit __future__.py when you add a new future feature anyway...).
> 
> The problem is that nobody is going to *remember* the subtleties.  IIRC,
> almost every bug I tracked down was due to *someone* neglecting to fiddle
> all the places that needed to be fiddled when changing future-flags.  So The
> Rule I implemented was two-fold:
[schnipp]

OK, I see.  In this spirit, wouldn't it be better to define the
PyCF_MASK and PyCF_MASK_OBSOLETE flags in the same place as the CO_*
flags they're made up of?

Cheers,
M.

-- 
  Richard Gabriel was wrong: worse is not better, lying is better.
  Languages and systems succeed in the marketplace to the extent that
  their proponents lie about what they can do.
                                       -- Tim Bradshaw, comp.lang.lisp


From greg@cosc.canterbury.ac.nz  Mon Aug 27 00:20:06 2001
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon, 27 Aug 2001 11:20:06 +1200 (NZST)
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: <2m7kvtcosu.fsf@starship.python.net>
Message-ID: <200108262320.LAA04767@s454.cosc.canterbury.ac.nz>

Michael Hudson <mwh@python.net>:

> Paul Prescod <paulp@ActiveState.com> writes:

> > Damian responded with a great quote: "...If
> > Performance wasn't an issue, I'd probably write Perl 6 in Python or
> > something like it with wonderfully beautiful OO abstractions and it
> > would run like a dog."

At least it would eliminate all the "why is Python
slower than Perl" ng questions.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From skip@pobox.com (Skip Montanaro)  Mon Aug 27 03:01:32 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Sun, 26 Aug 2001 21:01:32 -0500
Subject: [Python-Dev] Where are locals initialized?
Message-ID: <15241.43516.529415.828023@beluga.mojam.com>

I'm probably just too tired to be looking at this right now, but I'm having
no luck figuring out just where local variables are initialized.  In 1.5.2
eval_code2 is called with all the bits and pieces needed to create a frame
and does so.  In 2.1 eval_frame is called with an already initialized frame.
Either way I expected to see a call to PyFrame_LocalsToFast in or around
PyFrame_New, but the places where I see it don't seem like the logical place
(to my misfiring brain anyway).

Any pointers appreciated,

Skip



From barry@zope.com  Mon Aug 27 04:07:30 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Sun, 26 Aug 2001 23:07:30 -0400
Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences
References: <20010826065437.A8269@thyrsus.com>
 <Pine.LNX.4.32.0108260355580.8123-100000@ziggy.localdomain.fake>
Message-ID: <15241.47474.461384.774856@anthem.wooz.org>

    ESR> You're right.  But since a string can't be an element of a
    ESR> string, the case I'm really interested in would still work.

    KY> It's arguable.  A character can indeed be an element of a
    KY> string, and in Python characters are one-character strings.
    KY> So this violates

    KY>     x in s <=> there exists i such that s[i] == x

    KY> You could argue that strings should have special behaviour for
    KY> "in" to support the common case of finding a substring; then
    KY> you would have to argue on grounds of "practicality beats
    KY> purity".

And against TOOWTDI:

>>> 'cabcd'.find('ab') >= 0
1
>>> 'cabcd'.find('xx') >= 0
0

-Barry


From barry@zope.com  Mon Aug 27 04:08:33 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Sun, 26 Aug 2001 23:08:33 -0400
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects stringobject.c,2.124,2.125
References: <E15aLkW-0003Zp-00@usw-pr-cvs1.sourceforge.net>
 <20010825231735.B2111@lyra.org>
Message-ID: <15241.47537.351411.119384@anthem.wooz.org>

>>>>> "GS" == Greg Stein <gstein@lyra.org> writes:

    GS> For safety/propriety, you should call va_end() on the vargs
    GS> variable.

Good catch!  I meant to add that after noticing the original code
didn't contain it, and after double checking with the Linux manpage
for va_start.  I touched so much other code I forgot to go back and
add that call.  Will do so now...

    GS> You could use: s - PyString_AS_STRING(string)

Good call.  The original code had the former, but you're right that
this is better.

Thanks,
-Barry


From ask@valueclick.com  Mon Aug 27 10:51:27 2001
From: ask@valueclick.com (Ask Bjoern Hansen)
Date: Mon, 27 Aug 2001 02:51:27 -0700 (PDT)
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: <15236.3994.507401.287832@beluga.mojam.com>
Message-ID: <Pine.LNX.4.33.0108270249290.11087-100000@impatience.valueclick.com>

On Wed, 22 Aug 2001, Skip Montanaro wrote:

[...]
> libraries (and thus use by a critical mass) not in the core that much
> higher.  There are how many Python Server Pages like things?

#$&$&%.  Don't worry; even if we only look at those on CPAN we have
more templating systems in Perl than you would ever want to count.

> They probably all have some very nifty features, but tend to be
> used by rather small pockets of people, at least relative to the
> overall segment of the Python community doing webish stuff.
> I'm not entirely sure why that's the case, but I suspect it has
> to do in part with the barrier to discovering, downloading, and
> installing these packages.

Nah, some things just have to be implemented and reimplemented and
reimplemented and reimplemented in slightly different ways.

:-)


 - ask

-- 
ask bjoern hansen, http://ask.netcetera.dk/         !try; do();
more than a billion impressions per week, http://valueclick.com



From alex_c@MIT.EDU  Mon Aug 27 12:41:26 2001
From: alex_c@MIT.EDU (Alex Coventry)
Date: 27 Aug 2001 07:41:26 -0400
Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences
In-Reply-To: barry@zope.com's message of "Sun, 26 Aug 2001 23:07:30 -0400"
References: <15241.47474.461384.774856@anthem.wooz.org>
Message-ID: <etd1ylxn315.fsf@lola-granola.mit.edu>

Sometimes I find myself wishing that strings had a method "substringp"
that would do basically what Eric wants, like

>>> 'cabcd'.substringp('ca')
1
>>> 'cabcd'.substringp('xx')
0

substringp would probably be the wrong name for it, though.

Alex.


From barry@zope.com  Mon Aug 27 14:51:03 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 27 Aug 2001 09:51:03 -0400
Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences
References: <15241.47474.461384.774856@anthem.wooz.org>
 <etd1ylxn315.fsf@lola-granola.mit.edu>
Message-ID: <15242.20551.521357.800605@anthem.wooz.org>

>>>>> "AC" == Alex Coventry <alex_c@MIT.EDU> writes:

    AC> Sometimes I find myself wishing that strings had a method
    AC> "substringp" that would do basically what Eric wants, like

    >> 'cabcd'.substringp('ca')
    AC> 1
    >> 'cabcd'.substringp('xx')
    AC> 0

    AC> substringp would probably be the wrong name for it, though.

Hmm, "hassubstring()"?

not-implying-a-vote-on-adding-such-a-method-ly y'rs,
-Barry


From jeremy@zope.com  Mon Aug 27 15:27:17 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Mon, 27 Aug 2001 10:27:17 -0400 (EDT)
Subject: [Python-Dev] Where are locals initialized?
In-Reply-To: <15241.43516.529415.828023@beluga.mojam.com>
References: <15241.43516.529415.828023@beluga.mojam.com>
Message-ID: <15242.22725.241556.803488@slothrop.digicool.com>

It's in PyEval_EvalCodeEx().  Look for the SETLOCAL() macro.  The code
in lines 2305 to 2494 is all dealing with function arguments.

Jeremy



From nas@python.ca  Mon Aug 27 17:23:49 2001
From: nas@python.ca (Neil Schemenauer)
Date: Mon, 27 Aug 2001 09:23:49 -0700
Subject: [Python-Dev] python generator design bug?
Message-ID: <20010827092349.A23538@glacier.arctrix.com>

Eric Kidd thinks so:

    http://www.advogato.org/person/emk/

Here's an excerpt:

    [...] there's a subtle bug in the Python design. Consider tree
    traversal:

    def inorder(t):
        if (t.left != None):
            for (node in inorder(t.left)):
                yield node
        yield t
        if (t.right != None):
            for (node in inorder(t.right)):
                yield node

    If you study this carefully, you'll see that (unless the optimizer
    intervenes), Python has turned a perfectly good O(N) tree traversal
    into an O(N log N) traversal.

Thoughts?

  Neil


From guido@python.org  Mon Aug 27 17:38:15 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 27 Aug 2001 12:38:15 -0400
Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences
In-Reply-To: Your message of "Sun, 26 Aug 2001 04:01:01 PDT."
 <Pine.LNX.4.32.0108260355580.8123-100000@ziggy.localdomain.fake>
References: <Pine.LNX.4.32.0108260355580.8123-100000@ziggy.localdomain.fake>
Message-ID: <200108271638.MAA25363@cj20424-a.reston1.va.home.com>

> On Sun, 26 Aug 2001, Eric S. Raymond wrote:
> > You're right.  But since a string can't be an element of a string, the
> > case I'm really interested in would still work.
> 
> It's arguable.  A character can indeed be an element of a string, and
> in Python characters are one-character strings.  So this violates
> 
>     x in s   <=>  there exists i such that s[i] == x
> 
> You could argue that strings should have special behaviour for "in"
> to support the common case of finding a substring; then you would have
> to argue on grounds of "practicality beats purity".
> 
> I might support the argument on grounds of practicality beating purity,
> but i'd have to think carefully about violating the above definition.
> 
> I can see that "substring in string" would be very convenient, but then
> 
>     if char in '0123456789': ...
> 
> would suddenly have a very different meaning -- it would succeed for
> char = '12'.

We *could* allow this for strings, but not for other sequences.
For strings, type(s[0]) == type(s), and this avoids the ambiguity.
The fact that currently ``noncharacter in string'' raises an exception
makes it relatively painess to add semantics for it.  But I'll have to
think about why I ruled it out originall.  I'll track down Eric here
at LinuxWorld and discuss it with him, if we find time.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From esr@thyrsus.com  Mon Aug 27 19:01:34 2001
From: esr@thyrsus.com (Eric S. Raymond)
Date: Mon, 27 Aug 2001 14:01:34 -0400
Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences
In-Reply-To: <15242.20551.521357.800605@anthem.wooz.org>; from barry@zope.com on Mon, Aug 27, 2001 at 09:51:03AM -0400
References: <15241.47474.461384.774856@anthem.wooz.org> <etd1ylxn315.fsf@lola-granola.mit.edu> <15242.20551.521357.800605@anthem.wooz.org>
Message-ID: <20010827140134.A7687@thyrsus.com>

Barry A. Warsaw <barry@zope.com>:
>     AC> substringp would probably be the wrong name for it, though.
> 
> Hmm, "hassubstring()"?

How about "contains"?  E.g.

if a.contains(b):
	# do stuff
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

(Those) who are trying to read the Second Amendment out of the Constitution by
claiming it's not an individual right (are) courting disaster by encouraging
others to use the same means to eliminate portions of the Constitution they
don't like.
	-- Alan Dershowitz, Harvard Law School


From esr@thyrsus.com  Mon Aug 27 19:08:10 2001
From: esr@thyrsus.com (Eric S. Raymond)
Date: Mon, 27 Aug 2001 14:08:10 -0400
Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences
In-Reply-To: <200108271638.MAA25363@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Aug 27, 2001 at 12:38:15PM -0400
References: <Pine.LNX.4.32.0108260355580.8123-100000@ziggy.localdomain.fake> <200108271638.MAA25363@cj20424-a.reston1.va.home.com>
Message-ID: <20010827140810.B7687@thyrsus.com>

Guido van Rossum <guido@python.org>:
> We *could* allow this for strings, but not for other sequences.
> For strings, type(s[0]) == type(s), and this avoids the ambiguity.
> The fact that currently ``noncharacter in string'' raises an exception
> makes it relatively painess to add semantics for it.  But I'll have to
> think about why I ruled it out originall.  I'll track down Eric here
> at LinuxWorld and discuss it with him, if we find time.

Alas, due to a massive screwup by IDG that I only discovered two days
ago, I won't be at LinuxWorld this time.  Which bites; I was really looking
forward to going.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

"The bearing of arms is the essential medium through which the
individual asserts both his social power and his participation in
politics as a responsible moral being..."
        -- J.G.A. Pocock, describing the beliefs of the founders of the U.S.


From James_Althoff@i2.com  Mon Aug 27 19:17:25 2001
From: James_Althoff@i2.com (James_Althoff@i2.com)
Date: Mon, 27 Aug 2001 11:17:25 -0700
Subject: [Python-Dev] copy, len and the like as 'object' methods?
Message-ID: <OF860D700A.8F9D4B04-ON88256AB5.006128F5@i2.com>

Tim Peters wrote:
>Say what?!  What kind of stupid-ass language spells the len() function as
if
>it were a method?  That's carrying consistency to an irrational extreme --

Well, only those stupid-ass, irrational, and extremist languages that
include classes and methods as an integral and uniform part of the language
from the very beginning, I guess.

The "stupid-ass, irrational, extremist" side is merely saying that

mylist.append(x)
mylist.length()
mylist.extend(y)
mylist.length()

is more sensible (easier for new-to-python programmers to discover, more
consistent, more mainstream)

than

mylist.append(x)
len(mylist)
mylist.extend(y)
len(mylist)

and that len(mylist) as syntactic sugar for mylist.__len__() is sugar that
adds no value compared to just doing the straightforward thing of writing
mylist.length() or mylist.len().

But you all clearly like spelling it len(mylist) and it's already there, so
... whatever.

Jim



From tim.one@home.com  Mon Aug 27 20:03:00 2001
From: tim.one@home.com (Tim Peters)
Date: Mon, 27 Aug 2001 15:03:00 -0400
Subject: [Python-Dev] python generator design bug?
In-Reply-To: <20010827092349.A23538@glacier.arctrix.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEFCLIAA.tim.one@home.com>

[Neil Schemenauer]
> Eric Kidd thinks so:
>
>     http://www.advogato.org/person/emk/
>
> Here's an excerpt:
>
>     [...] there's a subtle bug in the Python design. Consider tree
>     traversal:
>
>     def inorder(t):
>         if (t.left != None):
>             for (node in inorder(t.left)):
>                 yield node
>         yield t
>         if (t.right != None):
>             for (node in inorder(t.right)):
>                 yield node
>
>     If you study this carefully, you'll see that (unless the optimizer
>     intervenes), Python has turned a perfectly good O(N) tree traversal
>     into an O(N log N) traversal.
>
> Thoughts?

Replied to this before on c.l.py (but to David Eppstein):

    http://aspn.activestate.com/ASPN/Mail/Message/624273

enumerate-a-btree-and-"log(n)"-is-1-ly y'rs  - tim



From tim.one@home.com  Mon Aug 27 21:00:36 2001
From: tim.one@home.com (Tim Peters)
Date: Mon, 27 Aug 2001 16:00:36 -0400
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: <OF860D700A.8F9D4B04-ON88256AB5.006128F5@i2.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEFHLIAA.tim.one@home.com>

[Tim, giving his "<wink>" key a rest]
> Say what?!  What kind of stupid-ass language spells the len() function as
> if it were a method?  That's carrying consistency to an irrational
> extreme --

[James_Althoff@i2.com]
> Well, only those stupid-ass, irrational, and extremist languages that
> include classes and methods as an integral and uniform part of
> the language from the very beginning, I guess.

I was more than half joking, but not entirely.

> The "stupid-ass, irrational, extremist" side is merely saying that
>
> mylist.append(x)
> mylist.length()
> mylist.extend(y)
> mylist.length()
>
> is more sensible (easier for new-to-python programmers to discover,
> more consistent, more mainstream)

Of course that's what they're saying, although "mainstream" is debatable:
pure-OO languages aren't exactly mainstream, and Python isn't a pure-OO
language.  It doesn't follow that Python is mainstream, but neither does it
follow that ".length()" isn't an eyesore <wink>.

> But you all clearly like spelling it len(mylist) and it's already
> there, so
> ... whatever.

I wouldn't want to write math.pi.sin() either -- functional notation for
pure (referentially transparent; side-effect free) functions has a long and
fine tradition, in and out of computer languages, and I *think* "get the
length of some list", not "list, of all the things I may ask you for, in
this case I want your length".  It simply reads better to me as a function.

Guido said he'd do it the other way had he to do it over, so I'll have to
rest on "it's already there".



From jack@oratrix.nl  Mon Aug 27 21:09:28 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Mon, 27 Aug 2001 22:09:28 +0200
Subject: [Python-Dev] CVS tags, unix Python and MacPython
Message-ID: <20010827200933.7A2FCB5E00@oratrix.oratrix.nl>

Folks,
I need some good ideas on how to organize tag names for MacPython
releases.

When the MacPython repository was still separate things were fairly
simple: when I did a MacPython release what ended up in the source
tree was simply the (correctly tagged) Mac subtree from that
repository and whatever happened to be current in the main repository
(usually a couple of weeks newer than the corresponding unix release),
with one or two files modified.

When the repositories were merged I switched over to using the same
tags as the unix release used. This has sort-of worked for the 2.1.1
release, although with some grumblings of the people doing the unix
2.1.1 release.

Things will get worse with the 2.2a2 release, as there are a couple of
things that I definitely want in there that aren't finished yet, so
it'll be at least another week until I'm done, and moreover I'll have
to be making a few (minor) mods to files outside the Mac subtree.

This problem will continue for as long as MacPython stays alive
(unless everyone is willing to delay all distributions until I'm
done with MacPython too:-).

I can see 3 possible solutions, I'd like to hear what people think is
the best course of action (and what the powers-that-be sanction):
- I can move the "r22a2" tag for the specific files. I don't think I
like this solution (despite its simplicity) because the files in
question are used by the OSX unix build too.
- I can add an "r22a2-mac" tag all over the tree.
- I can add an "r22a2" tag to the Mac subtree only, check out the
whole r22a2 tagged Python tree and manually check out those files for
which I need newer versions.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.cwi.nl/~jack        | ++++ see http://www.xs4all.nl/~tank/ ++++


From tim.one@home.com  Mon Aug 27 23:10:53 2001
From: tim.one@home.com (Tim Peters)
Date: Mon, 27 Aug 2001 18:10:53 -0400
Subject: [Python-Dev] PEP 264
In-Reply-To: <2m4rqu4qak.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEGELIAA.tim.one@home.com>

[Michael Hudson]
> OK, I see.  In this spirit, wouldn't it be better to define the
> PyCF_MASK and PyCF_MASK_OBSOLETE flags in the same place as the CO_*
> flags they're made up of?

-0.  I don't see that it matters.  Wouldn't object if someone else wanted
to, but they appear to make most sense where they are, next to the decl of
the PyCompilerFlags struct (the "CF" in PyCF_MASK refers to Compiler Flags,
not Code Flags -- these bits are doing double-duty in 2.2a2).



From guido@python.org  Tue Aug 28 18:03:08 2001
From: guido@python.org (Guido van Rossum)
Date: Tue, 28 Aug 2001 13:03:08 -0400
Subject: [Python-Dev] CVS tags, unix Python and MacPython
In-Reply-To: Your message of "Mon, 27 Aug 2001 22:09:28 +0200."
 <20010827200933.7A2FCB5E00@oratrix.oratrix.nl>
References: <20010827200933.7A2FCB5E00@oratrix.oratrix.nl>
Message-ID: <200108281703.NAA01931@cj20424-a.reston1.va.home.com>

> - I can move the "r22a2" tag for the specific files. I don't think I
> like this solution (despite its simplicity) because the files in
> question are used by the OSX unix build too.

That would be bad.

> - I can add an "r22a2-mac" tag all over the tree.

This sounds good to me.

> - I can add an "r22a2" tag to the Mac subtree only, check out the
> whole r22a2 tagged Python tree and manually check out those files for
> which I need newer versions.

It's up to you.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@scottb.demon.co.uk  Tue Aug 28 19:41:53 2001
From: barry@scottb.demon.co.uk (Barry Scott)
Date: Tue, 28 Aug 2001 19:41:53 +0100
Subject: [Python-Dev] copy, len and the like as 'object' methods?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEFHLIAA.tim.one@home.com>
Message-ID: <000601c12ff1$1b0571d0$060210ac@private>

> Tim said:
>                                                       I *think* "get the
> length of some list", not "list, of all the things I may ask you for, in
> this case I want your length".  It simply reads better to me as a function.

	length of list and type       len(list)
	read the file and type        file.read()

		maybe thinking...

	list how long are you         list.length()
	file I want to read you       file.read()

		BArry



From tim.one@home.com  Tue Aug 28 20:01:47 2001
From: tim.one@home.com (Tim Peters)
Date: Tue, 28 Aug 2001 15:01:47 -0400
Subject: [Python-Dev] FW: PyInline Released: Put C source code directly "inline" with your Python!
Message-ID: <LNBBLJKPBEHFEDALKOLCEEIOLIAA.tim.one@home.com>

FYI.  I haven't checked it out.

-----Original Message-----
From: python-list-admin@python.org
[mailto:python-list-admin@python.org]On Behalf Of Ken Simpson
Sent: Tuesday, August 28, 2001 2:06 PM
To: python-list@python.org
Subject: PyInline Released: Put C source code directly "inline" with
your Python!


The PyInline module allows you to put source code from other
programming languages directly "inline" in a Python script or
module. The code is automatically compiled as needed and then loaded
for immediate access from Python.

In short, PyInline lets you do things like this:

from PyInline import C
import __main__

# Let's put some C code "inline" with our Python:

x = C.Builder(code="""

#include <stdio.h>
void ja(char *str) {
  printf("Just another %s hacker\n", str);
}

""", targetmodule=__main__) # Add C methods to the __main__ module.

x.build()
ja("Inline")
---

PyInline is the Python equivalent of
Brian Ingerson's Inline module for Perl (http://inline.perl.org),
which was awarded the "Best Module" award at this year's Perl
Conference.

More information about PyInline can be found at the PyInline home page
on
SourceForge:

   http://pyinline.sourceforge.net

To discuss PyInline further, please join the pyinline-discuss mailing
list at
http://lists.sourceforge.net/lists/listinfo/pyinline-discuss
-- 
http://mail.python.org/mailman/listinfo/python-list



From barry@zope.com  Tue Aug 28 21:52:30 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Tue, 28 Aug 2001 16:52:30 -0400
Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences
References: <15241.47474.461384.774856@anthem.wooz.org>
 <etd1ylxn315.fsf@lola-granola.mit.edu>
 <15242.20551.521357.800605@anthem.wooz.org>
 <20010827140134.A7687@thyrsus.com>
Message-ID: <15244.1166.351735.428963@anthem.wooz.org>

>>>>> "ESR" == Eric S Raymond <esr@thyrsus.com> writes:

    >>  substringp would probably be the wrong name for it, though.
    >>  Hmm, "hassubstring()"?

    ESR> How about "contains"?  E.g.

    | if a.contains(b):
    | 	# do stuff

Heh, and from there it's a small leap to a.__contains__(b), a.k.a.
"b in a".

Sneaky Eric, very sneaky. :)

-Barry


From esr@thyrsus.com  Tue Aug 28 22:20:15 2001
From: esr@thyrsus.com (Eric S. Raymond)
Date: Tue, 28 Aug 2001 17:20:15 -0400
Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences
In-Reply-To: <15244.1166.351735.428963@anthem.wooz.org>; from barry@zope.com on Tue, Aug 28, 2001 at 04:52:30PM -0400
References: <15241.47474.461384.774856@anthem.wooz.org> <etd1ylxn315.fsf@lola-granola.mit.edu> <15242.20551.521357.800605@anthem.wooz.org> <20010827140134.A7687@thyrsus.com> <15244.1166.351735.428963@anthem.wooz.org>
Message-ID: <20010828172015.C7227@thyrsus.com>

Barry A. Warsaw <barry@zope.com>:
>     ESR> How about "contains"?  E.g.
> 
>     | if a.contains(b):
>     | 	# do stuff
> 
> Heh, and from there it's a small leap to a.__contains__(b), a.k.a.
> "b in a".
> 
> Sneaky Eric, very sneaky. :)

You know, that didn't consciously occur to me.  Which only goes to show
that my unconscious mind is even *more* evil...BWAHAHAHAHAA!
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

Society in every state is a blessing, but government even in its best
state is but a necessary evil; in its worst state an intolerable one;
for when we suffer, or are exposed to the same miseries *by a
government*, which we might expect in a country *without government*,
our calamities is heightened by reflecting that we furnish the means
by which we suffer."
	-- Thomas Paine


From m.favas@per.dem.csiro.au  Wed Aug 29 01:59:29 2001
From: m.favas@per.dem.csiro.au (Mark Favas)
Date: Wed, 29 Aug 2001 08:59:29 +0800
Subject: [Python-Dev] Change from "config.h" to "pyconfig.h" may be a ticking #include bomb
Message-ID: <3B8C3E71.A9A4C0BA@per.dem.csiro.au>

Some little while ago, Python 2.2's "configure" started to produce
pyconfig.h rather than config.h (a good thing). "make install" installs
this file into /usr/local/include/python2.2 where it can be used by
extensions, etc. However, any config.h file produced and installed
before the change to pyconfig.h remains in /usr/local/include/python2.2.
I've also been building the XML extension. Only when building this
extension on a brand new Python 2.2 platform did I find that XML's
extension/expat/lib/xmlparse.c #include's <config.h> rather than
<pyconfig.h> (probably not the only extension that does this). Old 2.2
installations found the old config.h file and compiled without error,
although they should have been using pyconfig.h. To pick up this sort of
bug, perhaps "make install" should attempt to remove config.h if it
exists in the target include directory?
 
-- 
Mark Favas  -   m.favas@per.dem.csiro.au
CSIRO, Private Bag No 5, Wembley, Western Australia 6913, AUSTRALIA


From jack@oratrix.nl  Wed Aug 29 13:29:51 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Wed, 29 Aug 2001 14:29:51 +0200
Subject: [Python-Dev] test_threaded_import
Message-ID: <20010829122951.A22DC303181@snelboot.oratrix.nl>

I spent the last couple of days hunting for the bug in MacPython threaded 
import, but it turned out the bug is in the test harness. (I did find another 
serious bug in the process, so the time wasn't wasted, but still)

The problem is that test_threaded_import cannot run inside an "import". It 
knows this, and so does regrtest, so they run the test after the import by 
calling test_threaded_import.test_main(), which does the actual test.

All good and fine if you run regrtest as a main script from the commandline, 
but of course it breaks if you run autotest, or if you "import regrtest" in an 
interactive window, because in these cases the import in the main thread 
already holds the lock.

I can simply add an "if not verbose", as in test_socketserver, but that isn't 
really the problem. Does anyone know a better solution?
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.cwi.nl/~jack        | ++++ see http://www.xs4all.nl/~tank/ ++++




From tim.one@home.com  Wed Aug 29 20:03:12 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 29 Aug 2001 15:03:12 -0400
Subject: [Python-Dev] test_threaded_import
In-Reply-To: <20010829122951.A22DC303181@snelboot.oratrix.nl>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEMHLIAA.tim.one@home.com>

[Jack Jansen]
> I spent the last couple of days hunting for the bug in MacPython
> threaded import, but it turned out the bug is in the test harness.
> (I did find another serious bug in the process, so the time wasn't
> wasted, but still)

Hmm!  You originally said this test hung "about 30% of the time", and didn't
respond when I asked how you ran it.  Given the way you are running it, it
should have hung 100% of the time.  So something here is still muddy.

> The problem is that test_threaded_import cannot run inside an
> "import". It knows this, and so does regrtest, so they run the test
> after the import by calling test_threaded_import.test_main(), which
> does the actual test.
>
> All good and fine if you run regrtest as a main script from the
> commandline,

Or if you run test_threaded_import directly (leaving regrtest out of it).

> but of course it breaks if you run autotest,

So it does!  I've never done that.

> or if you "import regrtest" in an interactive window,

Are you sure about that?  Works fine for me, and should work fine.

> because in these cases the import in the main thread
> already holds the lock.

It shouldn't hurt anything to import regrtest -- it's not until
regrtest.main() gets invoked that a problem can occur.

> I can simply add an "if not verbose", as in test_socketserver,
> but that isn't really the problem. Does anyone know a better
> solution?

I don't like that either -- test_socketserver almost never gets run on my
box simply because it's an oddball.  For the same reason I'm sure the
test_exceptions part of test_math never gets run on anyone's box.

Assuming you were suffering an illusion when saying "import regrest in an
interactive window" doesn't work, I'd be content to special-case the snot
out of autotest, like so:

Index: test_threaded_import.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test_threaded_import.py,v
retrieving revision 1.2
diff -c -r1.2 test_threaded_import.py
*** test_threaded_import.py     2001/05/22 18:28:25     1.2
--- test_threaded_import.py     2001/08/29 18:53:38
***************
*** 6,12 ****
  # randrange, and then Python hangs.

  import thread
! from test_support import verbose

  critical_section = thread.allocate_lock()
  done = thread.allocate_lock()
--- 6,12 ----
  # randrange, and then Python hangs.

  import thread
! from test_support import verbose, TestSkipped

  critical_section = thread.allocate_lock()
  done = thread.allocate_lock()
***************
*** 32,37 ****
--- 32,41 ----

  def test_main():        # magic name!  see above
      global N, done
+     import sys
+     for modname in sys.modules:
+         if modname.find('autotest') >= 0:
+             raise TestSkipped("can't run from autotest")
      done.acquire()
      for N in (20, 50) * 3:
          if verbose:



From jack@oratrix.nl  Wed Aug 29 21:16:43 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Wed, 29 Aug 2001 22:16:43 +0200
Subject: [Python-Dev] test_threaded_import
In-Reply-To: Message by "Tim Peters" <tim.one@home.com> ,
 Wed, 29 Aug 2001 15:03:12 -0400 , <LNBBLJKPBEHFEDALKOLCIEMHLIAA.tim.one@home.com>
Message-ID: <20010829201648.9ED5115F546@oratrix.oratrix.nl>

Recently, "Tim Peters" <tim.one@home.com> said:
> Hmm!  You originally said this test hung "about 30% of the time", and didn't
> respond when I asked how you ran it.  Given the way you are running it, it
> should have hung 100% of the time.  So something here is still muddy.

Elementary, my dear Watson. I do a "make install", which runs the
regression tests twice, succeeding both times. Then I do "python" and
"import autotest", at which point it fails, thereby failing
approximately 30% of the time. Of course, not being Sherlock Jansen, I
didn't realise autotest!=regrtest for this:-)

> > or if you "import regrtest" in an interactive window,
> 
> Are you sure about that?  Works fine for me, and should work fine.

Sherlock Peters, you're absolutely right, of course:-)
The regrtest ran afoul of the small stack space for threads in
MacPython (for which I checked in a fix this afternoon). I didn't try
it again when that bug was fixed.

I'll apply your patch, which seems to fix the problem fine for the
time being. Still, there's three issues that I think need to be solved
at some point:
- the fact that you (or some other thread) are holding the import lock
  is one of the few Python state bits that are not open to
  introspection.
- There may be denial of service attacks possible with this if a 16
  year old scriptkiddy can think of a way to exploit it in a
  restricted interpreter.
- The whole idea that a process deadlocked on semaphore locks is
  completely uninterruptible also smells bad.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.cwi.nl/~jack        | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 


From tim.one@home.com  Thu Aug 30 00:42:59 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 29 Aug 2001 19:42:59 -0400
Subject: [Python-Dev] test_threaded_import
In-Reply-To: <20010829201648.9ED5115F546@oratrix.oratrix.nl>
Message-ID: <LNBBLJKPBEHFEDALKOLCOENMLIAA.tim.one@home.com>

[Jack Jansen]
> ...
> Elementary, my dear Watson. I do a "make install", which runs the
> regression tests twice, succeeding both times. Then I do "python" and
> "import autotest", at which point it fails, thereby failing
> approximately 30% of the time. Of course, not being Sherlock Jansen, I
> didn't realise autotest!=regrtest for this:-)

Ah, a signficant digits error.  Had you said "approximately 33% of the
time", or even "approximately 3e1%", I would have guessed instantly <wink>.

> ...
> I'll apply your patch, which seems to fix the problem fine for the
> time being. Still, there's three issues that I think need to be solved
> at some point:
> - the fact that you (or some other thread) are holding the import lock
>   is one of the few Python state bits that are not open to
>   introspection.

The patch below adds introspection, sufficient to avoid special-casing
autotest.  Guido?  I'd upload the patch to SF instead, but, as you've (Jack)
noted, that's a black hole <0.9 wink>.

> - There may be denial of service attacks possible with this if a 16
>   year old scriptkiddy can think of a way to exploit it in a
>   restricted interpreter.

Someone else will have to worry about that; in a world where a 20-character
regexp can take centuries to match, I don't take DOS from a general-purpose
programming language seriously.

> - The whole idea that a process deadlocked on semaphore locks is
>   completely uninterruptible also smells bad.

That's threads for you:  start a thread, risk a deadlock.  There's no
portable way to interrupt a thread waiting on a lock, and, e.g., no *safe*
way at all on Windows short of killing the whole process.
Index: Python/import.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/import.c,v
retrieving revision 2.185
diff -c -r2.185 import.c
*** Python/import.c	2001/08/13 23:05:44	2.185
--- Python/import.c	2001/08/29 23:25:28
***************
*** 138,143 ****
--- 138,151 ----
  static long import_lock_thread = -1;
  static int import_lock_level = 0;

+ static PyObject *
+ imp_lock_held(PyObject *self, PyObject *args)
+ {
+ 	if (!PyArg_ParseTuple(args, ":lock_held"))
+ 		return NULL;
+ 	return PyInt_FromLong(import_lock_thread != -1);
+ }
+
  static void
  lock_import(void)
  {
***************
*** 179,184 ****
--- 187,200 ----
  #define lock_import()
  #define unlock_import()

+ static PyObject *
+ imp_lock_held(PyObject *self, PyObject *args)
+ {
+ 	if (!PyArg_ParseTuple(args, ":lock_held"))
+ 		return NULL;
+ 	return PyInt_FromLong(0);
+ }
+
  #endif

  /* Helper for sys */
***************
*** 2339,2350 ****
--- 2355,2373 ----
  The module name must include the full package name, if any.\
  ";

+ static char doc_lock_held[] = "\
+ lock_held() -> 0 or 1\n\
+ Return 1 if the import lock is currently held.\n\
+ On platforms without threads, return 0.\
+ ";
+
  static PyMethodDef imp_methods[] = {
  	{"find_module",		imp_find_module,	1, doc_find_module},
  	{"get_magic",		imp_get_magic,		1, doc_get_magic},
  	{"get_suffixes",	imp_get_suffixes,	1, doc_get_suffixes},
  	{"load_module",		imp_load_module,	1, doc_load_module},
  	{"new_module",		imp_new_module,		1, doc_new_module},
+ 	{"lock_held",		imp_lock_held,		1, doc_lock_held},
  	/* The rest are obsolete */
  	{"get_frozen_object",	imp_get_frozen_object,	1},
  	{"init_builtin",	imp_init_builtin,	1},



From guido@python.org  Thu Aug 30 16:01:24 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 30 Aug 2001 11:01:24 -0400
Subject: [Python-Dev] canned response added
Message-ID: <200108301501.f7UF1OQ18292@odiug.digicool.com>

At Jack's suggestion, I've used a canned response "Missing upload" to
the Bugs and Patches trackers at SourceForge.  This should save us a
bit of time dealing with this annoying SF bug.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fredrik@pythonware.com  Thu Aug 30 18:15:07 2001
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Thu, 30 Aug 2001 19:15:07 +0200
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Modules _sre.c,2.63,2.64
References: <E15cSwP-0003kc-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <007601c13177$539e0a20$42fb42d5@hagrid>

sjoerd wrote:
> 
> Modified Files:
> _sre.c 
> Log Message:
> Removed unreachable return to silence SGI compiler.
> 
> !     /* shouldn't end up here */
> !     return SRE_ERROR_ILLEGAL;
>   }
>   
> --- 1141,1145 ----
>       }
>   
> !     /* can't end up here */
>   }

I hate stuff like this: that line was there to make sure *I* don't
mess up when developing SRE, not to deal with potentially broken
compilers or misfired electrons.

isn't there any way to tell the SGI compiler to stop whining about
this?

</F>



From tim@zope.com  Thu Aug 30 19:50:15 2001
From: tim@zope.com (Tim Peters)
Date: Thu, 30 Aug 2001 14:50:15 -0400
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Modules _sre.c,2.63,2.64
In-Reply-To: <007601c13177$539e0a20$42fb42d5@hagrid>
Message-ID: <BIEJKCLHCIOIHAGOKOLHKEOFCFAA.tim@zope.com>

>> _sre.c
>> Log Message:
>> Removed unreachable return to silence SGI compiler.
>>
>> !     /* shouldn't end up here */
>> !     return SRE_ERROR_ILLEGAL;
>>   }
>>
>> --- 1141,1145 ----
>>       }
>>
>> !     /* can't end up here */
>>   }

[Fredrik Lundh]
> I hate stuff like this: that line was there to make sure *I* don't
> mess up when developing SRE, not to deal with potentially broken
> compilers or misfired electrons.
>
> isn't there any way to tell the SGI compiler to stop whining about
> this?

I've had better luck x-platform with, e.g.,

    assert(!"can't get here"); /* !string is always false */

In non-debug builds, it vanishes so there's nothing for the compiler to
gripe about; while in debug builds most compilers don't do enough flow
analysis to realize there *is* something to gripe about.

Overall, it would be more useful if SGI compilers in particular produced:

    C00666: Warning: -O may produce buggy code.

after every executable line <wink>.



From skip@pobox.com (Skip Montanaro)  Fri Aug 31 06:51:35 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Fri, 31 Aug 2001 00:51:35 -0500
Subject: [Python-Dev] timeoutsocket.py
Message-ID: <15247.9703.69434.567507@beluga.mojam.com>

I just submitted a feature request to add Tim O'Malley's timeoutsocket.py
module to the Python distribution.  Please have a look at

    http://sourceforge.net/tracker/index.php?func=detail&aid=457114&group_id=5470&atid=105470

and comment if you like.

Skip



From sjoerd.mullender@oratrix.com  Fri Aug 31 10:08:19 2001
From: sjoerd.mullender@oratrix.com (Sjoerd Mullender)
Date: Fri, 31 Aug 2001 11:08:19 +0200
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Modules _sre.c,2.63,2.64
In-Reply-To: Your message of Thu, 30 Aug 2001 14:50:15 -0400.
 <BIEJKCLHCIOIHAGOKOLHKEOFCFAA.tim@zope.com>
References: <BIEJKCLHCIOIHAGOKOLHKEOFCFAA.tim@zope.com>
Message-ID: <20010831090820.9BFC7301382@bireme.oratrix.nl>

Well, there is the possibility to add
	-woff 1110
to the compiler command line.

However, I don't agree that this is whining and I wouldn't like to see
any warnings suppressed like this.  Warnings signal a potential bug,
any so does this one.  If the flow of control did reach this
particular line, the compiler would also give a warning, namely that
the function has both return value and return void (or words to that
effect).

I use the compiler in non-debug mode (-g option, no -O option), and I
do get these warnings, so some compilers do do enough flow control in
non-debug mode.

On Thu, Aug 30 2001 "Tim Peters" wrote:

> >> _sre.c
> >> Log Message:
> >> Removed unreachable return to silence SGI compiler.
> >>
> >> !     /* shouldn't end up here */
> >> !     return SRE_ERROR_ILLEGAL;
> >>   }
> >>
> >> --- 1141,1145 ----
> >>       }
> >>
> >> !     /* can't end up here */
> >>   }
> 
> [Fredrik Lundh]
> > I hate stuff like this: that line was there to make sure *I* don't
> > mess up when developing SRE, not to deal with potentially broken
> > compilers or misfired electrons.
> >
> > isn't there any way to tell the SGI compiler to stop whining about
> > this?
> 
> I've had better luck x-platform with, e.g.,
> 
>     assert(!"can't get here"); /* !string is always false */
> 
> In non-debug builds, it vanishes so there's nothing for the compiler to
> gripe about; while in debug builds most compilers don't do enough flow
> analysis to realize there *is* something to gripe about.
> 
> Overall, it would be more useful if SGI compilers in particular produced:
> 
>     C00666: Warning: -O may produce buggy code.
> 
> after every executable line <wink>.
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> 

-- Sjoerd Mullender <sjoerd.mullender@oratrix.com>


From gward@python.net  Fri Aug 31 14:15:07 2001
From: gward@python.net (Greg Ward)
Date: Fri, 31 Aug 2001 09:15:07 -0400
Subject: [Python-Dev] Re: Negative index in insert()
In-Reply-To: <Pine.LNX.4.32.0108260226480.8123-100000@ziggy.localdomain.fake>; from ping@lfw.org on Sun, Aug 26, 2001 at 02:39:55AM -0700
References: <LNBBLJKPBEHFEDALKOLCOEBMLIAA.tim.one@home.com> <Pine.LNX.4.32.0108260226480.8123-100000@ziggy.localdomain.fake>
Message-ID: <20010831091507.A15968@gerg.ca>

On 26 August 2001, Ka-Ping Yee said:
> If this is fixed, i think we should do a complete job of it and
> make sure exceptions are raised when the index is out of range.
[...]
> This has the potential for breaking code, and so the transition
> should probably be done with a minor-version's worth of warning,
> but i think if we're going to change it we should make it right.

By "minor version" do you mean 2.2 -> 2.3, or 2.2 -> 2.2.1?  I presume
the former -- we're not allowed to make incompatible changes on point
releases anymore!  Of course, this implies a f***** statement.

(I'm not going to say the word, Tim said the first one who does dies...)

        Greg
-- 
Greg Ward - just another /P(erl|ython)/ hacker          gward@python.net
http://starship.python.net/~gward/
If you and a friend are being chased by a lion, it is not necessary to
outrun the lion.  It is only necessary to outrun your friend.


From gward@python.net  Fri Aug 31 14:35:19 2001
From: gward@python.net (Greg Ward)
Date: Fri, 31 Aug 2001 09:35:19 -0400
Subject: [Python-Dev] PEP 250
Message-ID: <20010831093519.A16031@gerg.ca>

Thomas --

have you been following the saga of patch #449054, part of the PEP 250
implementation.  If I understand things correctly, this patch only
partly implements PEP 250 -- the rest is waiting on a change to
bdist_wininst.  What's the status of this change?  Going to make it in?

Please see

  http://sourceforge.net/tracker/?func=detail&atid=305470&aid=449054&group_id=5470

if you haven't already...

        Greg
-- 
Greg Ward - geek-at-large                               gward@python.net
http://starship.python.net/~gward/
Just because you're paranoid doesn't mean they *aren't* out to get you.


From akuchlin@mems-exchange.org  Fri Aug 31 15:48:43 2001
From: akuchlin@mems-exchange.org (Andrew Kuchling)
Date: Fri, 31 Aug 2001 10:48:43 -0400
Subject: [Python-Dev] CVS: python/dist/src/Modules _sre.c,2.63,2.64
In-Reply-To: <20010831090820.9BFC7301382@bireme.oratrix.nl>; from sjoerd.mullender@oratrix.com on Fri, Aug 31, 2001 at 11:08:19AM +0200
References: <BIEJKCLHCIOIHAGOKOLHKEOFCFAA.tim@zope.com> <20010831090820.9BFC7301382@bireme.oratrix.nl>
Message-ID: <20010831104842.D30221@ute.mems-exchange.org>

On Fri, Aug 31, 2001 at 11:08:19AM +0200, Sjoerd Mullender wrote:
>any so does this one.  If the flow of control did reach this
>particular line, the compiler would also give a warning, namely that
>the function has both return value and return void (or words to that
>effect).

Your compiler might, but I don't know if Fredrik's will, and for the
SRE code Fredrik's compiler is more important.  Perhaps Tim's
suggested assert() can fix things so that both of you are happy.

--amk


From thomas.heller@ion-tof.com  Fri Aug 31 15:53:50 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Fri, 31 Aug 2001 16:53:50 +0200
Subject: [Python-Dev] PEP 250
References: <20010831093519.A16031@gerg.ca>
Message-ID: <017a01c1322c$bf558fd0$e000a8c0@thomasnotebook>

From: "Greg Ward" <gward@python.net>
> Thomas --
> 
> have you been following the saga of patch #449054, part of the PEP 250
> implementation.  If I understand things correctly, this patch only
> partly implements PEP 250 -- the rest is waiting on a change to
> bdist_wininst.  What's the status of this change?  Going to make it in?
Sure, will do it before 2.2a3. That's Sep 19, right?

Looking at the patch again, it seems sufficient to install into
lib/site-packages if sys.version >= 2.2. No fiddling with loading python
at runtime, importing distutils.sysconfig, and so on in bdist_wininst.

Thomas



From guido@python.org  Fri Aug 31 20:28:25 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 31 Aug 2001 15:28:25 -0400
Subject: [Python-Dev] Would it make sense to issue an extra alpha next week?
Message-ID: <200108311928.PAA21836@cj20424-a.reston1.va.home.com>

When I was updating the NEWS file this morning, I realized that we've
already added a truckload of nifty stuff in the 9 days since 2.2a2 was
released: int overflows return longs, classic division warnings,
subclassing builtins, super, getset, two fixes to float literals, the
new GC API, and PyString_FromFormat[V].

I expect that I'll be concentrating on documentation for the
type/class unification next.  While writing the first piece of
documentation, I realized that, sadly, several more advanced things
aren't available in 2.2a2 yet.

The next alpha is planned for Sept. 19, almost three weeks off still.

Does anybody object against the release of an extra release, 2.2a3,
around Sept. 5?  We could do 2.2a4 on Sept. 19, or a week later if the
schedule gets too crowded.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@home.com  Fri Aug 31 20:29:02 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 31 Aug 2001 15:29:02 -0400
Subject: [Python-Dev] -Dwarn, long->double overflow (was RE: [Python-checkins] CVS: python/dist/src/Misc NEWS,1.219,1.220)
In-Reply-To: <E15ct4r-00067t-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEEELJAA.tim.one@home.com>

[Guido]
> + + A new command line option, -D<arg>, is added to control run-time
> +   warnings for the use of classic division.
> ...
> +   Using -Dwarn issues a run-time warning about all uses of classic
> +   division for int, long, float and complex arguments.

I'm unclear on why we warn about classic division when a float or complex is
involved:

C:\Code\python\PCbuild>python -Dwarn
Python 2.2a2+ (#22, Aug 31 2001, 14:36:57) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> 2./3.
__main__:1: DeprecationWarning: classic float division
0.66666666666666663
>>>

Given that this is going to return the same result even in Python 3.0, what
is it warning me about?  That is, as an end user, I'm being warned about
behavior I can't do anything about, and behavior that Python isn't going to
change anyway.


Different issue:  I would like to add OverflowError when coercion of long to
float yields a senseless result.  PEP 238 mentions this as a possibility,
and

>>> from __future__ import division
>>> x = 1L << 3000
>>> x/(2*x)
-1.#IND
>>>

really sucks.  For that matter,

>>> float(1L << 3000)
1.#INF
>>>

has always sucked; future division just makes it suck harder <wink>.

Any objections to raising OverflowError here?

Note this is a bit painful for a bogus reason:  PyFloat_AsDouble returns a
double, and nothing in the codebase now checks it for a -1.0 error return
(despite that it can already produce one).  So half the effort would be
repairing code currently ignoring the possibility of error.


Third issue:  I don't see a *good* reason for the future-division

    x/(2*x)

above not to return 0.5.  long_true_divide could be made smart enough to do
that (more generally, to return a good float approximation whenever the true
result is representable as a float).



From guido@python.org  Fri Aug 31 20:39:21 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 31 Aug 2001 15:39:21 -0400
Subject: [Python-Dev] Re: -Dwarn, long->double overflow (was RE: [Python-checkins] CVS: python/dist/src/Misc NEWS,1.219,1.220)
In-Reply-To: Your message of "Fri, 31 Aug 2001 15:29:02 EDT."
 <LNBBLJKPBEHFEDALKOLCGEEELJAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCGEEELJAA.tim.one@home.com>
Message-ID: <200108311939.PAA21883@cj20424-a.reston1.va.home.com>

> [Guido]
> > + + A new command line option, -D<arg>, is added to control run-time
> > +   warnings for the use of classic division.
> > ...
> > +   Using -Dwarn issues a run-time warning about all uses of classic
> > +   division for int, long, float and complex arguments.
> 
> I'm unclear on why we warn about classic division when a float or complex is
> involved:
> 
> C:\Code\python\PCbuild>python -Dwarn
> Python 2.2a2+ (#22, Aug 31 2001, 14:36:57) [MSC 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> 2./3.
> __main__:1: DeprecationWarning: classic float division
> 0.66666666666666663
> >>>
> 
> Given that this is going to return the same result even in Python 3.0, what
> is it warning me about?  That is, as an end user, I'm being warned about
> behavior I can't do anything about, and behavior that Python isn't going to
> change anyway.

Sorry, I should have explained the intention.

This warning is intended for perusal by a tool (yet to create) that
reads the warnings and can automatically fix your code by inserting
the proper future division statements and/or replace / by //.  The
tool parses the source looking for / operators.  For each / it sees if
it has corresponding warnings, and if so, if the warnings are
consistent (only int/long or only float/complex), it does the right
thing.  If all / operators in a module have at least one warning, it
assumes that it has full coverage, and it adds a future division
statement to the module.  If there are some / operators for which it
doesn't have any warnings, it asks the user to create a better test
case.  For this, it needs to know about float and complex divisions as
well.

> Different issue: I would like to add OverflowError when coercion of
> long to float yields a senseless result.  PEP 238 mentions this as a
> possibility, and
> 
> >>> from __future__ import division
> >>> x = 1L << 3000
> >>> x/(2*x)
> -1.#IND
> >>>
> 
> really sucks.  For that matter,
> 
> >>> float(1L << 3000)
> 1.#INF
> >>>
> 
> has always sucked; future division just makes it suck harder <wink>.
> 
> Any objections to raising OverflowError here?

Sounds OK to me.

> Note this is a bit painful for a bogus reason:  PyFloat_AsDouble returns a
> double, and nothing in the codebase now checks it for a -1.0 error return
> (despite that it can already produce one).  So half the effort would be
> repairing code currently ignoring the possibility of error.
> 
> 
> Third issue:  I don't see a *good* reason for the future-division
> 
>     x/(2*x)
> 
> above not to return 0.5.  long_true_divide could be made smart enough to do
> that (more generally, to return a good float approximation whenever the true
> result is representable as a float).

Sure, but I also don't see a good reason to make this a priority.
It's a "don't care" corner of the language to me.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From gward@python.net  Fri Aug 31 21:33:29 2001
From: gward@python.net (Greg Ward)
Date: Fri, 31 Aug 2001 16:33:29 -0400
Subject: [Python-Dev] PEP 250
In-Reply-To: <017a01c1322c$bf558fd0$e000a8c0@thomasnotebook>; from thomas.heller@ion-tof.com on Fri, Aug 31, 2001 at 04:53:50PM +0200
References: <20010831093519.A16031@gerg.ca> <017a01c1322c$bf558fd0$e000a8c0@thomasnotebook>
Message-ID: <20010831163329.A16507@gerg.ca>


On 31 August 2001, Thomas Heller said:
> Sure, will do it before 2.2a3. That's Sep 19, right?

The sooner the better -- this is a "priority 9" patch!

> Looking at the patch again, it seems sufficient to install into
> lib/site-packages if sys.version >= 2.2. No fiddling with loading python
> at runtime, importing distutils.sysconfig, and so on in bdist_wininst.

Sure, whatever.  Up to your best judgment; I've never actually looked at
the C++ code that's compiled into that big blob of binary data in
bdist_wininst.py.  The C++ probably wouldn't mean much more to me than
the binary blob does.  ;-)

        Greg
-- 
Greg Ward - Unix geek                                   gward@python.net
http://starship.python.net/~gward/
My opinions may have changed, but not the fact that I am right.


From tim.one@home.com  Fri Aug 31 22:40:18 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 31 Aug 2001 17:40:18 -0400
Subject: [Python-Dev] RE: -Dwarn, long->double overflow (was RE: [Python-checkins] CVS: python/dist/src/Misc NEWS,1.219,1.220)
In-Reply-To: <200108311939.PAA21883@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEEOLJAA.tim.one@home.com>

[Tim]
>> I'm unclear on why we warn about classic division when a float
>> or complex is involved:

[Guido]
> ...
> This warning is intended for perusal by a tool (yet to create) that
> reads the warnings and can automatically fix your code by inserting
> the proper future division statements and/or replace / by //.

Got it now.  Thanks!  Cool idea, BTW.

>> I would like to add OverflowError when coercion of long to float
>> yields a senseless result.

> Sounds OK to me.

Anyone else object?

>> Third issue:  I don't see a *good* reason for the future-division
>>
>>     x/(2*x)
>>
>> above not to return 0.5 [when x is a large long].

> Sure, but I also don't see a good reason to make this a priority.
> It's a "don't care" corner of the language to me.

Then I'll leave it alone, but I can't understand this view (it's not like we
can claim we're victims of the platform C or libm here -- nonsense results
in such cases are on our heads, and an implementation that settles for total
loss of information when 53 good bits are obviously possible is at best
careless).



From gstein@lyra.org  Fri Aug 31 22:54:05 2001
From: gstein@lyra.org (Greg Stein)
Date: Fri, 31 Aug 2001 14:54:05 -0700
Subject: [Python-Dev] Would it make sense to issue an extra alpha next week?
In-Reply-To: <200108311928.PAA21836@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Aug 31, 2001 at 03:28:25PM -0400
References: <200108311928.PAA21836@cj20424-a.reston1.va.home.com>
Message-ID: <20010831145405.F3654@lyra.org>

+1

On Fri, Aug 31, 2001 at 03:28:25PM -0400, Guido van Rossum wrote:
> When I was updating the NEWS file this morning, I realized that we've
> already added a truckload of nifty stuff in the 9 days since 2.2a2 was
> released: int overflows return longs, classic division warnings,
> subclassing builtins, super, getset, two fixes to float literals, the
> new GC API, and PyString_FromFormat[V].
> 
> I expect that I'll be concentrating on documentation for the
> type/class unification next.  While writing the first piece of
> documentation, I realized that, sadly, several more advanced things
> aren't available in 2.2a2 yet.
> 
> The next alpha is planned for Sept. 19, almost three weeks off still.
> 
> Does anybody object against the release of an extra release, 2.2a3,
> around Sept. 5?  We could do 2.2a4 on Sept. 19, or a week later if the
> schedule gets too crowded.
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)

-- 
Greg Stein, http://www.lyra.org/