From arigo at tunes.org  Sat May  1 03:08:48 2004
From: arigo at tunes.org (Armin Rigo)
Date: Sat May  1 03:11:58 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <200404302313.i3UND0a08111@guido.python.org>
References: <200404231826.i3NIQxM18955@guido.python.org>
	<20040430104158.GA31871@vicky.ecs.soton.ac.uk>
	<200404301216.i3UCGXU07211@guido.python.org>
	<20040430130700.GA28030@vicky.ecs.soton.ac.uk>
	<200404302313.i3UND0a08111@guido.python.org>
Message-ID: <20040501070848.GA23938@vicky.ecs.soton.ac.uk>

Hello Guido,

On Fri, Apr 30, 2004 at 04:12:59PM -0700, Guido van Rossum wrote:
> That's not new, we all know that that's the downside of late binding.
> 
> My counter to that is that *any* use of genexps where the consumer
> doesn't consume (or discard) the iterator before the next line is
> reached is extremely advanced use.

My argument was that late-binding make it even more mysterious, not less
suprizing, because in the minority of cases where a listcomp is not used
immediately but still need a genexpr for performance, you have to think
carefully.

Well, I insist a bit because I cannot helping thinking that the arguments for
early-binding are "that's useful in a very small number of cases", and the
arguments for late-binding are "you don't want to do that anyway so let's pick
the semantic of free vars in functions".  I don't immediately think of genexps
as similar to functions, rather closer to listcomps, even if I know that they
are functions under the hood.


Armin


From dmgass at hotmail.com  Sat May  1 15:31:11 2004
From: dmgass at hotmail.com (Dan Gass)
Date: Sat May  1 15:31:15 2004
Subject: [Python-Dev] Proposal: A more powerful alternative to ConfigParser
Message-ID: <BAY18-F102p0V7nOh9K0000180b@hotmail.com>

Per suggestions posted on this mailing list I created a sourceforge project 
(config-py), registered it with PyPi and submitted the following 
announcement to python-announce.

I look forward to your feedback.

Thanks,
Dan Gass




Announcement
------------

config-py version 1.0 has been released and is available at SourceForge.  
This
is the initial release of a Python module for reading python based user
configuration files to control script execution.  (It is a more powerful
alternative to Python's ConfigParser module).  The long term goal is to 
refine
this module for eventual inclusion in the standard Python distribution.


Module Description
------------------

This python module provides a simple, but extremely flexible and powerful
method of controlling python script execution by use of configuration 
settings
stored in user configuration files.  This module is a more powerful
alternative to the ConfigParser module already available in the standard
Python distribution.

The standard ConfigParser module parses text based user configuration files
and provides access to that information.  This module reads python based
configuration files and provides access to the python objects contained
within.  In addition, this module supports hierarchical organization of the
settings within the configuration files so that settings may be accessed 
using
keys specified by the user from the command line or by other means.  By use 
of
python based configuration files, this module provides a simple solution to
most configuration problems but also has the flexibility to solve the most
complex configuration issues.


Other Important Information
---------------------------

License:            GNU General Public License (GPL)

Project Home Page:  http://config-py.sourceforge.net/

Download URL:       https://sourceforge.net/projects/config-py/

Author/Maintainer:  Dan Gass (dmgass@hotmail.com)

_________________________________________________________________
Check out the coupons and bargains on MSN Offers! http://youroffers.msn.com


From greg at cosc.canterbury.ac.nz  Sun May  2 23:03:11 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Sun May  2 23:07:21 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <20040430104158.GA31871@vicky.ecs.soton.ac.uk>
Message-ID: <200405030303.i4333Bxw019255@cosc353.cosc.canterbury.ac.nz>

Armin Rigo <arigo@tunes.org>:

> Early binding is closer to the idea that turning a listcomp into a
> genexprs should just work if you only iterate once on the result.

I'm not sure that's an idea we should be promulgating in the first
place. The original motivation for generator expressions was things
like

  total = sum(x**2 for x in stuff)

where not only will the sequence be used just once, but it will be
used *immediately* before doing anything else.

Those are the use cases we need to advertise this as targeting, I
think, not "any listcomp that's only used once".

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Sun May  2 23:05:46 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Sun May  2 23:07:24 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <5.2.1.1.0.20040430142620.030304e8@pop.bluewin.ch>
Message-ID: <200405030305.i4335kaL019266@cosc353.cosc.canterbury.ac.nz>

Samuele Pedroni <pedronis@bluewin.ch>:

> "There should be one-- and preferably only one --obvious way to do
> it."  of course this is the case for generator expressions and list
> comprehsensions because they are separated by performance
> characteristics and somehow by the fact that one directly produces a
> list. I suspect that we have to learn to live with the fact that
> they are distinguished by (just) that, both a nuisance (sometimes
> people don't want to bother to choose) and a feature.

I just had a thought.

What if generator expressions were allowed *only* as a
parameter to a function call?

Would that help avoid leading people into the trap of
using them where they're not appropriate?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Sun May  2 23:23:23 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Sun May  2 23:27:11 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <20040430093439.138D.JCARLSON@uci.edu>
Message-ID: <200405030323.i433NNKj019302@cosc353.cosc.canterbury.ac.nz>

Josiah Carlson <jcarlson@uci.edu>:

> Semantically, in the first case you are modifying your dictionary, which
> has consequences to anything that has a reference to the object. Mutable
> modification during runtime should be something we are used to by now.

The point is that anyone who's going to be confused by the difference
between early and late binding behaviour is likely to be equally
confused by this.

The issue is really about when expressions are evaluated, not when
names are bound. Patching up just one part of it (binding of free
variables) doesn't really fix the problem.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Sun May  2 23:27:41 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Sun May  2 23:30:42 2004
Subject: [Python-Dev] Accessing base class C struct of subclass instance
In-Reply-To: <6E9B692A-9AD0-11D8-961D-000393CBDF94@opnet.com>
Message-ID: <200405030327.i433RfBS019314@cosc353.cosc.canterbury.ac.nz>

Nick Bastin <nbastin@opnet.com>:

> If I define a new style class in C, and subclass it in python, how can 
> I get the base class instance object from a subclass instance in C?

What do you mean by "base class instance object"? That's
a meaningless combination of words to me.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Sun May  2 23:41:14 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Sun May  2 23:42:36 2004
Subject: [Python-Dev] Re: PEP 289 - Generator Expressions - Let's Move
	Forward
In-Reply-To: <rowen-08D028.14000830042004@sea.gmane.org>
Message-ID: <200405030341.i433fERl019326@cosc353.cosc.canterbury.ac.nz>

"Russell E. Owen" <rowen@cesmail.net>:

> Personally, I'd only like to see generator expressions added to Python 
> if they can allow list comprehensions to be deprecated -- to avoid 
> language clutter.

That's possible in the sense that a list comprehension
can be written as list(generator_expression).

But it will still be necessary to decide whether to
wrap a list() around it.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From guido at python.org  Mon May  3 00:08:35 2004
From: guido at python.org (Guido van Rossum)
Date: Mon May  3 00:08:06 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: Your message of "Mon, 03 May 2004 15:03:11 +1200."
	<200405030303.i4333Bxw019255@cosc353.cosc.canterbury.ac.nz> 
References: <200405030303.i4333Bxw019255@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200405030408.i4348Zk01989@guido.python.org>

> Armin Rigo <arigo@tunes.org>:
> 
> > Early binding is closer to the idea that turning a listcomp into a
> > genexprs should just work if you only iterate once on the result.

Greg Ewing:
> I'm not sure that's an idea we should be promulgating in the first
> place. The original motivation for generator expressions was things
> like
> 
>   total = sum(x**2 for x in stuff)
> 
> where not only will the sequence be used just once, but it will be
> used *immediately* before doing anything else.
> 
> Those are the use cases we need to advertise this as targeting, I
> think, not "any listcomp that's only used once".

Right -- thanks for saying it so clearly!  Early binding seems to be
trying to solve a problem with a use for genexprs that's questionable
at best.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon May  3 00:09:18 2004
From: guido at python.org (Guido van Rossum)
Date: Mon May  3 00:08:55 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: Your message of "Mon, 03 May 2004 15:05:46 +1200."
	<200405030305.i4335kaL019266@cosc353.cosc.canterbury.ac.nz> 
References: <200405030305.i4335kaL019266@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200405030409.i4349IP02000@guido.python.org>

> What if generator expressions were allowed *only* as a
> parameter to a function call?
> 
> Would that help avoid leading people into the trap of
> using them where they're not appropriate?

Nah, people would just use an identity function to work around the
restriction.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pedronis at bluewin.ch  Mon May  3 09:47:08 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Mon May  3 09:41:36 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <200405030408.i4348Zk01989@guido.python.org>
References: <Your message of "Mon, 03 May 2004 15:03:11 +1200."
	<200405030303.i4333Bxw019255@cosc353.cosc.canterbury.ac.nz>
	<200405030303.i4333Bxw019255@cosc353.cosc.canterbury.ac.nz>
Message-ID: <5.2.1.1.0.20040503154238.027cd650@pop.bluewin.ch>

At 21:08 02.05.2004 -0700, Guido van Rossum wrote:
> > Armin Rigo <arigo@tunes.org>:
> >
> > > Early binding is closer to the idea that turning a listcomp into a
> > > genexprs should just work if you only iterate once on the result.
>
>Greg Ewing:
> > I'm not sure that's an idea we should be promulgating in the first
> > place. The original motivation for generator expressions was things
> > like
> >
> >   total = sum(x**2 for x in stuff)
> >
> > where not only will the sequence be used just once, but it will be
> > used *immediately* before doing anything else.
> >

yes, but the question is how much easy is to grasp what "used 
*immediately*" means for a genexp, or IOW how much education is needed.

this looks innocent enough:

l = list_of_my_strings()
for pfx in discard_prefixes:
   l = ( x for x in l if not x.startswith(pfx) ) # this is not using l 
*immediately*
use_strings(l)


> > Those are the use cases we need to advertise this as targeting, I
> > think, not "any listcomp that's only used once".
>
>Right -- thanks for saying it so clearly!  Early binding seems to be
>trying to solve a problem with a use for genexprs that's questionable
>at best.


From jim.jewett at eds.com  Mon May  3 09:58:01 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Mon May  3 10:01:04 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move For ward
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D4A4@usahm010.exmi01.exch.eds.com>

Guido van Rossum:

> My counter to that is that *any* use of genexps where the consumer
> doesn't consume (or discard) the iterator before the next line is
> reached is extremely advanced use.

This might be worth putting in the PEP and the docs -- 

Normal usage of a generator expression is to immediately consume 
the entire sequence, or to discard the remainder.  

If portions of a generator *are* used later, programmers may need 
some extra precautions to ensure that all names still have the 
desired bindings.  

-jJ

From aahz at pythoncraft.com  Mon May  3 11:58:12 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon May  3 11:58:15 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <1082749292.9092.40.camel@localhost.localdomain>
References: <1082746969.40896859bcf09@mcherm.com>
	<1082749292.9092.40.camel@localhost.localdomain>
Message-ID: <20040503155812.GA9753@panix.com>

On Fri, Apr 23, 2004, Jeremy Hylton wrote:
>
> That leads me to wonder what exactly the rationale for generator
> expressions is.  The PEP says that "time, clarity, and memory are
> conserved by using an generator expression" but I can only see how
> memory is conserved.  That is, I don't find them any easier to read than
> list comprehensions and I don't understand the performance implications
> very well.  It's not obvious to me that their faster.

I've been skimming due to being out of town and catching up, but I
haven't seen a direct response to Jeremy's question about the rationale.
Jeremy, do you still want an answer?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I used to have a .sig but I found it impossible to please everyone..."  --SFJ

From arigo at tunes.org  Mon May  3 12:27:50 2004
From: arigo at tunes.org (Armin Rigo)
Date: Mon May  3 12:31:18 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <200405030303.i4333Bxw019255@cosc353.cosc.canterbury.ac.nz>
References: <20040430104158.GA31871@vicky.ecs.soton.ac.uk>
	<200405030303.i4333Bxw019255@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040503162750.GA11346@vicky.ecs.soton.ac.uk>

Hello Greg,

On Mon, May 03, 2004 at 03:03:11PM +1200, Greg Ewing wrote:
> I'm not sure that's an idea we should be promulgating in the first
> place. The original motivation for generator expressions was things
> like
> 
>   total = sum(x**2 for x in stuff)
> 
> where not only will the sequence be used just once, but it will be
> used *immediately* before doing anything else.

I understand perfecty well that anything else is advanced stuff, and that
people shouldn't do advanced stuff if they are not aware of all the issues.  
I also know that any kind of laziness together with mutating objects creates
delicate problems.  What I am looking for is a genexpr implementation that
would, as a bonus, be useful in advanced cases too instead of being just
unusable beyond these 95% of cases where the binding model doesn't matter.

The problem I have with late-bound names is that small, semi-advanced,
"innocent enough" examples like Samuele's keep popping up that would just give
wrong answers (not just crash).  Personally when I look at all these examples
I think they are elegant ways to express things that I could want to do, and I
could easily read and write this kind of code -- but it just doesn't do that I
expect it to.  The real answer is much more convoluted.

The only advantage I see to late binding is that they prevent people from
doing any advanced stuff with genexprs, and thus beginners never have to read
code containing advanced uses of genexprs (even clean-looking one like
Samuele's).


Armin


From guido at python.org  Mon May  3 13:14:04 2004
From: guido at python.org (Guido van Rossum)
Date: Mon May  3 13:14:12 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: Your message of "Mon, 03 May 2004 17:27:50 BST."
	<20040503162750.GA11346@vicky.ecs.soton.ac.uk> 
References: <20040430104158.GA31871@vicky.ecs.soton.ac.uk>
	<200405030303.i4333Bxw019255@cosc353.cosc.canterbury.ac.nz> 
	<20040503162750.GA11346@vicky.ecs.soton.ac.uk> 
Message-ID: <200405031714.i43HE4p03162@guido.python.org>

> The problem I have with late-bound names is that small,
> semi-advanced, "innocent enough" examples like Samuele's keep
> popping up that would just give wrong answers (not just crash).

Hm.  I don't think I've seen a single example that wasn't specifically
concocted by early-binding advocates (yours included -- you took
random listcomps and mused on what would happen if they'd be genexprs
instead).

> Personally when I look at all these examples I think they are
> elegant ways to express things that I could want to do, and I could
> easily read and write this kind of code -- but it just doesn't do
> that I expect it to.  The real answer is much more convoluted.

Remember that your brain is several times larger than that of most
Python users.

> The only advantage I see to late binding is that they prevent people
> from doing any advanced stuff with genexprs, and thus beginners
> never have to read code containing advanced uses of genexprs (even
> clean-looking one like Samuele's).

My problem with early binding is that it intruduces a new set of
behind-the-scenes semantics that you cannot always ignore -- while the
examples where it doesn't DWYM are even more obscure than the examples
that it is intended to save, it means that you still have to think
about the new semantics when doing advanced stuff -- and these new
semantics are unlike anything seen before in Python: transformations
that require a definition of "free variables" are not seen anywhere
else (nested scopes use a different concept).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From ark-mlist at att.net  Mon May  3 15:19:39 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Mon May  3 15:19:36 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <20040503155812.GA9753@panix.com>
Message-ID: <001501c43143$94a38210$6402a8c0@arkdesktop>

> > That leads me to wonder what exactly the rationale for generator
> > expressions is.  The PEP says that "time, clarity, and memory are
> > conserved by using an generator expression" but I can only see how
> > memory is conserved.  That is, I don't find them any easier to read than
> > list comprehensions and I don't understand the performance implications
> > very well.  It's not obvious to me that their faster.

> I've been skimming due to being out of town and catching up, but I
> haven't seen a direct response to Jeremy's question about the rationale.
> Jeremy, do you still want an answer?

I can see two potentially important cases where generator expressions win
big over list comprehensions:

	1) Where the code that is consuming the sequence yielded by the
generator expression terminates before consuming the entire sequence;

	2) Where the code that is consuming the sequence is an online
algorithm, and there is a potential delay between generating elements of the
sequence.

It is easier to find an example of the second case than of the first:

	foo(bar(line) for line in sys.stdin)

If foo expects a generator as its input:

	def foo(x):
		for line in x:
			print line

then using a generator expression instead of a list comprehension will cause
the program to print each line of output after reading the corresponding
line of input, rather than consuming all the input and then printing all the
output.


From arigo at tunes.org  Mon May  3 15:17:48 2004
From: arigo at tunes.org (Armin Rigo)
Date: Mon May  3 15:21:08 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <200405031714.i43HE4p03162@guido.python.org>
References: <20040430104158.GA31871@vicky.ecs.soton.ac.uk>
	<200405030303.i4333Bxw019255@cosc353.cosc.canterbury.ac.nz>
	<20040503162750.GA11346@vicky.ecs.soton.ac.uk>
	<200405031714.i43HE4p03162@guido.python.org>
Message-ID: <20040503191748.GA12891@vicky.ecs.soton.ac.uk>

Hello Guido,

On Mon, May 03, 2004 at 10:14:04AM -0700, Guido van Rossum wrote:
> (...) these new
> semantics are unlike anything seen before in Python: transformations
> that require a definition of "free variables" are not seen anywhere
> else (nested scopes use a different concept).

I think I see your point, although I'm not sure I see how nested scopes work
if not with free variables ( = variables used but not bound in the current
scope).


Armin


From tim.one at comcast.net  Mon May  3 16:31:48 2004
From: tim.one at comcast.net (Tim Peters)
Date: Mon May  3 16:31:54 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <200405031714.i43HE4p03162@guido.python.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEKJKKAB.tim.one@comcast.net>

[Guido]
> Hm.  I don't think I've seen a single example that wasn't specifically
> concocted by early-binding advocates (yours included -- you took
> random listcomps and mused on what would happen if they'd be genexprs
> instead).

That's a bit of a revisionist stretch:  the examples I trotted out last
October *made* me an early-binding advocate at the time.  For example,

   http://mail.python.org/pipermail/python-dev/2003-October/039323.html

That example doesn't have lists of generators either, which is another thing
you don't remember ever seeing <wink>.

It's still the case that I haven't seen a plausible example where
late-binding semantics both make a difference and are desirable, or where
early-binding semantics both make a difference and are undesirable --
although the magical implicitness of both schemes leaves me queasy.

...

> My problem with early binding is that it intruduces a new set of
> behind-the-scenes semantics that you cannot always ignore -- while the
> examples where it doesn't DWYM are even more obscure than the examples
> that it is intended to save, it means that you still have to think
> about the new semantics when doing advanced stuff -- and these new
> semantics are unlike anything seen before in Python:  transformations
> that require a definition of "free variables" are not seen anywhere
> else (nested scopes use a different concept).

Late-binding in genexps can't be ignored either when doing advanced stuff,
and are also unlike anything seen before in Python:  transforming what looks
like an expression into a delayed function body is also unprecendented (in
the absence of a lambda).  The fact that it "looks like" an expression is
exactly what's so surprising about that "the current values" of p and pipe
don't get used in the above example's

    # add a filter over the current pipe, and call that the new pipe
    pipe = (e for e in pipe if p(e))

It didn't strike me as an advanced use at the time, just as an example of
something that seemed like it should be natural.

It sounds like, today, "the current value" of pipe will be used after all,
but the evaluation of p will get delayed?  If so, that's not notably easy to
explain either (or so it seems to me).


From tdelaney at avaya.com  Mon May  3 17:56:42 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Mon May  3 17:56:50 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01729514@au3010avexu1.global.avaya.com>

Tim Peters wrote:

>     # add a filter over the current pipe, and call that the new pipe
>     pipe = (e for e in pipe if p(e))

This is a fairly convincing argument to me - that is something I do all
the time with listcomps. Saves having to invent extra names.

Tim Delaney

From tim.one at comcast.net  Mon May  3 22:30:30 2004
From: tim.one at comcast.net (Tim Peters)
Date: Mon May  3 22:30:38 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01729514@au3010avexu1.global.avaya.com>
Message-ID: <E1BKpha-0005ze-Sr@mail.python.org>

>>     # add a filter over the current pipe, and call that the new pipe
>>     pipe = (e for e in pipe if p(e))

[Delaney, Timothy C (Timothy)]
> This is a fairly convincing argument to me -

Except it's not an argument, it's just a six-month old example.  I don't
have an argument to make here.  I've observed repeatedly that in all
"reasonable" examples to date where the distinction makes a difference,
early binding does better than late binding, but I still haven't dreamt up a
"deep reason" for that.  The arguments on both sides have been singularly
uncompelling (to me, anyway).

> that is something I do all the time with listcomps.  Saves having to
> invent extra names.

I may have lost track, but I think the last plan Guido approved of here
would actually evaluate the target of the leftmost "for" ("pipe" in the
example above) immediately, leaving only "p" to get picked up later.  It's
impossible for me to guess how that might affect the things you do all the
time with listcomps; also impossible for me to guess whether you'd do
similar things with genexps.

This issue has always been long on specious arguments and short on concrete
examples.  But that's OK -- we'll get the concrete examples, but after it's
too late <wink>.



From jeremy at alum.mit.edu  Mon May  3 22:42:53 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon May  3 22:44:54 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move For ward
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D4A4@usahm010.exmi01.exch.eds.com>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D4A4@usahm010.exmi01.exch.eds.com>
Message-ID: <1083638573.17924.9.camel@localhost.localdomain>

On Mon, 2004-05-03 at 09:58, Jewett, Jim J wrote:
> Guido van Rossum:
> 
> > My counter to that is that *any* use of genexps where the consumer
> > doesn't consume (or discard) the iterator before the next line is
> > reached is extremely advanced use.
> 
> This might be worth putting in the PEP and the docs -- 

We are in rather dire need of docs and a revised PEP.

> Normal usage of a generator expression is to immediately consume 
> the entire sequence, or to discard the remainder.  
> 
> If portions of a generator *are* used later, programmers may need 
> some extra precautions to ensure that all names still have the 
> desired bindings.  

There may be a general rule that functions that make use of free
variables are best programmed in a functional style, where the function
doesn't depend on side-effects to those variables.

Jeremy



From greg at cosc.canterbury.ac.nz  Mon May  3 22:53:07 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon May  3 22:53:47 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <E1BKpha-0005ze-Sr@mail.python.org>
Message-ID: <200405040253.i442r7f5021005@cosc353.cosc.canterbury.ac.nz>

Tim Peters <tim.one@comcast.net>:

> This issue has always been long on specious arguments and short on concrete
> examples.  But that's OK -- we'll get the concrete examples, but after it's
> too late <wink>.

Maybe for a few releases they should have to be imported
from __hypotheticalfuture__, a module whose contents are
understood to be subject to merciless revision should
they turn out to be a bad idea...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From aahz at pythoncraft.com  Tue May  4 00:02:47 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue May  4 00:02:50 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <200405040253.i442r7f5021005@cosc353.cosc.canterbury.ac.nz>
References: <E1BKpha-0005ze-Sr@mail.python.org>
	<200405040253.i442r7f5021005@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040504040247.GA11979@panix.com>

On Tue, May 04, 2004, Greg Ewing wrote:
> Tim Peters <tim.one@comcast.net>:
>> 
>> This issue has always been long on specious arguments and short on concrete
>> examples.  But that's OK -- we'll get the concrete examples, but after it's
>> too late <wink>.
> 
> Maybe for a few releases they should have to be imported from
> __hypotheticalfuture__, a module whose contents are understood to be
> subject to merciless revision should they turn out to be a bad idea...

You're probably joking, but +1 from me.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Adopt A Process -- stop killing all your children!

From anthony at interlink.com.au  Tue May  4 00:54:08 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue May  4 00:54:41 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move For ward
In-Reply-To: <1083638573.17924.9.camel@localhost.localdomain>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D4A4@usahm010.exmi01.exch.eds.com>
	<1083638573.17924.9.camel@localhost.localdomain>
Message-ID: <409721F0.1020102@interlink.com.au>

Jeremy Hylton wrote:
> On Mon, 2004-05-03 at 09:58, Jewett, Jim J wrote:
> 
>>Guido van Rossum:
>>
>>
>>>My counter to that is that *any* use of genexps where the consumer
>>>doesn't consume (or discard) the iterator before the next line is
>>>reached is extremely advanced use.
>>
>>This might be worth putting in the PEP and the docs -- 
> 
> 
> We are in rather dire need of docs and a revised PEP.

Indeed. The timeframe we came up with at PyCon had the
generator expressions being landed "in the first week of
May". At the moment the discussion seems to be spinning
wheels -- it looks (to me) like there's not even 100%
agreement from everyone that they should even go into
2.4 at all.

I'm not fussed if we slip a week or two here, but if
it slips another 6 weeks, say, then the release time
line could be in some trouble.


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From gward at python.net  Tue May  4 08:16:00 2004
From: gward at python.net (Greg Ward)
Date: Tue May  4 08:16:04 2004
Subject: [Python-Dev] Re: Small additions to optparser
In-Reply-To: <20040429194735.GA7068@burma.localdomain>
References: <20040429194735.GA7068@burma.localdomain>
Message-ID: <20040504121600.GB1680@cthulhu.gerg.ca>

On 29 April 2004, Gustavo Niemeyer said:
> I have two small customizations I've used in most projects I've
> plugged optparse into. Since they're simple and generic, I'd
> like to integrate them back in optparser. These features are:
> 
> - Add a help formatter which uses capitalized headers.

Sure, looks simple enough.

> - Add a "help" keyword in OptionParser, allowing straightforward
>   addition of a custom help messages.

How is that different from the existing 'description' that David Goodger
added for Optik 1.4?

> Comments?
> 
> Also, how are we currently handling changes in optparser? Do we
> have an "official" maintainer (Greg?). IOW, should I:

As it says at the top of optparse.py:

  Originally distributed as Optik; see http://optik.sourceforge.net/ .

  If you have problems with this module, please do not file bugs,
  patches, or feature requests with Python; instead, use Optik's
  SourceForge project page:
    http://sourceforge.net/projects/optik

  For support, use the optik-users@lists.sourceforge.net mailing list
  (http://lists.sourceforge.net/lists/listinfo/optik-users).

So either post your patches to optik-users@lists.sourceforge.net, or
file 'em on the Optik tracker at SF.  Nothing wrong with cc'ing
python-dev at the same time, though.

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
All of science is either physics or stamp collecting.

From niemeyer at conectiva.com  Tue May  4 09:50:43 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Tue May  4 10:06:59 2004
Subject: [Python-Dev] Re: Small additions to optparser
In-Reply-To: <20040504121600.GB1680@cthulhu.gerg.ca>
References: <20040429194735.GA7068@burma.localdomain>
	<20040504121600.GB1680@cthulhu.gerg.ca>
Message-ID: <20040504135043.GA3096@burma.localdomain>

> > - Add a help formatter which uses capitalized headers.
> 
> Sure, looks simple enough.

Great! Thanks!

> How is that different from the existing 'description' that David
> Goodger added for Optik 1.4?

Here is real example. I have the following help text:

HELP = """\
Usage: repsys COMMAND [COMMAND ARGUMENTS]

Useful commands:
    co
    submit
    create
    getspec
    getsrpm
    changed
    authoremail

Run "repsys COMMAND --help" for more information.
"""

Using OptionParser(description=HELP) and "repsys --help", I get:

"""
Usage: repsys [options]

Usage: repsys COMMAND [COMMAND ARGUMENTS]  Useful commands:
co     submit     create     getspec     getsrpm     changed
authoremail  Run "repsys COMMAND --help" for more information.
Options:
  --version   show program's version number and exit
  -h, --help  show this help message and exit
  --debug
"""

While using OptionParser(help=HELP) and "repsys --help", I get:

"""
Usage: repsys COMMAND [COMMAND ARGUMENTS]

Useful commands:
    co
    submit
    create
    getspec
    getsrpm
    changed
    authoremail

Run "repsys COMMAND --help" for more information.
"""

[...]
> So either post your patches to optik-users@lists.sourceforge.net, or
> file 'em on the Optik tracker at SF.  Nothing wrong with cc'ing
> python-dev at the same time, though.

Will do that. Thanks!

-- 
Gustavo Niemeyer
http://niemeyer.net

From ark-mlist at att.net  Tue May  4 10:31:38 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Tue May  4 10:31:33 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move For ward
In-Reply-To: <1083638573.17924.9.camel@localhost.localdomain>
Message-ID: <003401c431e4$831962c0$6402a8c0@arkdesktop>

> There may be a general rule that functions that make use of free
> variables are best programmed in a functional style, where the function
> doesn't depend on side-effects to those variables.

The troublesome cases tend to be ones where the function depends on *the
lack* of side effects to free variables.



From arigo at tunes.org  Tue May  4 10:58:52 2004
From: arigo at tunes.org (Armin Rigo)
Date: Tue May  4 11:01:52 2004
Subject: [Python-Dev] negative modular exponentiation
Message-ID: <20040504145852.GA27574@vicky.ecs.soton.ac.uk>

Hello,

SF patch 936813 (fast modular exponentiation) introduces some optimization of
exponentiation with longs, including a new feature: in a pow(x,y,z), the
exponent y is allowed to be negative when it makes mathematical sense.  The
result of (x ** -y) % z is a number that is the inverse of (x ** y) % z,
i.e. multiplying the two gives 1 modulo z.  This is a very convenient feature 
if you are doing modulo arithmetic.  Quoting Trevor Perrin, the patch's 
author:

  Libraries like GMP and LibTomMath work the same way.
  Being able to take inverses mod a number is useful for
  cryptography (e.g. RSA, DSA, and Elgamal).

As this is a new feature it should be approved of.  If it is, I'd volunteer to
review the patch and make the necessary extension to the plain integer's pow()  
so that they work the same way.

E.g.

>>> pow(2,-1,9)
5

because (5*2)%9 == 1.  Currently this gives a TypeError (why not a
ValueError?) so changing the behaviour seems safe.


Armin


From guido at python.org  Tue May  4 11:33:53 2004
From: guido at python.org (Guido van Rossum)
Date: Tue May  4 11:33:59 2004
Subject: [Python-Dev] negative modular exponentiation
In-Reply-To: Your message of "Tue, 04 May 2004 15:58:52 BST."
	<20040504145852.GA27574@vicky.ecs.soton.ac.uk> 
References: <20040504145852.GA27574@vicky.ecs.soton.ac.uk> 
Message-ID: <200405041533.i44FXsD04839@guido.python.org>

> SF patch 936813 (fast modular exponentiation) introduces some
> optimization of exponentiation with longs, including a new feature:
> in a pow(x,y,z), the exponent y is allowed to be negative when it
> makes mathematical sense.  The result of (x ** -y) % z is a number
> that is the inverse of (x ** y) % z, i.e. multiplying the two gives
> 1 modulo z.  This is a very convenient feature if you are doing
> modulo arithmetic.  Quoting Trevor Perrin, the patch's author:
> 
>   Libraries like GMP and LibTomMath work the same way.
>   Being able to take inverses mod a number is useful for
>   cryptography (e.g. RSA, DSA, and Elgamal).
> 
> As this is a new feature it should be approved of.  If it is, I'd
> volunteer to review the patch and make the necessary extension to
> the plain integer's pow() so that they work the same way.
> 
> E.g.
> 
> >>> pow(2,-1,9)
> 5
> 
> because (5*2)%9 == 1.  Currently this gives a TypeError (why not a
> ValueError?) so changing the behaviour seems safe.

This seems of little practical value (especially since real crypto
wizards only complain that Python's longs are so much slower than
gmp), but I can't see it could possibly break something, so a +0 from
me.  (Hm, it could break the expectation that pow(x, y, z), when
defined, equals x**y%z.  At the very least the docs need to be
updated.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From anthony at interlink.com.au  Tue May  4 13:16:27 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue May  4 13:17:29 2004
Subject: [Python-Dev] Python 2.3.4 schedule
Message-ID: <4097CFEB.1090500@interlink.com.au>

My plan is to cut a release candidate for 2.3.4 next
Thursday (13th April) Australian time (so Wednesday night
for the merkins), and a release the following Thursday
(the 20th).

Can we say that the 2.3 branch is frozen from Tuesday? As
this is likely to be the final 2.3 release (modulo any
suprising new bugs), I'd like the time to try and nail
this one.

As per normal, I'll be on #python-dev on freenode for the
next two weeks or so.

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From mwh at python.net  Tue May  4 13:29:47 2004
From: mwh at python.net (Michael Hudson)
Date: Tue May  4 13:29:51 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: <4097CFEB.1090500@interlink.com.au> (Anthony Baxter's message
	of "Wed, 05 May 2004 03:16:27 +1000")
References: <4097CFEB.1090500@interlink.com.au>
Message-ID: <2my8o8p0c4.fsf@starship.python.net>

Anthony Baxter <anthony@interlink.com.au> writes:

> My plan is to cut a release candidate for 2.3.4 next
> Thursday (13th April) Australian time (so Wednesday night
> for the merkins), and a release the following Thursday
> (the 20th).

You mean May, presumably?

Cheers,
mwh

-- 
  In short, just business as usual in the wacky world of floating
  point <wink>.                        -- Tim Peters, comp.lang.python

From tim.one at comcast.net  Tue May  4 13:31:20 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue May  4 13:31:23 2004
Subject: [Python-Dev] negative modular exponentiation
In-Reply-To: <20040504145852.GA27574@vicky.ecs.soton.ac.uk>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEOPKKAB.tim.one@comcast.net>

[Armin Rigo]
> SF patch 936813 (fast modular exponentiation) introduces some
> optimization of exponentiation with longs, including a new feature:
> in a pow(x,y,z), the exponent y is allowed to be negative when it
> makes mathematical sense.  The result of (x ** -y) % z is a number that
> is the inverse of (x ** y) % z, i.e. multiplying the two gives 1
> modulo z.

...

> As this is a new feature it should be approved of.

It's fine by me, and new features are finer by me than adding ever more
obscure optimizations in longobject.c.  The Python long implementation has a
great track record wrt portability and lack of bugs, and gonzo optimizations
work against that.  For example, did Python really need the Karatsuba
multiplication gimmick?  I added the patch for it after a lot of reworking,
but overall I'm not sure it was a net win; and the current patch in question
boosts the smallest integer at which Karatsuba triggers to such a large
value that anyone reaching it would probably be much better off using a GMP
wrapper anyway.

> If it is, I'd volunteer to review the patch

One thing I noticed a few times in the patch was neglecting to check C calls
for error returns.  Even _PyLong_NumBits() can fail!

> and make the necessary extension to the plain integer's pow() so that
> they work the same way.
>
> E.g.
>
> >>> pow(2,-1,9)
> 5
>
> because (5*2)%9 == 1.  Currently this gives a TypeError (why not a
> ValueError?) so changing the behaviour seems safe.

ValueError would make more sense to me too.  I expect it was TypeError for
compatibility with older exceptions raised by the float version.


From fdrake at acm.org  Tue May  4 13:32:46 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue May  4 13:33:00 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: <2my8o8p0c4.fsf@starship.python.net>
References: <4097CFEB.1090500@interlink.com.au>
	<2my8o8p0c4.fsf@starship.python.net>
Message-ID: <200405041332.46873.fdrake@acm.org>

Anthony Baxter <anthony@interlink.com.au> writes:
 > My plan is to cut a release candidate for 2.3.4 next
 > Thursday (13th April) Australian time (so Wednesday night

On Tuesday 04 May 2004 01:29 pm, Michael Hudson wrote:
 > You mean May, presumably?

We just let the Aussies *think* they're a few hours ahead of us, but it looks 
like the truth is out...


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From tim.one at comcast.net  Tue May  4 13:47:28 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue May  4 13:47:32 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: <4097CFEB.1090500@interlink.com.au>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEPBKKAB.tim.one@comcast.net>

[Anthony Baxter]
> My plan is to cut a release candidate for 2.3.4 next
> Thursday (13th April) Australian time (so Wednesday night
                 ^^May in North America <wink>
> for the merkins), and a release the following Thursday
> (the 20th).
>
> Can we say that the 2.3 branch is frozen from Tuesday? As
> this is likely to be the final 2.3 release (modulo any
> suprising new bugs), I'd like the time to try and nail
> this one.

I tried to ask about 2.3.4 plans earlier here today, but because I used a
different account to mail to python-dev, it's still sitting in a moderator
queue.  The same msg can be found here:

    http://mail.zope.org/pipermail/zope-dev/2004-May/022838.html

Short course is that a serious bug with threads on Linux was reported almost
a year ago, against Python 2.2.2:

    http://www.python.org/sf/756924

but wasn't properly diagnosed before last night (by Andrew Langmead; he
added his diagnosis to the Python bug report today).

I know this affects the OP, and affects Zope on Linux systems using
LinuxThreads (but not those using NPTL).  I don't know whether anyone is
actively pursuing a workaround.  If someone is, or if a LinuxHead on
python-dev wants to, I think it's important to squash this bug in 2.3.4.

I don't have any other 2.3.4 issues in mind.


From Tino.Lange at isg.de  Tue May  4 17:06:12 2004
From: Tino.Lange at isg.de (Tino Lange)
Date: Tue May  4 17:07:15 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: <4097CFEB.1090500@interlink.com.au>
References: <4097CFEB.1090500@interlink.com.au>
Message-ID: <409805C4.1020609@isg.de>

Anthony Baxter wrote:

Hi Anthony, hi Python-Dev!

> My plan is to cut a release candidate for 2.3.4 next
> Thursday (13th April) Australian time (so Wednesday night
> for the merkins), and a release the following Thursday
> (the 20th).

> this is likely to be the final 2.3 release 

Is it possible to include the patch #945642 in 2.3.4?

If not, this would mean that *no version Python of 2.3(.x)* was ever 
able to handle nonblocking SSL socket connections properly (what the 
former Python-Versions 2.x could).

See: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=945642&group_id=5470

Cheers,

Tino





From tim.one at comcast.net  Tue May  4 17:53:47 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue May  4 17:53:59 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move For ward
In-Reply-To: <409721F0.1020102@interlink.com.au>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEALKLAB.tim.one@comcast.net>

[Jeremy]
>> We are in rather dire need of docs and a revised PEP.

[Anthony]
> Indeed. The timeframe we came up with at PyCon had the
> generator expressions being landed "in the first week of
> May". At the moment the discussion seems to be spinning
> wheels -- it looks (to me) like there's not even 100%
> agreement from everyone that they should even go into
> 2.4 at all.

Well, only one opinion really counts, and the discussion has more the feel
of whining at a wake about the deceased's bad qualities <wink>.

Alas, the PEP is indeed out of date, and I can't remember exactly which set
of gimmicks Guido favored.  The PEP currently says early-binding is the
rule, but I *think* Guido favored mostly-late binding -- which is late
binding, except that the iterable in the leftmost for-clause is evaluated at
once.  So

    ... (e1 for stuff in e2 for morestuff in e3 ...) ...

is effectively replaced by

    def __g(primary):
        for stuff in primary:
           for morestuff in e3:
              ...
                 yield e1

    ... __g(e2) ...


> I'm not fussed if we slip a week or two here, but if
> it slips another 6 weeks, say, then the release time
> line could be in some trouble.

Is anyone working (or willing to work) on the PEP, implementation, and docs
now?  I can't make time for it, but if I could I think the semantic issues
are settled (in Guido's mind, and possibly in some email of his I can't find
now).


From pedronis at bluewin.ch  Tue May  4 18:03:37 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Tue May  4 18:03:57 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move For ward
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEALKLAB.tim.one@comcast.net>
References: <409721F0.1020102@interlink.com.au>
Message-ID: <5.2.1.1.0.20040505000153.0303da68@pop.bluewin.ch>

At 17:53 04.05.2004 -0400, Tim Peters wrote:
>[Jeremy]
> >> We are in rather dire need of docs and a revised PEP.
>
>[Anthony]
> > Indeed. The timeframe we came up with at PyCon had the
> > generator expressions being landed "in the first week of
> > May". At the moment the discussion seems to be spinning
> > wheels -- it looks (to me) like there's not even 100%
> > agreement from everyone that they should even go into
> > 2.4 at all.
>
>Well, only one opinion really counts, and the discussion has more the feel
>of whining at a wake about the deceased's bad qualities <wink>.

well, the fact is that this particular deceased is going to stay with us <wink>



>Is anyone working (or willing to work) on the PEP, implementation, and docs
>now?  I can't make time for it, but if I could I think the semantic issues
>are settled (in Guido's mind, and possibly in some email of his I can't find
>now).

that's the one (I think)

http://mail.python.org/pipermail/python-dev/2004-April/044555.html



From jjl at pobox.com  Tue May  4 18:16:18 2004
From: jjl at pobox.com (John J Lee)
Date: Tue May  4 18:17:02 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: <4097CFEB.1090500@interlink.com.au>
References: <4097CFEB.1090500@interlink.com.au>
Message-ID: <Pine.LNX.4.58.0405042305020.659@alice>

On Wed, 5 May 2004, Anthony Baxter wrote:
[...]
> Can we say that the 2.3 branch is frozen from Tuesday? As
> this is likely to be the final 2.3 release (modulo any
> suprising new bugs), I'd like the time to try and nail
> this one.
[...]

Could somebody apply this before the 2.3 freeze?

http://www.python.org/sf/851736


Trivial fix to an obviously-broken class, has been sitting there, with
test, for some months.


John

From aml at world.std.com  Tue May  4 18:45:00 2004
From: aml at world.std.com (Andrew M. Langmead)
Date: Tue May  4 18:45:11 2004
Subject: [Python-Dev] signals and linuxthreads (was: Python 2.3.4 schedule)
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEPBKKAB.tim.one@comcast.net>
References: <4097CFEB.1090500@interlink.com.au>
	<LNBBLJKPBEHFEDALKOLCOEPBKKAB.tim.one@comcast.net>
Message-ID: <20040504224500.GA2826220@world.std.com>

On Tue, May 04, 2004 at 01:47:28PM -0400, Tim Peters wrote:
>     http://mail.zope.org/pipermail/zope-dev/2004-May/022838.html
> 
> Short course is that a serious bug with threads on Linux was reported almost
> a year ago, against Python 2.2.2:

I have a patch that seems to fix things
<http://world.std.com/~aml/python/linuxthreads-patch/linuxthreads.diffs>
and a c program to test for LinuxThreads behavior that would be suitable
for autoconf
<http://world.std.com/~aml/python/linuxthreads-patch/test.c>

I just need a bit of time testing the test suite changes for
how they run on other platforms, and to integrate the test program
into configure.in.

It puts all the changes into a REDIRECT_SIGNALS_TO_MAIN_THREAD
define, so there should be any effect on other platforms.

From guido at python.org  Tue May  4 19:44:53 2004
From: guido at python.org (Guido van Rossum)
Date: Tue May  4 19:48:53 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move For ward
In-Reply-To: Your message of "Tue, 04 May 2004 17:53:47 EDT."
	<LNBBLJKPBEHFEDALKOLCIEALKLAB.tim.one@comcast.net> 
References: <LNBBLJKPBEHFEDALKOLCIEALKLAB.tim.one@comcast.net> 
Message-ID: <200405042344.i44Nisv05570@guido.python.org>

> Alas, the PEP is indeed out of date, and I can't remember exactly
> which set of gimmicks Guido favored.  The PEP currently says
> early-binding is the rule, but I *think* Guido favored mostly-late
> binding -- which is late binding, except that the iterable in the
> leftmost for-clause is evaluated at once.  So
> 
>     ... (e1 for stuff in e2 for morestuff in e3 ...) ...
> 
> is effectively replaced by
> 
>     def __g(primary):
>         for stuff in primary:
>            for morestuff in e3:
>               ...
>                  yield e1
> 
>     ... __g(e2) ...

Exactly.

> > I'm not fussed if we slip a week or two here, but if
> > it slips another 6 weeks, say, then the release time
> > line could be in some trouble.
> 
> Is anyone working (or willing to work) on the PEP, implementation,
> and docs now?  I can't make time for it, but if I could I think the
> semantic issues are settled (in Guido's mind, and possibly in some
> email of his I can't find now).

How about this one. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim at zope.com  Tue May  4 11:03:41 2004
From: tim at zope.com (Tim Peters)
Date: Tue May  4 20:14:33 2004
Subject: [Python-Dev] RE: [Zope-dev] Segfault and Deadlock
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEKEKKAB.tim@zope.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEOEKKAB.tim@zope.com>

As Andrew Langmead has already discovered, the LinuxThreads issue with
SIGSEGV was reported on the Python bug tracker almost a year ago (well,
reported, but not diagnosed):

    SIGSEGV causes hung threads (Linux)
    http://www.python.org/sf/756924

Looks like:

    can't CNTRL-C when running os.system in a thread
    http://www.python.org/sf/756940

is related.

python-dev'ers, do we have a release manager for 2.3.4 (I didn't see a
resolution to the brouhaha at the end of March)?  If so, is 2.3.4 still
planned for this month?

tick-tock-tick-tock-ing-ly y'rs  - tim


From trevp at trevp.net  Tue May  4 22:49:38 2004
From: trevp at trevp.net (Trevor Perrin)
Date: Tue May  4 22:49:46 2004
Subject: [Python-Dev] negative modular exponentiation
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEOPKKAB.tim.one@comcast.net>
References: <20040504145852.GA27574@vicky.ecs.soton.ac.uk>
Message-ID: <5.2.0.9.0.20040504151541.02675cb8@pop.sbcglobal.yahoo.com>


Hi Armin, Guido, Tim, everyone,

Thanks for looking at this patch!  I've submitted a few other 
crypto-related patches too.  They're all waiting eagerly for good, loving 
people to adopt them :-)...

They've all got basically the same motivation.  If I explain that, it might 
help people decide whether they're worth it or not:

Rationale for some crypto patches in 2.4
-----------------------------------------
Almost all crypto protocols require the same fast or OS-dependent 
primitives: modular exponentiation, hashing, ciphers, and 
entropy-harvesting.  Everything else can be done in Python.

To get these primitives in Python you could install M2Crypto (needing 
OpenSSL and SWIG) or pycrypto (needing GMP).  But that complicates software 
distribution.  So I was hoping the few missing primitives could be added to 
2.4, enabling fast and portable pure-Python crypto.

I've submitted the below patches.  The first 2 aren't that important:

#923643, long <-> byte-string:  Convenience methods so longs are easier to 
read/write in network protocols:
   n = long(someString, 256)  # str -> long
   s = n.tostring()  # long -> str

#935454, sha256 module:  Since SHA-256 is preferred over SHA-1 for certain 
things these days.

#934711, platform-specific entropy:  An os.urandom() function providing the 
platform equivalent of /dev/urandom (CryptGenRandom on Windows, presumably 
SecureRandom on Java).

#936813, faster modular exponentiation...

The only thing left is block ciphers (AES, maybe 3DES).  Those would be 
easy (just copy from pycrypto) but I think this is courting enough 
controversy at the moment..


Anyways, about modular exponentiation:

On crypto-sized numbers (1000-8000 bits) on a P4, Python mod-exp is ~6-12x 
slower than GMP.  With this patch it's only ~3-4x slower, which is on par 
with another C library I tested (LibTomCrypt).

Mod-exp pretty much determines how long an SSL or IPsec handshake takes, or 
signing/decrypting a PGP message.  These often take a sizeable fraction of 
a second, so they're worth speeding up.

[Tim]
>The Python long implementation has a
>great track record wrt portability and lack of bugs, and gonzo optimizations
>work against that.

Good points.  The speedup is worth it to me, but it certainly adds 
complexity (though these are boring, straightforward optimizations that 
everyone does, and are described in lots of books.  I ain't good enough to 
do gonzo :-)

>For example, did Python really need the Karatsuba
>multiplication gimmick?  I added the patch for it after a lot of reworking,
>but overall I'm not sure it was a net win;

It seems a good speedup to me, particularly if the cutoff is increased - on 
4000-8000 bit numbers it gives around 10-30% faster multiplication, and on 
bigger numbers it's wildly faster.

>  and the current patch in question
>boosts the smallest integer at which Karatsuba triggers to such a large
>value that anyone reaching it would probably be much better off using a GMP
>wrapper anyway.

You'd be better off using GMP at pretty much any value.  But if you want to 
widely distribute a tool or product that uses crypto, it's a pain for every 
user to have to do that.


>One thing I noticed a few times in the patch was neglecting to check C calls
>for error returns.  Even _PyLong_NumBits() can fail!

Darn, thanks.  The 2 problems I see are in the long <-> byte-string code 
(which I left in the longobject.c patch), not the mod-exp code, which I 
went over more carefully.  I'll fix and re-submit the long<->byte-string 
patch..


Trevor 


From niemeyer at conectiva.com  Wed May  5 11:17:03 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Wed May  5 11:17:05 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: <4097CFEB.1090500@interlink.com.au>
References: <4097CFEB.1090500@interlink.com.au>
Message-ID: <20040505151703.GA2958@burma.localdomain>

Hello Anthony,

> Can we say that the 2.3 branch is frozen from Tuesday? As
> this is likely to be the final 2.3 release (modulo any
> suprising new bugs), I'd like the time to try and nail
> this one.

I'd like to include some of the proposed gettext improvements
into that candidate, but I'm not sure I'll get the necessary
time, since we're in a rush for a new distribution release.
Will try to prepare a patch in the weekend.

-- 
Gustavo Niemeyer
http://niemeyer.net

From guido at python.org  Wed May  5 11:28:17 2004
From: guido at python.org (Guido van Rossum)
Date: Wed May  5 11:28:27 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: Your message of "Wed, 05 May 2004 12:17:03 -0300."
	<20040505151703.GA2958@burma.localdomain> 
References: <4097CFEB.1090500@interlink.com.au>  
	<20040505151703.GA2958@burma.localdomain> 
Message-ID: <200405051528.i45FSH213712@guido.python.org>

> > Can we say that the 2.3 branch is frozen from Tuesday? As
> > this is likely to be the final 2.3 release (modulo any
> > suprising new bugs), I'd like the time to try and nail
> > this one.
> 
> I'd like to include some of the proposed gettext improvements
> into that candidate, but I'm not sure I'll get the necessary
> time, since we're in a rush for a new distribution release.
> Will try to prepare a patch in the weekend.

Huh?  We're talking about a bugfix release for 2.3 here.  Not a good
time for improvements, which sounds to me like a euphemism for new
features, not for bug fixes.

Even if they are backwards compatible, I'm still -1 on adding new
features to a bugfix release.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed May  5 11:28:35 2004
From: guido at python.org (Guido van Rossum)
Date: Wed May  5 11:29:27 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move For ward
In-Reply-To: Your message of "Tue, 04 May 2004 14:54:08 +1000."
	<409721F0.1020102@interlink.com.au> 
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D4A4@usahm010.exmi01.exch.eds.com>
	<1083638573.17924.9.camel@localhost.localdomain> 
	<409721F0.1020102@interlink.com.au> 
Message-ID: <200405051528.i45FSZh13723@guido.python.org>

> Indeed. The timeframe we came up with at PyCon had the
> generator expressions being landed "in the first week of
> May". At the moment the discussion seems to be spinning
> wheels -- it looks (to me) like there's not even 100%
> agreement from everyone that they should even go into
> 2.4 at all.

There doesn't need to be 100% agreement.  This thread (which is indeed
getting nowhere) started with my assertion that we should do it, using
late binding.  I haven't changed my mind yet.  I'm still waiting for
someone to volunteer to review the code once more and check it in.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed May  5 11:28:44 2004
From: guido at python.org (Guido van Rossum)
Date: Wed May  5 11:29:39 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: Your message of "Wed, 05 May 2004 03:16:27 +1000."
	<4097CFEB.1090500@interlink.com.au> 
References: <4097CFEB.1090500@interlink.com.au> 
Message-ID: <200405051528.i45FSiN13730@guido.python.org>

> My plan is to cut a release candidate for 2.3.4 next
> Thursday (13th April) Australian time (so Wednesday night
> for the merkins), and a release the following Thursday
> (the 20th).
> 
> Can we say that the 2.3 branch is frozen from Tuesday? As
> this is likely to be the final 2.3 release (modulo any
> suprising new bugs), I'd like the time to try and nail
> this one.

Either you meant May or that mail spent a really long time in some
mail queue... ;-)

One thing I'd love to see backported to 2.3.4 is the fix to
platform.py to properly recognize Windows XP when win32 is detected.
Mind if I backport this one personally?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pedronis at bluewin.ch  Wed May  5 11:44:54 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Wed May  5 11:45:00 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move For ward
In-Reply-To: <200405051528.i45FSZh13723@guido.python.org>
References: <Your message of "Tue,
	04 May 2004 14:54:08 +1000." <409721F0.1020102@interlink.com.au>
	<B8CDFB11BB44D411B8E600508BDF076C1E96D4A4@usahm010.exmi01.exch.eds.com>
	<1083638573.17924.9.camel@localhost.localdomain>
	<409721F0.1020102@interlink.com.au>
Message-ID: <5.2.1.1.0.20040505174120.030398a0@pop.bluewin.ch>

At 08:28 05.05.2004 -0700, Guido van Rossum wrote:
> > Indeed. The timeframe we came up with at PyCon had the
> > generator expressions being landed "in the first week of
> > May". At the moment the discussion seems to be spinning
> > wheels -- it looks (to me) like there's not even 100%
> > agreement from everyone that they should even go into
> > 2.4 at all.
>
>There doesn't need to be 100% agreement.  This thread (which is indeed
>getting nowhere)

one likely aspect here is that someone should write the doc for this feature,
and consider whether it should be in the tutorial (at least list comprehensions
are introduced there), and see how it reads.


>started with my assertion that we should do it, using
>late binding.  I haven't changed my mind yet.  I'm still waiting for
>someone to volunteer to review the code once more and check it in.
>
>--Guido van Rossum (home page: http://www.python.org/~guido/)
>
>_______________________________________________
>Python-Dev mailing list
>Python-Dev@python.org
>http://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: 
>http://mail.python.org/mailman/options/python-dev/pedronis%40bluewin.ch


From anthony at interlink.com.au  Wed May  5 12:14:14 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed May  5 12:14:27 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: <200405051528.i45FSiN13730@guido.python.org>
References: <4097CFEB.1090500@interlink.com.au>
	<200405051528.i45FSiN13730@guido.python.org>
Message-ID: <409912D6.6070105@interlink.com.au>

Guido van Rossum wrote:
>>My plan is to cut a release candidate for 2.3.4 next
>>Thursday (13th April) Australian time (so Wednesday night
>>for the merkins), and a release the following Thursday
>>(the 20th).
>>
>>Can we say that the 2.3 branch is frozen from Tuesday? As
>>this is likely to be the final 2.3 release (modulo any
>>suprising new bugs), I'd like the time to try and nail
>>this one.
> 
> 
> Either you meant May or that mail spent a really long time in some
> mail queue... ;-)

Yes, I meant May.

> One thing I'd love to see backported to 2.3.4 is the fix to
> platform.py to properly recognize Windows XP when win32 is detected.
> Mind if I backport this one personally?

Go for it.



-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From anthony at interlink.com.au  Wed May  5 12:16:34 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed May  5 12:16:50 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move For ward
In-Reply-To: <200405051528.i45FSZh13723@guido.python.org>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D4A4@usahm010.exmi01.exch.eds.com>
	<1083638573.17924.9.camel@localhost.localdomain>
	<409721F0.1020102@interlink.com.au>
	<200405051528.i45FSZh13723@guido.python.org>
Message-ID: <40991362.7050207@interlink.com.au>

Guido van Rossum wrote:
> There doesn't need to be 100% agreement.  This thread (which is indeed
> getting nowhere) started with my assertion that we should do it, using
> late binding.  I haven't changed my mind yet.  I'm still waiting for
> someone to volunteer to review the code once more and check it in.

Sure. And this was what I was referring to. Lotsa noise, little
traction. <wink>

This isn't meant as a slap at anyone - I am quite aware of time
limitations. I guess I was hoping to prod someone into stepping
forward and doing the work.


Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From niemeyer at conectiva.com  Wed May  5 12:54:09 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Wed May  5 12:54:04 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: <200405051528.i45FSH213712@guido.python.org>
References: <4097CFEB.1090500@interlink.com.au>
	<20040505151703.GA2958@burma.localdomain>
	<200405051528.i45FSH213712@guido.python.org>
Message-ID: <20040505165409.GA3644@burma.localdomain>

> Huh?  We're talking about a bugfix release for 2.3 here.  Not a good
> time for improvements, which sounds to me like a euphemism for new
> features, not for bug fixes.

Adding a gettext compatible lgettext() might be a good thing,
as well as bind_textdomain_codeset() (which pygtk users certainly
have missed already) but I have to admit this is not a bug fix
per se.

> Even if they are backwards compatible, I'm still -1 on adding new
> features to a bugfix release.

Acknowledged. Will prepare a patch for the next major asap.

-- 
Gustavo Niemeyer
http://niemeyer.net

From mal at egenix.com  Wed May  5 14:14:32 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed May  5 14:14:41 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: <200405051528.i45FSiN13730@guido.python.org>
References: <4097CFEB.1090500@interlink.com.au>
	<200405051528.i45FSiN13730@guido.python.org>
Message-ID: <40992F08.6080904@egenix.com>

Guido van Rossum wrote:
>>My plan is to cut a release candidate for 2.3.4 next
>>Thursday (13th April) Australian time (so Wednesday night
>>for the merkins), and a release the following Thursday
>>(the 20th).
>>
>>Can we say that the 2.3 branch is frozen from Tuesday? As
>>this is likely to be the final 2.3 release (modulo any
>>suprising new bugs), I'd like the time to try and nail
>>this one.
> 
> 
> Either you meant May or that mail spent a really long time in some
> mail queue... ;-)
> 
> One thing I'd love to see backported to 2.3.4 is the fix to
> platform.py to properly recognize Windows XP when win32 is detected.
> Mind if I backport this one personally?

This should only be a matter of copy the HEAD version to the release
branch. Note that the HEAD version is slightly different from
the one on the 2.3 branch - I haven't backported the cache
changes I made.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 05 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
EuroPython 2004, G?teborg, Sweden                           32 days left

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From bac at OCF.Berkeley.EDU  Wed May  5 14:23:16 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed May  5 14:23:30 2004
Subject: [Python-Dev] Passing compile(...,'exec') code to 'eval'
Message-ID: <40993114.5060708@ocf.berkeley.edu>

In preparations for my coding my thesis I have been trying to figure out 
how a local variable could be assigned to without me explicitly knowing 
during compilation.  Obviously 'exec' can.  But I wasn't sure about 'eval'.

Reading the docs, I didn't think it could since 'eval', when taking a 
string, only evaluates expressions.  But what about code objects?  The 
docs say, "The code object must have been  compiled passing 'eval' as 
the kind argument".  But I didn't read that when I started testing.

This is when I discovered you *can* pass in something using 'compile' 
with the kind argument of "exec"::

 >>> x
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
NameError: name 'x' is not defined
[16425 refs]
 >>> eval(compile("x = 1", "<string>", "exec"))
[16426 refs]
 >>> x
1

Is this a bug, or are the docs wrong?  I am hoping it is the former 
since if it is the latter my thesis just got a big caveat pasted into it 
about how 'eval' can cause problems and invalidate the type inferencing 
in irreparable ways.

-Brett

From jim.jewett at eds.com  Wed May  5 14:31:01 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Wed May  5 14:31:07 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D4A8@usahm010.exmi01.exch.eds.com>


Guido van Rossum:
>... mostly-late binding -- which is late binding, except that 
> the iterable in the leftmost for-clause is evaluated at once. 

Did you really mean leftmost, or all-but-the-rightmost?  

Special-casing the left-most causes the two-generator version
to behave as expected, but it makes the three-generator case
even more confusing.

Perhaps the entire early/late binding problem comes from trying 
to treat multiple generators as a special case.

For a single-generator expression, late binding makes sense
(or why use a free variable), but early binding would also
work most of the time.

For multiple generators, the current behavior is that the
final generator is exhausted by the first loop, so the other
generators really only need one value each.  The only problem
with that is that it isn't usually what people want.

If users do want a cross-product, then either the values have 
to be stored somewhere, or python has to create a whole series 
of "equivalent" generators.  Python can't do this any better
than the user, so why not make the storage explicit?  

	(x,y,z for x in e1 for y in e2 for z in e3)

<==>

	for x in e1.__iter__():
		for x in e2.__iter__()
			for z in e3.__iter__()
				yield (x,y,z)

It is the user's responsibility to ensure that e2 and e3 can
return an iterator multiple times, perhaps by wrapping them
in a list or tuple.

-jJ

From tim.one at comcast.net  Wed May  5 15:12:35 2004
From: tim.one at comcast.net (Tim Peters)
Date: Wed May  5 15:12:42 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D4A8@usahm010.exmi01.exch.eds.com>
Message-ID: <E1BLRou-0003Po-D5@mail.python.org>

[Jewett, Jim J]
> Guido van Rossum:
> >... mostly-late binding -- which is late binding, except that
> > the iterable in the leftmost for-clause is evaluated at once.

Guido didn't write that -- I did.  Guido agreed with it, though.

> Did you really mean leftmost,

Yes.

> or all-but-the-rightmost?

No.

...

Sorry, I didn't understand the rest of the msg, so I have to skip the rest.
Try a concrete example?



From jim.jewett at eds.com  Wed May  5 16:43:13 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Wed May  5 16:44:08 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D4A9@usahm010.exmi01.exch.eds.com>

> Try a concrete example?

	t1 = (1,2,3)
	t2 = ("a", "b", "c")
	t3 = ("first", "second", "third")

	(x,y,z for x in t1 for y in t2 for z in t3)

Do What I Mean is the crossproduct, 27 elements long,
which you get with a list comprehension.

This will happen if at least (all but one) generators
are pre-evaluated.  It will also happen if at least
(all but one) generators are restarted after they
raise StopIterator.

Doing strictly what happens today gets only

	(1,'a',"first"), (1,'a',"second"), (1,'a',"third")

because t3 has raised StopIteration by the time t1 or t2
advance.  Effectively, only the last iterator matters.

Keeping the last value of an iterator also adds

	(1, 'b', "third"), (1, 'c', "third"), 
	(2, 'c', "third"), (3, '3', "third")

for seven values -- but perhaps not a well-chosen seven.

Pre-evaluating only the leftmost gets all combinations of
(1,2,3) and ("first", "second", "third"), but still misses
items like (2, 'b', "second").  Since these are in the
middle of the series, they are the most likely to be 
forgotten during testing.

Restarting the second (and later) iterators for each value
of the first iterator does the right thing, without requiring
any magical pre-evaluation.  (Unless the iterator can't be
restarted -- in which case there is no right thing, and the
user should have decided what to cache.)

Thus my suggestion of
	(x,y,z for x in t1 for y in t2 for z in t3)
<==>
	for x in t1.__iter__():
		for y in t2.__iter__():
			for z in t3.__iter__():
				yield x,y,z

For the example data, this creates 3 separate iterators
on the letters and nine iterators on the words - but 
produces the expected result, without having to store
or pre-evaluate anything.

-jJ

From dave at boost-consulting.com  Wed May  5 17:02:17 2004
From: dave at boost-consulting.com (David Abrahams)
Date: Wed May  5 17:10:24 2004
Subject: [Python-Dev] horrible thought
Message-ID: <uy8o6bnae.fsf@boost-consulting.com>


Remember the vote on adding conditional expressions?

I'm afraid that the imminent proliferation of xxxcomps may indicate
the "right" syntax for this feature, if it ever does get added.

Something like

          (a else b if condition)

Now I've done my bad deed for the day.  Cheers,
Dave

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com


From tim.one at comcast.net  Wed May  5 17:24:40 2004
From: tim.one at comcast.net (Tim Peters)
Date: Wed May  5 17:38:57 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D4A9@usahm010.exmi01.exch.eds.com>
Message-ID: <E1BLTsi-0006oi-GM@mail.python.org>

[Jewett, Jim J]
> 	t1 = (1,2,3)
> 	t2 = ("a", "b", "c")
> 	t3 = ("first", "second", "third")
>
> 	(x,y,z for x in t1 for y in t2 for z in t3)
>
> Do What I Mean is the crossproduct, 27 elements long,
> which you get with a list comprehension.

Well, that expression by itself will result in a generator-iterator getting
created, with no actual results.  If you put it in a context that drives
generation, then it will deliver 27 results, and regardless of whether
early-binding, late-binding, or mostly-late binding is used (although the
latter two are sensitive to changes in the bindings for t2 and t3, if any
such occur between the time the genexp is created and the time the genexp is
driven).  For mostly-late binding, it's the same as this Python program
today (stuffing the genexp in a list() call to force it to do something):

"""
t1 = (1,2,3)
t2 = ("a", "b", "c")
t3 = ("first", "second", "third")

def __g(primary):
    for x in primary:
        for y in t2:
            for z in t3:
                yield x, y, z

print list(__g(t1))
"""

Try running that, and see what you get.

> This will happen if at least (all but one) generators
> are pre-evaluated.

There are no generators in your example, apart from the one created by the
genexp itself.  You may be confusing generators with iterable objects.

> It will also happen if at least (all but one) generators are 
> restarted after they raise StopIterator.
>
> Doing strictly what happens today gets only
> 
> 	(1,'a',"first"), (1,'a',"second"), (1,'a',"third")

No -- run the above.  You'll get the 27 you expect.

> because t3 has raised StopIteration by the time t1 or t2
> advance.

The innermost "for" loop swallows StopIteration each of the 9 times it
occurs, and the outer for loops never see it.  They keep going.  This is how
"for" loops always work (nothing is changing wrt that).



From guido at python.org  Wed May  5 19:20:00 2004
From: guido at python.org (Guido van Rossum)
Date: Wed May  5 19:20:20 2004
Subject: [Python-Dev] Passing compile(...,'exec') code to 'eval'
In-Reply-To: Your message of "Wed, 05 May 2004 11:23:16 PDT."
	<40993114.5060708@ocf.berkeley.edu> 
References: <40993114.5060708@ocf.berkeley.edu> 
Message-ID: <200405052320.i45NK0d14202@guido.python.org>

> This is when I discovered you *can* pass in something using 'compile' 
> with the kind argument of "exec"::
> 
>  >>> x
> Traceback (most recent call last):
>    File "<stdin>", line 1, in ?
> NameError: name 'x' is not defined
> [16425 refs]
>  >>> eval(compile("x = 1", "<string>", "exec"))
> [16426 refs]
>  >>> x
> 1
> 
> Is this a bug, or are the docs wrong?  I am hoping it is the former 
> since if it is the latter my thesis just got a big caveat pasted into it 
> about how 'eval' can cause problems and invalidate the type inferencing 
> in irreparable ways.

You shouldn't do this, and if you do, you will pay for your sin in a
special kind of hell set aside for people who take the implementation
as the spec.  It's hard to detect such undefined usage without making
eval() (and possibly everything) slower though.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido at python.org  Wed May  5 19:16:45 2004
From: guido at python.org (Guido van Rossum)
Date: Wed May  5 19:21:07 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: Your message of "Wed, 05 May 2004 13:54:09 -0300."
	<20040505165409.GA3644@burma.localdomain> 
References: <4097CFEB.1090500@interlink.com.au>
	<20040505151703.GA2958@burma.localdomain>
	<200405051528.i45FSH213712@guido.python.org> 
	<20040505165409.GA3644@burma.localdomain> 
Message-ID: <200405052316.i45NGkE14180@guido.python.org>

> > Huh?  We're talking about a bugfix release for 2.3 here.  Not a good
> > time for improvements, which sounds to me like a euphemism for new
> > features, not for bug fixes.
> 
> Adding a gettext compatible lgettext() might be a good thing,
> as well as bind_textdomain_codeset() (which pygtk users certainly
> have missed already) but I have to admit this is not a bug fix
> per se.

Definitely -1 for 2.3.4.

> > Even if they are backwards compatible, I'm still -1 on adding new
> > features to a bugfix release.
> 
> Acknowledged. Will prepare a patch for the next major asap.

Sure, if that's what Barry agreed to.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed May  5 20:30:25 2004
From: guido at python.org (Guido van Rossum)
Date: Wed May  5 20:30:38 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: Your message of "Wed, 05 May 2004 20:14:32 +0200."
	<40992F08.6080904@egenix.com> 
References: <4097CFEB.1090500@interlink.com.au>
	<200405051528.i45FSiN13730@guido.python.org> 
	<40992F08.6080904@egenix.com> 
Message-ID: <200405060030.i460UQA14346@guido.python.org>

> This should only be a matter of copy the HEAD version to the release
> branch. Note that the HEAD version is slightly different from
> the one on the 2.3 branch - I haven't backported the cache
> changes I made.

OK, done.  The cache fix is now in 2.3.4 too.

Note to Brett C and Raymond H: one of you added a call to reversed()
to platform.py; that's a no-no since the source code needs to remain
compatible with 1.5.2, per a comment in the source.  I've added
platform.py to PEP 291.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From ndbecker2 at verizon.net  Wed May  5 20:44:19 2004
From: ndbecker2 at verizon.net (Neal Becker)
Date: Wed May  5 20:44:29 2004
Subject: [Python-Dev] distutils problem: /usr/lib64/python2.3 on x86_64
Message-ID: <c7c1p3$b6d$1@sea.gmane.org>

I just tried to build wxPython on x86_64 (athlon64) on Fedora core3 test 3. 
It didn't work.

It looks like python is a 64-bit exec on this platform, and there
is /usr/lib64/python2.3/ with what looks like the usual stuff.  There is
also a /usr/lib/python2.3, with not much in it, but I guess if you install
a 32-bit python it would go there.

Anyway, distutils/sysconfig.py 
def get_python_lib(...
      if os.name == "posix":
        libpython = os.path.join(prefix,
                                 "lib", "python" + get_python_version())

This is wrong, and it doesn't seem to give me anyway to overide the choice. 
Is there a fix?


From pedronis at bluewin.ch  Wed May  5 20:59:40 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Wed May  5 20:59:46 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move Forward
In-Reply-To: <200404231826.i3NIQxM18955@guido.python.org>
Message-ID: <5.2.1.1.0.20040506021127.030398a0@pop.bluewin.ch>

At 11:26 23.04.2004 -0700, Guido van Rossum wrote:

>My reasons for (still!) preferring late binding are:
>
>(a) I find it hard to defend automatic variable capture given Python's
>     general late-binding semantics

I have been thinking about reason (a), the point is that a genexp is not 
simply a new way to introduce a closure, it is not simply sugar over lambda 
or def. A genexp is sugar for defining on the fly a generator closure and 
second instatiating it! Now there are two parts to that, it is true that 
closures in Python have late-bindings semantics but when they are deployed 
people in most cases use idioms that circuvent and neutralize late-bindings.

def make_show_action(msg):
    def action(evt,msg):
        show(msg)
    return action

used as

for act,msg in msgs:
   pnl.add(Button(act,action=make_show_action(msg))

or

for act,msg in msgs:
   pnl.add(Button(act,lambda msg=msg: show(msg)))

in fact here lambda has late-bindings semantics and we eschew them by not 
having any free variable at all.

Of course here enters (b), mixing for and genexp is advanced use. That's 
really a gut-feelings call at this point.

For most practical cases, for closures, late-bindings sematics seems 
something to keep under control, it's an implementation detail to allow 
recursive and mutally recursive nested functions.

So to repeat genexp have no simple direct correspondence to a lambda or a 
def. They are sugar for some more involved code, so I think it is more a 
call of what is the general idiomatic form for that kind of code, also 
maybe considering that the genexp is hiding the details of what is really 
going on (kind of supplying sheep's clothes to a wolf).

Maybe the sane thing is late-bindings, maybe early. But I don't think that 
as suggested by (a) that there is some deep design logic internal to Python 
at stake here.

At most the argument could be that late-bindings make things easier to 
explain because they are kind of like lambdas. Maybe, maybe not. Then there 
is (b).

>(b) I believe that use cases needing early binding are uncommon and
>     strained: they all involve creating a list of generator
>     expressions, which IMO is a pretty unusual thing to do

regards


From martin at v.loewis.de  Wed May  5 21:20:58 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed May  5 21:20:54 2004
Subject: [Python-Dev] distutils problem: /usr/lib64/python2.3 on x86_64
In-Reply-To: <c7c1p3$b6d$1@sea.gmane.org>
References: <c7c1p3$b6d$1@sea.gmane.org>
Message-ID: <409992FA.2090009@v.loewis.de>

Neal Becker wrote:
> This is wrong, and it doesn't seem to give me anyway to overide the choice. 
> Is there a fix?

Not that I know of. Did you even manage to install Python without
python.org/sf/926209

Regards,
Martin


From ndbecker2 at verizon.net  Wed May  5 21:54:31 2004
From: ndbecker2 at verizon.net (Neal Becker)
Date: Wed May  5 21:54:38 2004
Subject: [Python-Dev] Re: distutils problem: /usr/lib64/python2.3 on x86_64
References: <c7c1p3$b6d$1@sea.gmane.org> <409992FA.2090009@v.loewis.de>
Message-ID: <c7c5sn$hg5$1@sea.gmane.org>

"Martin v. L?wis" wrote:

> Neal Becker wrote:
>> This is wrong, and it doesn't seem to give me anyway to overide the
>> choice. Is there a fix?
> 
> Not that I know of. Did you even manage to install Python without
> python.org/sf/926209
> 

python-2.3 is already standard on Fedora Core3 Test3, I didn't build it.


From greg at cosc.canterbury.ac.nz  Thu May  6 02:06:28 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu May  6 02:07:01 2004
Subject: [Python-Dev] PEP 328: import...as with relative import
Message-ID: <200405060606.i4666Stx024952@cosc353.cosc.canterbury.ac.nz>

There's been some discussion on c.l.py about whether the
new relative import system should allow things like

  import ..abc as def

It seems the PEP currently disallows this, but nobody
seems to be able to adequately explain why. Is there
any technical reason it couldn't be done?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From mal at egenix.com  Thu May  6 03:11:53 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu May  6 03:11:53 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: <200405060030.i460UQA14346@guido.python.org>
References: <4097CFEB.1090500@interlink.com.au>	<200405051528.i45FSiN13730@guido.python.org>
	<40992F08.6080904@egenix.com>
	<200405060030.i460UQA14346@guido.python.org>
Message-ID: <4099E539.4090405@egenix.com>

Guido van Rossum wrote:
>>This should only be a matter of copy the HEAD version to the release
>>branch. Note that the HEAD version is slightly different from
>>the one on the 2.3 branch - I haven't backported the cache
>>changes I made.
> 
> OK, done.  The cache fix is now in 2.3.4 too.

Thanks.

> Note to Brett C and Raymond H: one of you added a call to reversed()
> to platform.py; that's a no-no since the source code needs to remain
> compatible with 1.5.2, per a comment in the source.  I've added
> platform.py to PEP 291.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 06 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
EuroPython 2004, G?teborg, Sweden                           31 days left

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From arigo at tunes.org  Thu May  6 04:51:16 2004
From: arigo at tunes.org (Armin Rigo)
Date: Thu May  6 04:54:36 2004
Subject: [Python-Dev] Passing compile(...,'exec') code to 'eval'
In-Reply-To: <40993114.5060708@ocf.berkeley.edu>
References: <40993114.5060708@ocf.berkeley.edu>
Message-ID: <20040506085116.GA17048@vicky.ecs.soton.ac.uk>

Hello Brett,

On Wed, May 05, 2004 at 11:23:16AM -0700, Brett C. wrote:
> Is this a bug, or are the docs wrong?  I am hoping it is the former 
> since if it is the latter my thesis just got a big caveat pasted into it 
> about how 'eval' can cause problems and invalidate the type inferencing 
> in irreparable ways.

There is another improbable way to bind variables with eval, with list
comprehensions:

>>> eval("[x for x in [3]]")
[3]
>>> x
3

If the goal is not security but type inference, I guess both this and the
compile() trick should be considered as outside any reasonable spec.


A bientot,

Armin.


From arigo at tunes.org  Thu May  6 05:10:18 2004
From: arigo at tunes.org (Armin Rigo)
Date: Thu May  6 05:13:34 2004
Subject: [Python-Dev] PEP 289 - Generator Expressions - Let's Move For ward
In-Reply-To: <200405051528.i45FSZh13723@guido.python.org>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D4A4@usahm010.exmi01.exch.eds.com>
	<1083638573.17924.9.camel@localhost.localdomain>
	<409721F0.1020102@interlink.com.au>
	<200405051528.i45FSZh13723@guido.python.org>
Message-ID: <20040506091018.GB17048@vicky.ecs.soton.ac.uk>

Hello,

On Wed, May 05, 2004 at 08:28:35AM -0700, Guido van Rossum wrote:
> There doesn't need to be 100% agreement.  This thread (which is indeed
> getting nowhere)

I'll leave it alone, then.  It can't go anywhere as long as we'll keep
opposing a bunch of concrete examples and an intuition.  I retract because
your intuition is firmly grounded, and I respect that even if I disagree -- 
I, for example, have the intuition that high-level languages should be faster
than low-level ones, despite all the examples :-)


Armin


From niemeyer at conectiva.com  Thu May  6 09:55:37 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Thu May  6 09:55:28 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: <200405052316.i45NGkE14180@guido.python.org>
References: <4097CFEB.1090500@interlink.com.au>
	<20040505151703.GA2958@burma.localdomain>
	<200405051528.i45FSH213712@guido.python.org>
	<20040505165409.GA3644@burma.localdomain>
	<200405052316.i45NGkE14180@guido.python.org>
Message-ID: <20040506135537.GA3440@burma.localdomain>

> > Acknowledged. Will prepare a patch for the next major asap.
> 
> Sure, if that's what Barry agreed to.

I said I'm going to prepare a patch. After that, I'll ask for
comments on this list. That's what I have always done about
things I'm not sure about, and that's what I would do even if
you hadn't mentioned the 2.3.4 issue.

-- 
Gustavo Niemeyer
http://niemeyer.net

From guido at python.org  Thu May  6 10:04:33 2004
From: guido at python.org (Guido van Rossum)
Date: Thu May  6 10:04:40 2004
Subject: [Python-Dev] PEP 328: import...as with relative import
In-Reply-To: Your message of "Thu, 06 May 2004 18:06:28 +1200."
	<200405060606.i4666Stx024952@cosc353.cosc.canterbury.ac.nz> 
References: <200405060606.i4666Stx024952@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200405061404.i46E4Xg16720@guido.python.org>

> There's been some discussion on c.l.py about whether the
> new relative import system should allow things like
> 
>   import ..abc as def
> 
> It seems the PEP currently disallows this, but nobody
> seems to be able to adequately explain why. Is there
> any technical reason it couldn't be done?

Because "import ..abc" is clearly nonsense, since after "import X" you
should be able to the imported entity as "X" in your program, and
"..abc" is not a valid expression.

This doesn't strictly apply to "import ..abc as xyz" ("def" was a poor
choice for "X" :-), but it doesn't seem worth stretching the syntax
when you can already do that using "from .. import abc as xyz".

--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Thu May  6 10:08:35 2004
From: barry at python.org (Barry Warsaw)
Date: Thu May  6 10:08:43 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: <200405052316.i45NGkE14180@guido.python.org>
References: <4097CFEB.1090500@interlink.com.au>
	<20040505151703.GA2958@burma.localdomain>
	<200405051528.i45FSH213712@guido.python.org>
	<20040505165409.GA3644@burma.localdomain>
	<200405052316.i45NGkE14180@guido.python.org>
Message-ID: <1083852514.28538.95.camel@anthem.wooz.org>

On Wed, 2004-05-05 at 19:16, Guido van Rossum wrote:

> > > Even if they are backwards compatible, I'm still -1 on adding new
> > > features to a bugfix release.
> > 
> > Acknowledged. Will prepare a patch for the next major asap.
> 
> Sure, if that's what Barry agreed to.

Martin and I are on the same page about this: no change to the existing
API, but +1 on adding new functions to provide the desired feature.  You
can assign a patch to me, but be sure it includes documentation and unit
tests.

-Barry



From gtalvola at nameconnector.com  Thu May  6 17:03:43 2004
From: gtalvola at nameconnector.com (Geoffrey Talvola)
Date: Thu May  6 17:03:52 2004
Subject: [Python-Dev] Python 2.3.4 schedule
Message-ID: <61957B071FF421419E567A28A45C7FE59AFB0A@mailbox.nameconnector.com>

Tino Lange wrote:
> Is it possible to include the patch #945642 in 2.3.4?
> 
> If not, this would mean that *no version Python of 2.3(.x)* was ever
> able to handle nonblocking SSL socket connections properly (what the
> former Python-Versions 2.x could).
> 
> See:
>
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=945642&group_id
=5470 

I'd like to see this in 2.3.4 too; since I was responsible for the breakage
in the first place, it will help assuage my guilt to see this fix get in...

- Geoff

From jim.jewett at eds.com  Thu May  6 17:19:51 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Thu May  6 17:22:26 2004
Subject: [Python-Dev] what level of backward compatibility?  how
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D4AD@usahm010.exmi01.exch.eds.com>

urlparse.py uses lists of strings to indicate which
protocols have which properties.

For instance:

    uses_relative = ['ftp', 'http', 'gopher', 'nntp', 'imap',	 
                     'wais', 'file', 'https', 'shttp', 'mms',
                     'prospero', 'rtsp', 'rtspu', '']

Logically, these are sets, rather than lists.

CVS change 143 changed the lists to sets.

Unfortunately, it breaks client code that did

    urlparse.uses_relative.append('my_protocol')

which was as close to a documented API as existed.

(1)  Is this an OK breakage with the 2.4 switch?
(2)  If not, should this change be backed out, or 
replaced with some set subclass that takes a few of
the list methods?

-jJ

From martin at v.loewis.de  Thu May  6 17:34:47 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu May  6 17:34:44 2004
Subject: [Python-Dev] what level of backward compatibility?  how
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D4AD@usahm010.exmi01.exch.eds.com>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D4AD@usahm010.exmi01.exch.eds.com>
Message-ID: <409AAF77.20900@v.loewis.de>

Jewett, Jim J wrote:
> Unfortunately, it breaks client code that did
> 
>     urlparse.uses_relative.append('my_protocol')
> 
> which was as close to a documented API as existed.

Why are you saying that (or: what does that mean)?

The uses_relative attribute of urlparse was never
documented, AFAICT.

> (1)  Is this an OK breakage with the 2.4 switch?

If it wasn't documented, it is OK to break it, but
it should be mentioned in whatsnew24.tex.

Regards,
Martin


From greg at cosc.canterbury.ac.nz  Thu May  6 20:39:12 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu May  6 20:39:20 2004
Subject: [Python-Dev] PEP 328: import...as with relative import
In-Reply-To: <200405061404.i46E4Xg16720@guido.python.org>
Message-ID: <200405070039.i470dChE026277@cosc353.cosc.canterbury.ac.nz>

Guido:

> This doesn't strictly apply to "import ..abc as xyz", but it doesn't
> seem worth stretching the syntax when you can already do that using
> "from .. import abc as xyz".

I can see your point of view. It just seems a bit strange coming at
from the other direction -- if you can say 'import a.b as z' there
doesn't seem to be any good reason not to say 'import ..a.b as z'. I
can imagine people getting tripped up by this.

I guess you're right that it doesn't matter much either way.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From guido at python.org  Fri May  7 00:24:55 2004
From: guido at python.org (Guido van Rossum)
Date: Fri May  7 00:25:06 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: Your message of "Thu, 06 May 2004 17:03:43 EDT."
	<61957B071FF421419E567A28A45C7FE59AFB0A@mailbox.nameconnector.com> 
References: <61957B071FF421419E567A28A45C7FE59AFB0A@mailbox.nameconnector.com>
Message-ID: <200405070424.i474OtT17542@guido.python.org>

> Tino Lange wrote:
> > Is it possible to include the patch #945642 in 2.3.4?
> > 
> > If not, this would mean that *no version Python of 2.3(.x)* was ever
> > able to handle nonblocking SSL socket connections properly (what the
> > former Python-Versions 2.x could).
> > 
> > See:
> >
> https://sourceforge.net/tracker/?func=detail&atid=305470&aid=945642&group_id
> =5470 
> 
> I'd like to see this in 2.3.4 too; since I was responsible for the breakage
> in the first place, it will help assuage my guilt to see this fix get in...

Haven't seen the patch, but the description of the fix makes me go "+1
please!" :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri May  7 00:29:01 2004
From: guido at python.org (Guido van Rossum)
Date: Fri May  7 00:29:13 2004
Subject: [Python-Dev] what level of backward compatibility? how
In-Reply-To: Your message of "Thu, 06 May 2004 23:34:47 +0200."
	<409AAF77.20900@v.loewis.de> 
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D4AD@usahm010.exmi01.exch.eds.com>
	<409AAF77.20900@v.loewis.de> 
Message-ID: <200405070429.i474T1t17567@guido.python.org>

> Jewett, Jim J wrote:
> > Unfortunately, it breaks client code that did
> > 
> >     urlparse.uses_relative.append('my_protocol')
> > 
> > which was as close to a documented API as existed.
> 
> Why are you saying that (or: what does that mean)?
> 
> The uses_relative attribute of urlparse was never
> documented, AFAICT.
> 
> > (1)  Is this an OK breakage with the 2.4 switch?
> 
> If it wasn't documented, it is OK to break it, but
> it should be mentioned in whatsnew24.tex.

I have a different POV.  I don't think there is a compelling reason to
change this attribute into a set (I doubt it's so time-critical as to
make a difference) and given that the attribute isn't flagged as
"private" by having a name starting with underscore, I think the
change ought to be reverted.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From cgipm at unixbeast.com  Fri May  7 00:33:16 2004
From: cgipm at unixbeast.com (Dave Hastings)
Date: Fri May  7 00:33:21 2004
Subject: [Python-Dev] New Subscriber
Message-ID: <20040507043316.GB45563@unixbeast.com>

Hello fellow developers.  My name is David Hastings and I am a new subscriber to the python-dev list.  I'm also a little new to python but I can get around in it.  I have been learning how to use the language for about a year now.  First, some information about myself.  I am a senior attending the University of Minnesota.  Today was the last day of classes for the spring semester and also the last day of my college career.  I am receiving a Bachelor of Science in Computer Science degree.  Man it feels good to be done !  This last semester a friend (Michael Sampson) and I took an independent study class where we sign up with a Professor and decide on a project and work on it until the end of the semester.  The project that was decided upon, with the advice of the Professor, was to make a CGIpm python module that reflected CGI.pm for perl.  We noticed that there is a current cgi module for python and even a cgipm module.  These modules are very useful, but did not include all the functions that CGI.pm for perl provides.  It turns out that my Professor really likes python and would have loved to see a CGIpm python module for python that was like the CGI.pm module for perl.  (I think he is a bit turned off by perl, more because it is a language that is not that easy to pick up unless you are familiar with unix, namely sed and awk.)  We decided to take the oportunity and wrote CGIpm.py.  I'm not going to go into a lot of details in this e-mail since it is an introductory e-mail, but the first version of CGIpm.py is complete.  I feel that this e-mail is a bit long winded as is, so I think I will end it.  I can not wait to start reading about all the new ideas, development, and contributions that this list will provide.

Dave 

From paul at prescod.net  Fri May  7 01:15:36 2004
From: paul at prescod.net (Paul Prescod)
Date: Fri May  7 01:15:52 2004
Subject: [Python-Dev] New Subscriber
In-Reply-To: <20040507043316.GB45563@unixbeast.com>
References: <20040507043316.GB45563@unixbeast.com>
Message-ID: <409B1B78.9020301@prescod.net>

Welcome to the Python world. If you are just new to Python, you will get 
a much more varied and interesting impression of Python from 
python-list[1] . Python-dev is where the people evolving the language 
argue about the precise semantics of generator expressions and what 
bytecodes will give the best performance. You are of course welcome to 
stick around but be aware that the discussion is very focused unlike on 
python-list where everything Pythonic is welcome.

  Paul Prescod

[1] http://mail.python.org/mailman/listinfo/python-list


From cgipm at unixbeast.com  Fri May  7 01:34:16 2004
From: cgipm at unixbeast.com (David Hastings)
Date: Fri May  7 01:40:17 2004
Subject: [Python-Dev] Re: New Subscriber
References: <20040507043316.GB45563@unixbeast.com>
	<409B1B78.9020301@prescod.net>
Message-ID: <loom.20040507T072908-344@post.gmane.org>

Paul Prescod <paul <at> prescod.net> writes:

> 
> Welcome to the Python world. If you are just new to Python, you will get 
> a much more varied and interesting impression of Python from 
> python-list[1] . Python-dev is where the people evolving the language 
> argue about the precise semantics of generator expressions and what 
> bytecodes will give the best performance. You are of course welcome to 
> stick around but be aware that the discussion is very focused unlike on 
> python-list where everything Pythonic is welcome.
> 
>   Paul Prescod
> 
> [1] http://mail.python.org/mailman/listinfo/python-list
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev <at> python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-
dev%40m.gmane.org
> 
> 

Oh.  I'm sorry.  I thought that "...and first drafts of PEPs are often posted 
here for initial review ..."  meant that I was supposed to post here if I had 
something new that might be welcomed.  I got that quote off of the help 
section in the python page in the section about the python-dev list.  Is there 
a better place where I should post this ?

thanks,
Dave






From paul at prescod.net  Fri May  7 01:49:22 2004
From: paul at prescod.net (Paul Prescod)
Date: Fri May  7 01:50:57 2004
Subject: [Python-Dev] Re: New Subscriber
In-Reply-To: <loom.20040507T072908-344@post.gmane.org>
References: <20040507043316.GB45563@unixbeast.com>	<409B1B78.9020301@prescod.net>
	<loom.20040507T072908-344@post.gmane.org>
Message-ID: <409B2362.5040108@prescod.net>

David Hastings wrote:

> ...
> 
> Oh.  I'm sorry.  I thought that "...and first drafts of PEPs are often posted 
> here for initial review ..."  meant that I was supposed to post here if I had 
> something new that might be welcomed.  I got that quote off of the help 
> section in the python page in the section about the python-dev list.  Is there 
> a better place where I should post this ?

I didn't know that you had a PEP in mind. Even so, it seems that your 
PEP would be about a new module (which is a tough sell) and a module 
that replaces an existing module (an even tougher sell). You should 
refine the idea in python-list before writing up a formal PEP.

  Paul Prescod



From jcarlson at uci.edu  Fri May  7 01:53:38 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Fri May  7 01:57:58 2004
Subject: [Python-Dev] Re: New Subscriber
In-Reply-To: <loom.20040507T072908-344@post.gmane.org>
References: <409B1B78.9020301@prescod.net>
	<loom.20040507T072908-344@post.gmane.org>
Message-ID: <20040506225112.F8C1.JCARLSON@uci.edu>

> Oh.  I'm sorry.  I thought that "...and first drafts of PEPs are often posted 
> here for initial review ..."  meant that I was supposed to post here if I had 
> something new that might be welcomed.  I got that quote off of the help 
> section in the python page in the section about the python-dev list.  Is there 
> a better place where I should post this ?

Having a PEP is different from what you posted.  Read a few of the PEPs
listed here: http://www.python.org/peps/ to get a feel for how PEPs are
written.

 - Josiah


From cgipm at unixbeast.com  Fri May  7 02:03:49 2004
From: cgipm at unixbeast.com (David Hastings)
Date: Fri May  7 02:03:56 2004
Subject: [Python-Dev] Re: New Subscriber
References: <20040507043316.GB45563@unixbeast.com>	<409B1B78.9020301@prescod.net>
	<loom.20040507T072908-344@post.gmane.org>
	<409B2362.5040108@prescod.net>
Message-ID: <loom.20040507T075920-258@post.gmane.org>

Paul Prescod <paul <at> prescod.net> writes:
> 
> 
> I didn't know that you had a PEP in mind. Even so, it seems that your 
> PEP would be about a new module (which is a tough sell) and a module 
> that replaces an existing module (an even tougher sell). You should 
> refine the idea in python-list before writing up a formal PEP.
> 
>   Paul Prescod
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev <at> python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-
dev%40m.gmane.org
> 
> 
Once again, I'm sorry.  I hope there is a way that these e-mails can be 
removed from the web page.  My intention wasn't to cause clutter.  Also, my 
intention was not to replace any modules.  I just wanted to build on it I 
guess.  We did actually use some of the funcionality of the cgi module.  I'm 
obviously in the wrong place.

Later all !
Dave




From fdrake at acm.org  Fri May  7 02:10:20 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri May  7 02:10:35 2004
Subject: [Python-Dev] what level of backward compatibility? how
In-Reply-To: <200405070429.i474T1t17567@guido.python.org>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D4AD@usahm010.exmi01.exch.eds.com>
	<409AAF77.20900@v.loewis.de>
	<200405070429.i474T1t17567@guido.python.org>
Message-ID: <200405070210.20467.fdrake@acm.org>

On Friday 07 May 2004 12:29 am, Guido van Rossum wrote:
 > I have a different POV.  I don't think there is a compelling reason to
 > change this attribute into a set (I doubt it's so time-critical as to
 > make a difference) and given that the attribute isn't flagged as
 > "private" by having a name starting with underscore, I think the
 > change ought to be reverted.

I'd like to suggest that we either document the "right way" to extend the 
information to support new URL schemes, and possibly add a function as the 
way to do that.  Having to update a whole set of lists from outside the 
module seems a poor and error-prone way to do this.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From ncoghlan at iinet.net.au  Fri May  7 08:07:04 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Fri May  7 08:07:46 2004
Subject: [Python-Dev] PEP 328: import...as with relative import
In-Reply-To: <200405070039.i470dChE026277@cosc353.cosc.canterbury.ac.nz>
References: <200405070039.i470dChE026277@cosc353.cosc.canterbury.ac.nz>
Message-ID: <409B7BE8.70904@iinet.net.au>

Greg Ewing wrote:
> I can see your point of view. It just seems a bit strange coming at
> from the other direction -- if you can say 'import a.b as z' there
> doesn't seem to be any good reason not to say 'import ..a.b as z'. I
> can imagine people getting tripped up by this.
> 
> I guess you're right that it doesn't matter much either way.

The other point in favour of the current way is that a bare import 
statement is *always* absolute, which further encourages absolute 
imports as the standard approach.

It's only once someone starts playing with the 'from' syntax for import 
that they need to know about the difference between absolute and 
relative import.

Regards,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268

From ncoghlan at iinet.net.au  Fri May  7 08:18:17 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Fri May  7 08:18:24 2004
Subject: [Python-Dev] horrible thought
In-Reply-To: <uy8o6bnae.fsf@boost-consulting.com>
References: <uy8o6bnae.fsf@boost-consulting.com>
Message-ID: <409B7E89.9070402@iinet.net.au>

David Abrahams wrote:

> Remember the vote on adding conditional expressions?
> 
> I'm afraid that the imminent proliferation of xxxcomps may indicate
> the "right" syntax for this feature, if it ever does get added.
> 
> Something like
> 
>           (a else b if condition)
> 
> Now I've done my bad deed for the day.

That's actually an interesting thought. Is there anything to be learned 
about conditionals by considering the syntactic sugar for list comps and 
gen comps?

List comp [f(x) for x in original if p(x)]:

   def _result(f, p, original):
     lc = []
     for x in original:
       if p(x):
         lc.append(f(x))
     return lc


Gen comp (f(x) for x in original if p(x)):

   def _result(f, p, original):
     for x in original:
       if p(x):
         yield f(x)

Conditional (syntactic sugar not defined):
   def _result(a, b, p):
     if p:
       return a
     else:
       return b

Nope, still suggests (a if p else b) to me.

With nesting then being (a if p1 else b if p2 else c if p3 else d).

OK, done now. We can just let that discussion go back into its box. . .

Regards,
Nick.


-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268

From jhylton at google.com  Fri May  7 08:55:45 2004
From: jhylton at google.com (Jeremy Hylton)
Date: Fri May  7 08:55:55 2004
Subject: [Python-Dev] what level of backward compatibility? how
In-Reply-To: <200405070210.20467.fdrake@acm.org>
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D4AD@usahm010.exmi01.exch.eds.com>
	<409AAF77.20900@v.loewis.de>
	<200405070429.i474T1t17567@guido.python.org>
	<200405070210.20467.fdrake@acm.org>
Message-ID: <1083934545.4530.64.camel@slothrop.corp.google.com>

On Fri, 2004-05-07 at 02:10, Fred L. Drake, Jr. wrote:
> On Friday 07 May 2004 12:29 am, Guido van Rossum wrote:
>  > I have a different POV.  I don't think there is a compelling reason to
>  > change this attribute into a set (I doubt it's so time-critical as to
>  > make a difference) and given that the attribute isn't flagged as
>  > "private" by having a name starting with underscore, I think the
>  > change ought to be reverted.
> 
> I'd like to suggest that we either document the "right way" to extend the 
> information to support new URL schemes, and possibly add a function as the 
> way to do that.  Having to update a whole set of lists from outside the 
> module seems a poor and error-prone way to do this.

It sounds like appending to the list was the right way up until now. 
That is, extending the list of protocols is a reasonable feature and
using a list with a non-underscore name provided one obvious was to do
that.

Jeremy



From guido at python.org  Fri May  7 10:31:46 2004
From: guido at python.org (Guido van Rossum)
Date: Fri May  7 10:31:51 2004
Subject: [Python-Dev] what level of backward compatibility? how
In-Reply-To: Your message of "Fri, 07 May 2004 02:10:20 EDT."
	<200405070210.20467.fdrake@acm.org> 
References: <B8CDFB11BB44D411B8E600508BDF076C1E96D4AD@usahm010.exmi01.exch.eds.com>
	<409AAF77.20900@v.loewis.de>
	<200405070429.i474T1t17567@guido.python.org> 
	<200405070210.20467.fdrake@acm.org> 
Message-ID: <200405071431.i47EVkw01614@guido.python.org>

>  > I have a different POV.  I don't think there is a compelling reason to
>  > change this attribute into a set (I doubt it's so time-critical as to
>  > make a difference) and given that the attribute isn't flagged as
>  > "private" by having a name starting with underscore, I think the
>  > change ought to be reverted.

[Fred]
> I'd like to suggest that we either document the "right way" to extend the 
> information to support new URL schemes, and possibly add a function as the 
> way to do that.  Having to update a whole set of lists from outside the 
> module seems a poor and error-prone way to do this.

Let's be conservative and not change the code just because the
(rarely-needed) API is a bit ugly.  Let's just document the existing
way and not declare old code that uses it out of date.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri May  7 10:38:57 2004
From: guido at python.org (Guido van Rossum)
Date: Fri May  7 10:39:03 2004
Subject: [Python-Dev] New Subscriber
In-Reply-To: Your message of "Thu, 06 May 2004 23:33:16 CDT."
	<20040507043316.GB45563@unixbeast.com> 
References: <20040507043316.GB45563@unixbeast.com> 
Message-ID: <200405071438.i47EcvT01660@guido.python.org>

Seems David Hastings got a rather cold welcome.  I would simply have
suggested that in general, these days, new modules aren't incorporated
into the standard library before they've seen a lot of use "in the
field"; and that the appropriate way for David to promote his module
would be a webpage (maybe on SourceForge?) and announcements to
comp.lang.python.announce.

Another suggestion for David is to rename the module -- "CGIpm" seems
a rather strange name for a module; the ".pm" in the original stands
for "Perl Module" which obviously isn't appropriate for a Python
translation. :-)

David, success with your new module, and say hi to your Prof!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one at comcast.net  Fri May  7 11:39:52 2004
From: tim.one at comcast.net (Tim Peters)
Date: Fri May  7 11:39:55 2004
Subject: [Python-Dev] Re: New Subscriber
In-Reply-To: <loom.20040507T075920-258@post.gmane.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEAGKMAB.tim.one@comcast.net>

[David Hastings]
> Once again, I'm sorry.

Stop apologizing -- it's embarrassing <wink>.

> I hope there is a way that these e-mails can be removed from the web
> page.  My intention wasn't to cause clutter.

It's not a problem.

> Also, my intention was not to replace any modules.  I just wanted to
> build on it I guess.  We did actually use some of the funcionality of
> the cgi module.  I'm obviously in the wrong place.

Not necessarily.  While enthusiasm is rare on python-dev some weeks, that
doesn't mean we all object to it <wink>.  My advice is to read a bit first
about how Python development gets done:

    http://www.python.org/dev/

You're already getting the sharp end of what the "Why Work on Python?" link
there describes as

    The most important skill Python can teach is the delicate skill of
    working in a diverse group.

I hope you stick around, and bring your module enhancements to a successful
release!


From FBatista at uniFON.com.ar  Fri May  7 15:03:27 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Fri May  7 15:06:06 2004
Subject: [Python-Dev] Re: New Subscriber
Message-ID: <A128D751272CD411BC9200508BC2194D03383963@escpl.tcp.com.ar>

[Tim Peters]

#- doesn't mean we all object to it <wink>.  My advice is to 
#- read a bit first
#- about how Python development gets done:
#- 
#-     http://www.python.org/dev/

David, I repost these Tim statements because as a new Python developer I
found something amusing.

If you follow the rules, you'll have a lot of valuable humans that donate
their time to help you to help everybody.

With follow the rules I mean to get used to how things get done here (for
example: learn how to write a PEP, that is a wonderful tool).

Welcome!

.	Facundo

From doko at cs.tu-berlin.de  Sun May  9 09:13:31 2004
From: doko at cs.tu-berlin.de (Matthias Klose)
Date: Sun May  9 09:13:35 2004
Subject: [Python-Dev] Source of the FAQ?
Message-ID: <16542.11899.318040.19846@gargle.gargle.HOWL>

Is the source text for the FAQ's available in some form, that could be
included in a packaged form for distribution?

Thanks, Matthias

From bac at OCF.Berkeley.EDU  Sun May  9 14:43:40 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun May  9 14:43:54 2004
Subject: [Python-Dev] Source of the FAQ?
In-Reply-To: <16542.11899.318040.19846@gargle.gargle.HOWL>
References: <16542.11899.318040.19846@gargle.gargle.HOWL>
Message-ID: <409E7BDC.30805@ocf.berkeley.edu>

Matthias Klose wrote:
> Is the source text for the FAQ's available in some form, that could be
> included in a packaged form for distribution?
> 

For basically all pages on the python.org web site, just change the 
extension from html to ht and you will get the text version that is 
worked on by the web team.  It will most likely either be HTML or reST. 
  Luckily most of the FAQs have been converted to reST for easier 
editing (thanks to AM Kuchling and anyone else who has helped out on the 
conversion).

-Brett

From bac at OCF.Berkeley.EDU  Sun May  9 17:04:51 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun May  9 17:05:05 2004
Subject: [Python-Dev] python-dev Summary for 2004-04-01 through 2004-04-30
	[rough draft]
Message-ID: <409E9CF3.5060504@ocf.berkeley.edu>

By not getting into any details on method decorators and relative 
imports I was actually able to finish this summary.  =)

I would like to get this out Wednesday night, so please send in your 
corrections by then.  Thanks.

-------------------------------

=====================
Summary Announcements
=====================
Well, as you may have noticed, I wasn't able to keep up the pace of 
releasing the summaries twice in a month.  Joys of school.  Hopefully 
over the summer I will be able to semi-monthly releases since I will 
only be working and not have homework hanging over my head 24/7.

To give everyone a a heads-up, a release candidate for Python 2.3.4 is 
going to be released May 13/14 (depending where you are on the globe). 
If you have time, please download it and run the regression test suite 
(instructions can be found in the module documentation for the 'test' 
package) and report any issues you have.


=========
Summaries
=========
--------------------------------------------------------
Method decorators and the discussion that will never end
--------------------------------------------------------
Method decorators and `PEP 318`_ were discussed ad nausea this past 
month.  Read the PEP for the updated details.

.. _PEP 318: http://www.python.org/peps/pep-0318.html

Contributing threads:
   - `PEP 318 -- a couple use cases 
<http://mail.python.org/pipermail/python-dev/2004-April/043902.html>`__
   - `PEP 318: Decorators last before colon/bakeoff 
<http://mail.python.org/pipermail/python-dev/2004-April/043911.html>`__
   - `PEP 318 bake-off? 
<http://mail.python.org/pipermail/python-dev/2004-April/043952.html>`__
   - `PEP 318: Is decorators the right term for these things? 
<http://mail.python.org/pipermail/python-dev/2004-April/043973.html>`__
   - `Re: PEP 318: Let's propose some useful built-in decorators 
<http://mail.python.org/pipermail/python-dev/2004-April/043975.html>`__
   - `Help with PEP 318 
<http://mail.python.org/pipermail/python-dev/2004-April/044011.html>`__
   - `PEP 318: Properties 
<http://mail.python.org/pipermail/python-dev/2004-April/044012.html>`__
   - `PEP 318: More examples of decorator use 
<http://mail.python.org/pipermail/python-dev/2004-April/044132.html>`__
   - `PEP 318: How I would implement decorators 
<http://mail.python.org/pipermail/python-dev/2004-April/044133.html>`__
   - `PEP 318 - check for consensus 
<http://mail.python.org/pipermail/python-dev/2004-April/044251.html>`__


------------------------------------------------------------
Relative imports: the *other* discussion that will never end
------------------------------------------------------------
`PEP 328`_, which covers relative imports, has been updated to cover its 
lengthy discussion.  It seems the PEP is pretty much finalized.

.. _PEP 328: http://www.python.org/peps/pep-0328.html

Contributing threads:
   - `Re: PEP 328 -- relative and multi-line import 
<http://mail.python.org/pipermail/python-dev/2004-April/043978.html>`__
   - ` Some comments on PEP 328 (absolute/relative imports) 
<http://mail.python.org/pipermail/python-dev/2004-April/043998.html>`__
   - `PEP 328: __path__ 
<http://mail.python.org/pipermail/python-dev/2004-April/044017.html>`__
   - `from ...sys import path 
<http://mail.python.org/pipermail/python-dev/2004-April/044024.html>`__


--------------------
stdlib: Python or C?
--------------------
The question of whether stdlib modules should be coded in Python or C 
came up again.  Armin Rigo posted some numbers comparing performance of 
heapq using the Python version both in a stock Python interpreter and in 
Psyco_ and then the C version.  Armin's quick timings showed that the 
Python version of heapq was actually after than the C version when used 
in Psyco.

If it is possible to get the Python version to go faster, should it and 
future modules be coded in Python?  The usual arguments over maintenance 
and readability were brought up.  In the end the Python version of heapq 
was put back into the library with the C version renamed to _heapq.

Contributing threads:
   - `Python is faster than C 
<http://mail.python.org/pipermail/python-dev/2004-April/044054.html>`__


-----------------------------------
Getting an Interest Group on Artima
-----------------------------------
Bill Venners of Artima_ asked if there was anyone who wanted to act as a 
moderator for an interest group for Python on the site.  If you are 
interested, read the email for details.

.. _Artima: http://www.artima.com/

Contributing threads:
   - `Python Interest Group 
<http://mail.python.org/pipermail/python-dev/2004-April/044131.html>`__


-------------------------------
Informative reprs for iterators
-------------------------------
Raymond Hettinger suggested having the repr of iterators for built-in 
types list the first three objects returned by the iterator.  That way 
you would actually know what was in the iterator.  It was suggested only 
for iterators where the length of the iterator was known and extracting 
the first three objects from the iterator to display them would be cheap.

In the end, though, it was not added for built-in types.  Some itertools 
iterators, though, did get more informative reprs.

Contributing threads:
   - `More imformative iterator representations 
<http://mail.python.org/pipermail/python-dev/2004-April/044136.html>`__
   - `Proposed iterator representations 
<http://mail.python.org/pipermail/python-dev/2004-April/044166.html>`__


------------------------------
Turning globals into constants
------------------------------
Raymond Hettinger came up with some code that could take a code object 
and make all globals constants, thus speeding up access.  It was 
proposed for the stdlib in `PEP 329`_, but was rejected in the end for 
being too "hackish".

It did bring up the idea of trying to come up with a way to flag 
built-ins that might be masked.  If the compiler knew what built-ins 
were not going to be masked a direct lookup in the built-in namespace 
would be all that was needed to access a built-in.  As it stands now, 
though, since an external module could inject a shadowing function into 
the global namespace of a module (such as having module bar having the 
code ``import foo; foo.len = lambda x: 42`` to shadow 'len' in the 
module foo), Python has to check the global namespace first, and then 
the built-in namespace for all non-local name accesses.  That is costly 
and it could possibly be avoided since the common case is to not shadow 
a built-in.

Guido suggested having programmers have to specify when a built-in could 
be shadowed.  This could be compared to the 'volatile' keyword in C; the 
programmer has to specify when the compiler should not try to optimize 
access.  This seemed like the best way to go since forcing the 
programmer to flag when a built-in will *not* be shadowed would be more 
work for the common case.  Obviously this would all break 
backwards-compatibility and thus would either require a __future__ 
statement initially or would be like true division and always be a 
__future__ import until Python 3.0 .  But it was all just being thrown 
around with no concrete ideas on implementation and such.

.. _PEP 329: http://www.python.org/peps/pep-0329.html

Contributing threads:
   - `Candidate Function Decorator 
<http://mail.python.org/pipermail/python-dev/2004-April/044204.html>`__
   - `PEP 329: Treating Builtins as Constants... 
<http://mail.python.org/pipermail/python-dev/2004-April/044396.html>`__
   - `peps 329, 266, 267 
<http://mail.python.org/pipermail/python-dev/2004-April/044492.html>`__


-----------------------------
Decimal "stuff" was discussed
-----------------------------
Two big topics came up about the Decimal package.  The first one was 
over whether there should be an exponent limit.  The discussion went 
back and forth, but in the end it was decided that having a limit is 
good since it will prevent undetected overflow and underflow.

The second topic was about adding extra methods.  This was shot down 
since the point of the Decimal module is to follow the standard as put 
forth at http://www2.hursley.ibm.com/decimal/decarith.html .  After the 
module has been in the library and seen some wide usage then possible 
additions beyond the standard could be made.

exponent limit
repr (passable to eval if possible)
sticking to spec initially

Contributing threads:
   - `Decimal data type issues 
<http://mail.python.org/pipermail/python-dev/2004-April/044208.html>`__
   - `Decimal conversion to string 
<http://mail.python.org/pipermail/python-dev/2004-April/044269.html>`__
   - `Decimal - context manipulation 
<http://mail.python.org/pipermail/python-dev/2004-April/044485.html>`__


-------------------------------------------------
Magic hashing numbers segues into JIT compilation
-------------------------------------------------
Raymond Hettinger suggested using a different number for hashing since 
it could be expressed in shifts and additions.  But Tim Peters said it 
was a bad idea to futz with the number since the current one had proven 
its usefulness for so long.

Since part of the justification for changing to another number was the 
number of cycles it would take to do the calculations, the discussion 
shifted to JIT compilation.  Clarification for JIT compared to 
specialized compilation (former does single compiled version of a 
function while the latter does multiple versions based on the argument 
types) came up along with wondering how some of the optimizations used 
in the Self_ language could be applied to Python.

.. _Self: http://research.sun.com/self/language.html

Contributing threads:
   - `String hash function multiplier 
<http://mail.python.org/pipermail/python-dev/2004-April/044235.html>`__
   - `Optimization targets - refcount 
<http://mail.python.org/pipermail/python-dev/2004-April/044304.html>`__


----------------------------------
Modules that need some documentin'
----------------------------------
Yours truly compiled a list of modules that need documentation that can 
be found both in the email starting this thread (although I made one 
mistake on that list of including the imp module) or at 
http://www.python.org/cgi-bin/moinmoin/ModulesThatNeedDocs where Michael 
Chermside was nice enough to put it on up on the wiki.

Writing docs for modules is not that hard and is a great way to help 
out.  See http://www.python.org/dev/doc/devel/doc/doc.html on how to do 
documentation using LaTeX.  And if that scares you, even just writing 
the docs in reST_ is helpful since someone else can volunteer to convert 
it to LaTeX.

Contributing threads:
   - `Possible modules that could use docs 
<http://mail.python.org/pipermail/python-dev/2004-April/044392.html>`__


--------------------------------------
A change in the name of the CVS server
--------------------------------------
SourceForge_ started forcing people to use cvs.sf.net:/cvsroot/python as 
the location of the CVS server as compared to the previously supported 
(but not "official") path of cvs.python.sf.net:/cvsroot/python .  Barry 
Warsaw posted a Python script by Greg Ward that would traverse a CVS 
checkout and change the name of the CVS server.  That script can be 
found at 
http://mail.python.org/pipermail/python-dev/2004-April/044593.html .

Contributing threads:
   - `SSH problems getting into SourceForge's CVS? 
<http://mail.python.org/pipermail/python-dev/2004-April/044592.html>`__


---------------------------
A ConfigParser replacement?
---------------------------
Dan Gass presented his `config.py`_ to python-dev to see if there was 
any interest in adding it.  While he was given the standard response 
that no new modules will be accepted for inclusion in the stdlib until 
it has seen widespread acceptance and use by the community, Barry Warsaw 
suggested a possible ConfigParser replacement shootout ala the 
`getopt-sig`_ and how it was decided that optparse (aka Optik_) should 
be the suggested command-line parser used.  This won't happen for 2.4, 
but if people are interested enough this could be done for 2.5 if 
someone decides to spear-head this.

People also immediately pointed out ZConfig_ (by our very own Fred 
Drake) as another possibility.

.. _config.py: http://config-py.sf.net/
.. _getopt-sig: http://mail.python.org/mailman/listinfo/getopt-sig
.. _Optik: http://optik.sourceforge.net/
.. _ZConfig: http://www.zope.org/Members/fdrake/zconfig/

Contributing threads:
   - `Proposal: A more powerful alternative to ConfigParser 
<http://mail.python.org/pipermail/python-dev/2004-April/044520.html>`__
   - `Proposal: A more powerful alternativeto... 
<http://mail.python.org/pipermail/python-dev/2004-April/044529.html>`__


-----------------------------------------
Moving forward with generator expressions
-----------------------------------------
Guido said he wanted to get generator expressions into CVS.  While this 
is good and all, it still doesn't address the whole argument over 
whether early or late binding for variables in the genexps should be 
used (in case you have not been following, early binding would have a 
genexp save the state of the variables used in the genexp at creation 
time while late binding will use the state of the variables at the time 
of each execution of the genexp).  Since Guido prefers the late binding 
for simplicity, he suggested using it for 2.4a1 and a2.  If it was found 
that late binding was bad then for 2.4b1 early binding could be used. 
No one had issues with that idea.

What people did was the whole thing about late bindings period. 
Basically the issue boils down to whether late binding is too much of a 
surprise for the cases where they can have issues.  But Greg Ewing 
pointed out that the use case that genexps were being created for were 
for arguments to functions; a use case where the genexp is used 
immediately and whose possible abuse of variables would not be an issue. 
  It was realized that genexps are not exactly the same as listcomps and 
thus not a direct replacement.  Genexps are meant for places where 
memory is going to be an issue; if a listcomp would do just a good of a 
job, then just go ahead and use a listcomp.

Contributing threads:
   - `PEP 289 - Generator Expressions - Let's Move Forward 
<http://mail.python.org/pipermail/python-dev/2004-April/044555.html>`__


---------------------------------------------------
How should gettext return strings from an .mo file?
---------------------------------------------------
Gustavo Niemeyer noticed that if you called 'gettext.gettext' it would 
return the exact bytes stored in the .mo file.  Apparently `GNU 
gettext`_ translates the bytes according to the current locale instead 
of just passing them through untouched.  Gustavo wanted to make Python 
match the GNU implementation.

Barry Warsaw and Martin v. Loewis disagreed, though.  Since Python's way 
of doing it has been that way for so long, they didn't want to break 
backwards-compatibility.  It was also stated that code should use 
'gettext.ugettext' anyway.

In the end it was agreed upon that if Gustavo wanted this functionality 
he should add another function to 'gettext' (such as 'lgettext') that 
did what he wanted.

.. _GNU gettext: http://www.gnu.org/software/gettext/gettext.html

Contributing threads:
   - `Small issues in gettext support 
<http://mail.python.org/pipermail/python-dev/2004-April/044569.html>`__


-----------------------------------------------
And how long have you been hacking on the core?
-----------------------------------------------
Anthony Baxter, fearless release manager, decided to run the Misc/ACKS 
file through ``cvs annotate`` which prints out who edited every line and 
when it was edited and then he sorted it.  This was to see when people 
were added to the ACKS file and thus find our roughly how long people 
have been hacking on the Python core.

The file wasn't started until January 26, 1994 so a lot of the 
"old-timers" don't have exact dates (of which Anthony is one of them). 
Yours truly was added on July 19, 2002.

Contributing threads:
   - `today's pointless, yet slightly terrifying, factoid(s) 
<http://mail.python.org/pipermail/python-dev/2004-April/044574.html>`__

From fumanchu at amor.org  Sun May  9 19:23:32 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Sun May  9 19:26:24 2004
Subject: [Python-Dev] python-dev Summary for 2004-04-01 through
	2004-04-30[rough draft]
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E5620C7@opus.amorhq.net>

Brett C wrote Yet Another Great Summary:

> Subject: [Python-Dev] python-dev Summary for 2004-04-01 
> through 2004-04-30[rough draft]
> 
> I would like to get this out Wednesday night, so please send in your 
> corrections by then.  Thanks.
>8
> --------------------
> stdlib: Python or C?
> --------------------
> The question of whether stdlib modules should be coded in Python or C 
> came up again.  Armin Rigo posted some numbers comparing 
> performance of 
> heapq using the Python version both in a stock Python 
> interpreter and in 
> Psyco_ and then the C version.  Armin's quick timings showed that the 
> Python version of heapq was actually after than the C version 
> when used in Psyco.

I think you mean "faster than" instead of "after than".

>8
> -----------------------------------------
> Moving forward with generator expressions
> -----------------------------------------
>8
> 
> What people did was the whole thing about late bindings period. 

I don't understand this sentence.

HTH,


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org

From greg at cosc.canterbury.ac.nz  Sun May  9 21:03:29 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Sun May  9 21:03:48 2004
Subject: [Python-Dev] PEP 328: import...as with relative import
In-Reply-To: <409B7BE8.70904@iinet.net.au>
Message-ID: <200405100103.i4A13Tjw031202@cosc353.cosc.canterbury.ac.nz>

Nick Coghlan <ncoghlan@iinet.net.au>:

> The other point in favour of the current way is that a bare import 
> statement is *always* absolute, which further encourages absolute 
> imports as the standard approach.

I don't understand why absolute imports should be regarded
as the "standard approach". Absolute imports are appropriate
for some things, relative imports are appropriate for other
things. You can't say that one is "standard" and the other
isn't.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From bob at redivi.com  Sun May  9 21:17:53 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sun May  9 21:18:03 2004
Subject: [Python-Dev] PEP 328: import...as with relative import
In-Reply-To: <200405100103.i4A13Tjw031202@cosc353.cosc.canterbury.ac.nz>
References: <200405100103.i4A13Tjw031202@cosc353.cosc.canterbury.ac.nz>
Message-ID: <DC3CC2D5-A21F-11D8-8F19-000A95686CD8@redivi.com>


On May 9, 2004, at 9:03 PM, Greg Ewing wrote:

> Nick Coghlan <ncoghlan@iinet.net.au>:
>
>> The other point in favour of the current way is that a bare import
>> statement is *always* absolute, which further encourages absolute
>> imports as the standard approach.
>
> I don't understand why absolute imports should be regarded
> as the "standard approach". Absolute imports are appropriate
> for some things, relative imports are appropriate for other
> things. You can't say that one is "standard" and the other
> isn't.

Either way, right now it's a real pain when you have a relative and 
absolute module with the same name.  You simply can't import the 
absolute one by standard means.  It would be at least nice if you could 
explicitly state that some import should be absolute (given the current 
relative-first search order).  I generally use absolute imports 
everywhere, because I've found it generally avoids problem -- but it 
certainly wasn't my first instinct when learning Python.

-bob


From aahz at pythoncraft.com  Sun May  9 21:44:09 2004
From: aahz at pythoncraft.com (Aahz)
Date: Sun May  9 21:44:13 2004
Subject: [Python-Dev] PEP 328: import...as with relative import
In-Reply-To: <200405100103.i4A13Tjw031202@cosc353.cosc.canterbury.ac.nz>
References: <409B7BE8.70904@iinet.net.au>
	<200405100103.i4A13Tjw031202@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040510014409.GA2603@panix.com>

On Mon, May 10, 2004, Greg Ewing wrote:
> Nick Coghlan <ncoghlan@iinet.net.au>:
>> 
>> The other point in favour of the current way is that a bare import 
>> statement is *always* absolute, which further encourages absolute 
>> imports as the standard approach.
> 
> I don't understand why absolute imports should be regarded as the
> "standard approach". Absolute imports are appropriate for some things,
> relative imports are appropriate for other things. You can't say that
> one is "standard" and the other isn't.

The way I'd put it is that relative imports are used only by experts:
people building chunks of Python code large enough to require packages.
Therefore absolute imports make more sense as the default.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Adopt A Process -- stop killing all your children!

From guido at python.org  Sun May  9 23:56:04 2004
From: guido at python.org (Guido van Rossum)
Date: Sun May  9 23:56:14 2004
Subject: [Python-Dev] PEP 328: import...as with relative import
In-Reply-To: Your message of "Mon, 10 May 2004 13:03:29 +1200."
	<200405100103.i4A13Tjw031202@cosc353.cosc.canterbury.ac.nz> 
References: <200405100103.i4A13Tjw031202@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200405100356.i4A3u5p13615@guido.python.org>

> I don't understand why absolute imports should be regarded
> as the "standard approach". Absolute imports are appropriate
> for some things, relative imports are appropriate for other
> things. You can't say that one is "standard" and the other
> isn't.

Maybe, but somewhere "underneath" all imports are converted to
absolute imports before they are carried out.  This is an important
normalization since every module ought to have exactly one true name,
and that must be its absolute name.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at iinet.net.au  Mon May 10 03:33:13 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Mon May 10 03:33:32 2004
Subject: [Python-Dev] Generator expression used in test suite?
Message-ID: <409F3039.2070009@iinet.net.au>

I just finished setting up a Mandrake 9.2 box for Python development, 
and got a doctest failure in test_deque (after running ./configure, 
make, make test).

It looks like one of the examples (roundrobin) is using a generator 
expression to populate the deque.

Since this will work once generator expressions hit CVS, is there any 
point in raising a bug report for it?

Regards,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268

From ncoghlan at iinet.net.au  Mon May 10 06:06:32 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Mon May 10 06:06:43 2004
Subject: [Python-Dev] SF Bug 933068 - Unexpected result from super() call
Message-ID: <409F5428.20709@iinet.net.au>

This bug report can probably be closed (the reported error is the result 
of supplying 'self' an an argument to the already bound method returned 
from the super call).

The relevant docs for super looked fairly clear to me, too, so I don't 
think it is a documentation bug, either.

Regards,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268


From barry at python.org  Mon May 10 09:50:03 2004
From: barry at python.org (Barry Warsaw)
Date: Mon May 10 09:50:11 2004
Subject: [Python-Dev] Generator expression used in test suite?
In-Reply-To: <409F3039.2070009@iinet.net.au>
References: <409F3039.2070009@iinet.net.au>
Message-ID: <1084197002.1735.451.camel@anthem.wooz.org>

On Mon, 2004-05-10 at 03:33, Nick Coghlan wrote:

> Since this will work once generator expressions hit CVS, is there any 
> point in raising a bug report for it?

Yes.  The standard test suite should never have failures.

-Barry



From ncoghlan at iinet.net.au  Mon May 10 10:27:47 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Mon May 10 10:27:53 2004
Subject: [Python-Dev] Generator expression used in test suite?
In-Reply-To: <1084197002.1735.451.camel@anthem.wooz.org>
References: <409F3039.2070009@iinet.net.au>
	<1084197002.1735.451.camel@anthem.wooz.org>
Message-ID: <409F9163.7020503@iinet.net.au>

Barry Warsaw wrote:

> On Mon, 2004-05-10 at 03:33, Nick Coghlan wrote:
> 
> 
>>Since this will work once generator expressions hit CVS, is there any 
>>point in raising a bug report for it?
> 
> 
> Yes.  The standard test suite should never have failures.

Done. Bug ID is 951270.

http://sourceforge.net/tracker/?func=detail&aid=951270&group_id=5470&atid=105470

The hard fix is to get generator expressions checked in, the easy fix is
to put '[' and ']' around the expression in the roundrobin example.

Regards,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268



From tjreedy at udel.edu  Mon May 10 12:19:49 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon May 10 12:20:00 2004
Subject: [Python-Dev] Is core dump always a bug?  Advice requested
Message-ID: <c7oa37$rq$1@sea.gmane.org>

The following was posted in the last 25 hours on c.l.p:

"Xaver Hinterhuber" <xaver_hinterhuber@web.de> wrote in message
news:c7o6al$i1j$1@online.de...
>     codeObject = new.code(
>       0, # argcount
>       0, # nlocals
>       0, # stacksize
>       0, # flags
>       codeString, # code
>       (), # consts
>       (), # names
>       (), # varnames
>       'content', # filename
>       'content', # name
>       3, # first line number
>       codeString # lnotab
>       )
>     f = new.function(codeObject, dict, 'f')
>     f()
>
> Everything runs fine, until the function is called with f().
> When python tries to execute f(), the core dump happens.
> I don't have any clue why python core dumps.
> The codeString is nothing complex, its a one-liner.

... but, as too typical of new posters, not posted ;-)

My question: I understand that core dumping by the Python interpreter is
normally considered a bug -- one that should be reported on SF -- and
fixed.  But does that principle apply to this sort of use (or probably
abuse) of the new module.  In other words, does the 'no core dump' contract
include a 'byte code compiled by the interpreter' proviso.  Or is ceval
intended to be robust against random strings masquerading as valid byte
code (which would seem require an InvalidByteCode exception).  Or in still
other words, is there a bug that I should encourage reporting of, or is
'don't do that' the extent of a proper response?

Terry J. Reedy





From fumanchu at amor.org  Mon May 10 12:44:19 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Mon May 10 12:47:10 2004
Subject: [Python-Dev] Is core dump always a bug?  Advice requested
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E5620CC@opus.amorhq.net>

Terry Reedy wrote:
> 
> The following was posted in the last 25 hours on c.l.p:
> 
> "Xaver Hinterhuber" <xaver_hinterhuber@web.de> wrote in message
> news:c7o6al$i1j$1@online.de...
> >     codeObject = new.code(
> >       0, # argcount
> >       0, # nlocals
> >       0, # stacksize
> >       0, # flags
> >       codeString, # code
> >       (), # consts
> >       (), # names
> >       (), # varnames
> >       'content', # filename
> >       'content', # name
> >       3, # first line number
> >       codeString # lnotab
> >       )
> >     f = new.function(codeObject, dict, 'f')
> >     f()
> >
> > Everything runs fine, until the function is called with f().
> > When python tries to execute f(), the core dump happens.
> > I don't have any clue why python core dumps.
> > The codeString is nothing complex, its a one-liner.
> 
> ... but, as too typical of new posters, not posted ;-)
> 
> My question: I understand that core dumping by the Python 
> interpreter is
> normally considered a bug -- one that should be reported on SF -- and
> fixed.  But does that principle apply to this sort of use (or probably
> abuse) of the new module.  In other words, does the 'no core 
> dump' contract
> include a 'byte code compiled by the interpreter' proviso.  
> Or is ceval
> intended to be robust against random strings masquerading as 
> valid byte
> code (which would seem require an InvalidByteCode exception). 
>  Or in still
> other words, is there a bug that I should encourage reporting 
> of, or is
> 'don't do that' the extent of a proper response?

There are other problems; once I passed a unicode string in co_names--it
took me quite a while to track down THAT mistake. :) Perhaps some more
rigorous assertions, returning ValueErrors? If not in C, then in the
'new' module, since that seems to be the vector of disaster for Xaver,
above. Perhaps this would also be an argument for keeping the 'new'
module around longer, as a 'safer' Pythonic wrapper over bare
code/function construction.


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org

From python at rcn.com  Mon May 10 13:25:00 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon May 10 13:26:35 2004
Subject: [Python-Dev] Is core dump always a bug?  Advice requested
In-Reply-To: <c7oa37$rq$1@sea.gmane.org>
Message-ID: <008801c436b3$b9689020$e841fea9@oemcomputer>

> is ceval
> intended to be robust against random strings masquerading as valid
byte
> code (which would seem require an InvalidByteCode exception).

I don't think ceval.c can protect itself against invalid bytecoding
without tons of additional checks.  For example, BUILD_LIST n, can
potentially pop past the beginning of the stack where n is a random
code.


Raymond


From altis at semi-retired.com  Mon May 10 13:57:08 2004
From: altis at semi-retired.com (Kevin Altis)
Date: Mon May 10 13:57:12 2004
Subject: [Python-Dev] Fwd: OSCON discount for core contributors
Message-ID: <74B72782-A2AB-11D8-8113-000A9598382A@semi-retired.com>

OSCON will be held at the Portland Marriott Downtown in Portland, 
Oregon, July 26-30, 2004.

Gina Blaber at O'Reilly is offering a special 20% discount for 20 core 
Python developers. Anyone on python-dev qualifies, so we'll handle this 
as first-come, first-served.

Note that if you have attended OSCON in the past you are already 
qualified for a 20% alumni discount in addition to the 20% early bird 
discount, so you don't need to use this special developer discount. If 
you qualify for the academic discounts, then those are better discount 
deals, and again you don't need to use this developer discount code. 
Regardless of which discount you use, if you plan to attend OSCON, 
you'll definitely save on registration by registering before June 18, 
2004 for an additional 20% early bird discount.

Registration:
http://conferences.oreillynet.com/pub/w/29/register.html

OSCON site:
http://conferences.oreillynet.com/os2004/

Python 12 track at OSCON:
http://conferences.oreillynet.com/pub/w/29/track_python.html
http://conferences.oreillynet.com/pub/w/29/tutorial_python.html

Hope to see you in Portland in July,

ka
---
Kevin Altis
altis@semi-retired.com
http://altis.pycs.net/
Python 12 co-chair

Begin forwarded message:

> From: "Gina Blaber" <gina@oreilly.com>
> Cc: gina@oreilly.com, lholder@oreilly.com
> Subject: OSCON discount for core contributors
>
> Hi Kevin,
>
> O'Reilly would like to help the core contributors of important open 
> source software projects to attend OSCON.   Therefore, I'd like to 
> offer a discount of 15% off the registration price to 20 core 
> contributors to  Python.  This discount can be combined with the 20% 
> Early Bird pricing, which ends June 18, 2004.
>
> The registration discount code for the 20 core contributors to use is 
> os04py.   I've cc'ed Linda Holder on this message; Linda can answer 
> any registration questions you may have.
>
> Thanks, and I look forward to seeing you at OSCON!
>
> -Gina Blaber
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/enriched
Size: 2142 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040510/37623b6e/attachment.bin
From python at rcn.com  Mon May 10 12:28:00 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon May 10 15:12:17 2004
Subject: [Python-Dev] Is core dump always a bug?  Advice requested
In-Reply-To: <c7oa37$rq$1@sea.gmane.org>
Message-ID: <007001c436ab$c375c7c0$e841fea9@oemcomputer>

> My question: I understand that core dumping by the Python interpreter
is
> normally considered a bug -- one that should be reported on SF -- and
> fixed.  But does that principle apply to this sort of use (or probably
> abuse) of the new module.  In other words, does the 'no core dump'
> contract
> include a 'byte code compiled by the interpreter' proviso.  Or is
ceval
> intended to be robust against random strings masquerading as valid
byte
> code (which would seem require an InvalidByteCode exception).  Or in
still
> other words, is there a bug that I should encourage reporting of, or
is
> 'don't do that' the extent of a proper response?
> 
> Terry J. Reedy

File the bug report and assign to me.


Raymond


From tjreedy at udel.edu  Mon May 10 16:07:56 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon May 10 16:11:05 2004
Subject: [Python-Dev] Re: Is core dump always a bug?  Advice requested
References: <c7oa37$rq$1@sea.gmane.org>
	<007001c436ab$c375c7c0$e841fea9@oemcomputer>
Message-ID: <c7oneu$8va$1@sea.gmane.org>


"Raymond Hettinger" <python@rcn.com> wrote in message
news:007001c436ab$c375c7c0$e841fea9@oemcomputer...
> File the bug report and assign to me.

Done.

http://sourceforge.net/tracker/index.php?func=detail&aid=951482&group_id=5470&atid=105470

tjr




From Tino.Lange at isg.de  Mon May 10 18:53:19 2004
From: Tino.Lange at isg.de (Tino Lange)
Date: Mon May 10 18:54:37 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: <200405070424.i474OtT17542@guido.python.org>
References: <61957B071FF421419E567A28A45C7FE59AFB0A@mailbox.nameconnector.com>
	<200405070424.i474OtT17542@guido.python.org>
Message-ID: <40A007DF.1050809@isg.de>

Guido van Rossum wrote:

[Patch #945642]
>>> (TL) Is it possible to include the patch #945642 in 2.3.4?
>> (GT) I'd like to see this in 2.3.4 too [...]
> (GvR) Haven't seen the patch, but the description of the fix makes me go "+1
> please!" :-)

Hi!

Hmmm, the 13th is coming soon and still noone has checked that in ...

But I think there was enough agreement that this really needs to be 
merged and checked in soon?

I understand that there's lots of things to do before such an 
maintenance release ... If you like I can offer to do the changes in CVS 
by myself - if you don't mind giving me checkin' rights for that. I will 
change _ssl.c accordingly for CVS HEAD and also backport it for the 
'release23-maint' branch. Of course I'll also try to check/test once 
again that it doesn't break anything.

No problem if you don't think that this is OK -- but then I'd like to 
ask again, that someone of you looks at it soon for 2.3.4 - according to 
the posted timeline there are only 3 days left...

Thanks and best regards!

Tino

From flaw at rhid.com  Mon May 10 21:23:04 2004
From: flaw at rhid.com (James William Pye)
Date: Mon May 10 21:23:05 2004
Subject: [Python-Dev] Access to generator type
Message-ID: <20040511012304.GE29924@void.ph.cox.net>

Greetings,

Can access to the generator type(gentype in ceval.c) and associated
data/structures be made available in a public interface/header(genobject.h)?
I know I can get the type by compiling a little generator and fetching its
type, but that seems to be a bit redundant as each project that needs it,
needs to make its own, nonstandardized version of access.

Regards,
        James William Pye
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 479 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040510/53cec2e2/attachment.bin
From greg at cosc.canterbury.ac.nz  Mon May 10 21:57:09 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon May 10 21:58:35 2004
Subject: [Python-Dev] Is core dump always a bug?  Advice requested
In-Reply-To: <c7oa37$rq$1@sea.gmane.org>
Message-ID: <200405110157.i4B1v9OC000490@cosc353.cosc.canterbury.ac.nz>

Terry Reedy <tjreedy@udel.edu>:

> Or is ceval intended to be robust against random strings masquerading
> as valid byte code (which would seem require an InvalidByteCode
> exception).

As long as it's possible to attempt to execute arbitrary strings
as bytecode, I'd say ceval should be robust against this.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From tim.one at comcast.net  Mon May 10 22:27:03 2004
From: tim.one at comcast.net (Tim Peters)
Date: Mon May 10 22:27:21 2004
Subject: [Python-Dev] Access to generator type
In-Reply-To: <20040511012304.GE29924@void.ph.cox.net>
Message-ID: <E1BNMzF-0007cv-Qk@mail.python.org>

[James William Pye]
> Can access to the generator type(gentype in ceval.c) and associated
> data/structures be made available in a public
> interface/header(genobject.h)?

Possibly, if someone wants it enough to make a patch (including doc
changes).  There are many types that aren't exposed at the C level, BTW.
Advertising their internals has an inhibiting effect on evolution.  The
generator type has been around long enough w/o change that the value in
keeping it private has diminished.

> I know I can get the type by compiling a little generator and fetching its
> type, but that seems to be a bit redundant as each project that needs it,
> needs to make its own, nonstandardized version of access.

Anyone who wants it can do

    from types import GeneratorType

today (types is a standard Python module).



From guido at python.org  Mon May 10 23:28:21 2004
From: guido at python.org (Guido van Rossum)
Date: Mon May 10 23:32:48 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: Your message of "Tue, 11 May 2004 13:57:09 +1200."
	<200405110157.i4B1v9OC000490@cosc353.cosc.canterbury.ac.nz> 
References: <200405110157.i4B1v9OC000490@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200405110328.i4B3SLB15406@guido.python.org>

> As long as it's possible to attempt to execute arbitrary strings
> as bytecode, I'd say ceval should be robust against this.

This ought to be a long-term project then: write a bytecode verifier.
It's not a trivial task!

--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at cosc.canterbury.ac.nz  Tue May 11 00:20:56 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue May 11 00:21:09 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405110328.i4B3SLB15406@guido.python.org>
Message-ID: <200405110420.i4B4Ku3P000714@cosc353.cosc.canterbury.ac.nz>

Guido:

> > As long as it's possible to attempt to execute arbitrary strings
> > as bytecode, I'd say ceval should be robust against this.
> 
> This ought to be a long-term project then: write a bytecode verifier.
> It's not a trivial task!

When I wrote that, I was assuming that ceval was already mostly robust
in this respect, and that what was being reported was a new hole
recently opened up.

But it appears I was gravely mistaken, and that ceval has been full of
gaping holes from the beginning.

I'm disappointed to learn this, because I had always regarded it as an
axiom that no Python-level code should be capable of crashing the
interpreter, and if it can, this represents a bug. However, it seems
this axiom has not been adhered to in the design of ceval.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From bob at redivi.com  Tue May 11 00:49:37 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue May 11 00:49:44 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405110420.i4B4Ku3P000714@cosc353.cosc.canterbury.ac.nz>
References: <200405110420.i4B4Ku3P000714@cosc353.cosc.canterbury.ac.nz>
Message-ID: <9B5AC266-A306-11D8-BCDD-000A95686CD8@redivi.com>

On May 11, 2004, at 12:20 AM, Greg Ewing wrote:

> Guido:
>
>>> As long as it's possible to attempt to execute arbitrary strings
>>> as bytecode, I'd say ceval should be robust against this.
>>
>> This ought to be a long-term project then: write a bytecode verifier.
>> It's not a trivial task!
>
> When I wrote that, I was assuming that ceval was already mostly robust
> in this respect, and that what was being reported was a new hole
> recently opened up.
>
> But it appears I was gravely mistaken, and that ceval has been full of
> gaping holes from the beginning.
>
> I'm disappointed to learn this, because I had always regarded it as an
> axiom that no Python-level code should be capable of crashing the
> interpreter, and if it can, this represents a bug. However, it seems
> this axiom has not been adhered to in the design of ceval.

Well with modules like ctypes and PyObjC becoming more common, it's 
pretty easy to make the interpreter crash whenever you really want it 
to :)

Python is no Java, there's not even a real attempt to bake security 
into it.. only good programming practices.  Doing extensive checking of 
bytecode at runtime would make Python's interpreter much slower than it 
already is.  I can't imagine that someone with a real need for mangling 
bytecode is going to need their hand held here.  In most cases such a 
person is probably already pretty familiar with the VM at the CPython 
implementation level anyways (since it's not really documented 
elsewhere, as far as I know).

As far as bytecode verification goes, I would imagine that running it 
through PyPy might be good enough to see if it's sane (assuming PyPy 
supports the same bytecode in a similar enough way).

-bob


From greg at cosc.canterbury.ac.nz  Tue May 11 01:04:43 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue May 11 01:05:16 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <9B5AC266-A306-11D8-BCDD-000A95686CD8@redivi.com>
Message-ID: <200405110504.i4B54hmp000811@cosc353.cosc.canterbury.ac.nz>

Bob Ippolito <bob@redivi.com>:

> Doing extensive checking of bytecode at runtime would make Python's
> interpreter much slower than it already is.

Another approach would be to make the bytecode a special object
that can only be created by the compiler (or other C code). If
you wanted custom bytecode, you'd have to build an AST and feed
it to the compiler, or write an extension module.

No doubt some bytecode hackers would find this too limiting,
however.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From anthony at interlink.com.au  Tue May 11 03:04:56 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue May 11 03:05:18 2004
Subject: [Python-Dev] Python 2.3.4 schedule
In-Reply-To: <40A007DF.1050809@isg.de>
References: <61957B071FF421419E567A28A45C7FE59AFB0A@mailbox.nameconnector.com>	<200405070424.i474OtT17542@guido.python.org>
	<40A007DF.1050809@isg.de>
Message-ID: <40A07B18.10005@interlink.com.au>

Tino Lange wrote:
> Guido van Rossum wrote:
> 
> [Patch #945642]
> 
>>>> (TL) Is it possible to include the patch #945642 in 2.3.4?
>>>
>>> (GT) I'd like to see this in 2.3.4 too [...]
>>
>> (GvR) Haven't seen the patch, but the description of the fix makes me 
>> go "+1
>> please!" :-)
> 
> 
> Hi!
> 
> Hmmm, the 13th is coming soon and still noone has checked that in ...
> 
> But I think there was enough agreement that this really needs to be 
> merged and checked in soon?
> 
> I understand that there's lots of things to do before such an 
> maintenance release ... If you like I can offer to do the changes in CVS 
> by myself - if you don't mind giving me checkin' rights for that. I will 
> change _ssl.c accordingly for CVS HEAD and also backport it for the 
> 'release23-maint' branch. Of course I'll also try to check/test once 
> again that it doesn't break anything.
> 
> No problem if you don't think that this is OK -- but then I'd like to 
> ask again, that someone of you looks at it soon for 2.3.4 - according to 
> the posted timeline there are only 3 days left...

That's correct. If this is really wanted for 2.3.4, someone
needs to step up and get it checked in. Today.



-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From guido at python.org  Tue May 11 10:12:00 2004
From: guido at python.org (Guido van Rossum)
Date: Tue May 11 10:12:07 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: Your message of "Tue, 11 May 2004 16:20:56 +1200."
	<200405110420.i4B4Ku3P000714@cosc353.cosc.canterbury.ac.nz> 
References: <200405110420.i4B4Ku3P000714@cosc353.cosc.canterbury.ac.nz> 
Message-ID: <200405111412.i4BEC0h16352@guido.python.org>

Greg:
> > > As long as it's possible to attempt to execute arbitrary strings
> > > as bytecode, I'd say ceval should be robust against this.
> > 
Guido:
> > This ought to be a long-term project then: write a bytecode verifier.
> > It's not a trivial task!

Greg:
> When I wrote that, I was assuming that ceval was already mostly robust
> in this respect, and that what was being reported was a new hole
> recently opened up.
> 
> But it appears I was gravely mistaken, and that ceval has been full of
> gaping holes from the beginning.
> 
> I'm disappointed to learn this, because I had always regarded it as an
> axiom that no Python-level code should be capable of crashing the
> interpreter, and if it can, this represents a bug. However, it seems
> this axiom has not been adhered to in the design of ceval.

No, the holes aren't in ceval.  The hole is the ability to create new
code objects from arbitrary arguments, which wasn't planned at all
when ceval was designed.  I was always reluctant to add this
capability, but finally allowed it, consiciously opening up the hole,
under pressure from various people who wanted to do bytecode level
hacking.

The responsibility to validate bytecode should really be in new.code()
(and probably in new.function, which requires the code to behave in
certain ways).

Note that while I like the general principle that Python code
shouldn't be able to cause segfaults, I don't think we should see that
property as a safety *guarantee*.  It's more a code-of-honor of the
Python implementation.  There are many violations of the principle,
mostly in platform-specific extensions (it is easy to crash Python
with certain win32 primitives, for example, and it seems unreasonable
to expect otherwise).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From dmgass at hotmail.com  Tue May 11 10:22:19 2004
From: dmgass at hotmail.com (Dan Gass)
Date: Tue May 11 10:22:23 2004
Subject: [Python-Dev] Applying the PSF license to the config-py project
Message-ID: <BAY18-F46QMAkaRYFIs000171e1@hotmail.com>

I've gotten negative feedback on my use of the GPL on this project, 
especially since I hope to get this module into the python core.

I am going to make another release soon with some minor (backward 
compatible) improvements and would like to change the license to the PSF.  I 
cannot find good definitive documentation on how to use this license in this 
situation.  What text do I need to place or reference in my code?  If 
someone could direct me to that documentation (if it exists) I would very 
much appreciate it.

BTW, feedback on this project has been positive but slow.  There have been 
55 downloads in the 8 days it has been out there.  
http://config-py.sourceforge.net/.  For better/worse I'm hoping to hear from 
the Zope/ZConfig guys :-)

Thanks,
Dan Gass

_________________________________________________________________
Express yourself with the new version of MSN Messenger! Download today - 
it's FREE! http://messenger.msn.com/go/onm00200471ave/direct/01/


From aahz at pythoncraft.com  Tue May 11 10:27:34 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue May 11 10:27:58 2004
Subject: [Python-Dev] Applying the PSF license to the config-py project
In-Reply-To: <BAY18-F46QMAkaRYFIs000171e1@hotmail.com>
References: <BAY18-F46QMAkaRYFIs000171e1@hotmail.com>
Message-ID: <20040511142734.GA15795@panix.com>

On Tue, May 11, 2004, Dan Gass wrote:
> 
> I am going to make another release soon with some minor (backward
> compatible) improvements and would like to change the license to the
> PSF.  I cannot find good definitive documentation on how to use this
> license in this situation.  What text do I need to place or reference
> in my code?  If someone could direct me to that documentation (if it
> exists) I would very much appreciate it.

Just say that it uses the PSF License 1.0.  That's technically inaccurate
(because there's not license with that precise name yet), but everyone
will know what you mean.  ;-)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Adopt A Process -- stop killing all your children!

From bob at redivi.com  Tue May 11 10:59:13 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue May 11 10:59:16 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405110504.i4B54hmp000811@cosc353.cosc.canterbury.ac.nz>
References: <200405110504.i4B54hmp000811@cosc353.cosc.canterbury.ac.nz>
Message-ID: <C3F879EC-A35B-11D8-BCDD-000A95686CD8@redivi.com>


On May 11, 2004, at 1:04 AM, Greg Ewing wrote:

> Bob Ippolito <bob@redivi.com>:
>
>> Doing extensive checking of bytecode at runtime would make Python's
>> interpreter much slower than it already is.
>
> Another approach would be to make the bytecode a special object
> that can only be created by the compiler (or other C code). If
> you wanted custom bytecode, you'd have to build an AST and feed
> it to the compiler, or write an extension module.
>
> No doubt some bytecode hackers would find this too limiting,
> however.

Then you should give up .pyc files, because I'd imagine they have the 
problems as ceval does if someone decided to change some stuff by hand.

-bob


From fdrake at acm.org  Tue May 11 11:19:37 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue May 11 11:19:47 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <C3F879EC-A35B-11D8-BCDD-000A95686CD8@redivi.com>
References: <200405110504.i4B54hmp000811@cosc353.cosc.canterbury.ac.nz>
	<C3F879EC-A35B-11D8-BCDD-000A95686CD8@redivi.com>
Message-ID: <200405111119.37843.fdrake@acm.org>

On Tuesday 11 May 2004 10:59 am, Bob Ippolito wrote:
 > Then you should give up .pyc files, because I'd imagine they have the
 > problems as ceval does if someone decided to change some stuff by hand.

No; if the bytecode verification is faster (I'd expect so), then you just need 
to verify the bytecode loaded from .pyc/.pyo files the same way new.code() 
and new.function() should.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From fumanchu at amor.org  Tue May 11 11:44:43 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Tue May 11 11:47:43 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E5620D2@opus.amorhq.net>

Greg:
> As long as it's possible to attempt to execute arbitrary strings
> as bytecode, I'd say ceval should be robust against this.
> 
> But it appears ... that ceval has been full of gaping holes
> from the beginning.

Guido:
> No, the holes aren't in ceval.  The hole is the ability to create new
> code objects from arbitrary arguments, which wasn't planned at all
> when ceval was designed.  I was always reluctant to add this
> capability, but finally allowed it, consiciously opening up the hole,
> under pressure from various people who wanted to do bytecode level
> hacking.
> 
> The responsibility to validate bytecode should really be in new.code()
> (and probably in new.function, which requires the code to behave in
> certain ways).

My turn for the time machine :) I wrote:

> Perhaps some more rigorous assertions, returning ValueErrors?
> If not in C, then in the 'new' module, since that seems to be
> the vector of disaster for Xaver, above. Perhaps this would also
> be an argument for keeping the 'new' module around longer,
> as a 'safer' Pythonic wrapper over bare code/function construction.

A good starting point would be some simple boundary and type checking on
arguments to new.code(). In this way, we could keep ceval fast and still
get the 'safety' benefit. If you like, I can produce some assertions
gleaned from my failures the last time I went bytecode hacking. ;)


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org

From skip at pobox.com  Tue May 11 15:33:41 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue May 11 15:33:16 2004
Subject: [Python-Dev] Applying the PSF license to the config-py project
In-Reply-To: <20040511142734.GA15795@panix.com>
References: <BAY18-F46QMAkaRYFIs000171e1@hotmail.com>
	<20040511142734.GA15795@panix.com>
Message-ID: <16545.10901.905836.115366@montanaro.dyndns.org>


    >> I cannot find good definitive documentation on how to use this
    >> license in this situation.  What text do I need to place or reference
    >> in my code?  If someone could direct me to that documentation (if it
    >> exists) I would very much appreciate it.

    aahz> Just say that it uses the PSF License 1.0.

As in:

    This package is licensed under the terms of the PSF License, version
    1.0.

?  I had exactly the same question a few days ago (for the watch project on
SF) but got distracted before I remembered to ask.  Maybe this is a question
worth answering on the PSF section of python.org?

Skip

From DavidA at ActiveState.com  Tue May 11 15:43:22 2004
From: DavidA at ActiveState.com (David Ascher)
Date: Tue May 11 15:44:23 2004
Subject: [Python-Dev] Applying the PSF license to the config-py project
In-Reply-To: <16545.10901.905836.115366@montanaro.dyndns.org>
References: <BAY18-F46QMAkaRYFIs000171e1@hotmail.com>	<20040511142734.GA15795@panix.com>
	<16545.10901.905836.115366@montanaro.dyndns.org>
Message-ID: <40A12CDA.2030000@ActiveState.com>

Skip Montanaro wrote:

>     >> I cannot find good definitive documentation on how to use this
>     >> license in this situation.  What text do I need to place or reference
>     >> in my code?  If someone could direct me to that documentation (if it
>     >> exists) I would very much appreciate it.
> 
>     aahz> Just say that it uses the PSF License 1.0.
> 
> As in:
> 
>     This package is licensed under the terms of the PSF License, version
>     1.0.
> 
> ?  I had exactly the same question a few days ago (for the watch project on
> SF) but got distracted before I remembered to ask.  Maybe this is a question
> worth answering on the PSF section of python.org?

Alternatively use the BSD or MIT license.  There's nothing magical about 
the PSF license as far as I'm concerned, and its particular wording 
implies ownership by the PSF, so it's not 'generic', and it refers to 
other licenses, so another "liberal" license like BSD/MIT works (GPL 
isn't the same) in terms of the code being 'adoptable' by the PSF.

--david



From pedronis at bluewin.ch  Tue May 11 15:45:00 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Tue May 11 15:44:58 2004
Subject: [Python-Dev] PEP 328 and __import__ contract
Message-ID: <5.2.1.1.0.20040511212342.02726fc8@pop.bluewin.ch>


PEP 328 seems at the moment to only describe the syntactic, surface and 
as-seen-by-the-user
aspects of the new behavior for absolute/relative imports.

What is going to happen to __import__ hook?

At the moment relative imports as the default are implemented at that level,
through get_parent and the first load_next in import_module_ex which is 
exported as __import__.

Will imports still be implemented in terms of/on top of __import__.

What will be the contract for __import__ then?
I suppose it could be kept the same, relaxing over time the obligation to 
implement
relative imports. At the moment a __import__ hook can be forced in absolute 
import mode
by passing an empty globals dict.

I think the PEP should cover some of this.

regards.



From barry at python.org  Tue May 11 15:51:18 2004
From: barry at python.org (Barry Warsaw)
Date: Tue May 11 15:51:29 2004
Subject: [Python-Dev] Applying the PSF license to the config-py project
In-Reply-To: <40A12CDA.2030000@ActiveState.com>
References: <BAY18-F46QMAkaRYFIs000171e1@hotmail.com>
	<20040511142734.GA15795@panix.com>
	<16545.10901.905836.115366@montanaro.dyndns.org>
	<40A12CDA.2030000@ActiveState.com>
Message-ID: <1084305077.28228.324.camel@anthem.wooz.org>

On Tue, 2004-05-11 at 15:43, David Ascher wrote:

> Alternatively use the BSD or MIT license.  There's nothing magical about 
> the PSF license as far as I'm concerned, and its particular wording 
> implies ownership by the PSF, so it's not 'generic', and it refers to 
> other licenses, so another "liberal" license like BSD/MIT works (GPL 
> isn't the same) in terms of the code being 'adoptable' by the PSF.

Of course, if you own the code, you can release it under any (and
multiple) license.  If/when the time comes, you will also be able to
donate it to the PSF at which point they/we will put it under the PSF
license.

-Barry



From tjreedy at udel.edu  Tue May 11 15:55:33 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue May 11 15:55:41 2004
Subject: [Python-Dev] Re: Mozpython help
References: <20040511185624.25266.qmail@web51007.mail.yahoo.com>
Message-ID: <c7rb3m$7ou$1@sea.gmane.org>

The pydev list is for discussing the development of future versions of
Python.  Your usage questions would be better directed to comp.lang.py or,
probably better, a mozpython specific group or list, assuming that there is
one.

tjr




From arigo at tunes.org  Tue May 11 16:56:02 2004
From: arigo at tunes.org (Armin Rigo)
Date: Tue May 11 16:59:12 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <DE1CF2B4FEC4A342BF62B6B2B334601E5620D2@opus.amorhq.net>
References: <DE1CF2B4FEC4A342BF62B6B2B334601E5620D2@opus.amorhq.net>
Message-ID: <20040511205602.GA1465@vicky.ecs.soton.ac.uk>

Hello,

As discussed already some time ago, on Linux you can just write random data
into /proc/self/mem.  There is no way the interpreter can be guaranteed not to
crash.

Moreover, writing a bytecode verifier is a difficult task which goes against
the purpose of new.code(), which was to create strange bytecodes that the
compiler would never produce itself.  It is possible to write a bytecode
verifier that accepts all bytecodes produced by the compiler and then some
more, but it is not possible to write any algorithm that can tell for sure
whether some bytecode is going to crash or can be proved safe (not even to
mention that there exist bytecodes which are safe but that no mathematician --
not just no machine -- will ever be able to prove so).

More concretely, what about issuing a warning the first time new.code() is
called?  People who really use new.code() in their application can silence it
with the warnings module.


A bientot,

Armin.

From michel at dialnetwork.com  Tue May 11 15:14:42 2004
From: michel at dialnetwork.com (Michel Pelletier)
Date: Tue May 11 17:24:35 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <E1BNZUo-0006Mb-6Q@mail.python.org>
References: <E1BNZUo-0006Mb-6Q@mail.python.org>
Message-ID: <200405111414.42559.michel@dialnetwork.com>

> Message: 6
> Date: Tue, 11 May 2004 07:12:00 -0700
> From: Guido van Rossum <guido@python.org>
> Subject: Re: [Python-Dev] Is core dump always a bug? Advice requested
> To: Greg Ewing <greg@cosc.canterbury.ac.nz>
> Cc: python-dev@python.org

> No, the holes aren't in ceval.  The hole is the ability to create new
> code objects from arbitrary arguments, which wasn't planned at all
> when ceval was designed.  I was always reluctant to add this
> capability, but finally allowed it, consiciously opening up the hole,
> under pressure from various people who wanted to do bytecode level
> hacking.

Please excuse me if I'm wrong, but wasn't the hole always there, you just made 
it wider?  It seems it has always been possible to handcraft an evil .pyc 
file and cause its import, it's just more laborious than using the new 
module.

> The responsibility to validate bytecode should really be in new.code()
> (and probably in new.function, which requires the code to behave in
> certain ways).

This seems to only cover the part of the hole you widened, not the whole hole.  
For example, Java verifies bytecode when a .class file is loaded  (See 
section 4.9 of the Java VM Specification).  At this time it does quite a bit 
of verification, the spec is very interesting and could be a possible 
starting point for a Python bytecode verifier.

It would seem a similar verification should happen any time the Python VM 
attempts to execute a code object for the first time, not just when the new 
module is used.

> Note that while I like the general principle that Python code
> shouldn't be able to cause segfaults, I don't think we should see that
> property as a safety *guarantee*.  It's more a code-of-honor of the
> Python implementation.  There are many violations of the principle,
> mostly in platform-specific extensions (it is easy to crash Python
> with certain win32 primitives, for example, and it seems unreasonable
> to expect otherwise).

While it's reasonable that C extensions can cause seg-faults, the problem to 
me seems to be the way in which bytecode can be used to exploit the 
interpreter without the involvement of extensions and not necessarily causing 
hard failures like seg faults.

-Michel

From greg at cosc.canterbury.ac.nz  Tue May 11 22:06:29 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue May 11 22:08:51 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405111119.37843.fdrake@acm.org>
Message-ID: <200405120206.i4C26TMm002483@cosc353.cosc.canterbury.ac.nz>

"Fred L. Drake, Jr." <fdrake@acm.org>:

> No; if the bytecode verification is faster (I'd expect so), then you just need 
> to verify the bytecode loaded from .pyc/.pyo files the same way new.code() 
> and new.function() should.

Just a thought, but is statically verifying the bytecode even
possible in principle? Seems to me it could be equivalent to
the halting problem.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From fdrake at acm.org  Tue May 11 22:14:40 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue May 11 22:14:51 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405120206.i4C26TMm002483@cosc353.cosc.canterbury.ac.nz>
References: <200405120206.i4C26TMm002483@cosc353.cosc.canterbury.ac.nz>
Message-ID: <200405112214.40228.fdrake@acm.org>

On Tuesday 11 May 2004 10:06 pm, Greg Ewing wrote:
 > Just a thought, but is statically verifying the bytecode even
 > possible in principle? Seems to me it could be equivalent to
 > the halting problem.

I don't see any reason to think them equivalent; we don't need to determine 
that the code will execute to completion, only that the bytecodes can be 
decoded without error.  Not trivial by any means, but I think it's a more 
constrained problem.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From pje at telecommunity.com  Tue May 11 23:33:53 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue May 11 23:33:16 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405112214.40228.fdrake@acm.org>
References: <200405120206.i4C26TMm002483@cosc353.cosc.canterbury.ac.nz>
	<200405120206.i4C26TMm002483@cosc353.cosc.canterbury.ac.nz>
Message-ID: <5.1.1.6.0.20040511232902.03a78810@mail.telecommunity.com>

At 10:14 PM 5/11/04 -0400, Fred L. Drake, Jr. wrote:
>On Tuesday 11 May 2004 10:06 pm, Greg Ewing wrote:
>  > Just a thought, but is statically verifying the bytecode even
>  > possible in principle? Seems to me it could be equivalent to
>  > the halting problem.
>
>I don't see any reason to think them equivalent; we don't need to determine
>that the code will execute to completion, only that the bytecodes can be
>decoded without error.  Not trivial by any means, but I think it's a more
>constrained problem.

Right; it should be possible, for example, to verify stack depth used, that 
stack growth isn't unlimited, that there is a return at the end, no invalid 
references to locals or co_names, and so on.  Basic "sanity" stuff, not 
in-depth analysis.

But it isn't clear to me that this is really necessary for new.code(), let 
alone .pyc/.pyo files.  And anybody that's crazy enough to send bytecode 
"over the wire" as part of an agent system or whatever darn well better 
have written their own bytecode verifier already, and maybe a sandbox or 
two as well.


From mhammond at skippinet.com.au  Tue May 11 22:23:33 2004
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Tue May 11 23:33:30 2004
Subject: [Python-Dev] os.popen and exit codes on windows
Message-ID: <0ff401c437c8$212ab390$0200a8c0@eden>

I'm looking at bug 602245 - http://www.python.org/sf/602245 - and would like
some guidance.

The issue is the return-code when closing the last handle from an os.popen()
function.

In a nutshell:  On Linux, the result of closing the last handle is the
result from the wait() function.  On Windows, the result is the integer
return code, exactly as specified by the child program - however, on
Windows, when the result is -1, an error is raised (and hence the bug
referenced above)

On Linux, the value from the wait() system call is split into three parts:
mask 0xff00 is the exit status
mask 0x007f is the signal causing the exit
mask 0x0080 is the cord dumped flag in combination with a signal

As part of fixing the bug for Windows, I have 2 basic choices:
1) Fix it by masking off the high bits, thereby returning an "unsigned"
result code - preventing -1, but still returning the result exactly as
specified by the child process.
2) Fix it by changing the result to resemble the Linux behaviour.  This
would mean programs using popen() are more portable.

My problem is that (2) will change the behaviour for existing programs.
Most programs will only check for non-zero, but we can't know that for sure.

One option is to do (1) for Python 2.3, and (2) for Python 2.4.  That
doesn't sound quite right either.  If (2) is best, we could probably forget
Python 2.3 all together, as the bug is quite old.

Any thoughts?  Comments directly in the bug are probably best.

Thanks,

Mark.


From tim.one at comcast.net  Tue May 11 23:39:57 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue May 11 23:40:04 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405120206.i4C26TMm002483@cosc353.cosc.canterbury.ac.nz>
Message-ID: <E1BNkbA-0003fY-8g@mail.python.org>

[Greg Ewing]
> Just a thought, but is statically verifying the bytecode even
> possible in principle? Seems to me it could be equivalent to
> the halting problem.

Verifying what, precisely?  Some things can clearly be checked.  For
examples, that all opcodes are defined, that no LOAD_CONST tries to index
beyond the actual length of co_consts, and that no jump tries to branch into
the middle of some multi-byte opcode sequence  Violations of such
simple-to-check kinds of things are what cause segfaults most often when
handing the PVM nonsense bytes.  Harder to do is flow-sensitive eval stack
simulation, to ensure that no path through the code can push more on the
eval stack than was allocated for it, and that there's enough stuff on the
stack at each point to satisfy each opcode that requires accessing the eval
stack.

A subset of what the Java bytecode verifier does is quite doable:

   http://java.sun.com/sfaq/verifier.html#1

Java has more things that need to be checked, because many of the JVM
instructions have typed arguments (and it has registers as well as a stack).
Most PVM arguments are PyObject*, and when specific types are required
there's already code in ceval.c to check that at runtime.



From greg at cosc.canterbury.ac.nz  Tue May 11 23:27:29 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue May 11 23:45:32 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405112214.40228.fdrake@acm.org>
Message-ID: <200405120327.i4C3RTNH002597@cosc353.cosc.canterbury.ac.nz>

"Fred L. Drake, Jr." <fdrake@acm.org>:

> I don't see any reason to think them equivalent; we don't need to determine 
> that the code will execute to completion, only that the bytecodes can be 
> decoded without error.

If arbitrary branching patterns are allowed, then in order to ensure
absence of stack overflow/underflow, etc., it will be necessary to do
things like proving that certain loops can't execute more than a
certain number of times. This smells halting-problem-complete to me.

It would be possible to prove that certain classes of bytecode
sequence are valid, and reject all others. But I suspect that would
reject a large number of valid-but-hard-to-prove sequences that
bytecode hackers would find "interesting".

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From bob at redivi.com  Wed May 12 00:05:49 2004
From: bob at redivi.com (Bob Ippolito)
Date: Wed May 12 00:05:57 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <5.1.1.6.0.20040511232902.03a78810@mail.telecommunity.com>
References: <200405120206.i4C26TMm002483@cosc353.cosc.canterbury.ac.nz>
	<200405120206.i4C26TMm002483@cosc353.cosc.canterbury.ac.nz>
	<5.1.1.6.0.20040511232902.03a78810@mail.telecommunity.com>
Message-ID: <A6E0E126-A3C9-11D8-BCDD-000A95686CD8@redivi.com>


On May 11, 2004, at 11:33 PM, Phillip J. Eby wrote:

> At 10:14 PM 5/11/04 -0400, Fred L. Drake, Jr. wrote:
>> On Tuesday 11 May 2004 10:06 pm, Greg Ewing wrote:
>>  > Just a thought, but is statically verifying the bytecode even
>>  > possible in principle? Seems to me it could be equivalent to
>>  > the halting problem.
>>
>> I don't see any reason to think them equivalent; we don't need to 
>> determine
>> that the code will execute to completion, only that the bytecodes can 
>> be
>> decoded without error.  Not trivial by any means, but I think it's a 
>> more
>> constrained problem.
>
> Right; it should be possible, for example, to verify stack depth used, 
> that stack growth isn't unlimited, that there is a return at the end, 
> no invalid references to locals or co_names, and so on.  Basic 
> "sanity" stuff, not in-depth analysis.
>
> But it isn't clear to me that this is really necessary for new.code(), 
> let alone .pyc/.pyo files.  And anybody that's crazy enough to send 
> bytecode "over the wire" as part of an agent system or whatever darn 
> well better have written their own bytecode verifier already, and 
> maybe a sandbox or two as well.

Exactly, why bother when there's plenty of valid bytecode that crashes? 
  Even without third party extension modules, there are usually platform 
specific files you can open and play with that'll cause a crash.  
os._exit and os.abort are essentially crashes too, and then there's the 
signal module, os.kill, etc.

The only example I can think of is if you were experimenting with 
genetic algorithms or something and you decided to randomly generate 
and mutate bytecode and see if it produces interesting results.. but in 
that case you probably want a much smaller and simpler VM anyways.

-bob


From fdrake at acm.org  Wed May 12 00:28:52 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed May 12 00:29:05 2004
Subject: [Python-Dev] Python 2.3.x doc tree closed
Message-ID: <200405120028.53006.fdrake@acm.org>

I'm cutting the docs for Python 2.3.4; please make no changes on the the 
release23-maint branch.

Thanks for everyone's help in getting this ready!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From anthony at interlink.com.au  Wed May 12 01:33:37 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed May 12 01:34:28 2004
Subject: [Python-Dev] release23-maint tree closed for 2.3.4rc1
Message-ID: <40A1B731.1050504@interlink.com.au>

Unless you're involved in the 2.3.4 release process
(so one of Anthony, Thomas, Fred or Jack(?)), please
consider the release23-maint branch frozen until after
Thursday the 20th of June. If you have something that
you want to see on the branch before then, please
email me first.

Thanks,
Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From anthony at interlink.com.au  Wed May 12 01:35:28 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed May 12 01:36:17 2004
Subject: [Python-Dev] Re: release23-maint tree closed for 2.3.4rc1
In-Reply-To: <40A1B731.1050504@interlink.com.au>
References: <40A1B731.1050504@interlink.com.au>
Message-ID: <40A1B7A0.2000606@interlink.com.au>

Anthony Baxter wrote:
> Unless you're involved in the 2.3.4 release process
> (so one of Anthony, Thomas, Fred or Jack(?)), please
> consider the release23-maint branch frozen until after
> Thursday the 20th of June. 

Argh. What is with my internal brain calendar this year??
Of course, I meant Thursday the 20th of _MAY_. May. It's
May. I know this. Really.


From mwh at python.net  Wed May 12 05:45:02 2004
From: mwh at python.net (Michael Hudson)
Date: Wed May 12 05:45:07 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <A6E0E126-A3C9-11D8-BCDD-000A95686CD8@redivi.com> (Bob
	Ippolito's message of "Wed, 12 May 2004 00:05:49 -0400")
References: <200405120206.i4C26TMm002483@cosc353.cosc.canterbury.ac.nz>
	<200405120206.i4C26TMm002483@cosc353.cosc.canterbury.ac.nz>
	<5.1.1.6.0.20040511232902.03a78810@mail.telecommunity.com>
	<A6E0E126-A3C9-11D8-BCDD-000A95686CD8@redivi.com>
Message-ID: <2misf2m129.fsf@starship.python.net>

Bob Ippolito <bob@redivi.com> writes:

> The only example I can think of is if you were experimenting with
> genetic algorithms or something and you decided to randomly generate
> and mutate bytecode and see if it produces interesting results.. but
> in that case you probably want a much smaller and simpler VM anyways.

As well as psychiatric help :-)

Cheers,
mwh

-- 
  (Unfortunately, while you get Tom Baker saying "then we 
   were attacked by monsters", he doesn't flash and make 
  "neeeeooww-sploot" noises.)
      -- Gareth Marlow, ucam.chat, from Owen Dunn's review of the year

From arigo at tunes.org  Wed May 12 07:01:39 2004
From: arigo at tunes.org (Armin Rigo)
Date: Wed May 12 07:04:43 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <E1BNkbA-0003fY-8g@mail.python.org>
References: <200405120206.i4C26TMm002483@cosc353.cosc.canterbury.ac.nz>
	<E1BNkbA-0003fY-8g@mail.python.org>
Message-ID: <20040512110139.GA14120@vicky.ecs.soton.ac.uk>

Hello Tim,

On Tue, May 11, 2004 at 11:39:57PM -0400, Tim Peters wrote:
> Verifying what, precisely?  (...) and that no jump tries to branch into
> the middle of some multi-byte opcode sequence

Why not?  :-)  (mwh can surely recommend me a psy)

> Harder to do is flow-sensitive eval stack
> simulation, to ensure that no path through the code can push more on the
> eval stack than was allocated for it, and that there's enough stuff on the
> stack at each point to satisfy each opcode that requires accessing the eval
> stack.

There are a few opcodes whose effect on the stack isn't self-contained, like
MAKE_FUNCTION which will pop N cell variables off the stack, for N loaded from
the code object which itself comes from the stack.  These ones are fun to
analyse.

Even without these, checking if a bytecode could possibly over/underflow the
stack is indeed equivalent to the halting problem; a silly example:

    <some algorithm which may stop or not>
    POP_TOP

This underflows the stack if and only if the algorithm stops.


Armin

From casey at zope.com  Wed May 12 09:34:14 2004
From: casey at zope.com (Casey Duncan)
Date: Wed May 12 09:32:59 2004
Subject: [Python-Dev] Re: Is core dump always a bug? Advice requested
References: <200405120206.i4C26TMm002483@cosc353.cosc.canterbury.ac.nz>
	<E1BNkbA-0003fY-8g@mail.python.org>
Message-ID: <20040512093414.15e73078.casey@zope.com>

On Tue, 11 May 2004 23:39:57 -0400
"Tim Peters" <tim.one@comcast.net> wrote:

> [Greg Ewing]
> > Just a thought, but is statically verifying the bytecode even
> > possible in principle? Seems to me it could be equivalent to
> > the halting problem.
> 
> Verifying what, precisely?  Some things can clearly be checked.  For
> examples, that all opcodes are defined, that no LOAD_CONST tries to
> index beyond the actual length of co_consts, and that no jump tries to
> branch into the middle of some multi-byte opcode sequence.

Damn, there goes my plans for phiX174.py

befunging-ly y'rs,

-Casey


From guido at python.org  Wed May 12 09:55:04 2004
From: guido at python.org (Guido van Rossum)
Date: Wed May 12 10:01:11 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: Your message of "Wed, 12 May 2004 12:01:39 BST."
	<20040512110139.GA14120@vicky.ecs.soton.ac.uk> 
References: <200405120206.i4C26TMm002483@cosc353.cosc.canterbury.ac.nz>
	<E1BNkbA-0003fY-8g@mail.python.org> 
	<20040512110139.GA14120@vicky.ecs.soton.ac.uk> 
Message-ID: <200405121355.i4CDt4n18585@guido.python.org>

> There are a few opcodes whose effect on the stack isn't
> self-contained, like MAKE_FUNCTION which will pop N cell variables
> off the stack, for N loaded from the code object which itself comes
> from the stack.  These ones are fun to analyse.

We're planning to get rid of that one (note that it was a late
addition; the original design was intended to be easy to analyze, but
others didn't see the constraints).  Any others that pose such problems?

> Even without these, checking if a bytecode could possibly over/underflow the
> stack is indeed equivalent to the halting problem; a silly example:
> 
>     <some algorithm which may stop or not>
>     POP_TOP
> 
> This underflows the stack if and only if the algorithm stops.

Uninteresting though, since no compiler will emit such code, and we
only need to accept a reasonable subset.  The same arguments could
prove that Java's bytecode verification is "impossible", but
nevertheless it's done.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tjreedy at udel.edu  Wed May 12 11:37:24 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed May 12 11:37:32 2004
Subject: [Python-Dev] Re: Is core dump always a bug? Advice requested
References: <200405120206.i4C26TMm002483@cosc353.cosc.canterbury.ac.nz>
	<E1BNkbA-0003fY-8g@mail.python.org>
Message-ID: <c7tgbm$r1f$1@sea.gmane.org>


"Tim Peters" <tim.one@comcast.net> wrote in message
news:E1BNkbA-0003fY-8g@mail.python.org...
> Verifying what, precisely?  Some things can clearly be checked.  For
> examples, that all opcodes are defined, that no LOAD_CONST tries to index
> beyond the actual length of co_consts, and that no jump tries to branch
into
> the middle of some multi-byte opcode sequence  Violations of such
> simple-to-check kinds of things are what cause segfaults most often when
> handing the PVM nonsense bytes.  Harder to do is flow-sensitive eval
stack
> simulation, to ensure that no path through the code can push more on the
> eval stack than was allocated for it, and that there's enough stuff on
the
> stack at each point to satisfy each opcode that requires accessing the
eval
> stack.
>
> A subset of what the Java bytecode verifier does is quite doable:

Since I opened this somewhat hidden hornet's nest, the (newby) author of
the code that caused the offending crash has admitted on clp that the code
he fed to new.code was standard ascii Python code, not bytecode.  In other
words, something that almost certainly looked like gibberish to ceval.  The
limited goal of usually rejecting such 'random' byte strings, if adopted,
*should* be doable since such should have many errors and we only need to
catch one, not all, to reject.

Protecting against maliciously or otherwise modified compiler output is a
whole different game.  Too bad there is no system-independent way to
'catch' crashes and exit more gracefully than usually useless core dumps,
blue screens of death, or whatever.

Terry J. Reedy




From jeremy at alum.mit.edu  Wed May 12 11:56:49 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed May 12 11:57:03 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405120327.i4C3RTNH002597@cosc353.cosc.canterbury.ac.nz>
Message-ID: <200405121556.i4CFusNw025611@mta7.pltn13.pbi.net>

> If arbitrary branching patterns are allowed, then in order to ensure
> absence of stack overflow/underflow, etc., it will be necessary to do
> things like proving that certain loops can't execute more than a
> certain number of times. This smells halting-problem-complete to me.

I think it would only need to check that any path through the loop has a no
net effect on stack depth -- before branching back to the beginning of the
loop, it must pop everything it pushed.

> It would be possible to prove that certain classes of bytecode
> sequence are valid, and reject all others. But I suspect that would
> reject a large number of valid-but-hard-to-prove sequences that
> bytecode hackers would find "interesting".

I find it hard to make a strong conjecture either way.  I think the only way
to find out would be to examine whatever it is that bytecode hackers do.
There could be an unsafe switch that turned off checking.

Jeremy


From mat at elandem.org  Wed May 12 13:07:06 2004
From: mat at elandem.org (Mat Ford)
Date: Wed May 12 13:17:55 2004
Subject: [Python-Dev] IPv6 support in Win binary again
Message-ID: <E1BNx84-000OE6-VA@lydian.elandem.org>

There was a thread back in October 2003 on the subject of including IPv6
support in the Windows binary:

	http://mail.python.org/pipermail/python-dev/2003-October/038623.html

Can anyone give me a status update on this? Is support there now? If not, is
there a predictable timeframe in which it can be expected to appear?

Thanks in advance,

Mat

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.678 / Virus Database: 440 - Release Date: 06/05/2004
 


From brian at sweetapp.com  Wed May 12 13:42:55 2004
From: brian at sweetapp.com (Brian Quinlan)
Date: Wed May 12 13:38:57 2004
Subject: [Python-Dev] factory functions in the datetime module (& others)
In-Reply-To: <20040512093414.15e73078.casey@zope.com>
References: <200405120206.i4C26TMm002483@cosc353.cosc.canterbury.ac.nz>	<E1BNkbA-0003fY-8g@mail.python.org>
	<20040512093414.15e73078.casey@zope.com>
Message-ID: <40A2621F.8070205@sweetapp.com>

It seems to me that the utility of subclassing certain library classes would 
be increased if the user could set a factory that the base class would use 
when creating new instances. That way, from the user's point of view, 
functionality would not "disappear" after operations e.g.

class MyDate(datetime.date):
     def foo(self):
         ...

nextday = MyDate(...) + datetime.timedelta(days=1)
nextday.foo() # exception

Storing the factory information in the base class has the disadvantage of 
causing a global behavior change but can probably be made speed efficient. 
Storing the factory in the subclass would require that the super class check 
for a class variable when creating instances. It would probably provide the 
most obvious semantics from the programmer's POV but would be organizationally 
odd i.e. each class would have to provide a factory for every overriden type 
that it's superclass creates.

Aside from datime.* this feature seems like it would be useful for other types 
where there is potential for making useful subclasses (unlike int) and where 
many operations result in new instances. sets come to mind.

Any thoughts?

Cheers,
Brian

From martin at v.loewis.de  Wed May 12 15:38:34 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed May 12 15:38:46 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405120206.i4C26TMm002483@cosc353.cosc.canterbury.ac.nz>
References: <200405120206.i4C26TMm002483@cosc353.cosc.canterbury.ac.nz>
Message-ID: <40A27D3A.6000603@v.loewis.de>

Greg Ewing wrote:
> Just a thought, but is statically verifying the bytecode even
> possible in principle? Seems to me it could be equivalent to
> the halting problem.

Reliably accepting all correct byte code, and rejecting all
incorrect one, would indeed be equivalent to the halting problem.

Therefore, byte code verification usually puts additional
constraints on byte code, rejecting some correct code as
unverifiable.

Then, in addition to the language spec, you also need to specify
what the verification rules are that an implementation needs to
follow.

The typical assumption is that you have "consistent" stack usage.
IOW, you need to be able to clearly identify start offsets of
byte code instructions, and associate a fixed stack depth with
each instruction. In typed languages, you also need to associate
a type with the TOS, and require that type to be consistent
across all possible execution paths. For example, in .NET
code verification, there is a requirement that the value stack
is empty at every branch target, and the start of every exception
handler.

IOW, it is only necessary that any byte code that the Python
byte code generator will ever emit verifies as correct. Accepting
additional byte codes is optional.

Regards,
Martin


From martin at v.loewis.de  Wed May 12 15:43:43 2004
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed May 12 15:43:50 2004
Subject: [Python-Dev] IPv6 support in Win binary again
In-Reply-To: <E1BNx84-000OE6-VA@lydian.elandem.org>
References: <E1BNx84-000OE6-VA@lydian.elandem.org>
Message-ID: <40A27E6F.70806@v.loewis.de>

Mat Ford wrote:
> There was a thread back in October 2003 on the subject of including IPv6
> support in the Windows binary:
> 
> 	http://mail.python.org/pipermail/python-dev/2003-October/038623.html
> 
> Can anyone give me a status update on this? Is support there now? 

No.

> If not, is
> there a predictable timeframe in which it can be expected to appear?

With the release of Python 2.4, which, at latest, should happen before
January, 2005.

In case you wonder what the problem is: if you compile with VC6, you
cannot provide IPv6 support unless you also have the platform SDK
installed. That is not part of the build process of Python 2.3, so
the official binaries of Python 2.3 will never ever support IPv6.
Using VC.NET is out of the question, since it would break the ABI
across minor releases, which is out of the question. Installing
the SDK for the build process might be an option, but the code base
has no way of detecting presence of the SDK, so this is essentially
also out of question.

Regards,
Martin


From tim.one at comcast.net  Wed May 12 16:17:58 2004
From: tim.one at comcast.net (Tim Peters)
Date: Wed May 12 16:18:06 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405121355.i4CDt4n18585@guido.python.org>
Message-ID: <E1BO0B1-0002go-5O@mail.python.org>

[Armin]
>> Even without these, checking if a bytecode could possibly over/underflow
>> the stack is indeed equivalent to the halting problem; a silly example:
>>
>>     <some algorithm which may stop or not>
>>     POP_TOP
>>
>> This underflows the stack if and only if the algorithm stops.

[Guido]
> Uninteresting though, since no compiler will emit such code, and we
> only need to accept a reasonable subset.  The same arguments could
> prove that Java's bytecode verification is "impossible", but
> nevertheless it's done.

That's right.  Anyone who intends to "do something" here (as opposed to just
talking about it) is strongly encouraged to read the Java docs I posted a
link to last time.  Their answer to my original question ("verify what,
precisely?") is "what the algorithm described here checks".  As also with
Java's compile-time guarantee of "no use of uninitialized variables",
there's no attempt to solve impossible problems.  Instead they give a
reasonable, conservative algorithm, and define "it's OK" as "what this
algorithm allows".  The difference between that and theoretical perfection
isn't an issue in practice.



From michel at dialnetwork.com  Wed May 12 16:13:27 2004
From: michel at dialnetwork.com (Michel Pelletier)
Date: Wed May 12 18:24:39 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
Message-ID: <200405121513.27827.michel@dialnetwork.com>

> > If arbitrary branching patterns are allowed, then in order to ensure
> > absence of stack overflow/underflow, etc., it will be necessary to do
> > things like proving that certain loops can't execute more than a
> > certain number of times. This smells halting-problem-complete to me.
>
> I think it would only need to check that any path through the loop has a no
> net effect on stack depth -- before branching back to the beginning of the
> loop, it must pop everything it pushed.

The ASM Java bytecode generation API has a similar flow control algorithm to 
determine the maximum stack size for java bytecode.  

http://asm.objectweb.org/

It uses the visitor pattern to visit classes, methods and instructions during 
bytecode generation that construct a control flow graph that is used at class 
generation time to calculate the maximum local variables and stack depth for 
methods (see Label.java and CodeWriter.java in their WebCVS). 

his information is embedded into the class, and is used at JVM verification 
time when the bytecode is reanalyzed to ensure that the numbers match the 
actual behavior of the bytecode.

I've seen a lot of concern about bytecode hacking crashing the interpreter, 
but the Java VM Spec goes further to state that there is a risk of 
exploitation.  I stared at ceval for a while to see if I could pick up such a 
hole.  It hurt.

I hear Phillip Eby when he says that anyone sending bytecode over the wire 
agent-fasion "darn well better" write their own bytecode verifier.  But in a 
world of massive paralleism and dynamic computing, that might not seem like 
such a bad feature to offer stock.

Would there be an interest in at least a PEP to consider a bytecode verifier?  
I'd be willing to take it on, but my experience is limited to JVM bytecode 
and verification.

-Michel


From jjl at pobox.com  Wed May 12 18:40:12 2004
From: jjl at pobox.com (John J Lee)
Date: Wed May 12 18:40:19 2004
Subject: [Python-Dev] cookielib module in 2.4?
Message-ID: <Pine.LNX.4.58.0405122246560.1681@alice>

I have a module that I'd like to get included in Python 2.4.  It adds
transparent HTTP cookie handling to urllib2 (and urllib too, I guess, if
somebody wants to write the little bit of code I imagine is required).
I'm prepared to maintain the thing, of course.  It has been distributed
for 2 years in a 1.5.2-compatible form, as the ClientCookie package, at
wwwsearch.sf.net.  I've just made a branch in which I have removed 1.5.2
(and various other) cruft, along with some superfluous features, and
turned it into a single-file module (plus a small patch to urllib2).  ATM
it's in a subversion branch here:

http://codespeak.net/svn/user/jjlee/wwwsearch/ClientCookie/branch/python-2.4_candidate/


I've renamed it 'cookielib'.  That's partly to fit in with the Python
standard library's naming convention for network client modules, partly so
that it's clear to people whether they're importing the old
1.5.2-compatible version or the 2.4 standard library version.  Actually,
the most frequent use case does not involve importing the module:

import urllib2
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor())
response = opener.open("http://example.com/")


So, assuming the module is likely to be accepted (given the answers to my
post here last year), I'd be happy if anybody here could

1. indicate if I'm roughly on the right track (especially welcome if
   you've had a look at the subversion branch above),

2. give me a rough idea how long I have to sort out the remaining issues
   and submit a patch


The major remaining issue is documentation, which is not yet in LaTeX
format (nor in Python std. lib. style) -- see my next post.  There are
also two or three niggles which I'd like to discuss with Jeremy Hylton,
Moshe Zadka, or anybody else who knows urllib2.  Finally, there's the fact
that the module contains untested thread-synchronisation code (I'm pretty
thread-ignorant).  Perhaps that's an academic issue ATM, though:  I failed
recently to eliminate deadlocks in a fairly simple bit of code using
urllib2 + threading, even when I verified there were no locks allocated by
any module other than threading itself...

TIA


John

From michel at dialnetwork.com  Wed May 12 16:31:43 2004
From: michel at dialnetwork.com (Michel Pelletier)
Date: Wed May 12 18:42:41 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
Message-ID: <200405121531.43532.michel@dialnetwork.com>


Armin said:
> > Even without these, checking if a bytecode could possibly 
over/underflowthe
> > stack is indeed equivalent to the halting problem; a silly example:
> > 
> >     <some algorithm which may stop or not>
> >     POP_TOP
> > 
> > This underflows the stack if and only if the algorithm stops.

Guido said:
> The same arguments could
> prove that Java's bytecode verification is "impossible", but
> nevertheless it's done.

But I belive Armin's argument that it equals the halting problem is incorrect.  
The net stack effect of bytecode can be verified regardless of the algorithm.  
If it is seen that two bytecodes produce an element on the stack and one 
bytecode consumes two, then the net effect is zero.  As Jeremy pointed out, 
iterations (of well formed code) have no net effect.  

Although Armin's example could certainly fail due to exhausting the stack of 
return frames (which is a runtime issue), or memory, or anything else, no 
well formed bytecode can exhaust the bytecode stack.  The way to know if it's 
well formed is to verify it using something like a control flow analysis 
similar to the way the Java ASM bytecode generation API does it.   The 
secondary benefit of this is that you are assured at runtime that the stack 
size the compiler computes for the bytecode is genuine.

-Michel


From jjl at pobox.com  Wed May 12 18:43:42 2004
From: jjl at pobox.com (John J Lee)
Date: Wed May 12 18:43:50 2004
Subject: [Python-Dev] Build a single file module-foo.html from Python LaTeX
	source?
Message-ID: <Pine.LNX.4.58.0405122340280.1681@alice>

Is there some magic make command that generates the module-foo.html for
just a single libfoo.tex file?  Doesn't necessarily have to generate
100%-identical output to the full run.

I want to write Python-LaTeX-format docs for a module module I hope to get
into Python 2.4, and document urllib2's handlers better.  It's nice to be
able to see the HTML output as you edit, without having to process the
whole std. lib. manual every time.


John

From aahz at pythoncraft.com  Wed May 12 20:01:35 2004
From: aahz at pythoncraft.com (Aahz)
Date: Wed May 12 20:03:59 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405121513.27827.michel@dialnetwork.com>
References: <200405121513.27827.michel@dialnetwork.com>
Message-ID: <20040513000135.GB11008@panix.com>

On Wed, May 12, 2004, Michel Pelletier wrote:
>
> Would there be an interest in at least a PEP to consider a bytecode
> verifier?  I'd be willing to take it on, but my experience is limited
> to JVM bytecode and verification.

PEPs for complex issues are always welcome -- they're a valuable
resource for the future.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Adopt A Process -- stop killing all your children!

From bac at OCF.Berkeley.EDU  Wed May 12 20:52:30 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed May 12 20:53:36 2004
Subject: [Python-Dev] cookielib module in 2.4?
In-Reply-To: <Pine.LNX.4.58.0405122246560.1681@alice>
References: <Pine.LNX.4.58.0405122246560.1681@alice>
Message-ID: <40A2C6CE.9060007@ocf.berkeley.edu>

John J Lee wrote:
[snip - cookielib explanation]
> 
> So, assuming the module is likely to be accepted (given the answers to my
> post here last year), I'd be happy if anybody here could
> 
> 1. indicate if I'm roughly on the right track (especially welcome if
>    you've had a look at the subversion branch above),
> 

Code, docs, tests.  But in order for the module to be accepted it needs 
to show that its usage is truly needed by having it be widely used in 
the Python community.  I don't do web programming so others will have to 
vouch for the amount of usage.

All explained in the "Intro to Development" doc found at 
http://www.python.org/dev/ .

> 2. give me a rough idea how long I have to sort out the remaining issues
>    and submit a patch

How ever long it takes.  Not sure if I really understand what you are 
wondering about.

-Brett

From greg at cosc.canterbury.ac.nz  Wed May 12 22:15:35 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed May 12 22:15:58 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405121556.i4CFusNw025611@mta7.pltn13.pbi.net>
Message-ID: <200405130215.i4D2FZ2S004169@cosc353.cosc.canterbury.ac.nz>

Jeremy Hylton <jeremy@alum.mit.edu>:

> I think it would only need to check that any path through the loop has a no
> net effect on stack depth -- before branching back to the beginning of the
> loop, it must pop everything it pushed.

If the branching is sufficiently spaghettoid, it might have
trouble identifying any coherent loops or paths.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From skip at pobox.com  Wed May 12 22:47:14 2004
From: skip at pobox.com (Skip Montanaro)
Date: Wed May 12 22:46:24 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405121531.43532.michel@dialnetwork.com>
References: <200405121531.43532.michel@dialnetwork.com>
Message-ID: <16546.57778.575514.924484@montanaro.dyndns.org>

    Michel> The way to know if it's well formed is to verify it using
    Michel> something like a control flow analysis similar to the way the
    Michel> Java ASM bytecode generation API does it.  

This is more-or-less what the Python byte code compiler already does to
determine the high water mark for the stack used by a chunk of code it's
compiling.  Look for c_stacklevel, c_maxstacklevel and the com_push() and
com_pop() functions in Python/compile.c.

Skip

From pje at telecommunity.com  Wed May 12 23:13:15 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed May 12 23:12:52 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405130215.i4D2FZ2S004169@cosc353.cosc.canterbury.ac.nz>
References: <200405121556.i4CFusNw025611@mta7.pltn13.pbi.net>
Message-ID: <5.1.1.6.0.20040512223230.0391f860@mail.telecommunity.com>

At 02:15 PM 5/13/04 +1200, Greg Ewing wrote:
>Jeremy Hylton <jeremy@alum.mit.edu>:
>
> > I think it would only need to check that any path through the loop has a no
> > net effect on stack depth -- before branching back to the beginning of the
> > loop, it must pop everything it pushed.
>
>If the branching is sufficiently spaghettoid, it might have
>trouble identifying any coherent loops or paths.

As I think Tim has already mentioned, all you do is track the stack depth 
for each instruction, and ensure it's consistent.  If there is an 
inconsistency in the predicted stack level for an instruction, as compared 
with any of the instructions that jump to it (no matter where they jump 
from), you have a problem.

Or, to look at it another way:

maxdepth = 0
startpoints = [(0,0)]
stackdepth = [None] * len(bytecode)

while startpoints:
     location,depth = startpoints.pop()

     while True:
        if location >= len(bytecode):
            raise VerificationError("Unexpected end of bytecode at",location)

        if stackdepth[location] is None:
            stackdepth[location] = depth
        else:
            if stackdepth[location]<>depth:
                raise VerificationError("Inconsistent stack depth at",location)
            else:
                break    # we've already processed this part of the code

        instruction = fetch(bytecode,location)
        instruction.verify_for(code_object,location)  # Check operand validity

        depth += instruction.effectOnStack()
        maxdepth = max(depth,maxdepth)

        if depth<0:
            raise VerificationError("Stack underflow at",location)

        if instruction.isterminal:  (e.g. return, raise)
            break        # Block is over, handle the next one if any

        elif instruction.isconditional:
            startpoints.append((instruction.alt_location(location),depth))

        location = instruction.next_location(location)

This algorithm should readily handle all forward and backward jumps, while 
ensuring that no stack growth or shrinkage can take place, because every 
instruction must have a single known stack depth, no matter how many 
forward or backward jumps it takes to get there.

Notice that the algorithm will "follow" every branch, but will not retrace 
the code at the target location once the stack depth is seen to be 
consistent.  It also tackles underflow and "falling off the end" of the 
bytecode, while simultaneously computing the maximum stack depth 
used.  Each instruction is fetched and processed for its depth effects and 
other characteristics exactly once, so the timing is O(n).  And, it has a 
nice place to put opcode-specific operand validation for opcodes that use 
locals, freevars, names, or constants.

Anyway, I think we can safely halt discussion of the halting problem 
now.  :)  And, since I've now reduced this from an interesting theoretical 
idea to talk about, into something resembling work (God forbid), anybody 
who wants to actually *implement* this crazy idea is now free to do so, 
without even having to read the paper Tim linked to.  :)

(Disclaimer: I didn't read the paper either, so there may be other 
interesting/applicable verifications there that I didn't sketch 
above.  This is just a spur-of-the-moment off-the-top-of-my-head draft thingy.)


From tim.one at comcast.net  Wed May 12 23:30:24 2004
From: tim.one at comcast.net (Tim Peters)
Date: Wed May 12 23:30:33 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405121513.27827.michel@dialnetwork.com>
Message-ID: <E1BO6vW-0007K6-9f@mail.python.org>

[Michel Pelletier]
> ...
> Would there be an interest in at least a PEP to consider a bytecode
> verifier?

I think so, yes.

> I'd be willing to take it on, but my experience is limited to JVM bytecode
> and verification.

Python should be easier, in large part because you have to give up sooner.
For example, Python has a single catch-all BINARY_ADD opcode for infix "+".
There aren't distinct opcodes for "adding ints" and "adding floats" and
"adding strings".  So, in Python, there's no need to even try to check the
types of BINARY_ADD's stack operands.  They're always pointer-to-PyObject,
and since pointer-to-PyObject is the only kind of thing that *can* be pushed
on the PVM stack, there's nothing for a modestly ambitious static verifier
to look for there.

Checks for type correctness in the PVM are done at runtime instead.  I think
it's fair to say that a bytecode verifier is overwhelmingly "just an
optimization":  if bytecode properties can be verified from static code
analysis, runtime code isn't needed to verify them dynamically, or, in the
absence of such runtime checks, static analysis plugs ways to provoke
segfaults.

Examples of things that aren't checked at all now in the PVM were given
before, like the PVM doesn't check that it allocated enough stack space, or
check that C-level indexing into the co_consts vector is in bounds.  Well,
in the latter case, that is checked (at runtime, on every co_consts access)
in a debug build, but not in a release build.



From tim.one at comcast.net  Thu May 13 00:05:46 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu May 13 00:05:55 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <5.1.1.6.0.20040512223230.0391f860@mail.telecommunity.com>
Message-ID: <E1BO7Tk-0001tJ-2s@mail.python.org>

[Phillip J. Eby]
...
> (Disclaimer: I didn't read the paper either, so there may be other
> interesting/applicable verifications there that I didn't sketch
> above.  This is just a spur-of-the-moment off-the-top-of-my-head draft
> thingy.)

It's important to read what people who have actually done it have to say
about subtleties.  For example, your algorithm considered 'return' to be 'a
terminal instruction'.  But what if it appeared in the 'try' clause of a
try/finally construct?  In Python's PVM, most *possible* control flow is
implicit (virtually any opcode can raise an exception, and from there
"magically jump" to the code at an enclosing 'except' or 'finally' clause);
and even an unexceptional bare 'return' can magically jump to an enclosing
'finally' block.  The Java reference discusses these things, and anyone
intending to do something here is still strongly urged to read it.



From michel at dialnetwork.com  Wed May 12 22:11:23 2004
From: michel at dialnetwork.com (Michel Pelletier)
Date: Thu May 13 00:09:50 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405121531.43532.michel@dialnetwork.com>
References: <200405121531.43532.michel@dialnetwork.com>
Message-ID: <200405122111.23306.michel@dialnetwork.com>

On Wednesday 12 May 2004 15:31, Michel Pelletier wrote:

> The secondary benefit of this is that you are assured at runtime that the
> stack size the compiler computes for the bytecode is genuine.

Thinking aloud, maybe this is how it can be an exploit: create a code object 
with a falsified maximum stack size, and bytecode that over/underflows the 
incorrectly allocated value stack so that the right bit pattern is executed 
to give execution control to malicious C-level code.

One way to inject the exploit would be a file permission weakness alowing a 
bogus, but commonly used Python module in .pyc form to be placed into the 
system interpreter path.  This would allow an exploiter to execute any 
machine instructions for any use of that module, system wide.  A trojan 
distutils product could also taint a system interpreter.

This would not otherwise be a big deal, because a file permission weakness 
would allow you to place malicious bytecode anyway, but a stack 
over/underflow would allow not just PVM instructions, but "real" processor 
machine instructions, thus subverting any protections the PVM or some other 
tool (rexec, Zope style security) could provide even at the bytecode level.  
Verifying that bytecode is well formed would prevent this.

All of this could be, as the saying goes, a "bag full of fart".  I'm examining 
the source now to see if I can come up with a proof of concept which I will 
not share publicly.

-Michel

From judson at mcs.anl.gov  Thu May 13 00:33:11 2004
From: judson at mcs.anl.gov (Ivan R. Judson)
Date: Thu May 13 00:34:07 2004
Subject: [Python-Dev] IPv6 support in Win binary again
In-Reply-To: <40A27E6F.70806@v.loewis.de>
Message-ID: <200405130433.i4D4WxU35612@mcs.anl.gov>


What is this implying about the 2.4 build assumptions?

Vs.net 2008 :)? Platform sdk? Something else?

--Ivan

> -----Original Message-----
> From: python-dev-bounces+judson=mcs.anl.gov@python.org 
> [mailto:python-dev-bounces+judson=mcs.anl.gov@python.org] On 
> Behalf Of Martin v. L?wis
> Sent: Wednesday, May 12, 2004 2:44 PM
> To: mat@elandem.org
> Cc: python-dev@python.org
> Subject: Re: [Python-Dev] IPv6 support in Win binary again
> 
> Mat Ford wrote:
> > There was a thread back in October 2003 on the subject of including 
> > IPv6 support in the Windows binary:
> > 
> > 	
> http://mail.python.org/pipermail/python-dev/2003-October/038623.html
> > 
> > Can anyone give me a status update on this? Is support there now? 
> 
> No.
> 
> > If not, is
> > there a predictable timeframe in which it can be expected to appear?
> 
> With the release of Python 2.4, which, at latest, should 
> happen before January, 2005.
> 
> In case you wonder what the problem is: if you compile with 
> VC6, you cannot provide IPv6 support unless you also have the 
> platform SDK installed. That is not part of the build process 
> of Python 2.3, so the official binaries of Python 2.3 will 
> never ever support IPv6.
> Using VC.NET is out of the question, since it would break the 
> ABI across minor releases, which is out of the question. 
> Installing the SDK for the build process might be an option, 
> but the code base has no way of detecting presence of the 
> SDK, so this is essentially also out of question.
> 
> Regards,
> Martin
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/judson%40mcs.anl.gov
> 
> 


From anthony at interlink.com.au  Thu May 13 00:45:13 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu May 13 00:46:14 2004
Subject: [Python-Dev] cookielib module in 2.4?
In-Reply-To: <40A2C6CE.9060007@ocf.berkeley.edu>
References: <Pine.LNX.4.58.0405122246560.1681@alice>
	<40A2C6CE.9060007@ocf.berkeley.edu>
Message-ID: <40A2FD59.9040906@interlink.com.au>

Brett C. wrote:
> John J Lee wrote:
>> 2. give me a rough idea how long I have to sort out the remaining issues
>>    and submit a patch
> 
> 
> How ever long it takes.  Not sure if I really understand what you are 
> wondering about.

If the question is "how long has he got before 2.4 alpha 1", the answer
is "about 6 weeks". I'm still aiming for a first alpha in the first week
of July or thereabouts.


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From arigo at tunes.org  Thu May 13 04:54:01 2004
From: arigo at tunes.org (Armin Rigo)
Date: Thu May 13 04:57:15 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405121531.43532.michel@dialnetwork.com>
References: <200405121531.43532.michel@dialnetwork.com>
Message-ID: <20040513085401.GA28305@vicky.ecs.soton.ac.uk>

Hello Michel,

On Wed, May 12, 2004 at 03:31:43PM -0500, Michel Pelletier wrote:
> But I belive Armin's argument that it equals the halting problem is
> incorrect.

You are missing my point.  Martin summarized what I intended to say: what is
equivalent to the halting problem is the question "can you say if the
following bytecode is going to execute some illegal opcode or not", which is a
purely theoretical question, as I am well aware.  The useful and checkable
question that interests us here is "does this bytecode follow the following
set of sane rules", for a well-specified set of rules.

> The net stack effect of bytecode can be verified regardless of the algorithm.

Interestingly, it is not the case currently that for any bytecode position the
stack depth is always the same.  Here is a short review.

* 'finally' blocks are executed with one, two or three values pushed on the
  stack; END_FINALLY will accordingly pop one, two or three values (depending
  on what the first popped value is).

* POP_BLOCK will empty the value stack up to the position stored in the block
  stack's popped block, for no reason that I am aware of.  I can't think of a
  compiler-emitted construct that uses this feature.  Maybe it was introduced
  to implement 'break' as a jump to the POP_BLOCK, but nowadays 'break' is
  more complicated anyway, using the general stack-unwinding and
  exception-handler-dispatching loop after the switch in ceval.c.

* MAKE_CLOSURE pops some cells to use for free variables, the number of which
  is found in the code object on the top of the stack.

I believe that all the other instructions have a stack effect that is either
constant or computable from their oparg.

If some long-term goal is to fix this, then MAKE_CLOSURE could be preceded by
a BUILD_TUPLE to collect all free vars into a tuple (which is what
MAKE_CLOSURE does anyway).  Fixing the 'finally' blocks isn't too complicated
-- always push three values on the stack, possibly None's.  A better 'fix'
would be that the block stack could, as we discussed at PyCon, be replaced by
a purely static structure in the code object.


Armin

From mwh at python.net  Thu May 13 05:40:26 2004
From: mwh at python.net (Michael Hudson)
Date: Thu May 13 05:40:36 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <E1BO6vW-0007K6-9f@mail.python.org> (Tim Peters's message of
	"Wed, 12 May 2004 23:30:24 -0400")
References: <E1BO6vW-0007K6-9f@mail.python.org>
Message-ID: <2mpt98ll6d.fsf@starship.python.net>

"Tim Peters" <tim.one@comcast.net> writes:

> Examples of things that aren't checked at all now in the PVM were given
> before, like the PVM doesn't check that it allocated enough stack space, or
> check that C-level indexing into the co_consts vector is in bounds.  Well,
> in the latter case, that is checked (at runtime, on every co_consts access)
> in a debug build, but not in a release build.

I think stack over/underflow is checked in a debug build too.

So here's a way to plug these holes: run a debug build the whole time
:-) It's only, what, a factor of 3 performance hit.

If people are seriously concerned about preventing Python coring in
all circumstances, we should look at the various ways of overflowing
the stack, too.  The recursion counter is a hack, and IMHO can't be
made to work sensibly.  For instance: list_sort() has quite a large
structure on the stack, so if you arrange to call it recursively it
falls off the end of the stack much much sooner than a simple Python
recursion.

I don't know what we can do about this.  Armin suggested another hack:
stick the address of a stack variable in Py_Main() in a global and
compare the address of another stack variable when you want to see how
much stack you have left.  Obvious problems include knowing what's
safe and which direction the stack is growing in... Even more scarily,
what SBCL (a Common Lisp implementation) does is mprotect() a VM page
and the end of the stack and deal with overflow in a SIGSEGV handler.
It's hard to see what else could be really safe (have I mentioned that
I hate C recently?).

Option 3, I guess, is integrate stackless :-)

Cheers,
mwh

-- 
8. A programming language is low level when its programs require
   attention to the irrelevant.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html

From tismer at stackless.com  Thu May 13 07:08:08 2004
From: tismer at stackless.com (Christian Tismer)
Date: Thu May 13 07:06:54 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405110420.i4B4Ku3P000714@cosc353.cosc.canterbury.ac.nz>
References: <200405110420.i4B4Ku3P000714@cosc353.cosc.canterbury.ac.nz>
Message-ID: <40A35718.8020700@stackless.com>

Greg Ewing wrote:

> Guido:
> 
> 
>>>As long as it's possible to attempt to execute arbitrary strings
>>>as bytecode, I'd say ceval should be robust against this.
>>
>>This ought to be a long-term project then: write a bytecode verifier.
>>It's not a trivial task!
> 
> 
> When I wrote that, I was assuming that ceval was already mostly robust
> in this respect, and that what was being reported was a new hole
> recently opened up.
> 
> But it appears I was gravely mistaken, and that ceval has been full of
> gaping holes from the beginning.
> 
> I'm disappointed to learn this, because I had always regarded it as an
> axiom that no Python-level code should be capable of crashing the
> interpreter, and if it can, this represents a bug. However, it seems
> this axiom has not been adhered to in the design of ceval.

I don't think so. ceval appears to be absolutely robust against byte
code that the bytecode compiler has generated.
I think the design error was to expose code object construction
without safety belts.

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From tismer at stackless.com  Thu May 13 08:58:30 2004
From: tismer at stackless.com (Christian Tismer)
Date: Thu May 13 08:57:09 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <2mpt98ll6d.fsf@starship.python.net>
References: <E1BO6vW-0007K6-9f@mail.python.org>
	<2mpt98ll6d.fsf@starship.python.net>
Message-ID: <40A370F6.4020501@stackless.com>

Michael Hudson wrote:

...

> I don't know what we can do about this.  Armin suggested another hack:
> stick the address of a stack variable in Py_Main() in a global and
> compare the address of another stack variable when you want to see how
> much stack you have left.  Obvious problems include knowing what's
> safe and which direction the stack is growing in... Even more scarily,
> what SBCL (a Common Lisp implementation) does is mprotect() a VM page
> and the end of the stack and deal with overflow in a SIGSEGV handler.
> It's hard to see what else could be really safe (have I mentioned that
> I hate C recently?).
> 
> Option 3, I guess, is integrate stackless :-)

Just as a note: Even Stackless can get into deep recursions
and then uses a hack similar to Armin's suggestion.
As a side effect, this made it quite simple to convince
cPickle to pickle very deeply nested structures.

On verification:

I think I'm all against writing a bytecode verifier,
because everything needed is already there. Hartmut Goebel
has written a nice Python decompyler, based upon John
Aycock's spark and prior work.
It produces output that often looks better than the original
source. And they verify the decompiled bytecode by compiling
it again.
The drawback is that this appears to be no longer an
open source project, see http://www.crazy-compilers.com/decompyle/
Maybe we should talk to Hartmut...

On sending code over the network:

Bytecode verification is fine, but you don't want to execute
foreign bytecode, even if it is valid. This is like executing
any binary program, which might do anything, being Python or not.
Since people are going to send programs over the network,
we will need some way to exchange compiled code in a trusted
manner, and I believe this take more considerations which is
a matter for the crypto people.

Anyway, here is

My Proposal (TM)
================

Let's assume that you don't trust any bytecode that has not yet
been verified for your machine.
Further, you generate a private key for your machine, or yourself.

For .pyc archives which you create yourself, your private key
is used to initialize a sha digest, and then the .pyc is run
through the digest, and the digest added to the .pyc.
When loading any .pyc, your private key is used again to
compute the digest, and the result verified against the
digest which is in the pyc (at its end I guess).

For code objects which come over a network, I suggest a similar
check, given that you have gotten the other side's key
and you can use it to verify foreign code. This works in
trusted networks, only. In public networks you would
need public/private key pairs... much harder.

Now, any bytecode that is not yet verified can be run through
decompyle just once, and the result is compiled again, signed
with your key, and stored.
This is more than requested, since it doesn't allow for any
bytecode sequence, but just something that has an existing
possible source code. Well, actually I think we want this,
because bytecode is an optimization artifact.

Summarizing, I think to add SHA keys to bytecode and .pyc
files, at least as an option, makes some sense and is fast
to check on every external access to bytecode.
Verification of healthy code on the VM level could be made
easy with decompyle.

cheers - chris
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From barry at python.org  Thu May 13 08:59:34 2004
From: barry at python.org (Barry Warsaw)
Date: Thu May 13 08:59:43 2004
Subject: [Python-Dev] cookielib module in 2.4?
In-Reply-To: <Pine.LNX.4.58.0405122246560.1681@alice>
References: <Pine.LNX.4.58.0405122246560.1681@alice>
Message-ID: <1084453174.28228.656.camel@anthem.wooz.org>

On Wed, 2004-05-12 at 18:40, John J Lee wrote:
> I have a module that I'd like to get included in Python 2.4.  It adds
> transparent HTTP cookie handling to urllib2

Although I haven't looked at your code, I think the functionality would
be a good addition to Python.  Brett's right though, it needs (Python
style) documentation and unit tests.

I wonder if it makes sense to combine the functionality of Cookie.py and
cookielib.py?  Perhaps not, but at the very least I'd like to see
something like this integrated into the eventual output of the web-sig
(probably not for 2.4).

-Barry



From anthony at python.org  Thu May 13 09:48:03 2004
From: anthony at python.org (Anthony Baxter)
Date: Thu May 13 10:19:04 2004
Subject: [Python-Dev] RELEASED Python 2.3.4, release candidate 1
Message-ID: <40A37C93.1090305@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Python development team and the Python community, I'm
happy to announce the release of Python 2.3.4 (release candidate 1).

Python 2.3.4 is a bug-fix release. See the release notes at the website
(also available as Misc/NEWS in the source distribution) for details of
the bugs squished in this release.

Assuming no major problems crop up, a final release of Python 2.3.4
will follow late next week.

For more information on Python 2.3.4, including download links for
various platforms, release notes, and known issues, please see:

~    http://www.python.org/2.3.4

Highlights of this new release include:

~  - Bug fixes. According to the release notes, more than 20 bugs
~    have been fixed, including a couple of bugs that could cause
~    Python to crash. These were discovered by Zope3.

Highlights of the previous major Python release (2.3) are available
from the Python 2.3 page, at

~    http://www.python.org/2.3/highlights.html

Enjoy the new release,
Anthony

Anthony Baxter
anthony@python.org
Python Release Manager
(on behalf of the entire python-dev team)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFAo3yTDt3F8mpFyBYRAicdAJwOngKitELJ25u3oCW+iQcc581wbQCgh1Ah
f/Ci2omZVG7p63xY9cfZiSw=
=e3Rv
-----END PGP SIGNATURE-----

From nas at arctrix.com  Thu May 13 10:24:53 2004
From: nas at arctrix.com (Neil Schemenauer)
Date: Thu May 13 10:24:58 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <20040513085401.GA28305@vicky.ecs.soton.ac.uk>
References: <200405121531.43532.michel@dialnetwork.com>
	<20040513085401.GA28305@vicky.ecs.soton.ac.uk>
Message-ID: <20040513142453.GA11925@mems-exchange.org>

On Thu, May 13, 2004 at 09:54:01AM +0100, Armin Rigo wrote:
> If some long-term goal is to fix this, then MAKE_CLOSURE could be
> preceded by a BUILD_TUPLE to collect all free vars into a tuple
> (which is what MAKE_CLOSURE does anyway).

I've done this in the ast branch.

  Neil

From jim.jewett at eds.com  Thu May 13 10:23:55 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Thu May 13 10:25:06 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D4B2@usahm010.exmi01.exch.eds.com>


Martin v. L?wis:

> Therefore, byte code verification usually puts additional
> constraints on byte code, rejecting some correct code as
> unverifiable.

An alternative would be to accept unverifiable code.

You won't stop all problems, but you can still catch
the most common mistakes.  For instance, the example
that started this thread was apparently an ASCII string
of uncomiled source code.

Just checking that all instructions are valid bytecodes
might have caught this.  Also checking that arguments 
are possible (no jumping beyond the code) would probably
have caught it.  

Verifying stack depth and such might be overkill for a 
first pass sanity check.

-jJ

From tim.one at comcast.net  Thu May 13 10:57:58 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu May 13 10:57:58 2004
Subject: [Python-Dev] IPv6 support in Win binary again
In-Reply-To: <200405130433.i4D4WxU35612@mcs.anl.gov>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEFHKPAB.tim.one@comcast.net>

[Ivan Judson]
> What is this implying about the 2.4 build assumptions?
>
> Vs.net 2008 :)? Platform sdk? Something else?

The Windows Python 2.4 distributed by the PSF will be built with VC 7.1 (aka
VC.NET 2003), assuming things go as planned.  When Martin said:

>> Using VC.NET is out of the question, since it would break the ABI
>> across minor releases, which is out of the question.

he should perhaps have said "micro" releases, under the usual

    major.minor.micro

release-numbering scheme.  In the Python world, a boost in the minor
(second) number is more what other projects consider to be a major release.
IOW, the Python numbering scheme is more like

    new_language.major.bugfix


From guido at python.org  Thu May 13 12:51:54 2004
From: guido at python.org (Guido van Rossum)
Date: Thu May 13 12:52:00 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: Your message of "Thu, 13 May 2004 09:54:01 BST."
	<20040513085401.GA28305@vicky.ecs.soton.ac.uk> 
References: <200405121531.43532.michel@dialnetwork.com>  
	<20040513085401.GA28305@vicky.ecs.soton.ac.uk> 
Message-ID: <200405131651.i4DGpsl21234@guido.python.org>

> Interestingly, it is not the case currently that for any bytecode
> position the stack depth is always the same.  Here is a short
> review.

Thanks for this overview!  I'd lost track of details like this over
the years.

> * 'finally' blocks are executed with one, two or three values pushed
>   on the stack; END_FINALLY will accordingly pop one, two or three
>   values (depending on what the first popped value is).

This is easily fixed, if necessary -- but is it necessary?

> * POP_BLOCK will empty the value stack up to the position stored in
>   the block stack's popped block, for no reason that I am aware of.
>   I can't think of a compiler-emitted construct that uses this
>   feature.  Maybe it was introduced to implement 'break' as a jump
>   to the POP_BLOCK, but nowadays 'break' is more complicated anyway,
>   using the general stack-unwinding and
>   exception-handler-dispatching loop after the switch in ceval.c.

I can't remember what my intention was; I probably wasn't so sure that
the stack level at the end of executing arbitrary code was always
the same.

> * MAKE_CLOSURE pops some cells to use for free variables, the number
>   of which is found in the code object on the top of the stack.

This is a recent addition, and should have been done differently.
It's not hard to fix, as you suggest (and as has been done in the AST
branch).

> I believe that all the other instructions have a stack effect that
> is either constant or computable from their oparg.

Me too.

> If some long-term goal is to fix this, then MAKE_CLOSURE could be
> preceded by a BUILD_TUPLE to collect all free vars into a tuple
> (which is what MAKE_CLOSURE does anyway).  Fixing the 'finally'
> blocks isn't too complicated -- always push three values on the
> stack, possibly None's.  A better 'fix' would be that the block
> stack could, as we discussed at PyCon, be replaced by a purely
> static structure in the code object.

Correct, I would like to see this happen.  There's dynamic stuff here
just to make the implementation easy rather than to support necessary
dynamism.  (The same could be said of some other parts of Python, but
there we've come to love the dynamism -- for the most part anyway. ;-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From jeremy at alum.mit.edu  Thu May 13 13:08:40 2004
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu May 13 13:08:45 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <2mpt98ll6d.fsf@starship.python.net>
Message-ID: <E1BOJhK-0001Lo-Kj@mail.python.org>

> I think stack over/underflow is checked in a debug build too.

> So here's a way to plug these holes: run a debug build the whole time
> :-) It's only, what, a factor of 3 performance hit.

That's what I do.  It's only a small constant factor.

Jeremy



From jjl at pobox.com  Thu May 13 14:36:56 2004
From: jjl at pobox.com (John J Lee)
Date: Thu May 13 14:37:03 2004
Subject: [Python-Dev] cookielib module in 2.4?
In-Reply-To: <1084453174.28228.656.camel@anthem.wooz.org>
References: <Pine.LNX.4.58.0405122246560.1681@alice>
	<1084453174.28228.656.camel@anthem.wooz.org>
Message-ID: <Pine.LNX.4.58.0405131927060.473@alice>

On Thu, 13 May 2004, Barry Warsaw wrote:

> On Wed, 2004-05-12 at 18:40, John J Lee wrote:
> > I have a module that I'd like to get included in Python 2.4.  It adds
> > transparent HTTP cookie handling to urllib2
>
> Although I haven't looked at your code, I think the functionality would
> be a good addition to Python.  Brett's right though, it needs (Python
> style) documentation and unit tests.

There are unit tests aplenty, and LaTeX docs are on the way.


> I wonder if it makes sense to combine the functionality of Cookie.py and
> cookielib.py?  Perhaps not, but at the very least I'd like to see

Not. :-)

There is little overlap between the two modules, and it would probably
complicate cookielib to combine it with Cookie (and might possibly make my
brain explode).


> something like this integrated into the eventual output of the web-sig
> (probably not for 2.4).

The web-sig list has been dead for some time.  I haven't seen any
client-side output from web-sig, other than my own work on a Browser
class.  Haven't seen any server-side output about which there's any
consensus, either, come to that.


John

From jjl at pobox.com  Thu May 13 14:42:35 2004
From: jjl at pobox.com (John J Lee)
Date: Thu May 13 14:42:43 2004
Subject: [Python-Dev] cookielib module in 2.4?
In-Reply-To: <40A2FD59.9040906@interlink.com.au>
References: <Pine.LNX.4.58.0405122246560.1681@alice>
	<40A2C6CE.9060007@ocf.berkeley.edu>
	<40A2FD59.9040906@interlink.com.au>
Message-ID: <Pine.LNX.4.58.0405131937090.473@alice>

On Thu, 13 May 2004, Anthony Baxter wrote:

> Brett C. wrote:
> > John J Lee wrote:
> >> 2. give me a rough idea how long I have to sort out the remaining issues
> >>    and submit a patch
> >
> >
> > How ever long it takes.  Not sure if I really understand what you are
> > wondering about.
>
> If the question is "how long has he got before 2.4 alpha 1", the answer
> is "about 6 weeks". I'm still aiming for a first alpha in the first week
> of July or thereabouts.

Question is : "how long before it's too late to get into 2.4?"

I take it from your answer that there's not a long freeze period planned
before first alpha, so I'll submit a patch in a week or two.

Thanks all


John

From michel at dialnetwork.com  Thu May 13 13:07:55 2004
From: michel at dialnetwork.com (Michel Pelletier)
Date: Thu May 13 15:09:34 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <E1BOCkZ-00065T-2k@mail.python.org>
References: <E1BOCkZ-00065T-2k@mail.python.org>
Message-ID: <200405131207.55026.michel@dialnetwork.com>

On Thursday 13 May 2004 04:43, python-dev-request@python.org wrote:
> From: "Phillip J. Eby" <pje@telecommunity.com>

> As I think Tim has already mentioned, all you do is track the stack depth
> for each instruction, and ensure it's consistent.  If there is an
> inconsistency in the predicted stack level for an instruction, as compared
> with any of the instructions that jump to it (no matter where they jump
> from), you have a problem.
>
> Or, to look at it another way:

<snip code example>

Thanks Phillip, this code is a great starting point.  I was trying to 
reproduce the algorithm given in the JVM Spec section 4.9, but it is 
seriously complicated by strict type issue which have no meaning here.

Also, as Armin pointed out, there are a couple instances of instructions that 
would be difficult to analize without some changes to their design 
(END_FINALLY and MAKE_CLOSURE).  But he suggests relatively simple solutions 
to both.

I am also working on a short list of static and structural constraints for 
bytecode, this is what I have so far (the ones i'm unsure about are 
questions):

Static Constraints on Bytecode Instructions

  1. The bytecode string must not be empty (len(co_code) > 0).

  2. Is there maximum co_code length?

  3. The first instruction in the bytecode string begins at index 0.

  4. Only valid bytecodes with correct number of operands can be in
     the bytecode string.

  5. The index of the next instruction equals the index of the current
     instruction plus the length of the current instruction and all
     its operands.

  6. The last byte of the last instruction must be len(co_code) - 1.

  7. The last instruction must be RETURN_VALUE?

  8. Exception handlers verification?  Maybe the compiler could generate a             
table of handlers, their handled types, and their instruction ranges for 
verification.  The run-time benefit would be that an exception handler could 
be quickly dispatched instead of falling through each except block in 
sequence looking for the matching handler as it looks to work now (not that 
it's a big performance bottleneck...)

Static Constraints on Bytecode Instruction Operands

  1. The target of a jump instruction must be within the code
     boundaries and must not fall between an instruction and its
     operands.

  2. The operand of a LOAD_CONST instruction must be an valid index
     into the co_consts.

  3. Can a similar bounds check be made at verfication time for
     LOAD_FAST?

  4. Can index into co_names, co_varnames, co_freevars, co_cellvars be
     verfied?

Structural Constraints between Bytecode Instructions

  1. Each instruction must only be executed with the appropriate
     number of arguments in the operand stack, regardless of the
     execution path that leads to its invocation.

  2. If an instruction can be executed along several different
     execution paths, the value stack must have the same depth prior
     to the execution of the instruction, regardless of the path
     taken.

  3. At no point during execution can the value stack grow to a
     depth greater than that implied by co_stacksize.
 
> This algorithm should readily handle all forward and backward jumps, while
> ensuring that no stack growth or shrinkage can take place, because every
> instruction must have a single known stack depth, no matter how many
> forward or backward jumps it takes to get there.
>
> Notice that the algorithm will "follow" every branch, but will not retrace
> the code at the target location once the stack depth is seen to be
> consistent.  It also tackles underflow and "falling off the end" of the
> bytecode, while simultaneously computing the maximum stack depth
> used.  Each instruction is fetched and processed for its depth effects and
> other characteristics exactly once, so the timing is O(n).  And, it has a
> nice place to put opcode-specific operand validation for opcodes that use
> locals, freevars, names, or constants.

Yea!  As Tim points out below there are some subtleties, but this is great.

> Anyway, I think we can safely halt discussion of the halting problem
> now.  :)  And, since I've now reduced this from an interesting theoretical
> idea to talk about, into something resembling work (God forbid), anybody
> who wants to actually *implement* this crazy idea is now free to do so,
> without even having to read the paper Tim linked to.  :)

Well I did read it, and I suggest anyone else also read chapter 4 of the Java 
VM Spec.  

-Michel



From michel at dialnetwork.com  Thu May 13 13:04:36 2004
From: michel at dialnetwork.com (Michel Pelletier)
Date: Thu May 13 15:09:47 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <E1BOCkZ-00065T-2k@mail.python.org>
References: <E1BOCkZ-00065T-2k@mail.python.org>
Message-ID: <200405131204.36176.michel@dialnetwork.com>

On Thursday 13 May 2004 04:43, python-dev-request@python.org wrote:

> It's important to read what people who have actually done it have to say
> about subtleties.  For example, your algorithm considered 'return' to be 'a
> terminal instruction'.  But what if it appeared in the 'try' clause of a
> try/finally construct?  In Python's PVM, most *possible* control flow is
> implicit (virtually any opcode can raise an exception, and from there
> "magically jump" to the code at an enclosing 'except' or 'finally' clause);
> and even an unexceptional bare 'return' can magically jump to an enclosing
> 'finally' block.  The Java reference discusses these things, and anyone
> intending to do something here is still strongly urged to read it.

But just looking at some code here, is RETURN_VALUE ever inside a SETUP_EXCEPT 
block?  The end of the SETUP_EXCEPT block apears to ABSOLUTE_JUMP past all 
the exception handlers to finally, and then to the RETURN_VALUE, which 
doesn't seem to fall under the protection of a try/except.

I'm not saying there are no subtleties as you describe, I'm just trying to 
quantify some of them and wondering if this is one.

-Michel

From martin at v.loewis.de  Thu May 13 15:23:35 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu May 13 15:23:41 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405131204.36176.michel@dialnetwork.com>
References: <E1BOCkZ-00065T-2k@mail.python.org>
	<200405131204.36176.michel@dialnetwork.com>
Message-ID: <40A3CB37.4090001@v.loewis.de>

Michel Pelletier wrote:
> But just looking at some code here, is RETURN_VALUE ever inside a SETUP_EXCEPT 
> block?  

What about this code:

 >>> def foo():
...   try:
...     try:
...       return 7
...     except:
...       pass
...   finally:
...     print "Hello"
...
 >>> dis.dis(foo)
   2           0 SETUP_FINALLY           22 (to 25)

   3           3 SETUP_EXCEPT             8 (to 14)

   4           6 LOAD_CONST               1 (7)
               9 RETURN_VALUE
              10 POP_BLOCK
              11 JUMP_FORWARD             7 (to 21)

   5     >>   14 POP_TOP
              15 POP_TOP
              16 POP_TOP

   6          17 JUMP_FORWARD             1 (to 21)
              20 END_FINALLY
         >>   21 POP_BLOCK
              22 LOAD_CONST               0 (None)

   8     >>   25 LOAD_CONST               2 ('Hello')
              28 PRINT_ITEM
              29 PRINT_NEWLINE
              30 END_FINALLY
              31 LOAD_CONST               0 (None)
              34 RETURN_VALUE



From barry at python.org  Thu May 13 15:23:48 2004
From: barry at python.org (Barry Warsaw)
Date: Thu May 13 15:23:57 2004
Subject: [Python-Dev] cookielib module in 2.4?
In-Reply-To: <Pine.LNX.4.58.0405131927060.473@alice>
References: <Pine.LNX.4.58.0405122246560.1681@alice>
	<1084453174.28228.656.camel@anthem.wooz.org>
	<Pine.LNX.4.58.0405131927060.473@alice>
Message-ID: <1084476227.28228.753.camel@anthem.wooz.org>

On Thu, 2004-05-13 at 14:36, John J Lee wrote:

> There are unit tests aplenty, and LaTeX docs are on the way.

Cool.

> > I wonder if it makes sense to combine the functionality of Cookie.py and
> > cookielib.py?  Perhaps not, but at the very least I'd like to see
> 
> Not. :-)
> 
> There is little overlap between the two modules, and it would probably
> complicate cookielib to combine it with Cookie (and might possibly make my
> brain explode).
> 
> 
> > something like this integrated into the eventual output of the web-sig
> > (probably not for 2.4).
> 
> The web-sig list has been dead for some time.  I haven't seen any
> client-side output from web-sig, other than my own work on a Browser
> class.  Haven't seen any server-side output about which there's any
> consensus, either, come to that.

Sigh.  Well, don't wait for them. ;)

-Barry



From tim.one at comcast.net  Thu May 13 15:32:30 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu May 13 15:32:35 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405131204.36176.michel@dialnetwork.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEHHKPAB.tim.one@comcast.net>

[Michel Pelletier]
> But just looking at some code here, is RETURN_VALUE ever inside a
> SETUP_EXCEPT block?  The end of the SETUP_EXCEPT block apears to
> ABSOLUTE_JUMP past all the exception handlers to finally, and then to
> the RETURN_VALUE, which doesn't seem to fall under the protection of a
> try/except.

I don't think I can understand what you're saying without a concrete
example.  Here's one:

"""
def f():
    try:
        return 42
    finally:
        print 'yup'

from dis import dis
dis(f)
"""

That displays:

"""
  2           0 SETUP_FINALLY            8 (to 11)

  3           3 LOAD_CONST               1 (42)
              6 RETURN_VALUE
              7 POP_BLOCK
              8 LOAD_CONST               0 (None)

  4     >>   11 LOAD_CONST               2 ('yup')

  5          14 PRINT_ITEM
             15 PRINT_NEWLINE
             16 END_FINALLY
             17 LOAD_CONST               0 (None)
             20 RETURN_VALUE
"""

The RETURN_VALUE at 6 doesn't return right away, although that's not obvious
from the byte code.  The code at 11, 14, 15, 16 executes first, and that
can't be guessed from staring at 6 in isolation.  I think the code at 7, 8,
17 and 20 is actually unreachable -- and that's so not obvious I had to say
"I think" <wink>.


From skip at pobox.com  Thu May 13 15:35:44 2004
From: skip at pobox.com (Skip Montanaro)
Date: Thu May 13 15:34:27 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405131207.55026.michel@dialnetwork.com>
References: <E1BOCkZ-00065T-2k@mail.python.org>
	<200405131207.55026.michel@dialnetwork.com>
Message-ID: <16547.52752.985771.539493@montanaro.dyndns.org>


    Michel>   7. The last instruction must be RETURN_VALUE?

Nope.  Relatively simple analysis of the basic blocks in the bytecode can
determine if the trailing push none/return value pair is dead code and
remove it if it is.

    Michel>   2. The operand of a LOAD_CONST instruction must be an valid
    Michel>      index into the co_consts.

    Michel>   3. Can a similar bounds check be made at verfication time for
    Michel>      LOAD_FAST?

Should be possible, yes.

    Michel>   4. Can index into co_names, co_varnames, co_freevars,
    Michel>      co_cellvars be verfied?

Also yes.  These are all fixed-length arrays (tuples actually) whose lengths
are determined at compile-time.

Skip

From fumanchu at amor.org  Thu May 13 15:46:32 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Thu May 13 15:49:30 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E5620EA@opus.amorhq.net>

Michel Pelletier wrote:

> I am also working on a short list of static and structural 
> constraints for 
> bytecode, this is what I have so far (the ones i'm unsure about are 
> questions):

A couple gotchas when useing new.code():

1. co_names, co_varnames, co_freevars, and co_cellvars take tuples of
strings; passing unicode strings crashes the interpreter. co_consts,
however, is unicode-safe.

2. Some co_flags should be checked to be consistent with other args; for
example, shouldn't CO_NOFREE be set if co_freevars is empty?


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org

From tim.one at comcast.net  Thu May 13 19:31:07 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu May 13 19:31:10 2004
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Lib/email
	Parser.py, 1.20, 1.20.12.1
In-Reply-To: <E1BOPNU-0001eN-PD@sc8-pr-cvs1.sourceforge.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEIPKPAB.tim.one@comcast.net>

[Barry]
> Log Message:
> _parsebody(): Do not create subparts unless the container has a main
> type of 'multipart' and the boundary is defined.  This fixes SF bug #
> 846938, and several recent email-sig bugs ...

Yadda yadda yadda.  At this point, the more obvious solution is to ignore
all that MIME crap, and just guess a good structure for the email.  The
spambayes Outlook addin does that all the time <wink>.


From barry at python.org  Thu May 13 19:45:44 2004
From: barry at python.org (Barry Warsaw)
Date: Thu May 13 19:46:08 2004
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Lib/email
	Parser.py, 1.20, 1.20.12.1
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEIPKPAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEIPKPAB.tim.one@comcast.net>
Message-ID: <1084491944.28228.854.camel@anthem.wooz.org>

On Thu, 2004-05-13 at 19:31, Tim Peters wrote:

> Yadda yadda yadda.  At this point, the more obvious solution is to ignore
> all that MIME crap, and just guess a good structure for the email.  The
> spambayes Outlook addin does that all the time <wink>.

Structure's for monkeyboys and pointy-bracket lovers, not for studly
w1nd0z 1nst4ll3r d00dz and 3m41l l33t h4x0rz.



From tdelaney at avaya.com  Thu May 13 19:51:09 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Thu May 13 19:51:19 2004
Subject: [Python-Dev] .pth files in current or script directory
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE0179A3E5@au3010avexu1.global.avaya.com>

A discussion (last week?) on c.l.py prompted a thought about .pth files.
Feel free to shoot it down ;)

The discussion in question concerned someone using a version control
system, and needing to change PYTHONPATH depending on which branch they
were in.

My immediately two thoughts were:

1. Use a build system (e.g. make) to set PYTHONPATH appropriately. This
is they type of thing we do.

2. Use a .pth file in the directory the code is run from (or possibly,
the same directory as the script).

Before posting the second, I thought I'd better try it out - fortunately
for me :) Putting a .pth in the current or script directory did nothing.

I'm wondering if anyone sees utility in being able to put a .pth file in
those locations and have them used in the same way as if they were in
the python executable directory.

For example, if you have a package 'p', with a 'test' subpackage
(containing scripts), it would be very simple to stick a 'test.pth' file
into the 'p.test' package, with the sole contents of 'test.pth' being
'..'. This would add the (absolute path of the) parent directory to the
root paths for module import.

I feel there's some kind of link here with PEP 328, but I'm not quite
sure where it fits.

Tim Delaney 

From greg at cosc.canterbury.ac.nz  Thu May 13 20:28:28 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu May 13 20:28:51 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <2mpt98ll6d.fsf@starship.python.net>
Message-ID: <200405140028.i4E0SSTe005757@cosc353.cosc.canterbury.ac.nz>

Michael Hudson <mwh@python.net>:

> Even more scarily, what SBCL (a Common Lisp implementation) does is
> mprotect() a VM page and the end of the stack and deal with overflow
> in a SIGSEGV handler.

Even that isn't necessarily reliable. What if a C call allocates
such a big stack frame that it jumps right over the protected
page? Eventually it will try to access the protected area, but
other chaos could occur in the meantime.

The only completely safe thing I can think of is to eliminate
C-level recursion altogether, a la Stackless.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Thu May 13 20:35:59 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu May 13 20:36:09 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <40A370F6.4020501@stackless.com>
Message-ID: <200405140035.i4E0ZxUd005767@cosc353.cosc.canterbury.ac.nz>

Christian Tismer <tismer@stackless.com>:

> Now, any bytecode that is not yet verified can be run through
> decompyle just once, and the result is compiled again, signed
> with your key, and stored.

If you're going to all that bother, why not just send
suitably-signed source code in the first place?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From guido at python.org  Thu May 13 20:25:53 2004
From: guido at python.org (Guido van Rossum)
Date: Thu May 13 21:28:58 2004
Subject: [Python-Dev] .pth files in current or script directory
In-Reply-To: Your message of "Fri, 14 May 2004 09:51:09 +1000."
	<338366A6D2E2CA4C9DAEAE652E12A1DE0179A3E5@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE0179A3E5@au3010avexu1.global.avaya.com>
Message-ID: <200405140025.i4E0Pro21903@guido.python.org>

> The discussion in question concerned someone using a version control
> system, and needing to change PYTHONPATH depending on which branch they
> were in.

Can't they manipulate sys.path *inside* the code in question?  That's
invariably the solution I end up using -- I never play with
PYTHONPATH any more.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From jepler at unpythonic.net  Thu May 13 21:44:13 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Thu May 13 21:44:29 2004
Subject: [Python-Dev] .pth files in current or script directory
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE0179A3E5@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE0179A3E5@au3010avexu1.global.avaya.com>
Message-ID: <20040514014413.GB26047@unpythonic.net>

I'm -NFS against reading .pth files from the current directory.
When a directory is large and is on a remote filesystem, enumerating it
at startup can really kill time -- heck, just putting an NFS directory
with a handful of entries in PYTHONPATH can slow Python's startup time
considerably, now imagine that you "cd /nfs/directory_with_1000_files"
and "time python -c 'pass'" .. blah!

Jeff

From tismer at stackless.com  Thu May 13 21:49:44 2004
From: tismer at stackless.com (Christian Tismer)
Date: Thu May 13 21:48:22 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405140035.i4E0ZxUd005767@cosc353.cosc.canterbury.ac.nz>
References: <200405140035.i4E0ZxUd005767@cosc353.cosc.canterbury.ac.nz>
Message-ID: <40A425B8.7080203@stackless.com>

Greg Ewing wrote:

> Christian Tismer <tismer@stackless.com>:
> 
> 
>>Now, any bytecode that is not yet verified can be run through
>>decompyle just once, and the result is compiled again, signed
>>with your key, and stored.
> 
> 
> If you're going to all that bother, why not just send
> suitably-signed source code in the first place?

Because I prefer suitably-signed bytecode, which would
usually only need the key verification, but no compile
step. There is also no redundancy due to formatting.
A program with a specific key always *is* that program.
I always thought that with a little addition, Python source
code could be completely replaced by bytecode.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From greg at cosc.canterbury.ac.nz  Thu May 13 23:46:21 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu May 13 23:46:47 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <40A425B8.7080203@stackless.com>
Message-ID: <200405140346.i4E3kLoL006025@cosc353.cosc.canterbury.ac.nz>

Christian Tismer <tismer@stackless.com>:

> Because I prefer suitably-signed bytecode, which would
> usually only need the key verification, but no compile
> step.

But you've just suggested decompiling and recompiling the bytecode in
order to verify it. This can't possibly be faster than just compiling
it.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From tismer at stackless.com  Fri May 14 11:02:37 2004
From: tismer at stackless.com (Christian Tismer)
Date: Fri May 14 11:01:23 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405140346.i4E3kLoL006025@cosc353.cosc.canterbury.ac.nz>
References: <200405140346.i4E3kLoL006025@cosc353.cosc.canterbury.ac.nz>
Message-ID: <40A4DF8D.7050708@stackless.com>

Greg Ewing wrote:

> Christian Tismer <tismer@stackless.com>:
> 
> 
>>Because I prefer suitably-signed bytecode, which would
>>usually only need the key verification, but no compile
>>step.
> 
> 
> But you've just suggested decompiling and recompiling the bytecode in
> order to verify it. This can't possibly be faster than just compiling
> it.

This is not what I said and also irrelevant, since it will only
happen once on untrusted bytecode. Afterwards it is trusted
and stored together with a keyed digest for later verification.

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From theller at python.net  Fri May 14 15:44:45 2004
From: theller at python.net (Thomas Heller)
Date: Fri May 14 15:44:52 2004
Subject: [Python-Dev] SF CVS question
Message-ID: <ekpm7pzm.fsf@python.net>

When doing 'cvs -n update', I get a warning like this:
  cvs server: WARNING: global `-l? option ignored.

Does anyone else get the same? Is there something wrong with my
configuration?  It also occurrs on other SF cvs repositories, not only
with python.

Thomas


From tim.one at comcast.net  Fri May 14 15:52:04 2004
From: tim.one at comcast.net (Tim Peters)
Date: Fri May 14 15:52:04 2004
Subject: [Python-Dev] SF CVS question
In-Reply-To: <ekpm7pzm.fsf@python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEPAKPAB.tim.one@comcast.net>

[Thomas Heller]
> When doing 'cvs -n update', I get a warning like this:
>   cvs server: WARNING: global `-l? option ignored.
>
> Does anyone else get the same? Is there something wrong with my
> configuration?  It also occurrs on other SF cvs repositories, not only
> with python.

LOL!  I never tried cvs -n before.  Here was my result:

C:\Code\python>cvs -n up
CVS.EXE update: inflate: unknown compression method
CVS.EXE [update aborted]: reading from server: Input/output error
Write failed flushing stdout buffer.
write stdout: Broken pipe

A regular "cvs up" (no -n) still works fine.


From michel at dialnetwork.com  Fri May 14 13:39:44 2004
From: michel at dialnetwork.com (Michel Pelletier)
Date: Fri May 14 15:55:08 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405131204.36176.michel@dialnetwork.com>
References: <E1BOCkZ-00065T-2k@mail.python.org>
	<200405131204.36176.michel@dialnetwork.com>
Message-ID: <200405141239.44886.michel@dialnetwork.com>

On Thursday 13 May 2004 12:04, Michel Pelletier wrote:

> But just looking at some code here, is RETURN_VALUE ever inside a
> SETUP_EXCEPT block?  The end of the SETUP_EXCEPT block apears to
> ABSOLUTE_JUMP past all the exception handlers to finally, and then to the
> RETURN_VALUE, which doesn't seem to fall under the protection of a
> try/except.

Never mind, I proved my self wrong.  RETURN_VALUE can often happen inside 
SETUP_EXCEPT and _FINALLY.  Expect the occasionally stupid question (I know, 
there are no stupid questions....)

-Michel

From tim.one at comcast.net  Fri May 14 15:58:47 2004
From: tim.one at comcast.net (Tim Peters)
Date: Fri May 14 15:58:46 2004
Subject: [Python-Dev] SF CVS question
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEPAKPAB.tim.one@comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEPBKPAB.tim.one@comcast.net>

> LOL!  I never tried cvs -n before.  Here was my result:
>
> C:\Code\python>cvs -n up
> CVS.EXE update: inflate: unknown compression method
> CVS.EXE [update aborted]: reading from server: Input/output error

...

I should note that I have a .cvsrc file containing the line

cvs -q -z6

If I rename .cvsrc so that CVS can't find it, then "cvs -n up" works fine
for me (at least in  SF's Python repository).


From theller at python.net  Fri May 14 16:01:42 2004
From: theller at python.net (Thomas Heller)
Date: Fri May 14 16:01:49 2004
Subject: [Python-Dev] SF CVS question
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEPAKPAB.tim.one@comcast.net> (Tim
	Peters's message of "Fri, 14 May 2004 15:52:04 -0400")
References: <LNBBLJKPBEHFEDALKOLCEEPAKPAB.tim.one@comcast.net>
Message-ID: <8yfu7p7d.fsf@python.net>

"Tim Peters" <tim.one@comcast.net> writes:

> [Thomas Heller]
>> When doing 'cvs -n update', I get a warning like this:
>>   cvs server: WARNING: global `-l? option ignored.
>>
>> Does anyone else get the same? Is there something wrong with my
>> configuration?  It also occurrs on other SF cvs repositories, not only
>> with python.
>
> LOL!  I never tried cvs -n before.  Here was my result:
>
> C:\Code\python>cvs -n up
> CVS.EXE update: inflate: unknown compression method
> CVS.EXE [update aborted]: reading from server: Input/output error
> Write failed flushing stdout buffer.
> write stdout: Broken pipe

That looks more like the more or less transient error I get from SF from
time to time.  cvs -n up is useful to display the state of the files in
the sandbox.  And pcl-cvs (from inside xemacs) uses this when you do a
M-x cvs-examine.
>
> A regular "cvs up" (no -n) still works fine.

Yes.  Sometimes ;-)

Thomas


From gregory.lielens at fft.be  Fri May 14 14:52:40 2004
From: gregory.lielens at fft.be (Gregory Lielens)
Date: Fri May 14 16:16:09 2004
Subject: [Python-Dev] modifying input behavior with
	PyOS_ReadlineFunctionPointer
Message-ID: <001701c439e4$a24034a0$1500a8c0@chikubi>

 
Hello all,

I have posted this to the general comp.lang.python list, 
But as it is really about an implementation detail, 
it is probably more a topic for the development list
I apologize in advance if this is not the correct place to post this....

We are currently developing python bindings to make a MPI parallel
application (Finite element solver) script driven. I have written a main
interpreter, able to execute scripts or interactive commands, in // ,
based on the following techniques:

*file execution: all python interpreter processes run the file using
PyRun_SimpleFile (file must be available to all processes)

*file execution using broadcast: process 0 read input file and store it
in
a file, broadcast it to all processes and the script is run using
PyRun_SimpleString (file must be available to process 0 only)

*interactive execution: All processes listen to user input using a
PyRun_InteractiveLoop. Process 0 read stdin and broadcast the line to
all
process, the other process listen to this broadcast. This is implemented
using the (very useful in this case :-) ) PyOS_ReadlineFunctionPointer ,
reassigning it to an input function which wrap the default
PyOS_ReadlineFunctionPointer (to be able to reuse nice readline
functionalities) and add the broadcasting mechanism...

All of this work perfectly, but I had to modify the python sources,
Parser/myreadline.c to be precise... Indeed, the behavior of
PyOS_Readline
is to use the user-modifiable PyOS_ReadlineFunctionPointer as input
mechanism, except when input is non-interactive in which case it fall
back to the non-user-modifiable PyOS_StdioReadline...As my processes are
non-interactive except for proc 0 (launched in background), proc 0
broadcast but the other ones do not listen. I thus had to remove this
test, so that the user-modifiable function is called in all cases.

/*Python code snipped from Parser/myreadline.c*/

  char *
PyOS_Readline(FILE *sys_stdin, FILE *sys_stdout, char *prompt) {
        char *rv;
 

        if (PyOS_ReadlineFunctionPointer == NULL) {
#ifdef __VMS
                PyOS_ReadlineFunctionPointer = vms__StdioReadline;
#else
                PyOS_ReadlineFunctionPointer = PyOS_StdioReadline;
#endif
        }
        }
        Py_BEGIN_ALLOW_THREADS
 

        /* This is needed to handle the unlikely case that the
         * interpreter is in interactive mode *and* stdin/out are not *
a
         tty.  This can happen, for example if python is run like *
this:
         python -i < test1.py
         */
        /* my modif: comment out the use of PyOS_StdioReadline for
           non-interactive input...*/
        /*if (!isatty (fileno (sys_stdin)) || !isatty
(fileno(sys_stdout)))
                rv = PyOS_StdioReadline (sys_stdin, sys_stdout, prompt);
        else*/
                rv = (*PyOS_ReadlineFunctionPointer)(sys_stdin,
                sys_stdout,
                                                     prompt);
        Py_END_ALLOW_THREADS
        return rv;
}
}
/*end of snipped code*/

This is a small modif, but annoying because we would prefer to use the
plain python distribution, and anyway I wonder if the current behavior
of
python is desirable, because basically it removes part of the
functionality of PyOS_ReadlineFunctionPointer: flexibility disappear for
non-interactive input.
In addition, this seems not so robust, it seems to me that it is not
correct for vms, where PyOS_StdioReadline will be used in
non-interactive
case while vms__StdioReadline will be used for interactive one....maybe
this is the intended behavior, but then the function naming is strange
:-)

So basically, shouldn't it be the responsibility of the code which
change
PyOS_ReadlineFunctionPointer to check if the function given is adapted
to
interactiveness of input? this seems the more flexible approach, and
probably involve only minor modif of the readline module. 
In addition, wouldn't it be nice to initialize
PyOS_ReadlineFunctionPointer 
to a default value (suitable reading function) at declaration, instead
of
defining it to NULL and let PyOS_Readline do the initialization when
needed? 
This way, user code can get back a meaningful reading function
storing the old value of PyOS_ReadlineFunctionPointer, use it to write
an
extended input function, and reassign it to
PyOS_ReadlineFunctionPointer...
This seems to me like the most flexible way to add user-tunable input
method... 
So, to all experienced python developers, having used this
PyOS_ReadlineFunctionPointer 
function or implemented this code, what do you think of these possible
slight
modifications? Or is there a better way to use it than my current
method?

Sorry for the long (and possibly strangely written, English is not my
native language :-) ) post,

Best regards,

Greg.



From vdbergh at luc.ac.be  Sat May 15 10:27:15 2004
From: vdbergh at luc.ac.be (Michel Van den Bergh)
Date: Sat May 15 10:24:59 2004
Subject: [Python-Dev] Re: Welcome to the "Python-Dev" mailing list
References: <mailman.270.1084630018.4178.python-dev@python.org>
Message-ID: <40A628C3.1050409@luc.ac.be>

Hello,

I apologize for posting so quickly on this list but I am having a problem.
I am the author of a game written in python (PyTraffic, it is on the vault)
and I am preparing a new version which will be much nicer in many ways.

It also includes some help in the form of a html file. Right now I am using
my own methods to display html files.  However I am also checking out 
the webbrowser
module.

Unfortunately this module has some obvious problems (tested on CVS 
version under Linux with mozilla)


(*) If you open an url (with the option "new window") and the browser is 
not already running
then it opens *two* windows: the home page and your url. This is 
different for example
from the default behaviour on Windows and also quite ugly.

(**) There is another related problem: if you open an url
and the browser is not already running then the code starts a new 
browser, does a timed sleep
and then issues a "-remote" command.

Using a timed sleep is very bad programming and *extremely fragile*. 
Indeed I had failures because the delay
(PROCESS_CREATION_DELAY) was not long enough. At the very least the -remote
command should be tried a couple of times.

BUT: there is no reason *at all* to issue a -remote command in this 
case. All browsers I know
accept a url as argument. So one can just start a new browser with the 
url as argument.

These problems are easy to fix but it is not clear from which version of 
webbrowser.py
to start. There are already two patches waiting (not fixing the above 
problems),
with no indication if they will ever be applied.

Any comments?

Best regards,
Michel





From mal at egenix.com  Sat May 15 10:30:15 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Sat May 15 10:30:20 2004
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Lib/email
	Parser.py, 1.20, 1.20.12.1
In-Reply-To: <1084491944.28228.854.camel@anthem.wooz.org>
References: <LNBBLJKPBEHFEDALKOLCOEIPKPAB.tim.one@comcast.net>
	<1084491944.28228.854.camel@anthem.wooz.org>
Message-ID: <40A62977.4050606@egenix.com>

Barry Warsaw wrote:
> On Thu, 2004-05-13 at 19:31, Tim Peters wrote:
> 
> 
>>Yadda yadda yadda.  At this point, the more obvious solution is to ignore
>>all that MIME crap, and just guess a good structure for the email.  The
>>spambayes Outlook addin does that all the time <wink>.
> 
> 
> Structure's for monkeyboys and pointy-bracket lovers, not for studly
> w1nd0z 1nst4ll3r d00dz and 3m41l l33t h4x0rz.

Hmm, I wonder if we should add another fun codec to the encodings
package ?! Anyone have a reference ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 15 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
EuroPython 2004, G?teborg, Sweden                           22 days left

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From paul at prescod.net  Sat May 15 13:40:33 2004
From: paul at prescod.net (Paul Prescod)
Date: Sat May 15 13:40:54 2004
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Lib/email
	Parser.py, 1.20, 1.20.12.1
In-Reply-To: <40A62977.4050606@egenix.com>
References: <LNBBLJKPBEHFEDALKOLCOEIPKPAB.tim.one@comcast.net>	<1084491944.28228.854.camel@anthem.wooz.org>
	<40A62977.4050606@egenix.com>
Message-ID: <40A65611.6090103@prescod.net>

M.-A. Lemburg wrote:

> Hmm, I wonder if we should add another fun codec to the encodings
> package ?! Anyone have a reference ?

Well Google turns these up:

http://www.bbc.co.uk/dna/h2g2/A787917
http://en.wikipedia.org/wiki/Leet
http://www.ninjalane.com/leet.aspx
http://home.cwru.edu/~jar20/l33tpaper.htm
http://www.ryanross.net/leet/

I wish I could find the URL where a guy talks to a wanna-be hacker with 
increasingly random syntax and the hacker is amazed at how l33t he is. 
"You are so l33t that I can't even understand what you are saying!"

  Paul Prescod



From jcarlson at uci.edu  Sat May 15 13:39:30 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Sat May 15 13:44:17 2004
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Lib/email
	Parser.py, 1.20, 1.20.12.1
In-Reply-To: <40A62977.4050606@egenix.com>
References: <1084491944.28228.854.camel@anthem.wooz.org>
	<40A62977.4050606@egenix.com>
Message-ID: <20040515102327.0A07.JCARLSON@uci.edu>

> >>Yadda yadda yadda.  At this point, the more obvious solution is to ignore
> >>all that MIME crap, and just guess a good structure for the email.  The
> >>spambayes Outlook addin does that all the time <wink>.
> > 
> > 
> > Structure's for monkeyboys and pointy-bracket lovers, not for studly
> > w1nd0z 1nst4ll3r d00dz and 3m41l l33t h4x0rz.
> 
> Hmm, I wonder if we should add another fun codec to the encodings
> package ?! Anyone have a reference ?

The only problem is that the mapping from English to 1337, or 1337 to
English on a per-character basis is not one-to-one:
s == s.encode('leet').decode('leet')
... would be false.

Generating the encoder would be relatively easy, but would only really produce
interesting results if, for example, i -> (l, 1, !, |, ;, :), and the
resulting mapping was taken in-context to other choices made in the
current word.

The decoder would be an exercise in frustration, assuming we don't care
about getting something exact, but something 'close enough'.  Assuming
purely English text, one would need a "possible decoding" of every word,
all of which could then be fed into a phonetic spell checker, which
would need to decide which of the possible decodings for this word was
likely the correct one. Sounds like a fun project, though as:
http://cockeyed.com/lessons/viagra/viagra.html
... has shown, there may not be a workable solution to the problem.

Pity though, it would be fun.

 - Josiah


From tjreedy at udel.edu  Sat May 15 13:56:27 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat May 15 13:56:34 2004
Subject: [Python-Dev] Re: Welcome to the "Python-Dev" mailing list
References: <mailman.270.1084630018.4178.python-dev@python.org>
	<40A628C3.1050409@luc.ac.be>
Message-ID: <c85lkc$96s$1@sea.gmane.org>


"Michel Van den Bergh" <vdbergh@luc.ac.be> wrote in message
news:40A628C3.1050409@luc.ac.be...
> the webbrowser
> module.
>
> Unfortunately this module has some obvious problems (tested on CVS
> version under Linux with mozilla)
...
> Any comments?

Submit a bug or patch item to SF.  Add comments to other patches re desire
for coordination of fixes.

TJR




From fdrake at acm.org  Sat May 15 14:01:00 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sat May 15 14:01:12 2004
Subject: [Python-Dev] Re: Welcome to the "Python-Dev" mailing list
In-Reply-To: <c85lkc$96s$1@sea.gmane.org>
References: <mailman.270.1084630018.4178.python-dev@python.org>
	<40A628C3.1050409@luc.ac.be> <c85lkc$96s$1@sea.gmane.org>
Message-ID: <200405151401.00897.fdrake@acm.org>

On Saturday 15 May 2004 01:56 pm, Terry Reedy wrote:
 > Submit a bug or patch item to SF.  Add comments to other patches re desire
 > for coordination of fixes.

Also make sure they're all assigned to the same person.  I think the others 
are currently assigned to me (since I'm to blame for the original version of 
the module).  Feel free to assign the new tracker item to me if the others 
are indeed assigned to me.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From tim.one at comcast.net  Sat May 15 14:06:37 2004
From: tim.one at comcast.net (Tim Peters)
Date: Sat May 15 14:06:47 2004
Subject: [Python-Dev] Berkeley test errors in 2.3.4c1
Message-ID: <E1BP3Yb-0000jT-7e@mail.python.org>

I tried 2.3.4rc1 on Win XP SP1, and got 30 errors in test_bsddb3.  Opened a
bug report:

    http://www.python.org/sf/954530

How's test_bsddb3 working on other 2.3.4c1 installations (I encourage adding
info to the bug report instead of via email)?




From greg at electricrain.com  Sat May 15 17:41:11 2004
From: greg at electricrain.com (Gregory P. Smith)
Date: Sat May 15 17:41:16 2004
Subject: [Python-Dev] Berkeley test errors in 2.3.4c1
In-Reply-To: <E1BP3Yb-0000jT-7e@mail.python.org>
References: <E1BP3Yb-0000jT-7e@mail.python.org>
Message-ID: <20040515214111.GB28971@zot.electricrain.com>

On Sat, May 15, 2004 at 02:06:37PM -0400, Tim Peters wrote:
> I tried 2.3.4rc1 on Win XP SP1, and got 30 errors in test_bsddb3.  Opened a
> bug report:
> 
>     http://www.python.org/sf/954530
> 
> How's test_bsddb3 working on other 2.3.4c1 installations (I encourage adding
> info to the bug report instead of via email)?

fixed.  it was broken test case due to a different default behaviour
on 2.3 than 2.4.  sorry about that.  i took the easy route and just
cauterized the broken test rather than writing a proper one for the
2.3 branch.


From michel.vandenbergh at luc.ac.be  Sat May 15 18:11:13 2004
From: michel.vandenbergh at luc.ac.be (michel.vandenbergh@luc.ac.be)
Date: Sat May 15 18:45:01 2004
Subject: [Python-Dev] Re: Welcome to the "Python-Dev" mailing list
In-Reply-To: <200405151401.00897.fdrake@acm.org>
References: <mailman.270.1084630018.4178.python-dev@python.org>
	<40A628C3.1050409@luc.ac.be> <c85lkc$96s$1@sea.gmane.org>
	<200405151401.00897.fdrake@acm.org>
Message-ID: <1084659073.40a6958115068@webmail.luc.ac.be>

Ok, I submitted a patch.

Best regard,
Michel


Quoting "Fred L. Drake, Jr." <fdrake@acm.org>:

> On Saturday 15 May 2004 01:56 pm, Terry Reedy wrote:
>  > Submit a bug or patch item to SF.  Add comments to other patches re
> desire
>  > for coordination of fixes.
> 
> Also make sure they're all assigned to the same person.  I think the others
> 
> are currently assigned to me (since I'm to blame for the original version of
> 
> the module).  Feel free to assign the new tracker item to me if the others 
> are indeed assigned to me.
> 
> 
>   -Fred
> 
> -- 
> Fred L. Drake, Jr.  <fdrake at acm.org>
> PythonLabs at Zope Corporation
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/vdbergh%40luc.ac.be
> 




From tim.one at comcast.net  Sat May 15 20:25:01 2004
From: tim.one at comcast.net (Tim Peters)
Date: Sat May 15 20:25:01 2004
Subject: [Python-Dev] Berkeley test errors in 2.3.4c1
In-Reply-To: <20040515214111.GB28971@zot.electricrain.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEFDLAAB.tim.one@comcast.net>

[Gregory P. Smith, on
 http://www.python.org/sf/954530
]

> fixed.  it was broken test case due to a different default behaviour
> on 2.3 than 2.4.  sorry about that.  i took the easy route and just
> cauterized the broken test rather than writing a proper one for the
> 2.3 branch.

Thanks!  I ran the original on WinXP from the released 2.3.4c1, but am not
able to build my own 2.3 Python on that box (well, not without more setup
time than I can give to it now).  Going back to a Win98SE box and building
from current release23-maint CVS, test_bsddb3 failed with 4 instances of a
different error (see attached).  But, IIRC, the test has always failed on
Win98SE in this way.  All the errors I reported above went away.
-------------- next part --------------
======================================================================
ERROR: test01_basics (bsddb.test.test_dbshelve.EnvBTreeShelveTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\CODE\23\lib\bsddb\test\test_dbshelve.py", line 75, in test01_basics
    self.do_open()
  File "C:\CODE\23\lib\bsddb\test\test_dbshelve.py", line 238, in do_open
    self.env.open(homeDir, self.envflags | db.DB_INIT_MPOOL | db.DB_CREATE)
DBAgainError: (11, 'Resource temporarily unavailable -- unable to join the environment')

======================================================================
ERROR: test01_basics (bsddb.test.test_dbshelve.EnvHashShelveTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\CODE\23\lib\bsddb\test\test_dbshelve.py", line 75, in test01_basics
    self.do_open()
  File "C:\CODE\23\lib\bsddb\test\test_dbshelve.py", line 238, in do_open
    self.env.open(homeDir, self.envflags | db.DB_INIT_MPOOL | db.DB_CREATE)
DBAgainError: (11, 'Resource temporarily unavailable -- unable to join the environment')

======================================================================
ERROR: test01_basics (bsddb.test.test_dbshelve.EnvThreadBTreeShelveTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\CODE\23\lib\bsddb\test\test_dbshelve.py", line 75, in test01_basics
    self.do_open()
  File "C:\CODE\23\lib\bsddb\test\test_dbshelve.py", line 238, in do_open
    self.env.open(homeDir, self.envflags | db.DB_INIT_MPOOL | db.DB_CREATE)
DBAgainError: (11, 'Resource temporarily unavailable -- unable to join the environment')

======================================================================
ERROR: test01_basics (bsddb.test.test_dbshelve.EnvThreadHashShelveTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\CODE\23\lib\bsddb\test\test_dbshelve.py", line 75, in test01_basics
    self.do_open()
  File "C:\CODE\23\lib\bsddb\test\test_dbshelve.py", line 238, in do_open
    self.env.open(homeDir, self.envflags | db.DB_INIT_MPOOL | db.DB_CREATE)
DBAgainError: (11, 'Resource temporarily unavailable -- unable to join the environment')

----------------------------------------------------------------------
Ran 216 tests in 264.300s

FAILED (errors=4)
From greg at electricrain.com  Sat May 15 20:28:13 2004
From: greg at electricrain.com (Gregory P. Smith)
Date: Sat May 15 20:28:17 2004
Subject: [Python-Dev] Berkeley test errors in 2.3.4c1
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEFDLAAB.tim.one@comcast.net>
References: <20040515214111.GB28971@zot.electricrain.com>
	<LNBBLJKPBEHFEDALKOLCAEFDLAAB.tim.one@comcast.net>
Message-ID: <20040516002813.GA10087@zot.electricrain.com>

On Sat, May 15, 2004 at 08:25:01PM -0400, Tim Peters wrote:
> [Gregory P. Smith, on
>  http://www.python.org/sf/954530
> ]
> 
> > fixed.  it was broken test case due to a different default behaviour
> > on 2.3 than 2.4.  sorry about that.  i took the easy route and just
> > cauterized the broken test rather than writing a proper one for the
> > 2.3 branch.
> 
> Thanks!  I ran the original on WinXP from the released 2.3.4c1, but am not
> able to build my own 2.3 Python on that box (well, not without more setup
> time than I can give to it now).  Going back to a Win98SE box and building
> from current release23-maint CVS, test_bsddb3 failed with 4 instances of a
> different error (see attached).  But, IIRC, the test has always failed on
> Win98SE in this way.  All the errors I reported above went away.

> ======================================================================
> ERROR: test01_basics (bsddb.test.test_dbshelve.EnvBTreeShelveTestCase)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "C:\CODE\23\lib\bsddb\test\test_dbshelve.py", line 75, in test01_basics
>     self.do_open()
>   File "C:\CODE\23\lib\bsddb\test\test_dbshelve.py", line 238, in do_open
>     self.env.open(homeDir, self.envflags | db.DB_INIT_MPOOL | db.DB_CREATE)
> DBAgainError: (11, 'Resource temporarily unavailable -- unable to join the environment')

Yeah I'd chalk that one up to Win98.

-g


From tim.one at comcast.net  Sun May 16 01:47:03 2004
From: tim.one at comcast.net (Tim Peters)
Date: Sun May 16 01:47:07 2004
Subject: [Python-Dev] Berkeley test errors in 2.3.4c1
In-Reply-To: <20040516002813.GA10087@zot.electricrain.com>
Message-ID: <mailman.2.1084686427.13604.python-dev@python.org>

[Gregory P. Smith, on test_bsddb3 Win98SE failures in test01_basics]
> Yeah I'd chalk that one up to Win98.

Me too <wink>.

Back over on WinXP and *current* (HEAD) CVS, there's one failure, which I
believe is real:

ERROR: test04_n_flag (bsddb.test.test_compat.CompatibilityTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Code\python\lib\bsddb\test\test_compat.py", line 86, in
test04_n_flag
    f = hashopen(self.filename, 'n')
  File "C:\Code\python\lib\bsddb\__init__.py", line 293, in hashopen
    d.open(file, db.DB_HASH, flags, mode)
DBInvalidArgError: (22, 'Invalid argument -- DB_TRUNCATE illegal with
locking specified')


I suspect Sleepycat screwed us there, changing the rules in midstream.
Someone on c.l.py appeared to track down the same thing here, but in an app
instead of in our test suite:

    http://mail.python.org/pipermail/python-list/2004-May/220168.html

    The change log of Berkeley DB 4.2.52 says "9. Fix a bug to now
    disallow DB_TRUNCATE on opens in locking environments, since we
    cannot prevent race conditions ..."



From mal at egenix.com  Sun May 16 08:31:58 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Sun May 16 08:32:06 2004
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Lib/email
	Parser.py, 1.20, 1.20.12.1
In-Reply-To: <40A65611.6090103@prescod.net>
References: <LNBBLJKPBEHFEDALKOLCOEIPKPAB.tim.one@comcast.net>	<1084491944.28228.854.camel@anthem.wooz.org>	<40A62977.4050606@egenix.com>
	<40A65611.6090103@prescod.net>
Message-ID: <40A75F3E.4010708@egenix.com>

Paul Prescod wrote:
> M.-A. Lemburg wrote:
> 
>> Hmm, I wonder if we should add another fun codec to the encodings
>> package ?! Anyone have a reference ?
> 
> 
> Well Google turns these up:
> 
> http://www.bbc.co.uk/dna/h2g2/A787917
> http://en.wikipedia.org/wiki/Leet
> http://www.ninjalane.com/leet.aspx
> http://home.cwru.edu/~jar20/l33tpaper.htm
> http://www.ryanross.net/leet/

Wow, nice collection of links, thanks.

> I wish I could find the URL where a guy talks to a wanna-be hacker with 
> increasingly random syntax and the hacker is amazed at how l33t he is. 
> "You are so l33t that I can't even understand what you are saying!"

... I wonder how these guys deal with the problem of
expressing numbers in their small little language. I suppose
they use the l33t version of roman literals ;-)

(c) |\/||\/|1\/, |?`/+|-|()|\| 5()F7\/\/4R3 F()|_||\||)471()|\|

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 16 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
EuroPython 2004, G?teborg, Sweden                           21 days left

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From tdelaney at avaya.com  Sun May 16 20:11:23 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Sun May 16 20:11:33 2004
Subject: [Python-Dev] .pth files in current or script directory
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE0179A652@au3010avexu1.global.avaya.com>

Guido van Rossum wrote:

> Can't they manipulate sys.path *inside* the code in question?  That's
> invariably the solution I end up using -- I never play with
> PYTHONPATH any more.

I do this too, but I always feel that I shouldn't have to, especially
when I'm not using `sys` for anything else. It also messes up my desire
to keep all the imports together - the `sys` magic needs to go first.

Jeff Epler wrote:

> I'm -NFS against reading .pth files from the current directory.
> When a directory is large and is on a remote filesystem, enumerating
> it at startup can really kill time -- heck, just putting an NFS
> directory with a handful of entries in PYTHONPATH can slow Python's
> startup time considerably, now imagine that you "cd
> /nfs/directory_with_1000_files" and "time python -c 'pass'" .. blah!

I don't use NFS so I've no experience with the performance problems
involved. Of course, Windows shares would have similar issues.

Hmm - would this objection go away if a specific filename were mandated
for these .pth files - e.g. pythonpath.pth? 

Tim Delaney

From jepler at unpythonic.net  Sun May 16 21:40:10 2004
From: jepler at unpythonic.net (Jeff Epler)
Date: Sun May 16 22:17:32 2004
Subject: [Python-Dev] .pth files in current or script directory
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE0179A652@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE0179A652@au3010avexu1.global.avaya.com>
Message-ID: <20040517014009.GC23803@unpythonic.net>

On Mon, May 17, 2004 at 10:11:23AM +1000, Delaney, Timothy C (Timothy) wrote:
> I don't use NFS so I've no experience with the performance problems
> involved. Of course, Windows shares would have similar issues.
> 
> Hmm - would this objection go away if a specific filename were mandated
> for these .pth files - e.g. pythonpath.pth? 

Yes, searching for a single named file is a much faster operation than
listing a directory, and would probably be a fairly small impact on
Python startup, much smaller than listing a directory with many files.
It's one round-trip to the server vs many round trips.

.. but that makes the feature somewhat different than .pth files in
site-packages (any name vs a single name), which may be a bad thing too.

Jeff

From tdelaney at avaya.com  Sun May 16 21:45:16 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Sun May 16 22:17:43 2004
Subject: [Python-Dev] .pth files in current or script directory
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE0179A6BB@au3010avexu1.global.avaya.com>

Jeff Epler wrote:

> On Mon, May 17, 2004 at 10:11:23AM +1000, Delaney, Timothy C
> (Timothy) wrote: 
>> I don't use NFS so I've no experience with the performance problems
>> involved. Of course, Windows shares would have similar issues.
>> 
>> Hmm - would this objection go away if a specific filename were
>> mandated for these .pth files - e.g. pythonpath.pth?
> 
> Yes, searching for a single named file is a much faster operation than
> listing a directory, and would probably be a fairly small impact on
> Python startup, much smaller than listing a directory with many files.
> It's one round-trip to the server vs many round trips.
> 
> .. but that makes the feature somewhat different than .pth files in
> site-packages (any name vs a single name), which may be a bad thing
> too. 

Yep - I thought about this. OTOH, the use case for .pth files for
site-packages is multiple libraries packaged separately. The use case
for this feature is a single code base i.e. there would be no need for
multiple .pth files since everything is under control of the developer.

Please don't get the idea I'm all gung-ho about this - I'm just trying
to see if there's enough reason to write a PEP for it (either to be
accepted or rejected).

Tim Delaney

From greg at electricrain.com  Sun May 16 15:22:56 2004
From: greg at electricrain.com (Gregory P. Smith)
Date: Sun May 16 23:18:25 2004
Subject: [Python-Dev] Berkeley test errors in 2.3.4c1
Message-ID: <20040516192256.GA20831@zot.electricrain.com>

On Sun, May 16, 2004 at 01:47:03AM -0400, Tim Peters wrote:
> [Gregory P. Smith, on test_bsddb3 Win98SE failures in test01_basics]
> > Yeah I'd chalk that one up to Win98.
> 
> Me too <wink>.
> 
> Back over on WinXP and *current* (HEAD) CVS, there's one failure, which I
> believe is real:
> 
> ERROR: test04_n_flag (bsddb.test.test_compat.CompatibilityTestCase)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "C:\Code\python\lib\bsddb\test\test_compat.py", line 86, in
> test04_n_flag
>     f = hashopen(self.filename, 'n')
>   File "C:\Code\python\lib\bsddb\__init__.py", line 293, in hashopen
>     d.open(file, db.DB_HASH, flags, mode)
> DBInvalidArgError: (22, 'Invalid argument -- DB_TRUNCATE illegal with
> locking specified')
> 
> 
> I suspect Sleepycat screwed us there, changing the rules in midstream.
> Someone on c.l.py appeared to track down the same thing here, but in an app
> instead of in our test suite:
> 
>     http://mail.python.org/pipermail/python-list/2004-May/220168.html
> 
>     The change log of Berkeley DB 4.2.52 says "9. Fix a bug to now
>     disallow DB_TRUNCATE on opens in locking environments, since we
>     cannot prevent race conditions ..."

heh.  nice.  yeah i saw that bugreport for pybsddb but hadn't looked
into it yet.  Binary packages of python 2.3.4 should be compiled using
BerkeleyDB 4.1 not 4.2 (as all previous python 2.3s shipped with 4.1).
Anyone who compiles it on their own against 4.2 will have to live with
this error in the old bsddb compatibility interface.

Taking a closer look at this to see if there is a workaround is on my
TODO list.

-g

From guido at python.org  Sun May 16 23:23:39 2004
From: guido at python.org (Guido van Rossum)
Date: Sun May 16 23:23:50 2004
Subject: [Python-Dev] .pth files in current or script directory
In-Reply-To: Your message of "Mon, 17 May 2004 10:11:23 +1000."
	<338366A6D2E2CA4C9DAEAE652E12A1DE0179A652@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE0179A652@au3010avexu1.global.avaya.com>
Message-ID: <200405170323.i4H3NdH02255@guido.python.org>

> > Can't they manipulate sys.path *inside* the code in question?  That's
> > invariably the solution I end up using -- I never play with
> > PYTHONPATH any more.
> 
> I do this too, but I always feel that I shouldn't have to, especially
> when I'm not using `sys` for anything else. It also messes up my desire
> to keep all the imports together - the `sys` magic needs to go first.

Yet, it beats depending on environment variables, so I always end up
doing it this way.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From eric at enthought.com  Mon May 17 00:34:59 2004
From: eric at enthought.com (eric jones)
Date: Mon May 17 00:35:09 2004
Subject: [Python-Dev] ANN: Job Openings at Enthought
Message-ID: <40A840F3.2050608@enthought.com>

Hello,

This is a bit of a status report I guess for SciPy (www.scipy.org) and
Enthought (www.enthought.com), but, really, it is a long winded job
posting. 

Among other things, Enthought develops scientific software
for clients. Python is central to to our strategy in this arena.  We have
long supported SciPy and believe strongly in Open Source Software. 
The purpose is to give people a feeling about what we do, the technical
challenges we face, and the skills needed to meet them.

SciPy
-----
There has been a lot of work on SciPy lately by the Travis Oliphant, Travis
Vaught, Pearu Peterson, Joe Cooper, and on the website by Jon-Eric 
Steinbomer
and Janet Swisher.  The site is now upgraded to Plone and we have a 
SciPy 0.3
package out.  If you're interested in seeing the activity level at 
scipy.org,
please visit.

http://www.scipy.org/map?rmurl=http%3A//scipy.net/scipyusage/

It looks like we're averaging about a 100 downloads a day if you add up all
source and binary distributions.  When SciPy 0.1 was released a couple a 
years
ago, I think we averaged about 10.  Its extremely difficult to 
extrapolate a user
base from such information, the obvious growth is good news.

Other Stuff
-----------
In addition to SciPy we have multiple projects going on right now that 
either
already plug into or will be plugins to a Python-based scientific 
application
framework called Envisage (sorta like an IDE for science apps) that we're
working on.  The idea is:

(1) To lower the bar as far as work required to get a GUI put
   on the front end of a scientific algorithm.
(2) Develop scientific plugins that can play well together so
   that, for instance, an electromagnetics simulator built by one
   person can be used in conjunction with an optimization plugin
   built by another person and results can be visualized with a
   3D plugin.

This is a hard, long path that requires much work and though.  We have
started the process and actually have built an app on a very early 
version of
the Envisage framework.  The app works great, but it's usage of the 
framework is a
little mixed at this point.  There were a lot of compromises we ended up 
making
on the framework side in order to ship on time.

Also, I am not sure whether "easy-to-program" and "flexible architecture for
compatible plugins" are not mutually exclusive.  I always have in my 
mind that I
want a smart scientist to be able to build the GUI in a short amount of time
after the algorithm (which is the "hard part") is finished.  I still 
harbor this
wish, but the lessons I've had over the last couple of years suggest 
that the GUI is
actually the most expensive part of development for large apps.  This is 
partially
due to the fact that commercial apps are rarely built around "research" 
algorithms
-- meaning that the algorithm code usually already exists in some form.  
Still,
UIs take tons of time.  Testing them is hard.  Flexibility requires more
complexity (factories, adapters, etc.) than seems necessary at first.  
Further,
building *good* UI's is really hard.  Scientist rarely have the patience or
perspective for it -- and yet it is very often difficult to build a good 
UI for
science apps without a scientist's understanding of the underlying problem.

We have a number of tools along with SciPy that we're building that are 
pieces
of this puzzle.

  1. Traits -- They at the heart of everything we do.  They provide
     some explicit typing, default GUI's, an observer event model for
     anything that uses them.  They can also require some getting used
     to.
  2. Kiva/Enable -- This is the generic drawing system for Python.
  3. Chaco -- Interactive 2D Plotting
  4. PyFace -- MVC layer on top of wxPython.  Supports trees, menus,
     and a few other things right now.  This will grow.
  5. Envisage -- Plugin based GUI framework for scientific applications.
     Beyond the basic GUI that comes from Envisage, we already have a couple
     plugins for profiling code using hotshot and also searching for memory
     leaks.  David Morrill wrote these and they are called gotcha and 
enroll.
     Martin Chilvers wrote a simple scintilla based plugin for editing 
scripts,
     and a PyCrust shell is available by default in the app (not
     a plugin at the moment).
    
     We would love to see a number of other plugins:
      a. Debugger.
      b. IPython-like PyCrust going.
      c. A more full featured script editing plugin.
         (there is a huge list of features here).
      d. Mayavi
      e. etc.

These are all used actively in commercial products that we deliver.  
However,
they are also in various stages of development and will continue to evolve
and grow.  Some are still pretty green.  Portions of these are openly 
available
now.  All five of the listed tool sets will be open at some point this 
summer
when they are cleaned up a bit.

Jobs
----
All of this leads to the fact that we are hiring and looking for smart,
pleasant, communicative people with integrity that are excited about 
building
this sort of stuff.  There is a lot of software architecture to be done that
needs talented designers.  There is UI design/development that needs 
scientists
that care about UIs or UI developers/designers that are really quick at 
grasping
scientific problems.  We also need really strong scientists/scientific
developers to do some of the backend coding for commercial products and also
SciPy development.  If you have a background in geophysics, electromagnetics
(especially multi-pole methods) that is a big plus. For this, a PhD is 
helpful, but
not required.  Parallel computing skills wouldn't hurt.  3D 
visualization a la
VTK/Python is a major need.  So is knowledge about 2D rendering to help 
finish
out Kiva and the Enable toolset. Strong Python skills, Python extension 
writing
knowledge and strong C/C++ skills are a big benefit.  Design/development 
with or
of Java NetBeans or Eclipse-like architecture is great -- or any other 
solid GUI
architecture.  Dedication to writing clean/clear code meant for other 
humans
to read is a requirement.

We have 3 or 4 scientific/python positions to fill over the coming 
months, and
we're looking for candidates that have the best mix of the above 
skills.  You'll
join our existing team of scientist/engineers, computer scientists, 
technical
writers, and an HCI specialist.  We like this mix and feel it is the 
best way to
build this sort of software. If you are interested in working at Enthought,
please send us your resume at jobs@enthought.com.  If not, please send this
posting to the smartest people you know that fit some part of the above 
description
(Python experience not explicitly required).  Salaries are competitive.  
Candidates must
be willing to relocate to Austin, Tx.
     
thanks,
eric jones

PS: There are additional positions listed at 
http://www.enthought.com/careers.htm
for those interested in business application development (not 
necessarily with Python).



From doko at cs.tu-berlin.de  Mon May 17 02:03:19 2004
From: doko at cs.tu-berlin.de (Matthias Klose)
Date: Mon May 17 02:03:29 2004
Subject: [Python-Dev] windist.exe status and license
In-Reply-To: <1084541142.696332.6706.nullmailer@calvin.fayauffre.org>
References: <1084541142.696332.6706.nullmailer@calvin.fayauffre.org>
Message-ID: <16552.21927.581799.337029@gargle.gargle.HOWL>

[CC to python-dev]

The windist.exe version shipped with current python releasese is
outdated and does not have any license information about windist
included.

Alexandre Fayolle writes:
> Package: python2.3-dev
> Version: 2.3.3-7
> Severity: normal
> 
> Hi,
> 
> I noticed the file /usr/lib/python2.3/distutils/command/wininst.exe is
> missing from my currently installed version of the python2.3-dev
> package. I'm pretty sure I used to be able to create windows installers
> for the modules I develop from my Debian machine. 
> 
> Is this removal caused by an omission, or is there some licensing issue ? 
> I could not find anything on that topic in the changelog, but maybe I
> didn't look hard enough. 

I cannot find anything about the licensing. The executable only shows

$Info: This file is packed with the UPX executable packer http://upx.tsx.org $
$Id: UPX 1.01 Copyright (C) 1996-2000 the UPX Team. All Rights Reserved. $

However I found http://upx.sourceforge.net/.

- Maybe the binary can be updated to the SF version?
- Add the upx license to the Lib/distutils/command directory?


[Debian only:

 The UPX license itself is discussed at
 http://lists.debian.org/debian-legal/2001/01/msg00025.html

 upx-nrv is currently in section non-free/utils, so it's better
 not to distribute it in the Debian package.
]

From skip at pobox.com  Mon May 17 09:21:32 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon May 17 09:21:21 2004
Subject: [Python-Dev] Python 2.3.4, release candidate 1 - BUG FIX
In-Reply-To: <20040517113327.GA25004@unpythonic.net>
References: <67B3A7DA6591BE439001F2736233351202B479BA@xch-nw-28.nw.nos.boeing.com>
	<mailman.31.1084583605.4157.python-list@python.org>
	<m3brknh1m7.fsf@pc150.maths.bris.ac.uk>
	<20040517113327.GA25004@unpythonic.net>
Message-ID: <16552.48220.302165.584705@montanaro.dyndns.org>


(Rerouting to python-dev...)

    Jeff> When a tar, gzipped tar, or bzip2'd tar contains a .spec file, you're
    Jeff> supposed to be able to build it into a binary RPM with
    Jeff>     rpmbuild -ta <tarball>
    Jeff> Unfortunately, with the spec file in this tarball, I get an error:
    Jeff>     $ rpmbuild -ta Python-2.3.4c1.tar.bz2 
    Jeff>     error: %changelog not in descending chronological order

I checked in the "obvious correct fix" to the spec file on CVS HEAD.  I
can't test it though, not having convenient access to a RedHat-ish system
these days.  I'll leave it for Anthony or one of the other 2.3.4 release
folks to backport it for the 2.3.4 release since that branch is closed.
(The first date should have a year of 2004.)

Skip


From mal at egenix.com  Mon May 17 09:42:10 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon May 17 09:42:16 2004
Subject: [Python-Dev] Relative vs. absolute imports
Message-ID: <40A8C132.3090705@egenix.com>

I have a few questions about the PEP 328 which I'd like discussed:

* Why do we absolutely need to change the current scheme
   of 'local, then absolute' for name resolution ?

   In the past I have often had the case where I wanted to
   move Python software written as top-level modules into
   their own package. The current logic makes this painless;
   the proposed move to 'absolute only' introduces serious
   editing which is bound to be error-prone.

   I don't object absolute imports at all, but I like the
   possibilitiy to package'ify existing Python module/module
   sets by simply copying them into a newly created package
   directory.

   Note that Python's own name lookup behaviour is very similar
   to the existing import scheme (nested scopes left aside).

* How will the proposed 'absolute only' strategy affect
   the __import__() API ?

   This is of special concern since quite a few applications
   out there rely on __import__() having the existing semantics
   and a change would cause these applications to fail. One
   such instance is the encodings package search function,
   which relies on first looking in the package for a suitable
   codec and then scans for a top-level installation.

* Will the DeprecationWarning only be raised for uses of
   'import xyz' and 'from xyz import a' or also for __import__() ?

* How do these semantics changes affect importers that hook
   into __import__ ? (imputil.py, ihooks.py, etc.)

* What's the affect on zip imports or tools like py2exe,
   McMillan's installer, etc. ?

BTW, is any of this already implemented to play with ?

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 17 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
EuroPython 2004, G?teborg, Sweden                           20 days left

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From pje at telecommunity.com  Mon May 17 10:20:37 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon May 17 10:19:24 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: <40A8C132.3090705@egenix.com>
Message-ID: <5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>

At 03:42 PM 5/17/04 +0200, M.-A. Lemburg wrote:
>I have a few questions about the PEP 328 which I'd like discussed:
>
>* Why do we absolutely need to change the current scheme
>   of 'local, then absolute' for name resolution ?

Because there's no way to override the current scheme without calling 
__import__ directly.


>* How will the proposed 'absolute only' strategy affect
>   the __import__() API ?

It doesn't change the API, only what arguments it gets called 
with.  Passing a different globals dictionary is all that's required to 
effect an absolute import.


>* Will the DeprecationWarning only be raised for uses of
>   'import xyz' and 'from xyz import a' or also for __import__() ?

It should not be for __import__().


>* How do these semantics changes affect importers that hook
>   into __import__ ? (imputil.py, ihooks.py, etc.)
>
>* What's the affect on zip imports or tools like py2exe,
>   McMillan's installer, etc. ?

Again, since the __import__ API isn't changing, it shouldn't affect any of 
this.


From bh at intevation.de  Mon May 17 10:26:06 2004
From: bh at intevation.de (Bernhard Herzog)
Date: Mon May 17 10:26:13 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: <5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	(Phillip J. Eby's message of "Mon, 17 May 2004 10:20:37 -0400")
References: <5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
Message-ID: <s9zfz9zi0zl.fsf@salmakis.intevation.de>

"Phillip J. Eby" <pje@telecommunity.com> writes:

> At 03:42 PM 5/17/04 +0200, M.-A. Lemburg wrote:
>>I have a few questions about the PEP 328 which I'd like discussed:
>>
>>* Why do we absolutely need to change the current scheme
>>   of 'local, then absolute' for name resolution ?
>
> Because there's no way to override the current scheme without calling
> __import__ directly.

Why does it have to be fixed in an incompatible way?

   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Skencil                                http://sketch.sourceforge.net/
Thuban                                  http://thuban.intevation.org/

From guido at python.org  Mon May 17 10:52:30 2004
From: guido at python.org (Guido van Rossum)
Date: Mon May 17 10:53:09 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: Your message of "Mon, 17 May 2004 16:26:06 +0200."
	<s9zfz9zi0zl.fsf@salmakis.intevation.de> 
References: <5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>  
	<s9zfz9zi0zl.fsf@salmakis.intevation.de> 
Message-ID: <200405171452.i4HEqVp03065@guido.python.org>

> Why does it have to be fixed in an incompatible way?

Because in the current situation the meaning of an import is
ambiguous.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From mal at egenix.com  Mon May 17 10:55:30 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon May 17 10:55:58 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: <5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
References: <5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
Message-ID: <40A8D262.7060505@egenix.com>

Phillip J. Eby wrote:
> At 03:42 PM 5/17/04 +0200, M.-A. Lemburg wrote:
> 
>> I have a few questions about the PEP 328 which I'd like discussed:
>>
>> * Why do we absolutely need to change the current scheme
>>   of 'local, then absolute' for name resolution ?
> 
> Because there's no way to override the current scheme without calling 
> __import__ directly.

Are you saying that you are not going to change the default __import__()
implementation, only the way it is called ? (I wonder how you'll
enforce the 'absolute only' strategy then)

>> * How will the proposed 'absolute only' strategy affect
>>   the __import__() API ?
> 
> It doesn't change the API, only what arguments it gets called with.  
> Passing a different globals dictionary is all that's required to effect 
> an absolute import.
> 
>> * Will the DeprecationWarning only be raised for uses of
>>   'import xyz' and 'from xyz import a' or also for __import__() ?
> 
> It should not be for __import__().
> 
> 
>> * How do these semantics changes affect importers that hook
>>   into __import__ ? (imputil.py, ihooks.py, etc.)
>>
>> * What's the affect on zip imports or tools like py2exe,
>>   McMillan's installer, etc. ?
> 
> 
> Again, since the __import__ API isn't changing, it shouldn't affect any 
> of this.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 17 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
EuroPython 2004, G?teborg, Sweden                           20 days left

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From pje at telecommunity.com  Mon May 17 11:04:15 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon May 17 11:03:04 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: <40A8D262.7060505@egenix.com>
References: <5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040517110205.029c9c20@mail.telecommunity.com>

At 04:55 PM 5/17/04 +0200, M.-A. Lemburg wrote:
>Phillip J. Eby wrote:
>>At 03:42 PM 5/17/04 +0200, M.-A. Lemburg wrote:
>>
>>>I have a few questions about the PEP 328 which I'd like discussed:
>>>
>>>* Why do we absolutely need to change the current scheme
>>>   of 'local, then absolute' for name resolution ?
>>Because there's no way to override the current scheme without calling 
>>__import__ directly.
>
>Are you saying that you are not going to change the default __import__()
>implementation, only the way it is called ? (I wonder how you'll
>enforce the 'absolute only' strategy then)

Huh?  You lost me.  What 'absolute only'?  There's "from .x import y" 
still, and that's relative.

As far as I know, nothing about __import__ is changing, only the spelling 
of import statements.


From edcjones at erols.com  Mon May 17 11:00:11 2004
From: edcjones at erols.com (Edward C. Jones)
Date: Mon May 17 11:06:58 2004
Subject: [Python-Dev] __str__ does not default to __repr__ if a base class
	has __str__
Message-ID: <40A8D37B.7030807@erols.com>

# In the code below, "a.__str__()" does not default to "a.__repr__() as
# might be expected. It finds the "__str__" in the base class X. The
# Reference Manual, section 3.3.1, says:
#    If a class defines __repr__() but not __str__(), then __repr__() is
#    also used when an 'informal' string representation of instances of
#    that class is required.
# If the reader is in lawyer mode, the documentation is correct.
# Is the current behavior what was intended? What should the behavior be?

class X(object):
    def __str__(self):
        return 'str for X'

    def __repr__(self):
        return 'repr for X'

class A(X):
    def __repr__(self):
        return 'repr for A'

#    __str__ = __repr__

x = X()
a = A()
print x.__str__()
print x.__repr__()
print a.__str__()
print a.__repr__()




From pedronis at bluewin.ch  Mon May 17 11:14:04 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Mon May 17 11:14:14 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: <40A8D262.7060505@egenix.com>
References: <5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
Message-ID: <5.2.1.1.0.20040517171026.02ff4198@pop.bluewin.ch>

At 16:55 17.05.2004 +0200, M.-A. Lemburg wrote:
>Phillip J. Eby wrote:
>>At 03:42 PM 5/17/04 +0200, M.-A. Lemburg wrote:
>>
>>>I have a few questions about the PEP 328 which I'd like discussed:
>>>
>>>* Why do we absolutely need to change the current scheme
>>>   of 'local, then absolute' for name resolution ?
>>Because there's no way to override the current scheme without calling 
>>__import__ directly.
>
>Are you saying that you are not going to change the default __import__()
>implementation, only the way it is called ? (I wonder how you'll
>enforce the 'absolute only' strategy then)

if you pass globals = {} you get absolute imports from __import__,

It is probably true that in the long run, given the new semantics, moving 
responsabilities
away from/changing __import__ contract could make sense.


From mal at egenix.com  Mon May 17 11:15:24 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon May 17 11:15:30 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: <5.1.1.6.0.20040517110205.029c9c20@mail.telecommunity.com>
References: <5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.1.1.6.0.20040517110205.029c9c20@mail.telecommunity.com>
Message-ID: <40A8D70C.5060309@egenix.com>

Phillip J. Eby wrote:
>>>> I have a few questions about the PEP 328 which I'd like discussed:
>>>>
>>>> * Why do we absolutely need to change the current scheme
>>>>   of 'local, then absolute' for name resolution ?
>>>
>>> Because there's no way to override the current scheme without calling 
>>> __import__ directly.
>>
>>
>> Are you saying that you are not going to change the default __import__()
>> implementation, only the way it is called ? (I wonder how you'll
>> enforce the 'absolute only' strategy then)
> 
> Huh?  You lost me.  What 'absolute only'?  There's "from .x import y" 
> still, and that's relative.
> 
> As far as I know, nothing about __import__ is changing, only the 
> spelling of import statements.

If "import os" is supposed to map to an absolute import
you will have to change the semantics of __import__ because
simply passing "os" to that API is going to implement the
standard 'local, then absolute' import scheme.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 17 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
EuroPython 2004, G?teborg, Sweden                           20 days left

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From pje at telecommunity.com  Mon May 17 11:38:40 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon May 17 11:37:31 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: <40A8D70C.5060309@egenix.com>
References: <5.1.1.6.0.20040517110205.029c9c20@mail.telecommunity.com>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.1.1.6.0.20040517110205.029c9c20@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040517113704.02d329b0@mail.telecommunity.com>

At 05:15 PM 5/17/04 +0200, M.-A. Lemburg wrote:
>Phillip J. Eby wrote:
>>>>>I have a few questions about the PEP 328 which I'd like discussed:
>>>>>
>>>>>* Why do we absolutely need to change the current scheme
>>>>>   of 'local, then absolute' for name resolution ?
>>>>
>>>>Because there's no way to override the current scheme without calling 
>>>>__import__ directly.
>>>
>>>
>>>Are you saying that you are not going to change the default __import__()
>>>implementation, only the way it is called ? (I wonder how you'll
>>>enforce the 'absolute only' strategy then)
>>Huh?  You lost me.  What 'absolute only'?  There's "from .x import y" 
>>still, and that's relative.
>>As far as I know, nothing about __import__ is changing, only the spelling 
>>of import statements.
>
>If "import os" is supposed to map to an absolute import
>you will have to change the semantics of __import__ because
>simply passing "os" to that API is going to implement the
>standard 'local, then absolute' import scheme.

As I believe I mentioned before, the relevant import opcodes will simply 
pass a different set of parameters to __import__.  Supplying a different 
"globals" dictionary suffices to implement an absolute import.


From mal at egenix.com  Mon May 17 11:37:31 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon May 17 11:37:41 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: <5.2.1.1.0.20040517171026.02ff4198@pop.bluewin.ch>
References: <5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.2.1.1.0.20040517171026.02ff4198@pop.bluewin.ch>
Message-ID: <40A8DC3B.1070107@egenix.com>

Samuele Pedroni wrote:
> At 16:55 17.05.2004 +0200, M.-A. Lemburg wrote:
> 
>> Phillip J. Eby wrote:
>>
>>> At 03:42 PM 5/17/04 +0200, M.-A. Lemburg wrote:
>>>
>>>> I have a few questions about the PEP 328 which I'd like discussed:
>>>>
>>>> * Why do we absolutely need to change the current scheme
>>>>   of 'local, then absolute' for name resolution ?
>>>
>>> Because there's no way to override the current scheme without calling 
>>> __import__ directly.
>>
>>
>> Are you saying that you are not going to change the default __import__()
>> implementation, only the way it is called ? (I wonder how you'll
>> enforce the 'absolute only' strategy then)
> 
> if you pass globals = {} you get absolute imports from __import__,

Ahh, a hidden feature :-) I didn't know that one yet.

> It is probably true that in the long run, given the new semantics, 
> moving responsabilities away from/changing __import__ contract 
 > could make sense.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 17 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
EuroPython 2004, G?teborg, Sweden                           20 days left

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From bh at intevation.de  Mon May 17 11:39:42 2004
From: bh at intevation.de (Bernhard Herzog)
Date: Mon May 17 11:39:49 2004
Subject: [Python-Dev] Relative vs. absolute imports
References: <5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<s9zfz9zi0zl.fsf@salmakis.intevation.de>
	<200405171452.i4HEqVp03065@guido.python.org>
Message-ID: <s9zsmdzgj0h.fsf@salmakis.intevation.de>

Guido van Rossum <guido@python.org> writes:

>> Why does it have to be fixed in an incompatible way?
>
> Because in the current situation the meaning of an import is
> ambiguous.

What exactly is ambiguous about it?  There are clear rules how the
lookup is performed and determining whether the import refers to another
module in the same package or a global module is just a simple directory
listing in most cases.

The only real problem with the current scheme is that sibling modules
can shadow global ones.  While that should be fixed it hardly justifies
an incompatible change, as it doesn't happen very often, and it would be
easy to fix without breaking compatibility.

   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Skencil                                http://sketch.sourceforge.net/
Thuban                                  http://thuban.intevation.org/

From pje at telecommunity.com  Mon May 17 11:50:22 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon May 17 11:49:13 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: <40A8DC3B.1070107@egenix.com>
References: <5.2.1.1.0.20040517171026.02ff4198@pop.bluewin.ch>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.2.1.1.0.20040517171026.02ff4198@pop.bluewin.ch>
Message-ID: <5.1.1.6.0.20040517114707.02716660@mail.telecommunity.com>

At 05:37 PM 5/17/04 +0200, M.-A. Lemburg wrote:
>Samuele Pedroni wrote:
>>At 16:55 17.05.2004 +0200, M.-A. Lemburg wrote:
>>>Are you saying that you are not going to change the default __import__()
>>>implementation, only the way it is called ? (I wonder how you'll
>>>enforce the 'absolute only' strategy then)
>>if you pass globals = {} you get absolute imports from __import__,
>
>Ahh, a hidden feature :-) I didn't know that one yet.

More of an emergent property.  Keep in mind that the current relative 
mechanism is always relative *to* something, and that "something" is 
defined by the globals['__name__'], if any.  If there is no '__name__', 
then the import has to be absolute.  For example, you would expect this:

     exec "import foo" in {},{}

to be an absolute import, would you not?


From tjreedy at udel.edu  Mon May 17 11:56:08 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon May 17 11:56:16 2004
Subject: [Python-Dev] Re: __str__ does not default to __repr__ if a base
	classhas __str__
References: <40A8D37B.7030807@erols.com>
Message-ID: <c8anap$2o2$1@sea.gmane.org>


"Edward C. Jones" <edcjones@erols.com> wrote in message
news:40A8D37B.7030807@erols.com...
> # In the code below, "a.__str__()" does not default to "a.__repr__() as
> # might be expected. It finds the "__str__" in the base class X. The
> # Reference Manual, section 3.3.1, says:
> #    If a class defines __repr__() but not __str__(), then __repr__() is
> #    also used when an 'informal' string representation of instances of
> #    that class is required.
> # If the reader is in lawyer mode, the documentation is correct.
> # Is the current behavior what was intended? What should the behavior be?
>
> class X(object):
>     def __str__(self):
>         return 'str for X'
>
>     def __repr__(self):
>         return 'repr for X'
>
> class A(X):
>     def __repr__(self):
>         return 'repr for A'
>
> #    __str__ = __repr__
>
> x = X()
> a = A()
> print x.__str__()
> print x.__repr__()
> print a.__str__()
> print a.__repr__()
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
>




From pedronis at bluewin.ch  Mon May 17 12:04:12 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Mon May 17 12:04:16 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: <5.1.1.6.0.20040517114707.02716660@mail.telecommunity.com>
References: <40A8DC3B.1070107@egenix.com>
	<5.2.1.1.0.20040517171026.02ff4198@pop.bluewin.ch>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.2.1.1.0.20040517171026.02ff4198@pop.bluewin.ch>
Message-ID: <5.2.1.1.0.20040517180028.02ff4198@pop.bluewin.ch>

At 11:50 17.05.2004 -0400, Phillip J. Eby wrote:
>At 05:37 PM 5/17/04 +0200, M.-A. Lemburg wrote:
>>Samuele Pedroni wrote:
>>>At 16:55 17.05.2004 +0200, M.-A. Lemburg wrote:
>>>>Are you saying that you are not going to change the default __import__()
>>>>implementation, only the way it is called ? (I wonder how you'll
>>>>enforce the 'absolute only' strategy then)
>>>if you pass globals = {} you get absolute imports from __import__,
>>
>>Ahh, a hidden feature :-) I didn't know that one yet.
>
>More of an emergent property.  Keep in mind that the current relative 
>mechanism is always relative *to* something, and that "something" is 
>defined by the globals['__name__'], if any.  If there is no '__name__', 
>then the import has to be absolute.  For example, you would expect this:
>
>     exec "import foo" in {},{}
>
>to be an absolute import, would you not?

yes, and

__import__('foo')

triggers also an absolute import because the default for globals is a 
conceptually empty namespace,
not the caller globals.

Respectively this can trigger a relative import:

__import__('foo',globals())

and the relevant bit is indeed __name__ . 


From tjreedy at udel.edu  Mon May 17 12:15:12 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon May 17 12:15:27 2004
Subject: [Python-Dev] Re: __str__ does not default to __repr__ if a base
	classhas __str__
References: <40A8D37B.7030807@erols.com>
Message-ID: <c8aoeh$5ue$1@sea.gmane.org>


"Edward C. Jones" <edcjones@erols.com> wrote in message
news:40A8D37B.7030807@erols.com...
> # In the code below, "a.__str__()" does not default to "a.__repr__() as
> # might be expected. It finds the "__str__" in the base class X. The
> # Reference Manual, section 3.3.1, says:
> #    If a class defines __repr__() but not __str__(), then __repr__() is
> #    also used when an 'informal' string representation of instances of
> #    that class is required.
> # If the reader is in lawyer mode, the documentation is correct.

I don't think one has to be in lawyer mode to think that 'attribute x is
defined by class C' == 'C.x returns an object without exception', whether
the production is by direct lookup, inheritance, or calculation.  It is
possible that 'defines' could be defined better.

The current behavior seems to be

try:
  f = i.__str__
except AttributeError:
  try:
    f = i.__repr__
  except AttributeError
    f = <default stringifier that produces '<class instance at 0x####>'
form >
s = f(i)

Your implied and underdefined alternative would require special casing of
__str__ lookup to not use inheritance.

> # Is the current behavior what was intended? What should the behavior be?

Since the current behavior is hard to see as a bug, I think it should stay
as is, if it has been consistently so forever, even if something else might
have been once preferred.

Terry J. Reedy




From alloydflanagan at comcast.net  Mon May 17 15:16:48 2004
From: alloydflanagan at comcast.net (alloydflanagan@comcast.net)
Date: Mon May 17 15:16:55 2004
Subject: [Python-Dev] Python-Dev] Re: __str__ does not default to __repr__
	if a base classhas __str__
Message-ID: <051720041916.3893.40A90FA00007E8A000000F352200751150020E090E020E04000B970104040E@comcast.net>

FWIW, I think the current behavior is correct.  If a __str__() method exists, it exists; we shouldn't treat an inherited method differently from one defined directly in the class.  So I would be very surprised if the inherited __str__() wasn't used.




From anon at spam-trap.richardcooper.net  Sun May 16 07:39:08 2004
From: anon at spam-trap.richardcooper.net (Richard Cooper)
Date: Mon May 17 15:37:29 2004
Subject: [Python-Dev] Re: .pth files in current or script directory
In-Reply-To: <0AA408F4E902C84C8F6BE093CCE9D7A88C2468@server1.artsalliance.co.uk>
Message-ID: <A4A1C3B2-A72D-11D8-9743-000A27E3999C@spam-trap.richardcooper.net>

Tim Delaney wrote:
> A discussion (last week?) on c.l.py prompted a thought about .pth 
> files.
> Feel free to shoot it down ;)
>
> The discussion in question concerned someone using a version control
> system, and needing to change PYTHONPATH depending on which branch they
> were in.

That was me, I think.

> My immediately two thoughts were:
>
> 1. Use a build system (e.g. make) to set PYTHONPATH appropriately. This
> is they type of thing we do.

I'm on windows and am quite attached to coding and running inside SciTE.
I think I could knock up a script to do the right thing and configure
SciTE to run it rather than python. However since I sometimes run from
the command line and my colleagues use different editors that's a whole
other can of worms.

> 2. Use a .pth file in the directory the code is run from (or possibly,
> the same directory as the script).

<sniped reason this doesn't work now>

Guido van Rossum wrote:
> Can't they manipulate sys.path *inside* the code in question?  That's
> invariably the solution I end up using -- I never play with
> PYTHONPATH any more.

I suppose I could, but adding a bit of boilerplate to the top of every
source file to support something that should be completely orthogonal
to the the language (using source control branches) seems a bit hacky-
workaround-tastic even compared to my current solution (a post-it
saying 'check PYTHONPATH' in big letters stuck to my monitor)

Anyway, enough discussion of my situation, which is probably more suited
to python-list. My real question is:

Is there any desire to address this problem in some future python? When
I see 2 python-dev'ers say "PYTHONPATH isn't up to it, here's what I do"
I smell something that should be fixed once rather that worked around
everywhere.

My initial idea was something along the lines of:

When a module is imported, its import search path includes everything it
does now PLUS the directory that is the top of the module's enclosing
package structure. That way you wouldn't have to add entries to
PYTHONPATH just so that modules could import (absolutely) from their own
package.

What do you think? Possible? Sensible? Any other ideas?

Rich


From edcjones at erols.com  Mon May 17 10:58:54 2004
From: edcjones at erols.com (Edward C. Jones)
Date: Mon May 17 18:07:38 2004
Subject: [Python-Dev] __str__ does not default to __repr__ if a base class
	has __str__
Message-ID: <40A8D32E.6050102@erols.com>

# In the code below, "a.__str__()" does not default to "a.__repr__() as
# might be expected. It finds the "__str__" in the base class X. The
# Reference Manual, section 3.3.1, says:
#     If a class defines __repr__() but not __str__(), then __repr__() is
#     also used when an 'informal' string representation of instances of
#     that class is required.
# If the reader is in lawyer mode, the documentation is correct.
# Is the current behavior what was intended? What should the behavior be?

class X(object):
    def __str__(self):
        return 'str for X'

    def __repr__(self):
        return 'repr for X'

class A(X):
    def __repr__(self):
        return 'repr for A'

#    __str__ = __repr__

x = X()
a = A()
print x.__str__()
print x.__repr__()
print a.__str__()
print a.__repr__()



From edcjones at erols.com  Mon May 17 10:57:57 2004
From: edcjones at erols.com (Edward C. Jones)
Date: Mon May 17 18:14:44 2004
Subject: [Python-Dev] __str__ does not default to __repr__ if a base class
	has __str__
Message-ID: <40A8D2F5.6000302@erols.com>

# In the code below, "a.__str__()" does not default to "a.__repr__() as
# might be expected. It finds the "__str__" in the base class X. The
# Reference Manual, section 3.3.1, says:
#     If a class defines __repr__() but not __str__(), then __repr__() is
#     also used when an 'informal' string representation of instances of
#     that class is required.
# If the reader is in lawyer mode, the documentation is correct.
# Is the current behavior what was intended? What should the behavior be?

class X(object):
    def __str__(self):
        return 'str for X'

    def __repr__(self):
        return 'repr for X'

class A(X):
    def __repr__(self):
        return 'repr for A'

#    __str__ = __repr__

x = X()
a = A()
print x.__str__()
print x.__repr__()
print a.__str__()
print a.__repr__()



From mal at egenix.com  Mon May 17 18:36:29 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon May 17 18:36:33 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: <5.1.1.6.0.20040517114707.02716660@mail.telecommunity.com>
References: <5.2.1.1.0.20040517171026.02ff4198@pop.bluewin.ch>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.2.1.1.0.20040517171026.02ff4198@pop.bluewin.ch>
	<5.1.1.6.0.20040517114707.02716660@mail.telecommunity.com>
Message-ID: <40A93E6D.8010605@egenix.com>

Phillip J. Eby wrote:
> At 05:37 PM 5/17/04 +0200, M.-A. Lemburg wrote:
> 
>> Samuele Pedroni wrote:
>>
>>> At 16:55 17.05.2004 +0200, M.-A. Lemburg wrote:
>>>
>>>> Are you saying that you are not going to change the default 
>>>> __import__()
>>>> implementation, only the way it is called ? (I wonder how you'll
>>>> enforce the 'absolute only' strategy then)
>>>
>>> if you pass globals = {} you get absolute imports from __import__,
>>
>>
>> Ahh, a hidden feature :-) I didn't know that one yet.
> 
> More of an emergent property.  Keep in mind that the current relative 
> mechanism is always relative *to* something, and that "something" is 
> defined by the globals['__name__'], if any.  If there is no '__name__', 
> then the import has to be absolute. 

Right. Point taken on that account. Perhaps a note should
go into the PEP that the internal mechanism is not going
to change and that instead only the syntax will get enhanced.

I still don't believe that we need to break code in order
to introduce relative imports, though, certainly not
before Py3k.

I also think that prepending "from ." to all
package local imports in order to overcome this looks like
silly (the dot is hardly visible in an editor), but that may
be just me.

BTW, what are you going to do about "from . import *" and
"from .. import *" ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 18 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
EuroPython 2004, G?teborg, Sweden                           19 days left

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From bac at OCF.Berkeley.EDU  Mon May 17 19:09:22 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon May 17 19:09:40 2004
Subject: [Python-Dev] Need traceback expert to review patch 954922
Message-ID: <40A94622.6040606@ocf.berkeley.edu>

Back in February, I tried to fix an issue of threads executed using 
'threading' that did not shut themselves down before the interpreter 
started tearing down globals, the thread triggering an exception, and 
then a new exception being raised while trying to report the exception 
raised in the thread by accessing globals (see 
http://mail.python.org/pipermail/python-dev/2004-February/042457.html 
for the previous discussion).  An initial fix was devised, but since 
there was no way to trigger the issue every time I was not sure if the 
fix was thorough enough.

Well, someone finally has an app that causes the error consistently and 
the previous fix did not go far enough.  But with the help of the app 
(mnet, specifically) I was finally able to get rid of all global 
accesses while reporting exceptions so that the original exception 
caught in the thread can get printed.  The patch is at 
http://python.org/sf/954922 while the bug report is at 
http://python.org/sf/754449 .

Trouble is that I had to basically fake traceback.format_exc() in the 
patch to a reasonable level since the 'traceback' module bounces around 
between its various functions too much to just store a local copy of the 
function and have it still work after globals are set to None.  So I 
need someone to just look over the code to make sure that the code is 
not doing something stupid when this code is called (default way is 
still used when 'sys' is defined and thus the interpreter is not being 
torn down).

And the reason I bring this up now is just in case there is any desire 
to put this in 2.3.4 .  I obviously know that I missed rc1, but I didn't 
get the details of triggering the problem soon enough and people running 
2.3.x are the ones bringing this up since this apparently did not happen 
under 2.2 for some reason.  I doubt Anthony wants to risk mucking with 
'threading' this late (although the patch is fairly self-contained but 
would require applying the previous patch to attempt to fix this as 
well), but I thought I would at least put it out there.

-Brett

From barry at python.org  Mon May 17 21:36:31 2004
From: barry at python.org (Barry Warsaw)
Date: Mon May 17 21:36:48 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: <40A93E6D.8010605@egenix.com>
References: <5.2.1.1.0.20040517171026.02ff4198@pop.bluewin.ch>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.2.1.1.0.20040517171026.02ff4198@pop.bluewin.ch>
	<5.1.1.6.0.20040517114707.02716660@mail.telecommunity.com>
	<40A93E6D.8010605@egenix.com>
Message-ID: <1084844191.19299.312.camel@anthem.wooz.org>

On Mon, 2004-05-17 at 18:36, M.-A. Lemburg wrote:

> I still don't believe that we need to break code in order
> to introduce relative imports, though, certainly not
> before Py3k.

One to to "fix" this in a backward compatible way is to instead
introduce a spelling for "really, import this absolutely", e.g.

from __python__ import re

or

import __python__.re

and leave the current import semantics alone, for Python 2.x.  I know
Guido's not in favor of this approach though.

-Barry



From greg at cosc.canterbury.ac.nz  Mon May 17 22:28:47 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon May 17 22:29:13 2004
Subject: [Python-Dev] Re: .pth files in current or script directory
In-Reply-To: <A4A1C3B2-A72D-11D8-9743-000A27E3999C@spam-trap.richardcooper.net>
Message-ID: <200405180228.i4I2Slel012600@cosc353.cosc.canterbury.ac.nz>

Richard Cooper <anon@server15.ukservers.net>:

> When a module is imported, its import search path includes everything it
> does now PLUS the directory that is the top of the module's enclosing
> package structure.

I don't see how that changes anything, since in order to get
imported in the first place, the module's top-level package has
to already be on the module search path somehow.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Mon May 17 22:33:42 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon May 17 22:33:57 2004
Subject: [Python-Dev] Need traceback expert to review patch 954922
In-Reply-To: <40A94622.6040606@ocf.berkeley.edu>
Message-ID: <200405180233.i4I2XgqL012607@cosc353.cosc.canterbury.ac.nz>

"Brett C." <bac@OCF.Berkeley.EDU>:

> Back in February, I tried to fix an issue of threads executed using
> 'threading' that did not shut themselves down before the interpreter
> started tearing down globals, the thread triggering an exception,
> and then a new exception being raised while trying to report the
> exception raised in the thread by accessing globals

Seems to me the way to fix this sort of thing properly is to
get rid of the need for explicitly clearing globals.

Now that we have a garbage collector, is it really necessary
any more? What would happen if we simply set sys.modules to
None and then invoked the collector?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From kbk at shore.net  Mon May 17 23:03:39 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Mon May 17 23:03:46 2004
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200405180303.i4I33doq020408@hydra.localdomain>


Patch / Bug Summary
___________________

Patches :  275 open ( +5) /  2380 closed (+14) /  2655 total (+19)
Bugs    :  753 open ( -1) /  4047 closed (+54) /  4800 total (+53)
RFE     :  133 open ( +0) /   126 closed ( +2) /   259 total ( +2)

New / Reopened Patches
______________________

CodeContext - an extension to show you where you are  (2004-04-16)
       http://python.org/sf/936169  reopened by  noamr

Explain 'in' keyword as it is first used  (2004-04-22)
CLOSED http://python.org/sf/940026  opened by  Gerrit Holl 

Replace if/elif chain with dispatch pattern in sre_compile  (2004-04-23)
CLOSED http://python.org/sf/941071  opened by  Raymond Hettinger 

Allow any encodings other than latin-1 in interactive interp  (2004-04-24)
       http://python.org/sf/941229  opened by  Hye-Shik Chang 

Minimal fix for bug 940578 (glob.glob on broken symlinks)  (2004-04-24)
       http://python.org/sf/941486  opened by  Cherniavsky Beni 

PEP309 Partial implementation  (2004-04-26)
       http://python.org/sf/941881  opened by  Hye-Shik Chang 

CodeContext - an extension to show you where you are  (2004-04-16)
       http://python.org/sf/936169  reopened by  noamr

Convert glob.glob to generator-based DFS  (2004-04-27)
       http://python.org/sf/943206  opened by  Cherniavsky Beni 

A simple 3-4% speed-up for PCs  (2004-04-28)
       http://python.org/sf/943898  opened by  Armin Rigo 

Add maketrans to string object  (2004-04-28)
       http://python.org/sf/943953  opened by  Gyro Funch 

urllib2 authentication mishandles empty pass bugfix 944082  (2004-04-29)
CLOSED http://python.org/sf/944110  opened by  sc0rp 

Bugfix for dircheck() in Objects/fileobject.c  (2004-04-30)
       http://python.org/sf/944928  opened by  Ulrich Berning 

nonblocking i/o with ssl socket not working at all  (2004-05-01)
       http://python.org/sf/945642  opened by  Tino Lange 

askstring => grab fail  (2004-05-01)
       http://python.org/sf/946153  opened by  wes 

Non-blocking Socket Server  (2004-05-01)
       http://python.org/sf/946207  opened by  Jesper ´JJ´ Jurcenoks 

do not add directory of sys.argv[0] into sys.path  (2004-05-02)
       http://python.org/sf/946373  opened by  wrobell 

AF_PACKET Hardware address support in socket module  (2004-05-03)
       http://python.org/sf/947352  opened by  Jason Andryuk 

Fix for #947380 - sys.path is wrong on windows sometimes  (2004-05-04)
       http://python.org/sf/947386  opened by  Ori Berger 

Apply freelist technique to empty lists  (2004-05-04)
CLOSED http://python.org/sf/947476  opened by  Raymond Hettinger 

Linux signals during threads  (2004-05-05)
       http://python.org/sf/948614  opened by  Andrew Langmead 

synchronous signals blocked in pthreads  (2004-05-06)
       http://python.org/sf/949332  opened by  Andrew Langmead 

fix bug in StringIO.truncate - length not changed  (2004-05-11)
       http://python.org/sf/951915  opened by  David Fraser 

fix typos  (2004-05-12)
CLOSED http://python.org/sf/952047  opened by  George Yoshida 

Fix os.stat handling of UNC roots  (2004-05-14)
       http://python.org/sf/954115  opened by  Greg Chapman 

Replacement for webbrowser.py which has problems.  (2004-05-15)
       http://python.org/sf/954628  opened by  Michel Van den Bergh 

Patches Closed
______________

CodeContext - an extension to show you where you are  (2004-04-16)
       http://python.org/sf/936169  closed by  kbk

Py_INCREF/DECREF available as macros only  (2004-04-20)
       http://python.org/sf/938302  closed by  theller

CodeContext - an extension to show you where you are  (2004-04-16)
       http://python.org/sf/936169  closed by  kbk

itertools roundrobin()  (2003-06-17)
       http://python.org/sf/756253  closed by  rhettinger

ConfigParser: fixes mixed-case interpolations (bug 857881)  (2003-12-24)
       http://python.org/sf/865455  closed by  fdrake

Replace if/elif chain with dispatch pattern in sre_compile  (2004-04-23)
       http://python.org/sf/941071  closed by  rhettinger

Explain 'in' keyword as it is first used  (2004-04-22)
       http://python.org/sf/940026  closed by  loewis

Apply freelist technique to empty lists  (2004-05-04)
       http://python.org/sf/947476  closed by  rhettinger

urllib2 CacheFTPHandler doesn't work on multiple dirs  (2003-12-01)
       http://python.org/sf/851736  closed by  mhammond

urllib2 authentication mishandles empty pass bugfix 944082  (2004-04-29)
       http://python.org/sf/944110  closed by  loewis

fix typos  (2004-05-11)
       http://python.org/sf/952047  closed by  fdrake

Error caused by patch #852334  (2004-04-10)
       http://python.org/sf/932796  closed by  fdrake

API Ref: PyErr_SetInterrupt not Obsolete  (2004-03-19)
       http://python.org/sf/919299  closed by  fdrake

ConfigParser should raise an error for duplicate options  (2003-12-04)
       http://python.org/sf/854484  closed by  fdrake

Fix docstring for posixpath.getctime  (2004-01-04)
       http://python.org/sf/870287  closed by  fdrake

Footnote on bug in Mailbox with Windows text-mode files  (2003-11-09)
       http://python.org/sf/838910  closed by  bwarsaw

New / Reopened Bugs
___________________

message.as_string hoses headers  (2004-04-20)
CLOSED http://python.org/sf/938890  opened by  Carl Karsten 

cPickle.Pickler: in "list" mode, no way to set protocol  (2004-04-22)
       http://python.org/sf/939395  opened by  John Belmonte 

Building python 2.3.3 RPM on Mandrake 9.2. fails  (2004-04-22)
       http://python.org/sf/939699  opened by  fabien wahl 

Logging failes for rotating file log (during rollOver)  (2004-04-22)
CLOSED http://python.org/sf/939836  opened by  smeyers2002 

xml parser bug  (2004-04-22)
CLOSED http://python.org/sf/940263  opened by  David Tefft 

pydoc.Helper.help() ignores input/output init parameters  (2004-04-22)
       http://python.org/sf/940286  opened by  Maxim Krikun 

string.lstrip problem with '_' string  (2004-04-22)
CLOSED http://python.org/sf/940316  opened by  Arnold 

Python crashes with bus error on sparc64.  (2004-04-22)
CLOSED http://python.org/sf/940354  opened by  Generic Player 

misbehaviour with map / lambda / isdigit()  (2004-04-23)
CLOSED http://python.org/sf/940522  opened by  dreary 

glob.glob inconsistent about broken symlinks  (2004-04-23)
       http://python.org/sf/940578  opened by  Cherniavsky Beni 

section 5.10: 'not' returns boolean, not int  (2004-04-23)
CLOSED http://python.org/sf/940579  opened by  Jon Ashley 

List with Canvas.create_line Option arrow=LAST Broke  (2004-04-24)
       http://python.org/sf/941262  opened by  Brian Brooks 

AIX shared library fix  (2004-04-24)
       http://python.org/sf/941346  opened by  Dick Dunbar 

Python crash on __init__/__getattr__/__setattr__ interaction  (2004-04-26)
       http://python.org/sf/942706  opened by  has 

Weakness in tuple hash  (2004-04-27)
       http://python.org/sf/942952  opened by  Steve Tregidgo 

urllib2 authentication mishandles empty password  (2004-04-29)
CLOSED http://python.org/sf/944082  opened by  sc0rp 

Signal reception problem. FreeBSD 4.8 and 4.9 after forkpty  (2004-04-28)
       http://python.org/sf/944119  opened by  Steve Levis 

Signal reception problem. FreeBSD 4.8 and 4.9 after forkpty  (2004-04-28)
       http://python.org/sf/944119  reopened by  levis501

No examples or usage docs for urllib2  (2004-04-29)
       http://python.org/sf/944394  opened by  Chris Withers 

urllib2 doesn't handle username/password in url  (2004-04-29)
       http://python.org/sf/944396  opened by  Chris Withers 

No documentation for urllib2 exception classes  (2004-04-29)
       http://python.org/sf/944407  opened by  Chris Withers 

csv writer bug on windows  (2004-04-29)
       http://python.org/sf/944890  opened by  Brian Kelley 

cgitb has backwards logic on log file creation  (2004-04-30)
CLOSED http://python.org/sf/945063  opened by  Danny Yoo 

platform.system() Windows inconsistency  (2004-04-30)
       http://python.org/sf/945665  opened by  Guido van Rossum 

dir(object) does not list __name__  (2004-05-01)
       http://python.org/sf/945861  opened by  Thomas Philips 

xmlrpclib omits charset in Content-Type HTTP header  (2004-05-01)
CLOSED http://python.org/sf/946130  opened by  Christian Schmidt 

CGI QUERY_STRING  (2004-05-01)
       http://python.org/sf/946247  opened by  Larry 

sys.path is wrong in some cases  (2004-05-04)
       http://python.org/sf/947380  opened by  Ori Berger 

os.utime() raises bad exception for unicode filenames  (2004-05-04)
CLOSED http://python.org/sf/947405  opened by  Mark Hammond 

urlib.urlopen() fails to raise exception  (2004-05-04)
       http://python.org/sf/947571  opened by  M.-A. Lemburg 

calendar.weekheader() undocumented  (2004-05-04)
       http://python.org/sf/947894  opened by  Leonardo Rochael Almeida 

calendar.weekheader(n): n should mean chars not bytes  (2004-05-04)
       http://python.org/sf/947906  opened by  Leonardo Rochael Almeida 

LaTeX not required  (2004-05-05)
CLOSED http://python.org/sf/948517  opened by  Aahz 

texi2html.py not working  (2004-05-06)
       http://python.org/sf/948881  opened by  George Yoshida 

PyExc_* not in index  (2004-05-06)
       http://python.org/sf/948970  opened by  Thomas Heller 

winzip cannot extrat tarfile created tars  (2004-05-06)
       http://python.org/sf/949052  opened by  Simon Eisenmann 

refresh() missing curses reference  (2004-05-06)
CLOSED http://python.org/sf/949100  opened by  Alexander Neumann 

Memory "read" instruction failed  (2004-05-06)
CLOSED http://python.org/sf/949301  opened by  AnToine 

Memory "read" instruction failed  (2004-05-06)
       http://python.org/sf/949301  reopened by  alexion

stringprep.py:  sets.Set() --> set()   (2004-05-06)
       http://python.org/sf/949329  opened by  Raymond Hettinger 

UnicodeEncodeError message has syntax error  (2004-05-07)
CLOSED http://python.org/sf/949605  opened by  Tom Lynn 

file write() method and non-blocking mode.  (2004-05-07)
       http://python.org/sf/949667  opened by  Donovan Baarda 

Invalid page in module PYTHON23.DLL  (2004-05-07)
CLOSED http://python.org/sf/949706  opened by  AnToine 

Problem w/6.27.2.2 GNUTranslations ungettext() example code  (2004-05-07)
       http://python.org/sf/949832  opened by  Martin Miller 

Is cmdqueue attribute used?  (2004-05-07)
CLOSED http://python.org/sf/950055  opened by  Raymond Hettinger 

itertools.chain doesn't "process" exceptions as they occur  (2004-05-07)
CLOSED http://python.org/sf/950057  opened by  Tristan Seligmann 

buffer() can point to freed memory  (2004-05-07)
CLOSED http://python.org/sf/950274  opened by  L. Peter Deutsch 

-fconstant-cfstrings required for Xcode 1.2  (2004-05-08)
       http://python.org/sf/950482  opened by  Bob Ippolito 

email: mishandles Content-Transfer-Enc. for text/* messages  (2004-05-09)
       http://python.org/sf/950747  opened by  Matthias Klose 

str(email) breaks message by adding newline  (2004-05-10)
CLOSED http://python.org/sf/951088  opened by  Gerrit Holl 

strptime doesn't account for %p in output  (2004-05-10)
CLOSED http://python.org/sf/951235  opened by  Skip Montanaro 

Generator expression used in doc example  (2004-05-11)
       http://python.org/sf/951270  opened by  Nick Coghlan 

ihooks chokes on BioPython package  (2004-05-10)
       http://python.org/sf/951275  opened by  mpalmer 

New module: doc versus action  (2004-05-10)
       http://python.org/sf/951482  opened by  Terry J. Reedy 

Crash when reading "import table" of certain windows DLLs  (2004-05-11)
       http://python.org/sf/951851  opened by  Mark Hammond 

__name__ should be in 2.3.10 Special Attributes  (2004-05-11)
CLOSED http://python.org/sf/952212  opened by  Robert Brewer 

Funny keywords in library index  (2004-05-12)
       http://python.org/sf/952737  opened by  Thomas Heller 

segfault in subclassing datetime.date & pickling  (2004-05-12)
       http://python.org/sf/952807  opened by  Thomas Wouters 

readline broke between 2.2.2 and 2.2.3  (2004-05-12)
CLOSED http://python.org/sf/952852  opened by  Jon Nelson 

"can't multiply sequence *by* non-int"  (2004-05-12)
CLOSED http://python.org/sf/952866  opened by  Tom Lynn 

execve rejects empty argument list  (2004-05-13)
       http://python.org/sf/952953  opened by  newsham 

cgi module documentation could mention getlist  (2004-05-13)
       http://python.org/sf/953177  opened by  Richard Jones 

asyncore misses socket closes when poll is used  (2004-05-13)
       http://python.org/sf/953599  opened by  Shane Kerr 

simple typo in pprint module  (2004-05-14)
CLOSED http://python.org/sf/954030  opened by  Michal Pasternak 

New email pkg FeedParser croaks on some spam  (2004-05-14)
CLOSED http://python.org/sf/954320  opened by  Skip Montanaro 

2.3 inspect.getframeinfo wrong tb line numbers  (2004-05-15)
       http://python.org/sf/954364  opened by  Robin Becker 

bsddb3 test errors  (2004-05-15)
CLOSED http://python.org/sf/954530  opened by  Tim Peters 

Error in urllib2 Examples  (2004-05-16)
       http://python.org/sf/954981  opened by  Phillip 

CFLAGS, CPPFLAGS, LDFLAGS are NOT used when building modules  (2004-05-17)
       http://python.org/sf/955197  opened by  Niki W. Waibel 

Bugs Closed
___________

Random.choice doesn't work with sets.  (2004-04-17)
       http://python.org/sf/936988  closed by  rhettinger

minidom.py writes TEXT_NODE wrong  (2004-04-18)
       http://python.org/sf/937282  closed by  loewis

re.sub on u'' gives NON-unicode ''  (2004-03-31)
       http://python.org/sf/926075  closed by  perky

message.as_string hoses headers  (2004-04-20)
       http://python.org/sf/938890  closed by  carlk

message.as_string hoses headers  (2004-04-20)
       http://python.org/sf/938890  closed by  tim_one

Shelve slow after 7/8000 key  (2004-01-21)
       http://python.org/sf/881522  closed by  tim_one

Slicing changes item-attributes of an instance of a UserList  (2004-04-19)
       http://python.org/sf/937700  closed by  hflori

xml parser bug  (2004-04-22)
       http://python.org/sf/940263  closed by  loewis

string.lstrip problem with '_' string  (2004-04-22)
       http://python.org/sf/940316  closed by  rhettinger

misbehaviour with map / lambda / isdigit()  (2004-04-23)
       http://python.org/sf/940522  closed by  tim_one

section 5.10: 'not' returns boolean, not int  (2004-04-23)
       http://python.org/sf/940579  closed by  rhettinger

Python crashes with bus error on sparc64.  (2004-04-22)
       http://python.org/sf/940354  closed by  genericplayer

Unbounded recursion in modulefinder  (2004-01-13)
       http://python.org/sf/876278  closed by  theller

Signal reception problem. FreeBSD 4.8 and 4.9 after forkpty  (2004-04-28)
       http://python.org/sf/944119  closed by  levis501

Logging failes for rotating file log (during rollOver)  (2004-04-22)
       http://python.org/sf/939836  closed by  smeyers2002

cgitb has backwards logic on log file creation  (2004-04-30)
       http://python.org/sf/945063  closed by  akuchling

os.tmpfile seek(0) fails on cygwin  (2004-01-01)
       http://python.org/sf/868743  closed by  jlt63

xmlrpclib omits charset in Content-Type HTTP header  (2004-05-02)
       http://python.org/sf/946130  closed by  loewis

os.utime() raises bad exception for unicode filenames  (2004-05-04)
       http://python.org/sf/947405  closed by  mhammond

5454 - documentation wrong for ossaudiodev mixer device  (2003-09-29)
       http://python.org/sf/814606  closed by  gward

Build fails in ossaudiodev.c with missing macros  (2003-10-29)
       http://python.org/sf/832236  closed by  gward

PyBool_* not documented in C API  (2004-02-02)
       http://python.org/sf/889154  closed by  fdrake

socket.gethostbyname raises gaierror, not herror  (2003-11-07)
       http://python.org/sf/837929  closed by  fdrake

pickle: __getinitargs__ not used for new-style objects  (2004-01-08)
       http://python.org/sf/873246  closed by  fdrake

urllib2 CacheFTPHandler doesn't work on multiple dirs  (2003-05-17)
       http://python.org/sf/738973  closed by  mhammond

os.chmod/os.utime/shutil do not work with unicode filenames  (2003-11-21)
       http://python.org/sf/846133  closed by  mhammond

long list in Pythonwin -> weird text  (2002-09-04)
       http://python.org/sf/604387  closed by  mhammond

LaTeX not required  (2004-05-05)
       http://python.org/sf/948517  closed by  fdrake

shutil.copyfile fails when dst exists read-only  (2003-09-22)
       http://python.org/sf/810879  closed by  akuchling

unable to specify another compiler  (2003-07-04)
       http://python.org/sf/765819  closed by  akuchling

urllib2 authentication mishandles empty password  (2004-04-29)
       http://python.org/sf/944082  closed by  loewis

pyexpat LexicalHandler swaps system_id and public_id  (2003-07-30)
       http://python.org/sf/780300  closed by  loewis

XMLFilterBase has bogus startElementNS  (2004-04-17)
       http://python.org/sf/936637  closed by  loewis

XMLGenerator ignores encoding in output  (2004-04-19)
       http://python.org/sf/938076  closed by  loewis

xml.sax.handler.ContentHandler missing argument  (2004-01-21)
       http://python.org/sf/881707  closed by  fdrake

file built-in object is after file() function  (2004-03-25)
       http://python.org/sf/922964  closed by  fdrake

refresh() missing curses reference  (2004-05-06)
       http://python.org/sf/949100  closed by  akuchling

Doc bug in PEP-0278  (2003-09-04)
       http://python.org/sf/800828  closed by  fdrake

Need "new style" note regarding descriptors  (2003-10-04)
       http://python.org/sf/817742  closed by  fdrake

need doc for new trace module  (2003-05-14)
       http://python.org/sf/737734  closed by  fdrake

Memory "read" instruction failed  (2004-05-06)
       http://python.org/sf/949301  closed by  theller

Memory "read" instruction failed  (2004-05-06)
       http://python.org/sf/949301  closed by  theller

UnicodeEncodeError message has syntax error  (2004-05-07)
       http://python.org/sf/949605  closed by  doerwalter

re seems broken on 64-bit machines/linux  (2004-04-08)
       http://python.org/sf/931848  closed by  loewis

Invalid page in module PYTHON23.DLL  (2004-05-07)
       http://python.org/sf/949706  closed by  theller

Is cmdqueue attribute used?  (2004-05-07)
       http://python.org/sf/950055  closed by  gvanrossum

itertools.chain doesn't "process" exceptions as they occur  (2004-05-07)
       http://python.org/sf/950057  closed by  rhettinger

Python 2.3.3: socketmodule doesn't build on UnixWare 7.1.3  (2004-04-05)
       http://python.org/sf/929689  closed by  loewis

buffer() can point to freed memory  (2004-05-08)
       http://python.org/sf/950274  closed by  nascheme

email library Parser can't parse some messages  (2004-02-12)
       http://python.org/sf/895510  closed by  bwarsaw

MIMEMultipart _subparts constructor unusable  (2004-01-25)
       http://python.org/sf/884030  closed by  bwarsaw

str(email) breaks message by adding newline  (2004-05-10)
       http://python.org/sf/951088  closed by  bwarsaw

strptime doesn't account for %p in output  (2004-05-10)
       http://python.org/sf/951235  closed by  bcannon

add support for cjkcodecs to Python email  (2003-12-01)
       http://python.org/sf/852347  closed by  bwarsaw

email.Utils.formataddr doesn't handle whitespace too well  (2004-01-12)
       http://python.org/sf/875646  closed by  bwarsaw

email package does not work with mailbox  (2002-07-26)
       http://python.org/sf/586899  closed by  bwarsaw

Memory error on AIX in email.Utils._qdecode  (2003-10-16)
       http://python.org/sf/824977  closed by  bwarsaw

__name__ should be in 2.3.10 Special Attributes  (2004-05-11)
       http://python.org/sf/952212  closed by  fdrake

readline broke between 2.2.2 and 2.2.3  (2004-05-12)
       http://python.org/sf/952852  closed by  jnelson

"can't multiply sequence *by* non-int"  (2004-05-12)
       http://python.org/sf/952866  closed by  rhettinger

email.Parser.Parser doesn't check for valid Content-Type  (2003-11-21)
       http://python.org/sf/846938  closed by  bwarsaw

simple typo in pprint module  (2004-05-14)
       http://python.org/sf/954030  closed by  montanaro

New email pkg FeedParser croaks on some spam  (2004-05-15)
       http://python.org/sf/954320  closed by  bwarsaw

bsddb3 test errors  (2004-05-15)
       http://python.org/sf/954530  closed by  greg

segmentation fault in test_re  (2004-01-03)
       http://python.org/sf/870120  closed by  hoel

ConfigParser bug  (2003-12-10)
       http://python.org/sf/857881  closed by  fdrake

New / Reopened RFE
__________________

Allow any lvalue for function definitions  (2004-05-09)
       http://python.org/sf/950644  opened by  David Albert Torpey 

RFE Closed
__________

optparse "append" action should always make the em  (2003-10-07)
       http://python.org/sf/819178  closed by  gward

http://doc.python.org  (2004-02-25)
       http://python.org/sf/904770  closed by  etrepum


From guido at python.org  Mon May 17 23:48:25 2004
From: guido at python.org (Guido van Rossum)
Date: Mon May 17 23:48:33 2004
Subject: [Python-Dev] __str__ does not default to __repr__ if a base class
	has __str__
In-Reply-To: Your message of "Mon, 17 May 2004 11:00:11 EDT."
	<40A8D37B.7030807@erols.com> 
References: <40A8D37B.7030807@erols.com> 
Message-ID: <200405180348.i4I3mPP03950@guido.python.org>

> # In the code below, "a.__str__()" does not default to "a.__repr__() as
> # might be expected. It finds the "__str__" in the base class X. The
> # Reference Manual, section 3.3.1, says:
> #    If a class defines __repr__() but not __str__(), then __repr__() is
> #    also used when an 'informal' string representation of instances of
> #    that class is required.
> # If the reader is in lawyer mode, the documentation is correct.
> # Is the current behavior what was intended? What should the behavior be?
> 
> class X(object):
>     def __str__(self):
>         return 'str for X'
> 
>     def __repr__(self):
>         return 'repr for X'
> 
> class A(X):
>     def __repr__(self):
>         return 'repr for A'
> 
> #    __str__ = __repr__
> 
> x = X()
> a = A()
> print x.__str__()
> print x.__repr__()
> print a.__str__()
> print a.__repr__()

This has never been different and the current outcome is exactly what
was intended.  The only case where __repr__ and __str__ act as each
other's default is when there's no base class.

And in any case, you're not supposed to call __str__ or __repr__
directly -- you should call str(x) or repr(x), etc. -- not that it
makes a difference here.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon May 17 23:50:11 2004
From: guido at python.org (Guido van Rossum)
Date: Mon May 17 23:50:19 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: Your message of "Mon, 17 May 2004 17:39:42 +0200."
	<s9zsmdzgj0h.fsf@salmakis.intevation.de> 
References: <5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<s9zfz9zi0zl.fsf@salmakis.intevation.de>
	<200405171452.i4HEqVp03065@guido.python.org> 
	<s9zsmdzgj0h.fsf@salmakis.intevation.de> 
Message-ID: <200405180350.i4I3oBD03973@guido.python.org>

> > Because in the current situation the meaning of an import is
> > ambiguous.
> 
> What exactly is ambiguous about it?

Looking at "import foo" you can't tell whether there's a module foo
expected in the current package or in the top-level module namespace.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon May 17 23:58:00 2004
From: guido at python.org (Guido van Rossum)
Date: Mon May 17 23:58:12 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: Your message of "Mon, 17 May 2004 21:36:31 EDT."
	<1084844191.19299.312.camel@anthem.wooz.org> 
References: <5.2.1.1.0.20040517171026.02ff4198@pop.bluewin.ch>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.2.1.1.0.20040517171026.02ff4198@pop.bluewin.ch>
	<5.1.1.6.0.20040517114707.02716660@mail.telecommunity.com>
	<40A93E6D.8010605@egenix.com> 
	<1084844191.19299.312.camel@anthem.wooz.org> 
Message-ID: <200405180358.i4I3w0f04074@guido.python.org>

> I know Guido's not in favor of this approach though.

We've been through this before.  Let's just please move forward.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From michel at dialnetwork.com  Mon May 17 22:07:09 2004
From: michel at dialnetwork.com (Michel Pelletier)
Date: Tue May 18 00:10:00 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405130325.i4D3PDTs013072@salk.daca.net>
References: <200405130325.i4D3PDTs013072@salk.daca.net>
Message-ID: <200405172107.09197.michel@dialnetwork.com>

On Wednesday 12 May 2004 22:30, Tim Peters wrote:
> [Michel Pelletier]
>
> > ...
> > Would there be an interest in at least a PEP to consider a bytecode
> > verifier?
>
> I think so, yes.

I'm working on a rough draft now.  Initially I would like to start small and 
easy:  a verifier written in python that provides a single function along the 
lines of the code provided by Phillip Eby that accepts the same arguments as 
dis(), a class, method, function, or code object.   It throws an exception 
and explanation for invalid bytecode and it will be up to programmers to call 
this method explicitly and choose to execute the code object or not.  This 
will probably cover almost all uses.

> Python should be easier, in large part because you have to give up sooner.
>
> Checks for type correctness in the PVM are done at runtime instead.  I
> think it's fair to say that a bytecode verifier is overwhelmingly "just an
> optimization":  if bytecode properties can be verified from static code
> analysis, runtime code isn't needed to verify them dynamically, or, in the
> absence of such runtime checks, static analysis plugs ways to provoke
> segfaults.

I think it's likely that both run-time check elimination and segfault plugging 
can come out ot this.  I'm not sure if it's possible for us to know what they 
all are up front, so I think a good requirement for a verification package 
will be a way to register new or project-specific verifications.

do you think there is a risk of exploitation?  for example, STORE_FAST, which 
does a direct set into PyObject **fastlocals, could be used to overwrite 
beyond the bounds of the array.  Can this or a stack over/underflow be used 
to execute arbitrary machine code?

> or
> check that C-level indexing into the co_consts vector is in bounds.  
> Well,
> in the latter case, that is checked (at runtime, on every co_consts access)
> in a debug build, but not in a release build.

I've investigated that a bit, and STORE_CONST and LOAD_CONST use GETITEM, 
which uses PyTuple_GetItem, which does a bounds check at run time even in a 
production build.  I think.  LOAD_FAST and STORE_FAST seem to do no runtime 
bounds check.

-Michel


From arigo at tunes.org  Tue May 18 04:57:33 2004
From: arigo at tunes.org (Armin Rigo)
Date: Tue May 18 05:01:02 2004
Subject: [Python-Dev] cmp(x,x)
Message-ID: <20040518085733.GA14842@vicky.ecs.soton.ac.uk>

Hello,

A minor semantic change that creeped in some time ago was an implicit
assumption that any object x should "reasonably" be expected to compare equal
to itself.  The arguments are summarized below (should this be documented,
inserted in NEWS, turned in a mini-PEP a posteriori ... ?).

The point is that comparisons now behave differently if they are issued by C
or Python code.  The expression 'x == y' will always call x.__eq__(y), because
in some settings (e.g. Numeric) the result is not just True or False but an
arbitrary object (e.g. a Numeric array of zeroes and ones).  You cannot just
say that 'x == x' should return True, because of that.  So the behavior of
PyObject_RichCompare() didn't change.

On the other hand, when C code does a comparison it uses
PyObject_RichCompareBool(), meaning it is only interested in a 1 or 0
answer.  So PyObject_RichCompareBool() is where the shortcut about comparing
an object with itself has been inserted.

The result is the following: say x has a method __cmp__() that always returns
-1.

>>> x == x             #   x.__cmp__() is called
False
>>> x < x              #   x.__cmp__() is called
True
>>> cmp(x,x)           #   x.__cmp__() is NOT called
0
>>> [x] == [x]         #   x.__cmp__() is NOT called
True

The only way to explain the semantic is that the expression 'x == x' always
call the special methods, but built-in functions are allowed to assume that
any object compares equal to itself.  In other words, C code usually checks
that objects are "identical or equal", which is equalivalent to the Python
expression 'x is y or x == y'.  For example, the equality of lists now works
as follows:

def __eq__(lst1, lst2):
    if len(lst1) != len(lst2):
        return False
    for x, y in zip(lst1, lst2):
        if not (x is y or x == y):
            return False
    else:
        return True

Should any of this be documented?

An alternative behavior would have been to leave PyObject_RichCompareBool()
alone and only insert the short-cut on specific object types' comparison
methods on a case-by-case basis.  For example, identical lists would just
compare equal, without going through the loop comparing each element.  This
would remove the surprize of x.__cmp__(x) being not always called.  The
semantics would be easier to explain too: two lists are equal if they are the
same list or if elements compare pairwise equal.  We would have (with x as
above):

>>> cmp(x,x)                # x.__cmp__(x) is called
-1
>>> [x] == [x]
False                       # because the lists contain a non-equal element
>>> lst = [x]; lst == lst
True                        # because it is the same list

Finally, whatever the final semantic is, we should make sure that existing
built-in objects behave in a consistent way.  Of course I'm thinking about
floats: on my Linux box,

>>> f = float('nan')
>>> cmp(f,f)
0                    # because f is f
>>> f == f
False                # because float.__eq__() is called

Note that as discussed below the following behavior is *expected* and in
accordance with standards:

>>> float('nan') is float('nan')
False
>>> float('nan') == float('nan')
False                             # not the same object

Unless there are serious objections I suggest to (i.e. I plan to) remove the
short-cut in PyObject_RichCompareBool() -- performance is probably not an
issue here -- and then review all built-in comparison methods and make sure
that they return "equal" for identical objects.

-+-  summary of arguments that lead to the original change (in cvs head).

The argument in favor of the change is to remove the complex and not useful
code trying to compare self-recursive structures: for example, in some setting
comparing __builtin__.__dict__ with itself would trigger recursive comparison
of __builtin__.__dict__ with itself in an endless loop.  The complex algorithm
was able to spot that.  The new semantics immediately assume
__builtin__.__dict__ to be equal to itself.  Removing the complex algorithm
means that you will now get an endless loop when comparing two *non-identical*
self-recursive structures with the same shape, which is most probably not a
problem in practice (on the contrary this implicit algorithm did hide a bug in
one of my program).

The argument _against_ used to be that e.g. it makes sense that the float
value 'nan' should be different from itself, as various standards require.  
This argument does not apply in Python: these standards are about comparing
*values*, not objects, so it makes perfect sense to say that even if x is the
result of a computation that yielded an unknown answer 'nan', this answer is
still equal to itself; what it is probably not equal to is *another* 'nan'
which was obtained differently.  In other words *two* float objects both
containing 'nan' should be different, but *one* 'nan' object is still equal to
itself.  This is sane as long as no code considers 'nan' as a singleton, or
tries to reuse 'nan' objects for different 'nan' values.

-+-

Armin

From arigo at tunes.org  Tue May 18 05:05:04 2004
From: arigo at tunes.org (Armin Rigo)
Date: Tue May 18 05:08:43 2004
Subject: [Python-Dev] Need traceback expert to review patch 954922
In-Reply-To: <200405180233.i4I2XgqL012607@cosc353.cosc.canterbury.ac.nz>
References: <40A94622.6040606@ocf.berkeley.edu>
	<200405180233.i4I2XgqL012607@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040518090504.GB14842@vicky.ecs.soton.ac.uk>

Hello Greg,

On Tue, May 18, 2004 at 02:33:42PM +1200, Greg Ewing wrote:
> Now that we have a garbage collector, is it really necessary
> any more? What would happen if we simply set sys.modules to
> None and then invoked the collector?

There is a patch on SF that does this.  Brett, can you see if this patch helps
in your case?

    http://www.python.org/sf/812369

The problem I can see with my patch is that it's not clear to me that it
eventually causes the destruction of enough objects.  For that matter it's not
clear why, or how many, objects we should try to destroy at shut-down time.  
I bet the goal is that if Python is embedded, and initialized and shut down
repeatedly, it should not leak memory.  Someone with such an application
should give my patch a serious try before I can start to believe that no
object is missed.  There are moreover a few other pending issues raised on the
patch's message board.


A bientot,

Armin.

From mwh at python.net  Tue May 18 07:04:46 2004
From: mwh at python.net (Michael Hudson)
Date: Tue May 18 07:04:51 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405172107.09197.michel@dialnetwork.com> (Michel Pelletier's
	message of "Mon, 17 May 2004 21:07:09 -0500")
References: <200405130325.i4D3PDTs013072@salk.daca.net>
	<200405172107.09197.michel@dialnetwork.com>
Message-ID: <2mlljqhu7l.fsf@starship.python.net>

Michel Pelletier <michel@dialnetwork.com> writes:

> do you think there is a risk of exploitation?  for example, STORE_FAST, which 
> does a direct set into PyObject **fastlocals, could be used to overwrite 
> beyond the bounds of the array.  Can this or a stack over/underflow be used 
> to execute arbitrary machine code?

If you're loading arbitrary bytecode, you will presumably at some
point be executing it, and that seems a much greater risk to me.

Cheers,
mwh

-- 
  We've had a lot of problems going from glibc 2.0 to glibc 2.1.
  People claim binary compatibility.  Except for functions they
  don't like.                       -- Peter Van Eynde, comp.lang.lisp

From mwh at python.net  Tue May 18 07:34:13 2004
From: mwh at python.net (Michael Hudson)
Date: Tue May 18 07:34:16 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040518085733.GA14842@vicky.ecs.soton.ac.uk> (Armin Rigo's
	message of "Tue, 18 May 2004 09:57:33 +0100")
References: <20040518085733.GA14842@vicky.ecs.soton.ac.uk>
Message-ID: <2md652hsui.fsf@starship.python.net>

Armin Rigo <arigo@tunes.org> writes:

> Hello,
>
> A minor semantic change that creeped in some time ago was an implicit
> assumption that any object x should "reasonably" be expected to compare equal
> to itself.  The arguments are summarized below (should this be documented,
> inserted in NEWS, turned in a mini-PEP a posteriori ... ?).


I have to admit that I haven't done my research, but I think you have
your history backwards: the recent change AIUI is that it's possible
that 'x == x' might be something *other* than Py_True...

Cheers,
mwh

-- 
  The ability to quote is a serviceable substitute for wit.
                                                -- W. Somerset Maugham

From arigo at tunes.org  Tue May 18 11:36:07 2004
From: arigo at tunes.org (Armin Rigo)
Date: Tue May 18 11:39:08 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <2md652hsui.fsf@starship.python.net>
References: <20040518085733.GA14842@vicky.ecs.soton.ac.uk>
	<2md652hsui.fsf@starship.python.net>
Message-ID: <20040518153607.GA10557@vicky.ecs.soton.ac.uk>

Hello Michael,

On Tue, May 18, 2004 at 12:34:13PM +0100, Michael Hudson wrote:
> > A minor semantic change that creeped in some time ago was an implicit
> > assumption that any object x should "reasonably" be expected to compare equal
> > to itself.
> 
> I have to admit that I haven't done my research, but I think you have
> your history backwards: the recent change AIUI is that it's possible
> that 'x == x' might be something *other* than Py_True...

Uh?  You're probably referring to a much older change.  It was possible from
at least Python 2.1, and I don't know if it has ever been impossible.  I am
speaking about these two changes in object.c:

----------------------------
revision 2.215
date: 2004/03/21 17:01:44;  author: rhettinger;  state: Exp;  lines: +11 -1
Add identity shortcut to PyObject_RichCompareBool.
----------------------------

and:

----------------------------
revision 2.211
date: 2003/10/28 12:05:47;  author: arigo;  state: Exp;  lines: +8 -180
Deleting cyclic object comparison.
SF patch 825639
http://mail.python.org/pipermail/python-dev/2003-October/039445.html
----------------------------


Armin

From arigo at tunes.org  Tue May 18 12:10:22 2004
From: arigo at tunes.org (Armin Rigo)
Date: Tue May 18 12:13:20 2004
Subject: [Python-Dev] Is core dump always a bug? Advice requested
In-Reply-To: <200405131651.i4DGpsl21234@guido.python.org>
References: <200405121531.43532.michel@dialnetwork.com>
	<20040513085401.GA28305@vicky.ecs.soton.ac.uk>
	<200405131651.i4DGpsl21234@guido.python.org>
Message-ID: <20040518161022.GB17637@vicky.ecs.soton.ac.uk>

Hello Guido,

On Thu, May 13, 2004 at 09:51:54AM -0700, Guido van Rossum wrote:
> > * 'finally' blocks are executed with one, two or three values pushed
> >   on the stack; END_FINALLY will accordingly pop one, two or three
> >   values (depending on what the first popped value is).
> 
> This is easily fixed, if necessary -- but is it necessary?

It is a minor inconvenience for analysis tools.  They more or less have to
consider that within a finally block the structure of the stack is initially
unknown (but contains at most N values, for some N) and that no opcode is
allowed to peek or modify this unknown portion of the stack until END_FINALLY
is reached.

Psyco is an example of tool which assumes that the stack depth is only
dependent on the bytecode position, and has a couple of hacks to deal with
finally blocks.

> > * POP_BLOCK will empty the value stack up to the position stored in
> >   the block stack's popped block, for no reason that I am aware of.
> 
> I can't remember what my intention was; I probably wasn't so sure that
> the stack level at the end of executing arbitrary code was always
> the same.

I guess it should be reviewed and removed.

> > ...  A better 'fix' would be that the block
> > stack could, as we discussed at PyCon, be replaced by a purely
> > static structure in the code object.
> 
> Correct, I would like to see this happen.  There's dynamic stuff here
> just to make the implementation easy rather than to support necessary
> dynamism.  (The same could be said of some other parts of Python, but
> there we've come to love the dynamism -- for the most part anyway. ;-)

I'd like to give it a try, but I'd like to see if someone has any idea about
the kind of structure that we are looking for here.  I was roughly thinking
about a binary tree, but we need something more compact than a naive tree.


Armin

From niemeyer at conectiva.com  Tue May 18 13:18:43 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Tue May 18 13:21:19 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040518085733.GA14842@vicky.ecs.soton.ac.uk>
References: <20040518085733.GA14842@vicky.ecs.soton.ac.uk>
Message-ID: <20040518171843.GA12816@burma.localdomain>

> >>> cmp(x,x)           #   x.__cmp__() is NOT called
> 0

This is odd. We'll have problems trying to implemnt a NaN
following the convention of being not-equal to itself,
for example.

> >>> [x] == [x]         #   x.__cmp__() is NOT called
> True

This is correct, IMO.

-- 
Gustavo Niemeyer
http://niemeyer.net

From jim.jewett at eds.com  Tue May 18 13:24:56 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Tue May 18 13:26:06 2004
Subject: [Python-Dev] cmp(x,x)
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D4BF@usahm010.exmi01.exch.eds.com>

Armin Rigo:

	say x has a method __cmp__() that always 
	returns -1.

	>>> x == x             #   x.__cmp__() is called
	False
	>>> x < x              #   x.__cmp__() is called
	True
	>>> cmp(x,x)           #   x.__cmp__() is NOT called
	0
	>>> [x] == [x]         #   x.__cmp__() is NOT called
	True

Note that the last (and only the last) is a change from 2.3
In 2.3, the (identity does not imply equality) trait rolls up 
to containers.

>>> l=[x]
>>> l == l
False

	The only way to explain the semantic is that the expression 
	'x == x' always call the special methods, but built-in functions 
	are allowed to assume that any object compares equal to itself.  

Note that "allowed" is as strong as you can get; is implies ==, but 
does not (currently) imply <= or preclude <.

	Should any of this be documented?

I think so; it is a problem only for pathological cases, but who would 
bother to look up rich comparison in the first place unless they were 
considering at least borderline strangeness.

	An alternative behavior would have been to leave 
	PyObject_RichCompareBool() alone and only insert the short-cut 
	on specific object types' comparison methods

I think it is reasonable for the default to be that identity implies
equality.  Either an explicitly defined __cmp__ should always be
called, or the notes should say that containers can treat identity
as good enough.  

	This is sane as long as no code considers 'nan' as a singleton, 
	or tries to reuse 'nan' objects for different 'nan' values.

Note that float('nan') raises on ValueError on at least the default 
windows 2.3 build; if a NaN is hard to get, I would expect it to be 
reused.

-jJ

From aahz at pythoncraft.com  Tue May 18 13:36:41 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue May 18 13:36:48 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: <40A93E6D.8010605@egenix.com>
References: <5.2.1.1.0.20040517171026.02ff4198@pop.bluewin.ch>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<5.2.1.1.0.20040517171026.02ff4198@pop.bluewin.ch>
	<5.1.1.6.0.20040517114707.02716660@mail.telecommunity.com>
	<40A93E6D.8010605@egenix.com>
Message-ID: <20040518173641.GA28095@panix.com>

On Tue, May 18, 2004, M.-A. Lemburg wrote:
>
> BTW, what are you going to do about "from . import *" and
> "from .. import *" ?

Those work exactly the same way they currently do: find the appropriate
namespace and import all available names.  It will raise some kind of
``ImportError`` if it goes outside the package, though (note added to
PEP).  I'll be adding some verbage about ``__import__()``; thanks for
raising the issue.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Adopt A Process -- stop killing all your children!

From aahz at pythoncraft.com  Tue May 18 14:01:53 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue May 18 14:02:34 2004
Subject: [Python-Dev] Re: .pth files in current or script directory
In-Reply-To: <A4A1C3B2-A72D-11D8-9743-000A27E3999C@spam-trap.richardcooper.net>
References: <0AA408F4E902C84C8F6BE093CCE9D7A88C2468@server1.artsalliance.co.uk>
	<A4A1C3B2-A72D-11D8-9743-000A27E3999C@spam-trap.richardcooper.net>
Message-ID: <20040518180153.GA2877@panix.com>

On Sun, May 16, 2004, Richard Cooper wrote:
>
> My initial idea was something along the lines of:
> 
> When a module is imported, its import search path includes everything it
> does now PLUS the directory that is the top of the module's enclosing
> package structure. That way you wouldn't have to add entries to
> PYTHONPATH just so that modules could import (absolutely) from their own
> package.

What's wrong with PEP 328?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Adopt A Process -- stop killing all your children!

From tim.one at comcast.net  Tue May 18 14:37:31 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue May 18 14:37:39 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040518171843.GA12816@burma.localdomain>
Message-ID: <mailman.37.1084905459.6944.python-dev@python.org>

> >>> cmp(x,x)           #   x.__cmp__() is NOT called
> 0

[Gustavo Niemeyer]
> This is odd. We'll have problems trying to implemnt a NaN
> following the convention of being not-equal to itself,
> for example.

That's what rich comparisons are for.  There's no problem getting

>>> x = some_nan
>>> x = x
False
>>>

in Python 2.4 (the result of that is *currently* a platform-dependent
crapshoot, though).

cmp(x, y) doesn't make sense for objects having only a partial ordering, so
I don't care cmp() does with a NaN.



From bac at OCF.Berkeley.EDU  Tue May 18 14:43:47 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue May 18 14:44:03 2004
Subject: [Python-Dev] Need traceback expert to review patch 954922
In-Reply-To: <20040518090504.GB14842@vicky.ecs.soton.ac.uk>
References: <40A94622.6040606@ocf.berkeley.edu>	<200405180233.i4I2XgqL012607@cosc353.cosc.canterbury.ac.nz>
	<20040518090504.GB14842@vicky.ecs.soton.ac.uk>
Message-ID: <40AA5963.80500@ocf.berkeley.edu>

Armin Rigo wrote:
> Hello Greg,
> 
> On Tue, May 18, 2004 at 02:33:42PM +1200, Greg Ewing wrote:
> 
>>Now that we have a garbage collector, is it really necessary
>>any more? What would happen if we simply set sys.modules to
>>None and then invoked the collector?
> 
> 
> There is a patch on SF that does this.  Brett, can you see if this patch helps
> in your case?
> 
>     http://www.python.org/sf/812369
> 

It segfaulted when it normally spits out a bunch of tracebacks about 
sys.stderr being None from the interpreter tearing itself down.

-Brett

From bac at OCF.Berkeley.EDU  Tue May 18 14:48:16 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue May 18 14:48:38 2004
Subject: [Python-Dev] Need traceback expert to review patch 954922
In-Reply-To: <40AA17DF.5050403@interlink.com.au>
References: <40A94622.6040606@ocf.berkeley.edu>
	<40AA17DF.5050403@interlink.com.au>
Message-ID: <40AA5A70.8040404@ocf.berkeley.edu>

Anthony Baxter wrote:

> Brett C. wrote:
> 
>> And the reason I bring this up now is just in case there is any desire 
>> to put this in 2.3.4 .  I obviously know that I missed rc1, but I 
>> didn't get the details of triggering the problem soon enough and 
>> people running 2.3.x are the ones bringing this up since this 
>> apparently did not happen under 2.2 for some reason.  I doubt Anthony 
>> wants to risk mucking with 'threading' this late (although the patch 
>> is fairly self-contained but would require applying the previous patch 
>> to attempt to fix this as well), but I thought I would at least put it 
>> out there.
> 
> 
> Not going to be there for 2.3.4.
> 

Figured as much.

> My approach to the bug fix releases is that, in general, "there's
> always a next release". That is, the bug fix releases are relatively
> frequent (5-6 months) and they're "what's in the branch at the moment".
> 

OK.  How long do you plan to do this for the 2.3 branch?

> Starting with the next (2.3.5/2.4.1) releases, I plan on sending out
> a 1 month warning to python-list/python-dev, so that people that care
> can make sure their pet bug fix is checked in and done before the
> release. I'm really not interested in trying to slot in last minute
> changes that may, or may not, break the release.
> 

Makes sense to me.

> In case it wasn't clear, I'm trying to get Python into a nice regular
> release cycle for the bugfix releases. While there will be the
> occasional "oh. my. god." type bug that drive a bug fix release
> now now now, I'd prefer that people can get used to a regular
> cycle for the bug fix releases. Part of this, of course, is being
> harsh on the no-new-features rule. I want people to feel confident
> that a bug fix release won't break their installed code base.

Getting into a regular groove for releases sounds great to me, 
especially if we can state confidently that a bugfix release will be 
along every 6 months (+/- a month).

-Brett

From arigo at tunes.org  Tue May 18 16:04:51 2004
From: arigo at tunes.org (Armin Rigo)
Date: Tue May 18 16:07:58 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040518171843.GA12816@burma.localdomain>
References: <20040518085733.GA14842@vicky.ecs.soton.ac.uk>
	<20040518171843.GA12816@burma.localdomain>
Message-ID: <20040518200451.GA4807@vicky.ecs.soton.ac.uk>

Hello Gustavo,

On Tue, May 18, 2004 at 02:18:43PM -0300, Gustavo Niemeyer wrote:
> This is odd. We'll have problems trying to implemnt a NaN
> following the convention of being not-equal to itself,
> for example.

I know the issue is a bit obfuscated, but please do read my mail entierely 
first: this is exactly my example :-)


Armin

From aahz at pythoncraft.com  Tue May 18 16:08:29 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue May 18 16:08:33 2004
Subject: [Python-Dev] Bugfix cycle
In-Reply-To: <40AA5A70.8040404@ocf.berkeley.edu>
References: <40A94622.6040606@ocf.berkeley.edu>
	<40AA17DF.5050403@interlink.com.au>
	<40AA5A70.8040404@ocf.berkeley.edu>
Message-ID: <20040518200828.GA14340@panix.com>

> Anthony Baxter wrote:
>>
>>Starting with the next (2.3.5/2.4.1) releases, I plan on sending out
>>a 1 month warning to python-list/python-dev, so that people that care
>>can make sure their pet bug fix is checked in and done before the
>>release. I'm really not interested in trying to slot in last minute
>>changes that may, or may not, break the release.

Cool!  That'll make some people happier.  I'm tempted to suggest adding
c.l.py.announce, but on second thought, anyone who's not paying
reasonably close attention to at least one of python-list/python-dev
probably doesn't have enough attention to properly champion a patch for
a bugfix release.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Adopt A Process -- stop killing all your children!

From niemeyer at conectiva.com  Tue May 18 19:32:00 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Tue May 18 19:31:32 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040518200451.GA4807@vicky.ecs.soton.ac.uk>
References: <20040518085733.GA14842@vicky.ecs.soton.ac.uk>
Message-ID: <20040518233200.GA15700@burma.localdomain>

> > This is odd. We'll have problems trying to implemnt a NaN
> > following the convention of being not-equal to itself,
> > for example.
> 
> I know the issue is a bit obfuscated, but please do read my mail
> entierely first: this is exactly my example :-)

Sorry.. I read your mail in a rush, and the main issue looked
to be related to these examples. Here we go again.

[...]
> >>> f = float('nan')
> >>> cmp(f,f)
> 0                    # because f is f
> >>> f == f
> False                # because float.__eq__() is called

I don't get the same results:

>>> f = float('nan')
>>> cmp(f,f)
0
>>> f == f
True

Also on a Linux box.

> Note that as discussed below the following behavior is *expected* and in
> accordance with standards:
> 
> >>> float('nan') is float('nan')
> False

I belive that testing identity on NaN might return True in cases like:

>>> f = float('nan')
>>> f is f
True

> >>> float('nan') == float('nan')
> False                             # not the same object

This is expected in an IEEE754 compliant implementation. Something
we currently are not (I get different results, as shown).

> Unless there are serious objections I suggest to (i.e. I plan to) remove the
> short-cut in PyObject_RichCompareBool() -- performance is probably not an
> issue here -- and then review all built-in comparison methods and make sure
> that they return "equal" for identical objects.

+1

-- 
Gustavo Niemeyer
http://niemeyer.net

From greg at cosc.canterbury.ac.nz  Tue May 18 21:39:04 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue May 18 21:39:40 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040518085733.GA14842@vicky.ecs.soton.ac.uk>
Message-ID: <200405190139.i4J1d40t014215@cosc353.cosc.canterbury.ac.nz>

Armin Rigo <arigo@tunes.org>:

> Unless there are serious objections I suggest to (i.e. I plan to) remove the
> short-cut in PyObject_RichCompareBool() -- performance is probably not an
> issue here -- and then review all built-in comparison methods and make sure
> that they return "equal" for identical objects.

Sounds right to me.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Tue May 18 22:38:02 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue May 18 22:38:32 2004
Subject: [Python-Dev] Need traceback expert to review patch 954922
In-Reply-To: <20040518090504.GB14842@vicky.ecs.soton.ac.uk>
Message-ID: <200405190238.i4J2c2aO014302@cosc353.cosc.canterbury.ac.nz>

Armin Rigo <arigo@tunes.org>:

> I bet the goal is that if Python is embedded, and initialized and shut down
> repeatedly, it should not leak memory.

I think that's going to be impossible to achieve as long
as there is no protocol for shutting down extension modules,
which may be holding references to things in C globals.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From tim.one at comcast.net  Tue May 18 22:44:05 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue May 18 22:44:12 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040518153607.GA10557@vicky.ecs.soton.ac.uk>
Message-ID: <mailman.41.1084934652.6944.python-dev@python.org>

[Armin Rigo]
>>> A minor semantic change that creeped in some time ago was an implicit
>>> assumption that any object x should "reasonably" be expected to
>>> compare equal to itself.

[Michael Hudson]
>> I have to admit that I haven't done my research, but I think you have
>> your history backwards: the recent change AIUI is that it's possible
>> that 'x == x' might be something *other* than Py_True...

[Armin]
> Uh?  You're probably referring to a much older change.  It was possible
> from at least Python 2.1, and I don't know if it has ever been
> impossible.

So-called "rich comparisons" were new in 2.1.  Before then, all comparisons,
however spelled, ended up invoking a spelling of cmp(), and
PyObject_Compare() has had its current

	if (v == w)
		return 0;

special case ever since revison 1.1 of object.c (that was about 14(!) years
ago now).

So, yes, before Python 2.1, it wasn't possible for x == x (or x <= x or x >=
x) to return false, or for x < x, x > x, or x != x to return true, or for
cmp(x, x) to return anything other than 0.  Rich comparisons changed all
that excepting the last.  At first in 2.1, and through all released Pythons
since then, the "v==w implies equal" special case has remained in
PyObject_Compare, but was left out of implementations of the 6 "new",
true/false relational operators.

> I am speaking about these two changes in object.c:
> ----------------------------
> revision 2.215
> date: 2004/03/21 17:01:44;  author: rhettinger;  state: Exp;  lines: +11 -
> 1
> Add identity shortcut to PyObject_RichCompareBool.
> ----------------------------

Raymond added that after you, Guido and I agreed to it at the PyCon sprints
in March.  It was specifically to ease your troubles with comparing
recursive dictionaries, which started after the hideous cyclic-compare hacks
were removed.  As CVS records, removing them was also your fault <wink>:

> and:
> 
> ----------------------------
> revision 2.211
> date: 2003/10/28 12:05:47;  author: arigo;  state: Exp;  lines: +8 -180
> Deleting cyclic object comparison.
> SF patch 825639
> http://mail.python.org/pipermail/python-dev/2003-October/039445.html



From tim.one at comcast.net  Tue May 18 23:24:50 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue May 18 23:24:51 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040518085733.GA14842@vicky.ecs.soton.ac.uk>
Message-ID: <mailman.42.1084937091.6944.python-dev@python.org>

[Armin Rigo]
> ...
> Note that as discussed below the following behavior is *expected* and in
> accordance with standards:
>
> >>> float('nan') is float('nan')
> False

That result simply isn't defined, even on platforms where float('nan')
doesn't blow up:  floats are immutable objects, and outside of interning
strings, Python doesn't define anything about the identity of immutable
objects returned from computation.

> >>> float('nan') == float('nan')
> False                             # not the same object

The one is a platform-dependent crapshoot, and again even on platforms where
float('nan') doesn't blow up.  In current CVS HEAD, Michael made it much
more *likely* to return False than in any Python released to date, but
there's still no guarantee.  For example, even with current CVS, it returns
True if compiled with MSVC 6, but False if compiled with MSVC 7.1.  Python
simply doesn't define anything about behavior in the presence of NaNs,
infinities or signed zeroes (neither as floats nor as string representations
(like 'nan')).

> Unless there are serious objections I suggest to (i.e. I plan to) remove
> the short-cut in PyObject_RichCompareBool() -- performance is probably
> not an issue here -- and then review all built-in comparison methods and
> make sure that they return "equal" for identical objects.

If your recursive dict compares work OK then, fine by me.  The exact details
of comparison semantics have always been muddy, and I expect always will be
(the combination of backward compatibility hacks and new features makes
comparison reality as complicated as the code implementing it -- that's some
of the most complicated stuff in the entire codebase).



From anthony at interlink.com.au  Tue May 18 10:04:15 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed May 19 04:02:01 2004
Subject: [Python-Dev] Need traceback expert to review patch 954922
In-Reply-To: <40A94622.6040606@ocf.berkeley.edu>
References: <40A94622.6040606@ocf.berkeley.edu>
Message-ID: <40AA17DF.5050403@interlink.com.au>

Brett C. wrote:
> And the reason I bring this up now is just in case there is any desire 
> to put this in 2.3.4 .  I obviously know that I missed rc1, but I didn't 
> get the details of triggering the problem soon enough and people running 
> 2.3.x are the ones bringing this up since this apparently did not happen 
> under 2.2 for some reason.  I doubt Anthony wants to risk mucking with 
> 'threading' this late (although the patch is fairly self-contained but 
> would require applying the previous patch to attempt to fix this as 
> well), but I thought I would at least put it out there.

Not going to be there for 2.3.4.

My approach to the bug fix releases is that, in general, "there's
always a next release". That is, the bug fix releases are relatively
frequent (5-6 months) and they're "what's in the branch at the moment".

Starting with the next (2.3.5/2.4.1) releases, I plan on sending out
a 1 month warning to python-list/python-dev, so that people that care
can make sure their pet bug fix is checked in and done before the
release. I'm really not interested in trying to slot in last minute
changes that may, or may not, break the release.

In case it wasn't clear, I'm trying to get Python into a nice regular
release cycle for the bugfix releases. While there will be the
occasional "oh. my. god." type bug that drive a bug fix release
now now now, I'd prefer that people can get used to a regular
cycle for the bug fix releases. Part of this, of course, is being
harsh on the no-new-features rule. I want people to feel confident
that a bug fix release won't break their installed code base. In
addition, I never ever want to see

try:
     True, False
except NameError:
     True, False = 1, 0

or it's like ever again. I've seen it far far too often in code that
wants to be "2.2" compatible, but the new True/False in the bugfix
release of 2.2 made life unpleasant.

I'm open to suggestions for this - although emails about "but I really
really want SF bug #XYZ fixed for 2.3.4" is not what I'm after.

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From theller at python.net  Wed May 19 04:44:04 2004
From: theller at python.net (Thomas Heller)
Date: Wed May 19 04:44:21 2004
Subject: [Python-Dev] windist.exe status and license
In-Reply-To: <16552.21927.581799.337029@gargle.gargle.HOWL> (Matthias
	Klose's message of "Mon, 17 May 2004 08:03:19 +0200")
References: <1084541142.696332.6706.nullmailer@calvin.fayauffre.org>
	<16552.21927.581799.337029@gargle.gargle.HOWL>
Message-ID: <k6z8u7qj.fsf@python.net>

Matthias Klose <doko@cs.tu-berlin.de> writes:

> [CC to python-dev]
>
> The windist.exe version shipped with current python releasese is
> outdated and does not have any license information about windist
> included.
>
> Alexandre Fayolle writes:
>> Package: python2.3-dev
>> Version: 2.3.3-7
>> Severity: normal
>> 
>> Hi,
>> 
>> I noticed the file /usr/lib/python2.3/distutils/command/wininst.exe is
>> missing from my currently installed version of the python2.3-dev
>> package. I'm pretty sure I used to be able to create windows installers
>> for the modules I develop from my Debian machine. 
>> 
>> Is this removal caused by an omission, or is there some licensing issue ? 
>> I could not find anything on that topic in the changelog, but maybe I
>> didn't look hard enough. 
>
> I cannot find anything about the licensing. The executable only shows
>
> $Info: This file is packed with the UPX executable packer http://upx.tsx.org $
> $Id: UPX 1.01 Copyright (C) 1996-2000 the UPX Team. All Rights Reserved. $
>
> However I found http://upx.sourceforge.net/.
>
> - Maybe the binary can be updated to the SF version?
> - Add the upx license to the Lib/distutils/command directory?
>
>
> [Debian only:
>
>  The UPX license itself is discussed at
>  http://lists.debian.org/debian-legal/2001/01/msg00025.html
>
>  upx-nrv is currently in section non-free/utils, so it's better
>  not to distribute it in the Debian package.
> ]

Does this/should this affect the official 2.3.4 release, or is this only
for debian packaging?

upx is licensed GPL, with this exception which applies (imo) to the
wininst.exe binary:

<quote>
SPECIAL EXCEPTION FOR COMPRESSED EXECUTABLES
============================================

   The stub which is imbedded in each UPX compressed program is part
   of UPX and UCL, and contains code that is under our copyright. The
   terms of the GNU General Public License still apply as compressing
   a program is a special form of linking with our stub.

   Hereby Markus F.X.J. Oberhumer and Laszlo Molnar grant you special
   permission to freely use and distribute all UPX compressed programs
   (including commercial ones), subject to the following restrictions:

   1. You must compress your program with a completely unmodified UPX
      version; either with our precompiled version, or (at your option)
      with a self compiled version of the unmodified UPX sources as
      distributed by us.
   2. This also implies that the UPX stub must be completely unmodfied, i.e.
      the stub imbedded in your compressed program must be byte-identical
      to the stub that is produced by the official unmodified UPX version.
   3. The decompressor and any other code from the stub must exclusively get
      used by the unmodified UPX stub for decompressing your program at
      program startup. No portion of the stub may get read, copied,
      called or otherwise get used or accessed by your program.
</quote>

Especially 3. puts restrictions on the binary. The full upx license is
here: <http://upx.sourceforge.net/upx-license.html>

Hm, maybe it would be better to *not* compress wininst.exe by upx any
longer. The uncompressed exe is 57344 bytes, the compressed on 21504
bytes.  Any bdist_wininst installer would be 36 kB larger, which isn't a
large deal.

Thomas


From theller at python.net  Wed May 19 05:04:27 2004
From: theller at python.net (Thomas Heller)
Date: Wed May 19 05:04:41 2004
Subject: [Python-Dev] windist.exe status and license
In-Reply-To: <k6z8u7qj.fsf@python.net> (Thomas Heller's message of "Wed, 19
	May 2004 10:44:04 +0200")
References: <1084541142.696332.6706.nullmailer@calvin.fayauffre.org>
	<16552.21927.581799.337029@gargle.gargle.HOWL>
	<k6z8u7qj.fsf@python.net>
Message-ID: <ekpgu6sk.fsf@python.net>

(followup to myself)

> Hm, maybe it would be better to *not* compress wininst.exe by upx any
> longer. The uncompressed exe is 57344 bytes, the compressed on 21504
> bytes.  Any bdist_wininst installer would be 36 kB larger, which isn't a
> large deal.

After 5 minutes, it became crystal clear to me that this is the obvious
solution.  Who cares about 36 kB, and that's probably less than the size
of the GPL.

I'll change this for the 2.3.4 release.

Thomas


From mwh at python.net  Wed May 19 05:11:25 2004
From: mwh at python.net (Michael Hudson)
Date: Wed May 19 05:11:33 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040518233200.GA15700@burma.localdomain> (Gustavo Niemeyer's
	message of "Tue, 18 May 2004 20:32:00 -0300")
References: <20040518085733.GA14842@vicky.ecs.soton.ac.uk>
	<20040518233200.GA15700@burma.localdomain>
Message-ID: <2mwu38hjcy.fsf@starship.python.net>

Gustavo Niemeyer <niemeyer@conectiva.com> writes:

> This is expected in an IEEE754 compliant implementation.

What's one of them? <wink>

> Something we currently are not (I get different results, as shown).

Are you using 2.3 or 2.4 here?

Cheers,
mwh

-- 
  surely, somewhere, somehow, in the history of computing, at least
  one manual has been written that you could at least remotely
  attempt to consider possibly glancing at.              -- Adam Rixey

From anthony at interlink.com.au  Wed May 19 05:59:55 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed May 19 06:00:14 2004
Subject: [Python-Dev] Need traceback expert to review patch 954922
In-Reply-To: <40AA5A70.8040404@ocf.berkeley.edu>
References: <40A94622.6040606@ocf.berkeley.edu>	<40AA17DF.5050403@interlink.com.au>
	<40AA5A70.8040404@ocf.berkeley.edu>
Message-ID: <40AB301B.7000304@interlink.com.au>

Brett C. wrote:
> Anthony Baxter wrote:
>> My approach to the bug fix releases is that, in general, "there's
>> always a next release". That is, the bug fix releases are relatively
>> frequent (5-6 months) and they're "what's in the branch at the moment".
>>
> 
> OK.  How long do you plan to do this for the 2.3 branch?

I'm anticipating doing this until 2.4.1. That is, the upcoming
release train will be:

    2.3.4
    2.4a{1,2,...}
    2.4b{1,2,...}
    2.4rc{1,2,...}
    2.4
    2.3.5
    2.4.1

Unless there's a pressing need for it, I don't see the point to
cutting a 2.3.6 or later, once 2.4 is onto the release24-maint
branch. Obviously, if there's one of the oh-my-gods type bugs in
2.3, this will probably require a 2.3.6. Hopefully, by the time
I finish the 2.4 cycle, there'll be some sane release automation
tools finished, too.

Once we've forked release24-maint, release23-maint can start to
die. There's no point backporting patches to two separate -maint
branches.


> Getting into a regular groove for releases sounds great to me, 
> especially if we can state confidently that a bugfix release will be 
> along every 6 months (+/- a month).

Yep. I plan to update the bug fix release PEP and post it out to
c.l.py and python-dev for feedback.

As ever, I'm open to feedback on any of the above. Except for the
"can we please have this whizzo feature in the next bug fix release"
feedback. You know the answer to that one ;)


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From arigo at tunes.org  Wed May 19 06:28:20 2004
From: arigo at tunes.org (Armin Rigo)
Date: Wed May 19 06:31:19 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040519032538.5EB062F33B@bespin.org>
References: <20040518085733.GA14842@vicky.ecs.soton.ac.uk>
	<20040519032538.5EB062F33B@bespin.org>
Message-ID: <20040519102820.GA7677@vicky.ecs.soton.ac.uk>

Hello Tim,

On Tue, May 18, 2004 at 11:24:50PM -0400, Tim Peters wrote:
> > >>> float('nan') is float('nan')
> > False
> 
> That result simply isn't defined, even on platforms where float('nan')
> doesn't blow up:  floats are immutable objects, and outside of interning
> strings, Python doesn't define anything about the identity of immutable
> objects returned from computation.

Ok, I won't go into that argument again then.

Raymond told me that the PyObject_RichCompareBool() short-cut is essential for
performance, as a very large number of internal comparisons end up comparing
identical objects (e.g. with dicts).

I don't really care about what occurs with float('nan'), but it seems
marginally strange that the outcome could, with some compilers, depend on the
way you compare them: even though x == x and cmp(x,x) are known to do
different things, as rich comparison goes, you wouldn't expect this to show up
with floats.  But anyway that's fine by me (and could be easily fixed by
patching float_compare and float_richcompare to check for identity).

A slightly stranger irregularity (though not a problem in general) is that
comparing two objects of user-defined classes may invoke neither the __cmp__
nor the __eq__ special method at all.  It should at least be documented
somewhere, in my opinion.  The language reference currently explicitely states
that x==y calls x.__eq__(y) and falls back to y.__eq__(x).


Armin

From niemeyer at conectiva.com  Wed May 19 09:38:12 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Wed May 19 09:37:42 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <2mwu38hjcy.fsf@starship.python.net>
References: <20040518085733.GA14842@vicky.ecs.soton.ac.uk>
	<20040518233200.GA15700@burma.localdomain>
	<2mwu38hjcy.fsf@starship.python.net>
Message-ID: <20040519133812.GA12100@burma.localdomain>

> > This is expected in an IEEE754 compliant implementation.
> 
> What's one of them? <wink>

I can tell you.. or I'd have to kill you. ;-)

> > Something we currently are not (I get different results, as shown).
> 
> Are you using 2.3 or 2.4 here?

2.3.. I wasn't aware about your recent change. Are you planning
to include C99 stuff as well?

-- 
Gustavo Niemeyer
http://niemeyer.net

From mwh at python.net  Wed May 19 09:44:08 2004
From: mwh at python.net (Michael Hudson)
Date: Wed May 19 09:44:13 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040519133812.GA12100@burma.localdomain> (Gustavo Niemeyer's
	message of "Wed, 19 May 2004 10:38:12 -0300")
References: <20040518085733.GA14842@vicky.ecs.soton.ac.uk>
	<20040518233200.GA15700@burma.localdomain>
	<2mwu38hjcy.fsf@starship.python.net>
	<20040519133812.GA12100@burma.localdomain>
Message-ID: <2mpt90h6qf.fsf@starship.python.net>

Gustavo Niemeyer <niemeyer@conectiva.com> writes:

>> > This is expected in an IEEE754 compliant implementation.
>> 
>> What's one of them? <wink>
>
> I can tell you.. or I'd have to kill you. ;-)
>
>> > Something we currently are not (I get different results, as shown).
>> 
>> Are you using 2.3 or 2.4 here?
>
> 2.3.. I wasn't aware about your recent change. Are you planning
> to include C99 stuff as well?

No.  My recent change was mainly motivated by the wart that in Python
2.3 on linux float('nan') == x came out True for any float x at all!

I'm not enough of a numerics guy to know what people who would like
real 754 support would actually want.

Cheers,
mwh

-- 
  Finding a needle in a haystack is a lot easier if you burn down
  the haystack and scan the ashes with a metal detector.
      -- the Silicon Valley Tarot (another one nicked from David Rush)

From bac at OCF.Berkeley.EDU  Wed May 19 14:17:41 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed May 19 14:18:01 2004
Subject: [Python-Dev] Need traceback expert to review patch 954922
In-Reply-To: <40AB301B.7000304@interlink.com.au>
References: <40A94622.6040606@ocf.berkeley.edu>	<40AA17DF.5050403@interlink.com.au>
	<40AA5A70.8040404@ocf.berkeley.edu>
	<40AB301B.7000304@interlink.com.au>
Message-ID: <40ABA4C5.6070300@ocf.berkeley.edu>

Anthony Baxter wrote:

> Brett C. wrote:
> 
>> Anthony Baxter wrote:
>>
>>> My approach to the bug fix releases is that, in general, "there's
>>> always a next release". That is, the bug fix releases are relatively
>>> frequent (5-6 months) and they're "what's in the branch at the moment".
>>>
>>
>> OK.  How long do you plan to do this for the 2.3 branch?
> 
> 
> I'm anticipating doing this until 2.4.1. That is, the upcoming
> release train will be:
> 
>    2.3.4
>    2.4a{1,2,...}
>    2.4b{1,2,...}
>    2.4rc{1,2,...}
>    2.4
>    2.3.5
>    2.4.1
> 
> Unless there's a pressing need for it, I don't see the point to
> cutting a 2.3.6 or later, once 2.4 is onto the release24-maint
> branch. Obviously, if there's one of the oh-my-gods type bugs in
> 2.3, this will probably require a 2.3.6. Hopefully, by the time
> I finish the 2.4 cycle, there'll be some sane release automation
> tools finished, too.
> 

Anything us normal folk can help you with in terms of the aforementioned 
tools?

Maybe this could encompass a tool to run the testing suite, re-run the 
failed tests with verbose output, and then email them to a Mailman list 
with system info so we have a partially automated testing framework for 
alphas, betas, and release candidates (Christopher Blunck proposed this 
idea back in March; email at 
http://mail.python.org/pipermail/python-dev/2004-March/043492.html) 
since we currently don't have a testing server farm running regularly 
anymore?


-Brett

From bh at intevation.de  Wed May 19 14:53:21 2004
From: bh at intevation.de (Bernhard Herzog)
Date: Wed May 19 14:53:26 2004
Subject: [Python-Dev] Relative vs. absolute imports
References: <5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<s9zfz9zi0zl.fsf@salmakis.intevation.de>
	<200405171452.i4HEqVp03065@guido.python.org>
	<s9zsmdzgj0h.fsf@salmakis.intevation.de>
	<200405180350.i4I3oBD03973@guido.python.org>
Message-ID: <s9zpt90s0ym.fsf@salmakis.intevation.de>

Guido van Rossum <guido@python.org> writes:

>> > Because in the current situation the meaning of an import is
>> > ambiguous.
>> 
>> What exactly is ambiguous about it?
>
> Looking at "import foo" you can't tell whether there's a module foo
> expected in the current package or in the top-level module namespace.

Well, looking at "print foo" I can't tell whether its a local variable,
a global variable or a builtin either.  Nobody's going to suggest we
should introduce special syntax to distinguish them, I hope.  Module
import isn't much different and it's only a little harder to figure out
whether its a relative import or not, and you only have to distinguish
them when the importer is a module in a package. 

I've been using packages in the current form ever since they've been
introduced in Python 1.5.  In all that time, relative imports have never
been a problem for me, IIRC, and I use them for all sibling imports.

Absolute imports have their own difficulties, too, btw, and relative
imports don't add much complexity on top of that.  I don't think the
ambiguity, if you want to call it that, is enough of a problem to
justify the incompatibility.

   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Skencil                                http://sketch.sourceforge.net/
Thuban                                  http://thuban.intevation.org/

From pje at telecommunity.com  Wed May 19 15:21:32 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed May 19 15:21:25 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: <s9zpt90s0ym.fsf@salmakis.intevation.de>
References: <5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<s9zfz9zi0zl.fsf@salmakis.intevation.de>
	<200405171452.i4HEqVp03065@guido.python.org>
	<s9zsmdzgj0h.fsf@salmakis.intevation.de>
	<200405180350.i4I3oBD03973@guido.python.org>
Message-ID: <5.1.1.6.0.20040519151756.037f6b50@mail.telecommunity.com>

At 08:53 PM 5/19/04 +0200, Bernhard Herzog wrote:
>Guido van Rossum <guido@python.org> writes:
>
> >> > Because in the current situation the meaning of an import is
> >> > ambiguous.
> >>
> >> What exactly is ambiguous about it?
> >
> > Looking at "import foo" you can't tell whether there's a module foo
> > expected in the current package or in the top-level module namespace.
>
>Well, looking at "print foo" I can't tell whether its a local variable,
>a global variable or a builtin either.  Nobody's going to suggest we
>should introduce special syntax to distinguish them, I hope.  Module
>import isn't much different

Yes it is, actually.  You can answer the "print foo" question simply by 
inspecting the code of the function that contains it.  (And if it's not in 
a function, you know it's a global, unless it's never assigned, in which 
case it's a builtin.)

But the "import foo" question cannot be answered by referring to the source 
of the module in which it occurs.


>Absolute imports have their own difficulties, too, btw, and relative
>imports don't add much complexity on top of that.  I don't think the
>ambiguity, if you want to call it that, is enough of a problem to
>justify the incompatibility.

IIRC, nobody has suggested eliminating them prior to 3.0, which will have 
lots of other backward-incompatible changes anyway.


From aahz at pythoncraft.com  Wed May 19 15:32:05 2004
From: aahz at pythoncraft.com (Aahz)
Date: Wed May 19 15:33:26 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: <5.1.1.6.0.20040519151756.037f6b50@mail.telecommunity.com>
References: <5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<s9zfz9zi0zl.fsf@salmakis.intevation.de>
	<200405171452.i4HEqVp03065@guido.python.org>
	<s9zsmdzgj0h.fsf@salmakis.intevation.de>
	<200405180350.i4I3oBD03973@guido.python.org>
	<5.1.1.6.0.20040519151756.037f6b50@mail.telecommunity.com>
Message-ID: <20040519193204.GA7731@panix.com>

On Wed, May 19, 2004, Phillip J. Eby wrote:
> At 08:53 PM 5/19/04 +0200, Bernhard Herzog wrote:
>>
>>Absolute imports have their own difficulties, too, btw, and relative
>>imports don't add much complexity on top of that.  I don't think the
>>ambiguity, if you want to call it that, is enough of a problem to
>>justify the incompatibility.
> 
> IIRC, nobody has suggested eliminating them prior to 3.0, which will have 
> lots of other backward-incompatible changes anyway.

Actually, the current plan is to eliminate relative imports that don't
use the new syntax in Python 2.6.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Adopt A Process -- stop killing all your children!

From bh at intevation.de  Wed May 19 15:34:49 2004
From: bh at intevation.de (Bernhard Herzog)
Date: Wed May 19 15:34:55 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: <5.1.1.6.0.20040519151756.037f6b50@mail.telecommunity.com>
	(Phillip J. Eby's message of "Wed, 19 May 2004 15:21:32 -0400")
References: <5.1.1.6.0.20040517101458.0635d450@mail.telecommunity.com>
	<s9zfz9zi0zl.fsf@salmakis.intevation.de>
	<200405171452.i4HEqVp03065@guido.python.org>
	<s9zsmdzgj0h.fsf@salmakis.intevation.de>
	<200405180350.i4I3oBD03973@guido.python.org>
	<5.1.1.6.0.20040519151756.037f6b50@mail.telecommunity.com>
Message-ID: <s9z8yfoxlba.fsf@salmakis.intevation.de>

"Phillip J. Eby" <pje@telecommunity.com> writes:

> Yes it is, actually.  You can answer the "print foo" question simply by
> inspecting the code of the function that contains it.  (And if it's not
> in a function, you know it's a global, unless it's never assigned, in
> which case it's a builtin.)
>
> But the "import foo" question cannot be answered by referring to the
> source of the module in which it occurs.

Of course it's different, but it's not all that much different.  To
determine whether an import is relative or not you look into the
directory containing the module with the import statement.  Shouldn't be
more than a few keystrokes in a decent editor.

> IIRC, nobody has suggested eliminating them prior to 3.0, which will
> have lots of other backward-incompatible changes anyway.

The new import scheme will be the only one available in Python 2.6 and
later according to the PEP and the current style of relative imports
will produce DeprecationWarnings in 2.5.

   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Skencil                                http://sketch.sourceforge.net/
Thuban                                  http://thuban.intevation.org/

From ncoghlan at iinet.net.au  Wed May 19 20:22:46 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed May 19 20:22:56 2004
Subject: [Python-Dev] LC_NUMERIC, gtk and Patch 774665
Message-ID: <40ABFA56.9050805@iinet.net.au>

A problem was recently reported on wxPython-dev that smells strongly of 
this issue with GTK changing the LC_NUMERIC locale to something other 
than 'C' causing subsequent severe problems in the handling of floats by 
the Python interpreter.

I went looking for Christian's proposed PEP (to make Python more locale 
agnostic when it comes to LC_NUMERIC), and didn't find it listed in PEP 
0. The last activity on Gustavo's patch was in December 2003, and the 
last comment in September.

What's the current state of this?

Regards,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268



From mcherm at mcherm.com  Wed May 19 20:38:43 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed May 19 20:38:47 2004
Subject: [Python-Dev] Relative vs. absolute imports
Message-ID: <1085013523.40abfe13c8346@mcherm.com>


Phillip Eby:
> Yes it is, actually.  You can answer the "print foo" question simply by
> inspecting the code of the function that contains it.  (And if it's not
> in a function, you know it's a global, unless it's never assigned, in
> which case it's a builtin.)
>
> But the "import foo" question cannot be answered by referring to the
> source of the module in which it occurs.

Bernhard Herzog:
> Of course it's different, but it's not all that much different.  To
> determine whether an import is relative or not you look into the
> directory containing the module with the import statement.  Shouldn't be
> more than a few keystrokes in a decent editor.

On the contrary, I find a big difference between something which can
be learned by inspecting the source and something which requires
access to the filesystem the source is running on. Many times that
filesystem isn't even available: "Susan emailed me this module she
uses to fnob large datasets, and I can't understand why it isn't
working for me!"

-- Michael Chermside


From ncoghlan at iinet.net.au  Wed May 19 21:48:33 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed May 19 21:48:43 2004
Subject: [Python-Dev] __str__ does not default to __repr__ if a base class
	has __str__
In-Reply-To: <200405180348.i4I3mPP03950@guido.python.org>
References: <40A8D37B.7030807@erols.com>
	<200405180348.i4I3mPP03950@guido.python.org>
Message-ID: <40AC0E71.8080003@iinet.net.au>

Guido van Rossum wrote:
>>class X(object):
>>    def __str__(self):
>>        return 'str for X'
>>
>>    def __repr__(self):
>>        return 'repr for X'
>>
>>class A(X):
>>    def __repr__(self):
>>        return 'repr for A'
>>
>>#    __str__ = __repr__
> 
> This has never been different and the current outcome is exactly what
> was intended.  The only case where __repr__ and __str__ act as each
> other's default is when there's no base class.

The OP's example also shows how easy it is to create a class which 
returns the same thing for str(foo) and repr(foo). Simply write one of 
either __repr__ or __str__ and then assign the same function to the 
other one.

Regards,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268


From anthony at interlink.com.au  Wed May 19 21:54:20 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed May 19 22:19:55 2004
Subject: [Python-Dev] python 2.3.4 delayed for a week
Message-ID: <40AC0FCC.2040705@interlink.com.au>

Due to circumstances beyond our control, Thomas is unable to
cut the windows installer for 2.3.4 for a week - we'll delay
the release until he's back on deck.
-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From flaw at rhid.com  Thu May 20 12:42:46 2004
From: flaw at rhid.com (James William Pye)
Date: Thu May 20 12:42:52 2004
Subject: [Python-Dev] Access to generator type
In-Reply-To: <20040511022708.598B0720E9E@rhid.com>
References: <20040511012304.GE29924@void.ph.cox.net>
	<20040511022708.598B0720E9E@rhid.com>
Message-ID: <20040520164246.GO62439@void.ph.cox.net>

On 05/10/04:19/1, Tim Peters wrote:
> Possibly, if someone wants it enough to make a patch (including doc
> changes).  There are many types that aren't exposed at the C level, BTW.
> Advertising their internals has an inhibiting effect on evolution.  The
> generator type has been around long enough w/o change that the value in
> keeping it private has diminished.

Okay, here's a preliminary patch with genobject.[ch]:

http://sourceforge.net/tracker/index.php?func=detail&aid=957398&group_id=5470&atid=305470

I assumed(yay) that py-dev would want it inline with how other core objects
are managed(*object.[ch]), so I spun it out into its own files in
/Object/genobject.c and /Include/genobject.h..

Of course, genobject.c references eval_frame(in ceval.c), which raises a
problem; currently it is resolved in my tree by making a wrapper in
ceval.c(PyEval_EvaluateFrame(){eval_frame()}). This is unlikely to be an
acceptable solution, and I'm not sure how py-dev wants this to be handled.

Keep it in its own genobject.[ch] and get eval_frame how?
Or keep it in ceval.[ch]?

The latter seems to be the right choice to me, but less pertty.

Regards,
        James William Pye
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 479 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040520/0898db7c/attachment.bin
From gjc at inescporto.pt  Thu May 20 12:52:47 2004
From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro)
Date: Thu May 20 12:53:08 2004
Subject: [Python-Dev] LC_NUMERIC, gtk and Patch 774665
In-Reply-To: <40ABFA56.9050805@iinet.net.au>
References: <40ABFA56.9050805@iinet.net.au>
Message-ID: <1085071967.1037.7.camel@spectrum.inescn.pt>

A Qui, 2004-05-20 ?s 01:22, Nick Coghlan escreveu:
> A problem was recently reported on wxPython-dev that smells strongly of 
> this issue with GTK changing the LC_NUMERIC locale to something other 
> than 'C' causing subsequent severe problems in the handling of floats by 
> the Python interpreter.
> 
> I went looking for Christian's proposed PEP (to make Python more locale 
> agnostic when it comes to LC_NUMERIC), and didn't find it listed in PEP 
> 0. The last activity on Gustavo's patch was in December 2003, and the 
> last comment in September.
> 
> What's the current state of this?

  AFAICT, there was interest in including this patch even without a full
PEP.  Christian is currently on vacation.  There is a unoficial PEP
draft at http://www.async.com.br/~kiko/pep-numeric.txt

  I stopped working on the patch because there was nothing more to do. 
But I'm willing to do some more effort as necessary to include this in
Python 2.4.

  Best regards.

-- 
Gustavo Jo?o Alves Marques Carneiro
<gjc@inescporto.pt> <gustavo@users.sourceforge.net>
The universe is always one step beyond logic.


From bh at intevation.de  Thu May 20 16:15:01 2004
From: bh at intevation.de (Bernhard Herzog)
Date: Thu May 20 16:15:29 2004
Subject: [Python-Dev] Relative vs. absolute imports
References: <1085013523.40abfe13c8346@mcherm.com>
Message-ID: <s9ziseq6eka.fsf@salmakis.intevation.de>

Michael Chermside <mcherm@mcherm.com> writes:

> Bernhard Herzog:
>> Of course it's different, but it's not all that much different.  To
>> determine whether an import is relative or not you look into the
>> directory containing the module with the import statement.  Shouldn't be
>> more than a few keystrokes in a decent editor.
>
> On the contrary, I find a big difference between something which can
> be learned by inspecting the source and something which requires
> access to the filesystem the source is running on. 

Me too, but, in the case of imports, regardless of whether they're
absolute or relative you will always have to refer to information ouside
of the module you're looking at.  In most cases you'll need to look at
the filesystem among other things.

However, telling relative and absolute imports apart is easy compared to
the difficulty of dealing with imports in general, just like telling
local and global variables apart is easy compared to the difficulty of
dealing with variables in general.  That's what I'm comparing.

> Many times that
> filesystem isn't even available: "Susan emailed me this module she
> uses to fnob large datasets, and I can't understand why it isn't
> working for me!"

Sure, but errors are always hard to find when you look at a piece of
code without access to the relevant context.  PEP 328 won't have much if
any impact on this problem.

   Bernhard

-- 
Intevation GmbH                                 http://intevation.de/
Skencil                                http://sketch.sourceforge.net/
Thuban                                  http://thuban.intevation.org/

From ncoghlan at iinet.net.au  Thu May 20 19:02:46 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Thu May 20 19:03:07 2004
Subject: [Python-Dev] Relative vs. absolute imports
In-Reply-To: <40A8C132.3090705@egenix.com>
References: <40A8C132.3090705@egenix.com>
Message-ID: <40AD3916.6070004@iinet.net.au>

M.-A. Lemburg wrote:
>   I don't object absolute imports at all, but I like the
>   possibilitiy to package'ify existing Python module/module
>   sets by simply copying them into a newly created package
>   directory.

Doesn't PEP 328 actually make it *easier* to 'packagise' an existing module?

If it wasn't in a package before, all of its imports *must* be absolute. 
With PEP 328, you can copy it into a package, and it will continue to 
work as before. With the current scheme, I understand that the newly 
packagised module may break if any if its new sibling modules shadow 
something in the standard library, site-packages or sys.path.

Cheers,
Nick.
Neologisms ahoy!

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268


From Mark.Lake at aigfpc.com  Thu May 20 13:31:09 2004
From: Mark.Lake at aigfpc.com (Mark.Lake@aigfpc.com)
Date: Fri May 21 10:19:27 2004
Subject: [Python-Dev] sets, scanf
Message-ID: <75ABDD2D2D569A439D18FFB90DC14945AC46CC@USEVS1.aigfpc.com>

Just a couple of modest suggestions for new features.  Nothing
earth-shattering here but I think these would give a lot of bang for the
buck.

1.  sscanf functionality: I know this has been discussed for years, but it's
time to bite the bullet and do it. 

2. Sets are a useful addition to the language, but sorted sets (and
dictionaries) would be more useful.  C++/Java/C# all provide these types. In
particular I'd like to see:

A.  Dictionaries support union, intersection, and symmetric difference
operations as sets do.
B.  (Sorted) Dictionaries and sets support slicing operations: E.g.
dict['a':'z'] should return a sub-dictionary with all the keys between 'a'
and 'z'.
C. Binary searching for items in sorted sequence and mapping types.
D. Ability to handle multiple getitems at once in sequence and mapping types
(Akin to what can be done in numarray).  Certainly there is no good reason
why lists/tuples shouldn't support list[iterable] (not to be confused with
list(iterable)) which returns or dictionaries support something like
dict.getm(iterable).

Mark Lake


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040520/cb63f0a8/attachment.html
From rledwith at cas.org  Wed May 19 10:06:16 2004
From: rledwith at cas.org (Robert Ledwith)
Date: Fri May 21 10:20:13 2004
Subject: [Python-Dev] cmp(x,x)
Message-ID: <200405191406.i4JE6GIR020652@mailhost.cas.org>

> Armin Rigo wrote:

> ...
> Unless there are serious objections I suggest to (i.e. I plan to) remove the
> short-cut in PyObject_RichCompareBool() -- performance is probably not an
> issue here -- and then review all built-in comparison methods and make sure
> that they return "equal" for identical objects.

I have a serious objection to this because it is a major performance
problem.  In my Python-2.3.3 code I had implemented an equivalent comparison
test because of a performance bottleneck.  The bottleneck involved 
looking up tuples in huge
dictionaries of tuples (millions of tuples per dictionary). 
Putting in a simple equality test reduced the run time of programs by as
much as 50%; since each program took days to run, this was a substantial
improvement for me.  I was considering posting this as a bugfix when
I discovered that it had already been incorporated into the CVS.

For the sake of people such as myself who use Python to perform large-scale
symbolic processing,
please consider other approaches to solving the NaN situation before
removing this code.

:s Robert H. Ledwith               INET: rledwith@cas.org
:s Unless explicitly stated to the contrary, nothing in this posting should
:s be construed as representing my employer's opinions.


From seba_woj at interia.pl  Fri May 21 10:36:41 2004
From: seba_woj at interia.pl (Sebastian)
Date: Fri May 21 10:36:47 2004
Subject: [Python-Dev] SystemError: com_backpatch: offset too large
Message-ID: <20040521143641.A721316341@front.interia.pl>

I've got a problem with python script.
My script return exceptinon:

SystemError: com_backpatch: offset too large

I'm beginer in python programming, and don't known what do with this message.

From andrew-pythondev at puzzling.org  Fri May 21 11:02:52 2004
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Fri May 21 11:03:13 2004
Subject: [Python-Dev] sets, scanf
In-Reply-To: <75ABDD2D2D569A439D18FFB90DC14945AC46CC@USEVS1.aigfpc.com>
References: <75ABDD2D2D569A439D18FFB90DC14945AC46CC@USEVS1.aigfpc.com>
Message-ID: <20040521150252.GK22686@frobozz>

On Thu, May 20, 2004 at 01:31:09PM -0400, Mark.Lake@aigfpc.com wrote:
[...]
> 
>    2. Sets are a useful addition to the language, but sorted sets (and
>    dictionaries) would be more useful.  C++/Java/C# all provide these types.
>    In particular I'd like to see:
> 
>    A.  Dictionaries support union, intersection, and symmetric difference
>    operations as sets do.

What's the intersection of {'a': 1} and {'a': 2}?  {'a': (1, 2)}?  What
about of {1.0: 'one point oh'} and {1: 'one'}?

Also, this would place a significant burden on people trying to implement
their own classes that present a dictionary-like interface, although perhaps
UserDict.DictMixin could help there...

A PEP for this that's more than a sentence long would probably be a good
idea :)

[...]
>    C. Binary searching for items in sorted sequence and mapping types.

See the bisect module.  I'm not sure what binary searching of a mapping type
would mean, though (I guess you mean for your proposed sorted dictionary
type?).

>    D. Ability to handle multiple getitems at once in sequence and mapping
>    types (Akin to what can be done in numarray).  Certainly there is no good
>    reason why lists/tuples shouldn't support list[iterable] (not to be
>    confused with list(iterable)) which returns or dictionaries support
>    something like dict.getm(iterable).

Well, one reason is that it places a greater burden on people implementing
__getitem__, and it doesn't really offer much improvement over:
    [l[x] for x in iterable]

Or for dictionaries:
    dict(d[x] for x in iterable)  # Assuming 2.4's generator expressions.

-Andrew.


From john at neggie.net  Fri May 21 11:07:28 2004
From: john at neggie.net (John Belmonte)
Date: Fri May 21 11:07:39 2004
Subject: [Python-Dev] hash comments in interactive interpreter
Message-ID: <40AE1B30.4000806@neggie.net>

Hello,

I'm wondering what the rationale is for hash comments yielding a 
continuation prompt (PS2) in the interactive interpreter.  I'd like to 
put such comments in interactive examples, but the output is hard to follow:

   >>> a = 5
   >>> # and now for some multiplication
   ...
   >>> a * 10
   50

What I would prefer can, ironically, be shown by the interactive 
interpreter emulator:

   >>> import code
   >>> code.interact()
   Python 2.3.3 (#2, May  1 2004, 06:12:12)
   [GCC 3.3.3 (Debian 20040401)] on linux2
   Type "help", "copyright", "credits" or "license" for more information.
   (InteractiveConsole)
   >>> a = 5
   >>> # and now for some multiplication
   >>> a * 10
   50

Another apparent inconsistency is that end of line comments don't behave 
this way:

   >>> a = 5
   >>> a * 10  # end of line comment
   50

Is there something preventing this anomaly in the interactive 
interpreter from being corrected?

-John


-- 
http:// ift ile.org/


From mcherm at mcherm.com  Fri May 21 11:07:53 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Fri May 21 11:07:56 2004
Subject: [Python-Dev] cmp(x,x)
Message-ID: <1085152072.40ae1b490002e@mcherm.com>

Armin Rigo writes:
> Unless there are serious objections I suggest to (i.e. I plan to) remove the
> short-cut in PyObject_RichCompareBool() -- performance is probably not an
> issue here -- and then review all built-in comparison methods and make sure
> that they return "equal" for identical objects.

Robert H. Ledwith writes:
> I have a serious objection to this because it is a major performance
> problem.

Performance is certainly a legitimate concern. But wouldn't your
performance problems be fixed just as well if the tuple class implemented
"identical-objects-are-immediately-equal" instead of putting it in the
general comparison logic for all objects? Because if I understand
correctly, that's what Armin is suggesting. Tuples would still be
compared just as rapidly.

-- Michael Chermside


From mwh at python.net  Fri May 21 11:11:46 2004
From: mwh at python.net (Michael Hudson)
Date: Fri May 21 11:18:53 2004
Subject: [Python-Dev] hash comments in interactive interpreter
In-Reply-To: <40AE1B30.4000806@neggie.net> (John Belmonte's message of "Fri,
	21 May 2004 11:07:28 -0400")
References: <40AE1B30.4000806@neggie.net>
Message-ID: <2m65apu85p.fsf@starship.python.net>

John Belmonte <john@neggie.net> writes:

> Hello,
>
> I'm wondering what the rationale is for hash comments yielding a
> continuation prompt (PS2) in the interactive interpreter.  I'd like to
> put such comments in interactive examples, but the output is hard to
> follow:
[...]
> Is there something preventing this anomaly in the interactive
> interpreter from being corrected?

Lack of time and effort?  The parser is fairly hairy.

Cheers,
mwh

-- 
40. There are two ways to write error-free programs; only the third
    one works.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html

From arigo at tunes.org  Fri May 21 11:23:49 2004
From: arigo at tunes.org (Armin Rigo)
Date: Fri May 21 11:26:56 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <1085152072.40ae1b490002e@mcherm.com>
References: <1085152072.40ae1b490002e@mcherm.com>
Message-ID: <20040521152349.GA6340@vicky.ecs.soton.ac.uk>

Hello Michael,

On Fri, May 21, 2004 at 08:07:53AM -0700, Michael Chermside wrote:
> Performance is certainly a legitimate concern.

Ok, then the current situation looks like the best one.  I suggest we just
drop a note in the documentation saying that if there are objects for which
x.__eq__(x) isn't necessarily true, you shouldn't expect them to be handled in
a fully consistent way.

> But wouldn't your
> performance problems be fixed just as well if the tuple class implemented
> "identical-objects-are-immediately-equal" instead of putting it in the
> general comparison logic for all objects? Because if I understand
> correctly, that's what Armin is suggesting. Tuples would still be
> compared just as rapidly.

There is an extra costly indirection until the execution reaches that point.


Armin

From niemeyer at conectiva.com  Fri May 21 13:36:54 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Fri May 21 13:36:39 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040521152349.GA6340@vicky.ecs.soton.ac.uk>
References: <1085152072.40ae1b490002e@mcherm.com>
	<20040521152349.GA6340@vicky.ecs.soton.ac.uk>
Message-ID: <20040521173654.GA1170@burma.localdomain>

Hello again Armin,

> > Performance is certainly a legitimate concern.
> 
> Ok, then the current situation looks like the best one.  I suggest we
> just drop a note in the documentation saying that if there are objects
> for which x.__eq__(x) isn't necessarily true, you shouldn't expect
> them to be handled in a fully consistent way.

Is it true that the current list comparison algorithm
goes trough every element doing identity comparison, and
if every element in the first list *is* the element at
the same position in the second list, and the list has
the same size, then lists are considered equal?

If that's true, then I belive there's no reason for
not comparing if the first list *is* the second list at
the top of list_richcompare(), right?

-- 
Gustavo Niemeyer
http://niemeyer.net

From tim.one at comcast.net  Fri May 21 15:21:09 2004
From: tim.one at comcast.net (Tim Peters)
Date: Fri May 21 15:21:22 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040521152349.GA6340@vicky.ecs.soton.ac.uk>
Message-ID: <mailman.89.1085167282.6944.python-dev@python.org>

[Armin Rigo]
> Ok, then the current situation looks like the best one.  I suggest we
> just drop a note in the documentation saying that if there are objects
> for which x.__eq__(x) isn't necessarily true, you shouldn't expect them
> to be handled in a fully consistent way.

If that's *all* we say, it's not really helpful, is it?  Besides, __eq__()
isn't even required to return a truth value.  If we believe the CPython
implementation is correct, then the truth is more that the implementation is
allowed to return True for

    x == y

if

    x is y

is true, regardless of what x.__eq__(y), y.__eq__(x), x.__cmp__(y) or
y.__cmp__(x) would do if called directly.

If that's what we really mean, then better to say that.  It then sounds
exactly like the cheap-ass micro optimization it is <wink>.

[Michael Chermside]
>> But wouldn't your> performance problems be fixed just as well if the
>> tuple class implemented "identical-objects-are-immediately-equal"
>> instead of putting it in the general comparison logic for all objects?
>> Because if I understand correctly, that's what Armin is suggesting.
>> Tuples would still be compared just as rapidly.

> There is an extra costly indirection until the execution reaches that
> point.

Well, gasoline is costly, and so is a Mercedes-Benz.  Michael was replying
to a fellow worried about factor-of-2 slowdowns.  The indirection here isn't
in that ballpark of expense.  Still, it's not worth any expense unless it's
useful.

NaN comparison is a red herring not only because Python doesn't have a sane
story in any area involving NaNs, but also because the 32 possible 754
comparison predicates can't be mapped onto 6 operator symbols.  Note that in
C99, the infix C relational operators raise an exception (754's "invalid
operation") when applied to a NaN, and C99 adds a pile of new macros for
non-exceptional NaN comparison (my favorite is the fetchingly named
"islessgreater(x, y)" -- which "is similar to (x) < (y) || (x) > (y);
however, islessgreater(x,y) does not raise the invalid exception when x and
y are unordered (nor does it evaluate x and y twice)").

In a sane design for handling NaNs, it's important that the "legacy"
comparison operators raise an exception in their presence, because code
written under the assumption of trichotomy is a bug minefield when unordered
NaNs become possible comparands.

In short, if Python ever wants "correct" NaN comparison, it needs to add new
operators or functions too.



From guido at python.org  Fri May 21 15:29:51 2004
From: guido at python.org (Guido van Rossum)
Date: Fri May 21 15:30:02 2004
Subject: [Python-Dev] hash comments in interactive interpreter
In-Reply-To: Your message of "Fri, 21 May 2004 11:07:28 EDT."
	<40AE1B30.4000806@neggie.net> 
References: <40AE1B30.4000806@neggie.net> 
Message-ID: <200405211929.i4LJTpJ22082@guido.python.org>

> I'm wondering what the rationale is for hash comments yielding a 
> continuation prompt (PS2) in the interactive interpreter.  I'd like to 
> put such comments in interactive examples, but the output is hard to follow:
> 
>    >>> a = 5
>    >>> # and now for some multiplication
>    ...
>    >>> a * 10
>    50

It's a bug -- but not easy to fix or it would have been fixed long
ago.  Probably the lexter could be hacked to fix this, but be careful
that a change doesn't botch all the other special cases!

--Guido van Rossum (home page: http://www.python.org/~guido/)

From casey at zope.com  Fri May 21 15:37:08 2004
From: casey at zope.com (Casey Duncan)
Date: Fri May 21 15:36:48 2004
Subject: [Python-Dev] Re: cmp(x,x)
References: <20040521152349.GA6340@vicky.ecs.soton.ac.uk>
	<E1BRFba-00015m-00@deer.gmane.org>
Message-ID: <20040521153708.02f1fa1d.casey@zope.com>

On Fri, 21 May 2004 15:21:09 -0400
"Tim Peters" <tim.one@comcast.net> wrote:

> [Armin Rigo]
[..]
> NaN comparison is a red herring not only because Python doesn't have a
> sane story in any area involving NaNs, but also because the 32
> possible 754 comparison predicates can't be mapped onto 6 operator
> symbols.  Note that in C99, the infix C relational operators raise an
> exception (754's "invalid operation") when applied to a NaN, and C99
> adds a pile of new macros for non-exceptional NaN comparison (my
> favorite is the fetchingly named"islessgreater(x, y)" -- which "is
> similar to (x) < (y) || (x) > (y); however, islessgreater(x,y) does
> not raise the invalid exception when x and y are unordered (nor does
> it evaluate x and y twice)").

Finally, a use for <>. Barry will be so happy ;^)

-Casey


From barry at python.org  Fri May 21 15:46:45 2004
From: barry at python.org (Barry Warsaw)
Date: Fri May 21 16:01:31 2004
Subject: [Python-Dev] Re: cmp(x,x)
In-Reply-To: <20040521153708.02f1fa1d.casey@zope.com>
References: <20040521152349.GA6340@vicky.ecs.soton.ac.uk>
	<E1BRFba-00015m-00@deer.gmane.org>
	<20040521153708.02f1fa1d.casey@zope.com>
Message-ID: <1085168805.7927.57.camel@localhost>

On Fri, 2004-05-21 at 15:37, Casey Duncan wrote:

> Finally, a use for <>. Barry will be so happy ;^)

And Just, and Fred.

We constitute the Pointy Bracket Troika with Fred as our King.

all-things-being-unequal-ly y'rs,
-Barry



From barry at python.org  Fri May 21 15:50:21 2004
From: barry at python.org (Barry Warsaw)
Date: Fri May 21 16:22:57 2004
Subject: [Python-Dev] Re: cmp(x,x)
In-Reply-To: <200405211607.28390.fred@zope.com>
References: <20040521152349.GA6340@vicky.ecs.soton.ac.uk>
	<20040521153708.02f1fa1d.casey@zope.com>
	<1085168805.7927.57.camel@localhost> <200405211607.28390.fred@zope.com>
Message-ID: <1085169021.7927.67.camel@localhost>

On Fri, 2004-05-21 at 16:07, Fred Drake wrote:

> No, no, no.  "<>" is a misapplication of pointy brackets.  Note that there's 
> nothing in the middle; what's the point of bracketing if there's nothing to 
> bracket?
> 
> The inequality operator should be spelled "!=", mostly so it's not confused 
> with the structured-bracketing operation.

Okay, Fred's out.  He gets to rewrite the Windows installer in XML.

most-people-are-wrong-ly y'rs,
-Barry



From tjreedy at udel.edu  Fri May 21 18:15:20 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri May 21 18:21:41 2004
Subject: [Python-Dev] Re: SystemError: com_backpatch: offset too large
References: <20040521143641.A721316341@front.interia.pl>
Message-ID: <c8lv1p$uq5$1@sea.gmane.org>


"Sebastian" <seba_woj@interia.pl> wrote in message
news:20040521143641.A721316341@front.interia.pl...
> I've got a problem with python script.
> My script return exceptinon:
>
> SystemError: com_backpatch: offset too large
>
> I'm beginer in python programming, and don't known what do with this
message

Pydev mailing list and the gmane mirror are for discussion of future
Python.  Questions about using current Python should be directed to
comp.lang.python or the main Python mailing list.

For this question, you need to include more information: the system you are
running on; the version of Python you are running; if short, the code you
ran.  If the code is long, try to cut it down to get something short that
gives the same error.  At least say what modules you imported.  Also, can
you rerun with the same inputs and get the same error in the same place, or
is it variable.

Post that information in the proper place (not here) and you should get
answers or at least more suggestions.

Terry J. Reedy




From fred at zope.com  Fri May 21 16:07:28 2004
From: fred at zope.com (Fred Drake)
Date: Fri May 21 21:04:50 2004
Subject: [Python-Dev] Re: cmp(x,x)
In-Reply-To: <1085168805.7927.57.camel@localhost>
References: <20040521152349.GA6340@vicky.ecs.soton.ac.uk>
	<20040521153708.02f1fa1d.casey@zope.com>
	<1085168805.7927.57.camel@localhost>
Message-ID: <200405211607.28390.fred@zope.com>

On Friday 21 May 2004 03:46 pm, Barry Warsaw wrote:
 > On Fri, 2004-05-21 at 15:37, Casey Duncan wrote:
 > > Finally, a use for <>. Barry will be so happy ;^)
 >
 > And Just, and Fred.

No, no, no.  "<>" is a misapplication of pointy brackets.  Note that there's 
nothing in the middle; what's the point of bracketing if there's nothing to 
bracket?

The inequality operator should be spelled "!=", mostly so it's not confused 
with the structured-bracketing operation.


  -Fred

-- 
Fred L. Drake, Jr.  <fred at zope.com>
PythonLabs at Zope Corporation


From greg at cosc.canterbury.ac.nz  Sun May 23 20:40:21 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Sun May 23 20:40:43 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040521173654.GA1170@burma.localdomain>
Message-ID: <200405240040.i4O0eLAr023799@cosc353.cosc.canterbury.ac.nz>

Gustavo Niemeyer <niemeyer@conectiva.com>:

> Is it true that the current list comparison algorithm
> goes trough every element doing identity comparison, and
> if every element in the first list *is* the element at
> the same position in the second list, and the list has
> the same size, then lists are considered equal?

> If that's true, then I belive there's no reason for
> not comparing if the first list *is* the second list at
> the top of list_richcompare(), right?

What if the list contains a NaN?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From mcherm at mcherm.com  Mon May 24 08:50:07 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon May 24 08:50:13 2004
Subject: [Python-Dev] cmp(x,x)
Message-ID: <1085403007.40b1ef7f255c1@mcherm.com>

I wrote:
> Performance is certainly a legitimate concern.

Armin replied:
> Ok, then the current situation looks like the best one.  I
> suggest we just
> drop a note in the documentation saying that if there are
> objects for which
> x.__eq__(x) isn't necessarily true, you shouldn't expect them
> to be handled in
> a fully consistent way.
  [...]
> There is an extra costly indirection until the execution
> reaches that point.

Armin:

Performance is a *legitimate* concern, but not an *overwhelming*
one. If we are in danger of having to walk over the entirety of a
(possibly large) recusive structure (eg: tuple of tuples) to do
a comparison, then that's dangerously slow, and it's worth mucking
with the definition of equality to avoid it. But a single indirection
is "costly" only to someone capable of reaching your level of
close-to-the-machine performance understanding. I say, go with your
original plan -- the current behavior is a mess.

-- Michael Chermside




From mcherm at mcherm.com  Mon May 24 09:07:19 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon May 24 09:07:24 2004
Subject: [Python-Dev] factory functions in the datetime module (& others)
Message-ID: <1085404039.40b1f387f1b5f@mcherm.com>

Brian Quinlan writes:
> It seems to me that the utility of subclassing certain
> library classes would
> be increased if the user could set a factory that the base
> class would use
> when creating new instances.
    [...]
> Any thoughts?

Hmm... I guess I'm -0 on the idea... it wouldn't really
bother me, but I think it's a feature we might as well
leave out.

I guess my philosophy on this comes down to a fundamental
approach to subclassing basic built-in types. Types like int,
float, string, datetime, list and tuple are simple,
fundamental types built into Python (or its standard libraries).
The usual way to use these is by composition -- create a new
type that contains fields which are ints, floats, etc. And
that is enough to solve nearly all problems -- in fact, up
until Python 2.1, that was pretty much the only choice.

But Python wants to go the extra mile to give the programmer
flexibility, so we've allowed even these basic types to
be subclassed by the developer. The question is just how far
we should go in supporting (and encouraging) this use. I feel
that 99% of all uses of these basic types should be by
composition, and only 1% (or less) by inheritance. So while
I am in favor of *permitting* the subclassing, I'd be wary
of tweaking other parts of the language to support it. I
agree that providing customizable factory functions for these
would be very helpful to users of fundamental type subclasses,
but I think they are (and ought to be) a fairly small minority.

As for HOW to do it... a global factory seems like a recipe for
encouraging different packages to conflict with each other --
it only works so long as only ONE module tries to install a
subtype (and perhaps not even then). A per-object approach is
questionable in terms of performance... and it is not
acceptable (to me) to slow the performance of code which uses
fundamental types by composition (essentially all Python code)
for a feature like this.

Just my thoughts on the matter... I certainly agree that
subclassing fundamental types is less powerful because there is
no way to register some sort of factory function, but I'm not
sure it's an important-enough feature to be worth the cost of
supporting registrable factory functions.

-- Michael Chermside


From mcherm at mcherm.com  Mon May 24 11:32:22 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon May 24 11:32:26 2004
Subject: [Python-Dev] RE: cmp(x,x)
Message-ID: <1085412742.40b21586b9536@mcherm.com>

  [...discussion of Armin's idea of moving the short-circuiting
      of equality tests comparing an object to itself from
      richcompare (where it affects all objects) to individual
      object implementations (like tuple)...]

Raymond writes:
> Actually, it's a ton of indirection (many steps) not just a single C
> lookup.  Equality testing is one of the most common and most expensive
> ops in Python.

Okay, I may be mistaken about the performance impact. Can anyone do
actual measurements? Given that equality testing is already quite slow,
I'd be surprised if there were a noticable performance difference --
but I'm often surprised by performance behavior... show me.

> My main concern is that Armin is about to do backflips (inserting new
> code into the details of every object's comparison code) just to
> facilitate the otherwise unholy goal of having an non-reflexsive
> equality comparison.

I think that the REAL goal is consistancy in when user-defined
equality comparison overrides (__eq__, __ne__, __cmp__) are invoked
and when they're not. I find this program surprising:


    Python 2.3.4c1 (#52, May 12 2004, 19:37:24) [MSC v.1200 32 bit (Intel)] on
win32

    Type "help", "copyright", "credits" or "license" for more information.
    >>> class NoisyCompare(int):
    ...     def __cmp__(self, other):
    ...         print 'comparing'
    ...         return int.__cmp__(self, other)
    ...
    >>> n = NoisyCompare(3)
    >>> n == n # CASE 1
    comparing
    True
    >>> n < n # CASE 2
    comparing
    False
    >>> cmp(n,n) # CASE 3
    0
    >>> [n] == [n] # CASE 4
    comparing
    True

(Actually, I find it surprising for two reasons... because case 3
does NOT print 'comparing', and because case 4 DOES (and Armin said
it wouldn't).

Allowing NANs to have wierd behavior is a side effect, probably
less important than allowing USERS to define their own classes that
misbehave.

> My main concern is that Armin is about to do backflips (inserting new
> code into the details of every object's comparison code) just to
> facilitate the otherwise unholy goal of having an non-reflexsive
> equality comparison.

Now hang on... why is this in EVERY object's code? I would think that
list, tuple, and dict need it because they often contain recursive
structures. But for most types, why do it at all?

> IMO, it is not unreasonable to insist that equality be reflexsive.  That
> is somewhat basic to the whole notion of equality and certainly reflects
> the assumptions made throughtout the code base.

No, I don't think it's unreasonable either. But is it necessary? We
should try to have a single, simple rule to explain when user-defined
operator-overrides of comparison functions are invoked. If the rule
is that they are ALWAYS invoked, and that the short-circuiting of
identity comparisons (which is certainly necessary for things like
tuple and list) is simply part of the implementation of that type, then
the rule is VERY simple. If the rule is that the user-defined function
is always invoked on differing objects and MAY be invoked if an
object is compared to itself, then that's a lot less clean. Anything
else I can think of is probably too complex to consider.

By the way, Armin introduced this by saying:
> A minor semantic change that creeped in some time ago was an implicit
> assumption that any object x should "reasonably" be expected to compare
> equal to itself.

How long ago was that?

-- Michael Chermside


From arigo at tunes.org  Mon May 24 12:28:31 2004
From: arigo at tunes.org (Armin Rigo)
Date: Mon May 24 12:31:40 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <200405240040.i4O0eLAr023799@cosc353.cosc.canterbury.ac.nz>
References: <20040521173654.GA1170@burma.localdomain>
	<200405240040.i4O0eLAr023799@cosc353.cosc.canterbury.ac.nz>
Message-ID: <20040524162831.GA20681@vicky.ecs.soton.ac.uk>

Hello Greg,

On Mon, May 24, 2004 at 12:40:21PM +1200, Greg Ewing wrote:
> What if the list contains a NaN?

Uh, right.  The whole issue with NaNs is muddy.  Let's focus on Numeric as a 
much saner example of why you need 'a==a' to return something else than True.


Armin

From tim.one at comcast.net  Mon May 24 12:38:15 2004
From: tim.one at comcast.net (Tim Peters)
Date: Mon May 24 12:38:24 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040524162831.GA20681@vicky.ecs.soton.ac.uk>
Message-ID: <mailman.126.1085416704.6944.python-dev@python.org>

[Greg Ewing]
>> What if the list contains a NaN?

[Armin Rigo]
> Uh, right.  The whole issue with NaNs is muddy.  Let's focus on Numeric as
> a much saner example of why you need 'a==a' to return something else than
> True.

Yes.  Please <wink>.



From python at rcn.com  Mon May 24 16:57:47 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon May 24 16:59:35 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040524162831.GA20681@vicky.ecs.soton.ac.uk>
Message-ID: <007b01c441d1$c4d0b900$c327c797@oemcomputer>

> On Mon, May 24, 2004 at 12:40:21PM +1200, Greg Ewing wrote:
> > What if the list contains a NaN?
> 
> Uh, right.  The whole issue with NaNs is muddy.  Let's focus on
Numeric as
> a
> much saner example of why you need 'a==a' to return something else
than
> True.

More specifically, find a valid use case where bool(a==a) returns False
(because Py_RichCompareBool() still does a boolean coercion on the
result of a.__eq__(b)).

For the use case to be compelling, it would need to justify breaking
things like:

    mylist.append(x)
    assert x in mylist

I don't believe that you will find sane examples.
Identity-implying-equality is a useful and important invariant.  Don't
give it up easily.



Raymond


From bob at redivi.com  Mon May 24 18:04:32 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon May 24 18:04:45 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <007b01c441d1$c4d0b900$c327c797@oemcomputer>
References: <007b01c441d1$c4d0b900$c327c797@oemcomputer>
Message-ID: <55C8D724-ADCE-11D8-BDF7-000A95686CD8@redivi.com>


On May 24, 2004, at 4:57 PM, Raymond Hettinger wrote:

>> On Mon, May 24, 2004 at 12:40:21PM +1200, Greg Ewing wrote:
>>> What if the list contains a NaN?
>>
>> Uh, right.  The whole issue with NaNs is muddy.  Let's focus on
> Numeric as
>> a
>> much saner example of why you need 'a==a' to return something else
> than
>> True.
>
> More specifically, find a valid use case where bool(a==a) returns False
> (because Py_RichCompareBool() still does a boolean coercion on the
> result of a.__eq__(b)).
>
> For the use case to be compelling, it would need to justify breaking
> things like:
>
>     mylist.append(x)
>     assert x in mylist
>
> I don't believe that you will find sane examples.
> Identity-implying-equality is a useful and important invariant.  Don't
> give it up easily.

I think the use case mentioned was something like Numeric, where it may 
be useful for anArray == anotherArray to return an array of bool, not a 
single True or False.  It would also be useful for defining objects 
that implement some domain specific language in Python.. for example, 
something like an AppleScript or SQL bridge where you are basically 
building an AST, compiling it to some other form, and sending it over 
the wire.  It may be convenient if you could do this without actually 
parsing the code as a string.

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2357 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040524/5f9acbf9/smime.bin
From raymond.hettinger at verizon.net  Mon May 24 19:13:44 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Mon May 24 19:16:00 2004
Subject: [Python-Dev] Standard Library Tour Part II
Message-ID: <001101c441e4$c325cd80$a2b1958d@oemcomputer>

Comments are invited on the attached draft for a second overview of the
standard library.

The first overview of basic modules at
http://docs.python.org/tut/node12.html was well received and seemed to
get people off to a running start.  

The primary goal of this follow-up is to reduce the transition time for
a programmer to start writing real applications using Python.  

A second goal is give information to people who work through tutorials
to determine the capabilities of various languages.

A third goal is provide a jumping-off point for someone learning brand
new concepts such as weak references or threading.  The current docs
merely describe the cockpit controls and assume that you already know
how to fly an aircraft.


Raymond

=========================================================

Brief Tour of Modules in the Standard Libary (part II)


This second tour covers more advanced modules that support professional
programming needs.  Their use rarely arises for small scripts.


Output Formatting
-----------------

The repr module provides an version of repr() for abbreviated displays
of large or deeply nested containers::

    >>> import repr   
    >>> repr.repr(set('supercalafraglisticexpallidoches'))
    "set(['a', 'c', 'd', 'e', 'f', 'g', ...])"

The pprint module offers more sophisticated control over printing both
built-in and user defined objects in a way that is readable by the
interpreter.  When the result is longer than one line, the "pretty
printer"
adds line breaks and indentation to more clearly reveal data structure::

    >>> import pprint
    >>> t = [[[['black', 'cyan'], 'white', ['green', 'red']],
[['magenta',
    ...     'yellow'], 'blue']]]
    ...
    >>> pprint.pprint(t, width=30)
    [[[['black', 'cyan'],
       'white',
       ['green', 'red']],
      [['magenta', 'yellow'],
       'blue']]]

The textwrap module formats paragraphs of text to fit a given screen
width::

    >>> import textwrap
    >>> doc = """The wrap() method is just like fill() except that it
returns
    ... a list of strings instead of one big string with newlines to
separate
    ... the wrapped lines."""
    ...
    >>> print textwrap.fill(doc, width=40)
    The wrap() method is just like fill()
    except that it returns a list of strings
    instead of one big string with newlines
    to separate the wrapped lines.

The locale module accesses a database of cultural specific data formats.
The grouping attribute of locale's format function provides a direct way
of formatting numbers with group separators.

    >>> import locale
    >>> locale.setlocale(locale.LC_ALL, 'English_United States.1252')
    'English_United States.1252'
    >>> conv = locale.localeconv()     # get a mapping of conventions
    >>> x = 1234567.8
    >>> locale.format("%d", x, grouping=True)
    '1,234,567'
    >>> locale.format("%s%.*f", (conv['currency_symbol'],
    ...	      conv['int_frac_digits'], x), grouping=True)
    '$1,234,567.80'


Working with Binary Data Record Layouts
---------------------------------------

The struct module provides pack() and unpack() functions for working
with variable length binary record formats.  The following example shows
how to loop through header information in a ZIP file (with pack codes
"H" and "L" representing two and four byte numbers respectively)::

    import struct
    data = open('myfile.zip', 'rb').read()
    start = 0
    for i in range(3):                      # show the first 3 file
headers
        start += 14
        fields = struct.unpack('LLLHH', data[start:start+16])
        crc32, comp_size, uncomp_size, filenamesize, extra_size =
fields

        start += 16
        filename = data[start:start+filenamesize]
        start += filenamesize
        extra = data[start:start+extra_size]
        print filename, hex(crc32), comp_size, uncomp_size

        start += extra_size + comp_size     # skip to the next header


Multi-threading
---------------

Threading is a technique for decoupling tasks which are not sequentially
dependent.  Python threads are driven by the operating system and run
in a single process and share memory space in a single interpreter.

Threads can be used to improve the responsiveness of applications
that accept user input while other tasks run in the background.
The following code shows how the high level threading module can run
tasks in background while the main program continues to run::

    import threading, zipfile

    class AsyncZip(threading.Thread):
        def __init__(self, infile, outfile):
            Thread.__init__(self)        
            self.infile = infile
            self.outfile = outfile
        def run(self):
            f = zipfile.ZipFile(self.outfile, 'w', zipfile.ZIP_DEFLATED)
            f.write(self.infile)
            f.close()
            print 'Finished background zip of: ', self.infile

    AsyncZip('mydata.txt', 'myarchive.zip').start()
    print 'The main program continues to run'

The principal challenge of multi-thread applications is coordinating
threads that share data or other resources.  To that end,
the threading module provides a number of synchronization primitives
including locks, events, condition variables, and semaphores.

While those tools are powerful, minor design errors can result in
problems that are difficult to reproduce.  A simpler and more robust
approach to task coordination is concentrating all access to a resource
in a single thread and then using the Queue module to feed that thread
with requests from other threads.  Applications that use Queue objects
for
inter-thread communication and coordination tend to be easier to design,
more readable, and more reliable.


Logging
-------

The logging module offers a full featured and flexible logging system.
At its simplest, log messages are sent to a file or to sys.stderr.
Other options include routing messages through email, datagrams,
sockets, or to an HTTP Server.  Messages are assigned priorities
including DEBUG, INFO, WARNING, ERROR, and CRITICAL.  Filtering allows
messages to be directed based on their priority (for example:
email only critcal messages, ignore debug messages, and write everything
else to a file):

    [XXX add example]
                           

Weak References
---------------

Python does automatic memory management (reference counting for most
objects and garbage collection to eliminate cycles).  The memory is
freed shortly after the last reference to it has been eliminated.

This approach works fine for most applications but occasionally there
is a need to track objects only as long as they are being used by
something else.  Unfortunately, just tracking them creates a reference
that makes them permanent.  The weakref module provides tools tracking
objects without creating a reference.  When the object is no longer
needed, it is automatically removed from a weakref table and a callback
is triggered for weakref objects.  Typically applications include
caching objects that are expensive to create:

    >>> import weakref, gc
    >>> class A:
            def __init__(self, value):
                    self.value = value
            def __repr__(self):
                    return str(self.value)

            
    >>> a = A(10)			# create a reference
    >>> d = weakref.WeakValueDictionary()
    >>> d['primary'] = a      # does not create a reference
    >>> d['primary']		# fetch the object if it is still alive
    10
    >>> del a			# remove the one reference
    >>> gc.collect()          # make garbage collection run right away
    0
    >>> d['primary'] = a      # the entry was automatically removed

    Traceback (most recent call last):
      File "<pyshell#108>", line 1, in -toplevel-
        d['primary'] = a            # does not create a reference
    NameError: name 'a' is not defined


Tools for working with lists
----------------------------

Many data structure needs can be met with the built-in lists.  However,
sometimes there is a need for alternative implementations with different
performance trades-offs.

The array module provides an array() object that is like a list that
stores
only homongenous data but stores it more compactly.  The following
example shows an array of numbers stored as two byte unsigned binary
numbers (typecode "H") rather than the usual 16 bytes per entry
for regular lists of python int objects::

    >>> from array import array
    >>> a = array('H', [4000, 10, 700, 22227])
    >>> a[1:3]
    array('H', [10, 700])

The collections module provides a deque() object that is like a list
with
faster appends and pops from the left side but slower lookups in the
middle.
These objects are well suited for implementing queues and breadth first
tree searches::

    >>> from collections import deque
    >>> d = deque(["task1", "task2", "task3"])
    >>> d.append("task4")
    >>> print "Handling task,", d.popleft()
    task1

    unsearched = deque([initialnode])
    def breadth_first_search(unsearched):
        node = unsearched.popleft()
        for m in gen_moves(node):
            if is_goal(m):
                return m
            unsearched.append(m)

In addition to alternative list implementations, the library also offers
other
tools such as the bisect module with functions for manipulating sorted
lists::

    >>> import bisect
    >>> scores = [(100, 'perl'), (200, 'tcl'), (400, 'lua'), (500,
'python')]
    >>> bisect.insort(scores, (300, 'ruby'))
    >>> scores
    [(100, 'perl'), (200, 'tcl'), (300, 'ruby'), (400, 'lua'), (500,
'python')]

The heapq module provides functions for implementing heaps based on
regular
lists.  The lowest valued entry is always kept at position zero.  This
is useful for applications which repeatedly access the smallest element
but do not want to run a full list sort::

    >>> from heapq import heapify, heappop, heappush
    >>> data = [1, 3, 5, 7, 9, 2, 4, 6, 8, 0]
    >>> heapify(data)                      # rerrange the list into heap
order
    >>> heappush(data, -5)                 # add a new entry
    >>> [heappop(data) for i in range(3)]  # fetch the three smallest
entries
    [-5, 0, 1]    


[??? Should the tempfile module be in the tour ???]
--------------------------------------------------- 


From python at rcn.com  Mon May 24 19:52:29 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon May 24 19:54:19 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <55C8D724-ADCE-11D8-BDF7-000A95686CD8@redivi.com>
Message-ID: <000701c441ea$2cd31440$a2b1958d@oemcomputer>

[Bob Ippolito]
> I think the use case mentioned was something like Numeric, where it
may
> be useful for anArray == anotherArray to return an array of bool, not
a
> single True or False

The code in question is in Py_RichCompareBool() which *always* just
returns True or False.  That routine is called by list.__contains__()
and many other functions that expect a yes or no answer.  

The regular rich comparison function, Py_RichCompare() is the same as it
always was and can still return arrays of bools, complex numbers, or
anything at all.

IOW, we're still looking for a use case that warrants removing the
identity-implies-equality guarantee out of Py_RichCompareBool().  My
argument focused on the invariant assumptions that would be broken if
this was done.  Second, I don't believe that a sane use case will be
found for an object x that returns False for "bool(x==x)" and that
returns False for "x in [x]".  Stretching ones mind to find such an
object seems like an exercise in masochism.


Raymond


From arigo at tunes.org  Mon May 24 20:14:25 2004
From: arigo at tunes.org (Armin Rigo)
Date: Mon May 24 20:17:29 2004
Subject: [Python-Dev] Standard Library Tour Part II
In-Reply-To: <001101c441e4$c325cd80$a2b1958d@oemcomputer>
References: <001101c441e4$c325cd80$a2b1958d@oemcomputer>
Message-ID: <20040525001425.GA2600@vicky.ecs.soton.ac.uk>

Hello Raymond,

On Mon, May 24, 2004 at 07:13:44PM -0400, Raymond Hettinger wrote:
>     >>> del a			# remove the one reference
>     >>> gc.collect()          # make garbage collection run right away
>     0
>     >>> d['primary'] = a      # the entry was automatically removed
> 
>     Traceback (most recent call last):
>       File "<pyshell#108>", line 1, in -toplevel-
>         d['primary'] = a            # does not create a reference
>     NameError: name 'a' is not defined

Oups, wrong line and hence wrong exception for what you're trying to show.


A bientot,

Armin

From python at rcn.com  Mon May 24 20:43:09 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon May 24 20:45:01 2004
Subject: [Python-Dev] Standard Library Tour Part II
In-Reply-To: <20040525001425.GA2600@vicky.ecs.soton.ac.uk>
Message-ID: <001201c441f1$40d8f2a0$a2b1958d@oemcomputer>

[ Armin Rigo ]
> >         d['primary'] = a            # does not create a reference
> >     NameError: name 'a' is not defined
> 
> Oups, wrong line and hence wrong exception for what you're trying to
show.

Fixed.

Thanks,


Raymond


From anthony at interlink.com.au  Tue May 25 11:27:56 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue May 25 11:29:03 2004
Subject: [Python-Dev] 2.3.4 release this thursday
Message-ID: <40B365FC.5040709@interlink.com.au>

Thomas is back on deck, so I'm aiming to do the 2.3.4 release
this Thursday (AU time).

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From niemeyer at conectiva.com  Tue May 25 11:30:20 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Tue May 25 11:29:53 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040524163915.10E18472F5@perninha.conectiva.com.br>
References: <20040524162831.GA20681@vicky.ecs.soton.ac.uk>
	<20040524163915.10E18472F5@perninha.conectiva.com.br>
Message-ID: <20040525153020.GB4093@burma.localdomain>

> [Greg Ewing]
> >> What if the list contains a NaN?
> 
> [Armin Rigo]
> > Uh, right.  The whole issue with NaNs is muddy.  Let's focus on Numeric as
> > a much saner example of why you need 'a==a' to return something else than
> > True.
> 
> Yes.  Please <wink>.

I belive that either my question was misinterpreted, or I'm
missing something.

I'm not suggesting to limit 'a==a' to return True/False only. I'm
suggesting to introduce a shortcut in the *list* comparison (more
specifically, in list_richcompare), since this is what is
currently being done, but with a more expensive logic.

Going through the function, in a case where v is w and op is
either Py_EQ or Py_NE, we have

1) Both objects are lists, go on.
2) Both objects have the same size, go on.
3) Searching for the first index where items are different, using
   PyObject_RichCompareBool(), won't find anything. Go on.
4) No more items to compare. Check sizes.
5) Sizes match. Check comparison operator.
6) Return either Py_True or Py_False.

This whole logic might be replaced by simply:

    if (v == w && (op == Py_EQ || op == Py_NE)) {
        res = (op == Py_EQ ? Py_True : Py_False);
        Py_INCREF(res);
        return res;
    }

-- 
Gustavo Niemeyer
http://niemeyer.net

From niemeyer at conectiva.com  Tue May 25 11:40:49 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Tue May 25 11:40:13 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040525153020.GB4093@burma.localdomain>
References: <20040524162831.GA20681@vicky.ecs.soton.ac.uk>
	<20040524163915.10E18472F5@perninha.conectiva.com.br>
	<20040525153020.GB4093@burma.localdomain>
Message-ID: <20040525154049.GC4093@burma.localdomain>

> Going through the function, in a case where v is w and op is
> either Py_EQ or Py_NE, we have

Looking again, this is true for any operation, not only
Py_EQ or Py_NE. So the shortcut might be:

    if (v == w) {
        if (op == Py_EQ || op == Py_LE || op == Py_GE)
            res = Py_True;
        else
            res = Py_False;
         Py_INCREF(res);
         return res;
    }

-- 
Gustavo Niemeyer
http://niemeyer.net

From tim.one at comcast.net  Tue May 25 12:05:48 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue May 25 12:06:08 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040525153020.GB4093@burma.localdomain>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEEDLEAB.tim.one@comcast.net>

[Gustavo Niemeyer]
> I belive that either my question was misinterpreted, or I'm
> missing something.

Probably both, but who cares <wink>.

> I'm not suggesting to limit 'a==a' to return True/False only. I'm
> suggesting to introduce a shortcut in the *list* comparison (more
> specifically, in list_richcompare), since this is what is
> currently being done, but with a more expensive logic.

What does "currently" mean to you?  2.3.3 and HEAD differ here.

> Going through the function, in a case where v is w and op is
> either Py_EQ or Py_NE, we have

These cases aren't interesting on HEAD.  Nobody calls list_richcompare
directly.  If something calls PyObject_RichCompareBool() with Py_EQ or
Py_NE, then on HEAD the result is returned instantly, because of this code
at the start of PyObject_RichCompareBool():

	/* Quick result when objects are the same.
	   Guarantees that identity implies equality. */
	if (v == w) {
		if (op == Py_EQ)
			return 1;
		else if (op == Py_NE)
			return 0;
	}

That code doesn't care whether v and w are lists or ints or Unicode strings
or floats (etc).  2.3.3 doesn't have this code.

> 1) Both objects are lists, go on.

On HEAD, we don't get into list_richcompare.

> 2) Both objects have the same size, go on.
> 3) Searching for the first index where items are different, using
>    PyObject_RichCompareBool(), won't find anything. Go on.

If v and w were created via, e.g.,

    v = []
    v.append(v)
    w = v

then step #3 is a lot more complicated in 2.3.3 (because we recurse into
list_richcompare repeatedly then, and a bunch of gone-on-HEAD machinery
tries to notice that and guess the right result).

> 4) No more items to compare. Check sizes.
> 5) Sizes match. Check comparison operator.
> 6) Return either Py_True or Py_False.
>
> This whole logic might be replaced by simply:
>
>     if (v == w && (op == Py_EQ || op == Py_NE)) {
>         res = (op == Py_EQ ? Py_True : Py_False);
>         Py_INCREF(res);
>         return res;
>     }

On HEAD, it has been replaced, but for all object types with one blow.


From ark-mlist at att.net  Tue May 25 12:07:09 2004
From: ark-mlist at att.net (Andrew Koenig)
Date: Tue May 25 12:07:10 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <000701c441ea$2cd31440$a2b1958d@oemcomputer>
Message-ID: <006701c44272$554e5b40$6602a8c0@arkdesktop>

> IOW, we're still looking for a use case that warrants removing the
> identity-implies-equality guarantee out of Py_RichCompareBool().

I can think of two use cases:

	1) IEEE-like behavior.  Before you say that there is nothing wrong
with making floating-point comparison yield true for NaN==NaN, let me remind
you of the beast called "Signaling NaN", which should raise an exception on
any operation, including comparison.

	2) A class might have comparison operations that put trace
information in a log file for testing purposes.  Such a class might be
intended to help in testing searching algorithms, for example.  I think it
would be useful to be able to know whether x[i] is ever compared with x[i]
by putting appropriate tracing comparison operations in the elements of x.


From fdrake at acm.org  Tue May 25 12:12:43 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue May 25 12:12:53 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Doc/tut tut.tex,
	1.228, 1.229
In-Reply-To: <E1BSeTg-00084S-FR@sc8-pr-cvs1.sourceforge.net>
References: <E1BSeTg-00084S-FR@sc8-pr-cvs1.sourceforge.net>
Message-ID: <200405251212.43580.fdrake@acm.org>

On Tuesday 25 May 2004 12:08 pm, rhettinger@users.sourceforge.net wrote:
 > Modified Files:
 > 	tut.tex
 > Log Message:
 > Follow official recommendation for domain names in examples.

Yay!  Thanks, Raymond!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From jim.jewett at eds.com  Tue May 25 12:30:38 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Tue May 25 12:31:05 2004
Subject: [Python-Dev] cmp(x,x)
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D4E0@usahm010.exmi01.exch.eds.com>

Gustavo Niemeyer: <niemeyer@conectiva.com>

> Looking again, this is true for any operation, not only
> Py_EQ or Py_NE.

CVS has previously restricted the shortcut to those two,
but I agree it should be all or nothing.

Even unordered objects normally compare equal to themselves;
if "is" does not imply "__eq__", then we are by definition 
dealing with an exceptional case.  (NaN, or SQL NULL, or DUMMY,
or ...)  Since different exceptional cases will have different
rules, it might be reasonable to ask the creator to use a custom
comparison function.  They will often have to do this anyway.

Instead of 
	if (a == b):

they can write
	if (a is not NULL) and (a == b):
or
	if (a == b) and (not isNaN(a)):

-jJ

From aahz at pythoncraft.com  Tue May 25 12:32:52 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue May 25 12:32:55 2004
Subject: [Python-Dev] 2.3.4 release this thursday
In-Reply-To: <40B365FC.5040709@interlink.com.au>
References: <40B365FC.5040709@interlink.com.au>
Message-ID: <20040525163252.GA9862@panix.com>

On Wed, May 26, 2004, Anthony Baxter wrote:
>
> Thomas is back on deck, so I'm aiming to do the 2.3.4 release
> this Thursday (AU time).

I'm not objecting, precisely, but this weekend is Memorial Day weekend
in the USA, so there will be fewer resources available if anything goes
wrong.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"He's Quebecois. He puts mayonnaise on *everything*."  --siobhan@virulent.org

From niemeyer at conectiva.com  Tue May 25 12:50:14 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Tue May 25 12:49:38 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEEDLEAB.tim.one@comcast.net>
References: <20040525153020.GB4093@burma.localdomain>
	<LNBBLJKPBEHFEDALKOLCMEEDLEAB.tim.one@comcast.net>
Message-ID: <20040525165014.GD4093@burma.localdomain>

> > I'm not suggesting to limit 'a==a' to return True/False only. I'm
> > suggesting to introduce a shortcut in the *list* comparison (more
> > specifically, in list_richcompare), since this is what is
> > currently being done, but with a more expensive logic.
> 
> What does "currently" mean to you?  2.3.3 and HEAD differ here.

I meant HEAD.

> > Going through the function, in a case where v is w and op is
> > either Py_EQ or Py_NE, we have
> 
> These cases aren't interesting on HEAD.  Nobody calls list_richcompare
> directly.  If something calls PyObject_RichCompareBool() with Py_EQ or
> Py_NE, then on HEAD the result is returned instantly, because of this
> code at the start of PyObject_RichCompareBool():

I'm wondering what is causing this confusion then:

% ./python
Python 2.4a0 (#5, May 25 2004, 13:45:26)
[GCC 3.3.3 (release)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
[...]
>>> l = []
>>> l == l
Got in list_richcompare()!
True
>>> if l == l: print "foo"
...
Got in list_richcompare()!
foo

-- 
Gustavo Niemeyer
http://niemeyer.net

From tim.one at comcast.net  Tue May 25 12:58:07 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue May 25 12:58:13 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D4E0@usahm010.exmi01.exch.eds.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEEKLEAB.tim.one@comcast.net>

[Gustavo Niemeyer]
>> Looking again, this is true for any operation, not only
>> Py_EQ or Py_NE.

[Jim J Jewett]
> CVS has previously restricted the shortcut to those two,
> but I agree it should be all or nothing.

It could for lists specifically, but not for objects in general.  The rich
comparison PEP explicitly promised that, for example, x <= y is not
necessarily the same as "x < y or x == y":  from x == y, we're not allowed
to infer that x <= y, or x >= y, or not x < y, or not x > y.  For that
matter, the PEP

    http://www.python.org/peps/pep-0207.html

also promised

   3 The == and != operators are not assumed to be each other's
     complement

so it's also not kosher to infer x != y is False from that x == y is True.

Any shortcuts based on object identity are optimizations, and dubious ones
because they violate promises made in that PEP.  There's a strong pragmatic
("go fast") case for starting to infer "x == y" and "not x != y" from "x is
y", but that doesn't extend to inferring others from "x is y"
(PyObject_RichCompareBool gets invoked with Py_EQ and Py_NE all over the
place in the core; it's never invoked with Py_GT, Py_GE, or Py_LE; there are
a few invocations with Py_LT, mostly involved with sorting).


From mcherm at mcherm.com  Tue May 25 13:07:31 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue May 25 13:07:34 2004
Subject: [Python-Dev] cmp(x,x)
Message-ID: <1085504851.40b37d5322d1a@mcherm.com>

Raymond writes:
> The code in question is in Py_RichCompareBool() which *always* just
> returns True or False.  That routine is called by list.__contains__()
> and many other functions that expect a yes or no answer.
>
> The regular rich comparison function, Py_RichCompare() is the same as it
> always was and can still return arrays of bools, complex numbers, or
> anything at all.

All right, I took a look at exactly where PyObject_RichCompareBool is
called. Here is a COMPLETE list of all uses in 2.3.3 (that's what I
had lying around):

  * In listobject.c, to test containment in a list (ie: "x in [x]").

  * In tupleobject.c to perform containment tests on tuples
    (ie: "x in (a,b,c)").

  * In arraymodule.c, in array_contains to test containment in an array.

  * In listobject.c, in listindex(), listremove(), and listcount() to find
    the first occurance, delete the first occurance, or count the number of
    occurances of an object in the list.

  * In arraymodule.c, in array_index(), array_remove(), and array_count() to
    find the first occurance, delete the first occurance, or count the
    number of occurances of an object in the array.

  * In abstract.c, in _PySequence_IterSearch() to test containment of, find
    the first occurance of, or count the occurances of some object.

  * In listobject.c, in list_richcompare(), to skip past identical list
    elements when comparing lists.

  * In tupleobject.c, in tuplerichcompare() to skip past identical tuple
    elements when comparing tuples.

  * In arraymodule.c, in array_richcompare, to skip past identical array
    elements when comparing arrays.

  * In listobject.c, used in sorting if the user does not provide a
    user-defined comparison function.

  * In bltinmodule.c, in min_max(), to compare two objects within the min
    and max functions.

  * In iterobject.c, to test whether an object is the sentinal object.

  * In dictobject.c, in lookdict(), to test whether an key in the dict
    matches the key being looked up.

  * In dictobject.c, in characterize() ... I'm not quite sure what this
    is doing.

  * In dictobject.c when comparing two dicts for equality.

  * In typeobject.c, in method_is_overloaded, to test whether the methods
    defined in a class and some subclass are actually the same object.

  * In ceval.c, for matching keyword arguments to code object varnames
    in PyEval_EvalCodeEx.

  * In ceval.c in _PyEval_SliceIndex() to compare a bound to the number
    0L.

  * And finally, it is part of the published API so it could appear
    anywhere in extension modules... but I would guess that
    people use PyObject_RichCompare normally and only call
    PyObject_RichCompareBool if they really want a boolean.

What I learned by writing this list is that Raymond is right...
being able to assume that "x is x" implies "x == x" is very useful
for implementors. And if that assumption is made in
PyObject_RichCompareBool() and NOT in PyObject_RichCompare(), then
it hardly ever needs to conflict with the desire for user-defined
comparison functions to return peculiar things (like Numeric arrays,
values that are artifically not equal (like NANs or a poorly-designed
user-implemented Max object).

Okay, I know this is too much analysis, but I've started now, so
I'm going to go through with it and send off this email. Here's my
thought: Raymond has convinced me that we want to make this assumption
in the interpreter. I want the rules for when a user-defined comparison
function (__cmp__, __eq__, or __ne__) is invoked and when it isn't.
Maybe we can achieve both.

Here's that list above, grouped by function:

  1. Test Containment in a sequence, also index(), count(), and remove() on
     sequences.

  2. Comparing sequences or dicts to each other.

  3. Sorting lists when no user-defined comparison function is given.

  4. in min() and max()

  5. checking for sentinal in iteration

  6. looking up dict keys

  7. characterize() in dictobject (what's this for?)

  8. Checking if a method is the same to tell if it's overloaded.

  9. Matching keyword arguments to code object varnames in EvalCode

 10. Compare a slice bound to 0L.

 11. Extension modules that call PyObject_RichCompareBool instead of
     PyObject_RichCompare.

This is the list of all places where RichCompareBool is called instead
of RichCompare, and thus of all places where a user-defined comparison
function might (surprisingly) not be called. Some are not relevent
(eg: #9 compares only strings, a built-in type). For some, I can
concoct artifical examples where someone would care (eg: #5: creating
a sentinal for iter() that tries to be so VERY clever that it allows
the sentinal object itself to occur in the sequence sometimes
without stopping the iteration; or #3: trying to understand the
behavior of timsort by creating objects that log comparisons rather
than by reading the code). But these feel artificial to me. The big
ones seem to be #1 and #2.

I'd even say that it makes "intuitive sense" to me somehow that
containment, index(), count() and remove() all act as if identity
implied equality (although I can think of only *slightly* absurd
examples where a user might try to alter this behavior. But when
comparing sequences or dicts to each other (something people do
a LOT), I would intuitively expect that any contained objects
with customized comparison methods would have those methods
invoked.

Okay... I've talked myself into a corner now. Raymond has convinced
me that my original idea was misguided, and I've looked closely at
the problem, but I don't see an "obvious" solution. I'm tending
to think it's best to put the test for identity in
PyObject_RichCompareBool, but then how do we explain (in simple terms)
when user-defined comparison methods are invoked and when they're
not necessarily?

Well, even without an answer, I'd better send this email off and
get back to my own work. I'm not sure if I've gotten anywhere with
this or not.

-- Michael Chermside


From tim.one at comcast.net  Tue May 25 13:28:52 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue May 25 13:28:59 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <20040525165014.GD4093@burma.localdomain>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEEOLEAB.tim.one@comcast.net>

[Gustavo Niemeyer]
> I'm wondering what is causing this confusion then:
>
> % ./python
> Python 2.4a0 (#5, May 25 2004, 13:45:26)
> [GCC 3.3.3 (release)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> [...]
> >>> l = []
> >>> l == l
> Got in list_richcompare()!
> True
> >>> if l == l: print "foo"
> ...
> Got in list_richcompare()!
> foo

My mistake!  And my apologies.  PyObject_RichCompare() (not ...Bool()) gets
called in this case, and it can't infer anything from object identity alone.
The list object may, but it's quite possible that trying to would cost more
cycles than it saved.


From jim.jewett at eds.com  Tue May 25 13:58:52 2004
From: jim.jewett at eds.com (Jewett, Jim J)
Date: Tue May 25 14:01:09 2004
Subject: [Python-Dev] cmp(x,x)
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D4E3@usahm010.exmi01.exch.eds.com>


[Jim J Jewett]
>> CVS has previously restricted the shortcut to those two,
>> but I agree it should be all or nothing.

[Tim Peters]
> The rich comparison PEP explicitly promised that, for example, 
> x <= y is not necessarily the same as "x < y or x == y":  

> ...it's also not kosher to infer x != y is False from 
> that x == y is True.

With consistent orderings, "x <= y" is strictly weaker than 
"x < y or x == y"; they differ only when a pair of objects 
don't know *which* of "<" and "==" is true.

If we do not assume consistency, then "is" does not even
imply "==", let alone "not !="

> Any shortcuts based on object identity are optimizations, 
> and dubious ones

Agreed. 

> because they violate promises made in that PEP. 

PEP 207 http://www.python.org/peps/pep-0207.html said that 
you can't (generally) reason from one rich compare to 
another.  It said nothing about reasoning from "is" to any 
of the comparisons directly.  So this optimization may 
violate the spirit of __eq__, but it doesn't violate the
letter of the PEP.

> There's a strong pragmatic ("go fast") case for starting to
> infer "x == y" and "not x != y" from "x is y", but that
> doesn't extend to inferring others from "x is y"

For normal objects, consistency holds.

When consistency does not hold, even Py_EQ and Py_NE are wrong.  
Doing (only) the other operations correctly won't fix the whole 
program; it will just make the bugs less obvious (and therefore 
harder to catch).

-jJ

From jim.jewett at EDS.COM  Tue May 25 15:42:50 2004
From: jim.jewett at EDS.COM (Jewett, Jim J)
Date: Tue May 25 15:44:07 2004
Subject: [Python-Dev] cmp(x,x)
Message-ID: <B8CDFB11BB44D411B8E600508BDF076C1E96D4E5@usahm010.exmi01.exch.eds.com>

Michael Chermside:

[Where PyObject_RichCompareBool is called, and the is shortcut
might be a problem]

Most of these operations -- including the container comparison --
are not well-defined if "is" does not imply equality.

> I would intuitively expect that any contained objects
> with customized comparison methods would have those methods
> invoked.

Yes, but you couldn't reliably predict the order they would
be compared (I *choose* to start with the least significant!
Oh yeah?  Well I timsort with much galloping!), so inconsistent 
results are not guaranteed.

The real problem is that you won't get the expected side effects.

> I'm tending to think it's best to put the test for identity in
> PyObject_RichCompareBool, but then how do we explain (in simple
> terms) when user-defined comparison methods are invoked and when 
> they're not necessarily?

Say that Python only promises to make the comparisons you explicitly
ask for.  Implicit comparisons (including cmp, sort, and comparisons 
between subobjects) may be optimized away.  For
example:
	a < b

will compare a and b, but:
	[1, a] < [2, b] 

or even:
	[a] < [b] 

might not. 

-jJ

From python at rcn.com  Tue May 25 15:57:28 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue May 25 15:59:26 2004
Subject: [Python-Dev] cmp(x,x)
In-Reply-To: <B8CDFB11BB44D411B8E600508BDF076C1E96D4E4@usahm010.exmi01.exch.eds.com>
Message-ID: <002401c44292$824d9f00$2d24a044@oemcomputer>

Forgive me, but perhaps a Letterman style top ten list may be a better
way to summarize what I've heard so far.

Proposal:   Remove 8 lines of code discussed and approved by Raymond,
Tim, Guido, and Armin at PyCon.  Replicate those lines and push them
down in every object known to man.

Advantages:
10) Facilitating insane use cases
9) Increasing the size of code base
8) Avoiding that silly concept of code factoring
7) Reducing performance
6) Breaking foolish invariants like "[x]==[x] and x in [x]"
5) Tricking myself into thinking that some sort of consistency has been
achieved though some existing calls to Py_RichCompareBool() already have
their own identity checks slightly upstream of the call.
 . . .
And the number one reason for reverting good code is: 
    It feels so groovy to call something a hack!


Hopefully you find this funny and use it help get things back on course.
With so many contrarians the mailing list, it's nearly impossible to get
agreement on the simplest things.  It's amazing how much support there
is for something with no practical use cases.


Raymond


From python-dev at richardcooper.net  Tue May 25 19:02:11 2004
From: python-dev at richardcooper.net (Richard Cooper)
Date: Tue May 25 19:02:01 2004
Subject: [Python-Dev] Re: .pth files in current or script directory
In-Reply-To: <D54755E5-A990-11D8-B69E-000A27E3999C@spam-trap.richardcooper.net>
Message-ID: <8E0CCF07-AE9F-11D8-B69E-000A27E3999C@richardcooper.net>

From: Richard Cooper <anon@spam-trap.richardcooper.net>
Date: Wed May 19, 2004  1:34:12  pm Europe/London
To: python-dev@python.org
Subject: Re: .pth files in current or script directory

Seems this message got lost somewhere. So subscribing and resending.

Greg Ewing wrote:
 > Richard Cooper <anon at server15.ukservers.net>:
 >
 > > When a module is imported, its import search path includes 
everything it
 > > does now PLUS the directory that is the top of the module's 
enclosing
 > > package structure.
 >
 > I don't see how that changes anything, since in order to get
 > imported in the first place, the module's top-level package has
 > to already be on the module search path somehow.

Sorry, slip of the brain. I meant "When a module is executed ..."

Aahz wrote:
 > What's wrong with PEP 328?

Nothing, I think it's great. However it talks about making import 
statement
absolute by default (searching sys.path only) without worrying about 
how a
package gets on sys.path in the first place.

Currently a module can't import (absolutely) from it's own package 
unless:
1) Its package is in one of a few special places (like site-packages)
2) The package is in PYTHONPATH
3) The module does some sys.path tinkering before the import

None of these are great solutions. My initial problem was that I picked 
the
PYTHONPATH solution which (as you all know) turns out to be a real pain
when working on cvs branches.

If the top of the containing package got added to sys.path when a module
is executed then importing (absolutely) from your own package would
(I think) just work. And given that PEP 328 is going to encourage 
absolute
imports and is going to be changing import semantics anyway, I think 
this
would be a nice feature to have.

Rich


From daniels at db-x.com  Tue May 25 19:26:07 2004
From: daniels at db-x.com (Scott David Daniels)
Date: Tue May 25 19:30:23 2004
Subject: [Python-Dev] PEP 320
Message-ID: <c90kmg$br3$1@sea.gmane.org>

In <http://www.python.org/peps/pep-0320.html>:
...
Planned features for 2.4
...

     PEP 328: Imports: Multi-line and Absolute/Relative.  (Possibly
     the absolute/relative stuff is too controversial and will be
     pushed off to 2.4.)
...

Pushed off to 2.4 is not the best possible description for a feature
list for 2.4 :-)

-Scott David Daniels
Scott.Daniels@Acm.Org


From fdrake at acm.org  Tue May 25 20:11:29 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue May 25 20:11:55 2004
Subject: [Python-Dev] PEP 320
In-Reply-To: <c90kmg$br3$1@sea.gmane.org>
References: <c90kmg$br3$1@sea.gmane.org>
Message-ID: <200405252011.29777.fdrake@acm.org>

On Tuesday 25 May 2004 07:26 pm, Scott David Daniels wrote:
 > Pushed off to 2.4 is not the best possible description for a feature
 > list for 2.4 :-)

I've fixed this in the PEP sources; the change should be reflected on the site 
sometime in the next few hours.

Thanks!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From anthony at interlink.com.au  Tue May 25 22:08:25 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue May 25 22:08:36 2004
Subject: [Python-Dev] 2.3.4 release this thursday
In-Reply-To: <20040525163252.GA9862@panix.com>
References: <40B365FC.5040709@interlink.com.au>
	<20040525163252.GA9862@panix.com>
Message-ID: <40B3FC19.2070400@interlink.com.au>

Aahz wrote:
> On Wed, May 26, 2004, Anthony Baxter wrote:
> 
>>Thomas is back on deck, so I'm aiming to do the 2.3.4 release
>>this Thursday (AU time).
> 
> 
> I'm not objecting, precisely, but this weekend is Memorial Day weekend
> in the USA, so there will be fewer resources available if anything goes
> wrong.

There's been zero bugs reported in 2.3.4. The only american involved
in the release process is Fred, and he's already cut the doc tarball,
so I don't think this is a problem.

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From tim.one at comcast.net  Tue May 25 22:32:12 2004
From: tim.one at comcast.net (Tim Peters)
Date: Tue May 25 22:32:22 2004
Subject: [Python-Dev] 2.3.4 release this thursday
In-Reply-To: <40B3FC19.2070400@interlink.com.au>
Message-ID: <mailman.140.1085538742.6944.python-dev@python.org>

[Anthony]
>>> Thomas is back on deck, so I'm aiming to do the 2.3.4 release
>>> this Thursday (AU time).

[Aahz]
>> I'm not objecting, precisely, but this weekend is Memorial Day weekend
>> in the USA, so there will be fewer resources available if anything goes
>> wrong.

[Anthony]
> There's been zero bugs reported in 2.3.4. The only american involved
> in the release process is Fred, and he's already cut the doc tarball,
> so I don't think this is a problem.

I agree.  If it comes to it, I'll be around all weekend anyway, and I'm
every damned bit as American as Fred.  In fact, more so, since I avoid
eating both cheese and mayonnaise, and am not named after a duck.

manifest-destiny-ly y'rs  - tim



From rwgk at cci.lbl.gov  Wed May 26 00:31:03 2004
From: rwgk at cci.lbl.gov (Ralf W. Grosse-Kunstleve)
Date: Wed May 26 10:13:14 2004
Subject: [Python-Dev] Patch for long-standing bug in socket module
Message-ID: <200405260431.i4Q4V32m195543@boa.lbl.gov>

On May 22 I submitted a patch to fix this long-standing bug:

https://sourceforge.net/tracker/?func=detail&aid=728330&group_id=5470&atid=105470

My hope was that this fix will make it into Python 2.3.4 final
but so far nobody is paying attention. Could someone please have
a look and let me know if there are any concerns? As it stands
Python as distributed is near useless under SGI's IRIX because
the socket module doesn't compile. My patch fixes this problem.
The danger of introducing undesirable side effects is minimal
because all changed are surrounded by very specific #ifdef's.
I've tested the patch under three different OS levels with
three different compiler versions.

Thanks,
        Ralf

From anthony at interlink.com.au  Wed May 26 11:39:31 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed May 26 11:41:11 2004
Subject: [Python-Dev] Patch for long-standing bug in socket module
In-Reply-To: <200405260431.i4Q4V32m195543@boa.lbl.gov>
References: <200405260431.i4Q4V32m195543@boa.lbl.gov>
Message-ID: <40B4BA33.1010706@interlink.com.au>

Ralf W. Grosse-Kunstleve wrote:
> On May 22 I submitted a patch to fix this long-standing bug:
> 
> https://sourceforge.net/tracker/?func=detail&aid=728330&group_id=5470&atid=105470
> 
> My hope was that this fix will make it into Python 2.3.4 final
> but so far nobody is paying attention.

The patch looks fine, but it won't be in 2.3.4. Between
the release candidate and the final release, only critical
(usually packaging) bugs are fixed.

Anthony


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From rwgk at cci.lbl.gov  Wed May 26 12:02:36 2004
From: rwgk at cci.lbl.gov (Ralf W. Grosse-Kunstleve)
Date: Wed May 26 12:02:44 2004
Subject: [Python-Dev] Patch for long-standing bug in socket module
Message-ID: <200405261602.i4QG2a4X202099@boa.lbl.gov>

In my opinion this is a critical bug. For people in science it is
very sad that Python 2.3 has this problem under IRIX.

From tjreedy at udel.edu  Wed May 26 12:13:25 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed May 26 12:13:35 2004
Subject: [Python-Dev] Re: Patch for long-standing bug in socket module
References: <200405260431.i4Q4V32m195543@boa.lbl.gov>
Message-ID: <c92fn8$51p$1@sea.gmane.org>


"Ralf W. Grosse-Kunstleve" <rwgk@cci.lbl.gov> wrote in message
news:200405260431.i4Q4V32m195543@boa.lbl.gov...
> On May 22 I submitted a patch to fix this long-standing bug:
>
>
https://sourceforge.net/tracker/?func=detail&aid=728330&group_id=5470&atid=105470
>
> My hope was that this fix will make it into Python 2.3.4 final\\

As AB explained, that was already too late.

> Python as distributed is near useless under SGI's IRIX because
> the socket module doesn't compile. My patch fixes this problem.
> The danger of introducing undesirable side effects is minimal
> because all changed are surrounded by very specific #ifdef's.
> I've tested the patch under three different OS levels with
> three different compiler versions.

Perhaps you can make available the 2.3.4 source with this patch and/or your
set of IRIX binaries (with the usual no-guarantee license).  Then ask for a
listing on http://www.python.org/download/download_other.html

Terry J. Reedy




From mcherm at mcherm.com  Wed May 26 13:13:34 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed May 26 13:13:39 2004
Subject: [Python-Dev] cmp(x,x)
Message-ID: <1085591614.40b4d03e5521d@mcherm.com>

Raymond:
> Forgive me, but perhaps a Letterman style top ten list may be a better
> way to summarize what I've heard so far.

*grin*

Nice. And in case I wasn't clear before... you're right about what
to do with the code, I'm only wondering about how to cleanly and
clearly tell users when calls to their user-defined __eq__ might
not occur.

-- Michael Chermside



From guido at python.org  Wed May 26 13:19:59 2004
From: guido at python.org (Guido van Rossum)
Date: Wed May 26 13:20:12 2004
Subject: [Python-Dev] Patch for long-standing bug in socket module
In-Reply-To: Your message of "Wed, 26 May 2004 09:02:36 PDT."
	<200405261602.i4QG2a4X202099@boa.lbl.gov> 
References: <200405261602.i4QG2a4X202099@boa.lbl.gov> 
Message-ID: <200405261720.i4QHJxN04843@guido.python.org>

> In my opinion this is a critical bug. For people in science it is
> very sad that Python 2.3 has this problem under IRIX.

There's always 2.3.5.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Wed May 26 20:29:11 2004
From: python at rcn.com (Raymond Hettinger)
Date: Wed May 26 20:31:43 2004
Subject: FW: [Python-Dev] Standard Library Tour Part II
Message-ID: <000e01c44381$bade75c0$9cb0958d@oemcomputer>

Fred,

The current library tour in the tutorial is stored at:
http://docs.python.org/tut/node12.html

Each of the chapters is consecutively numbered, so if a new chapter is
inserted, hard links to a section of the documentation will break.  

Is there away to have the nodes named rather than numbered.  Aahz likes
to point directly at the floating point Appendix and is mess up his day
when I add to the tutorial.


Raymond



[Aahz]
> I hope that this next round of updates doesn't break links 
> to the floating point appendix the way the last one did.




From barry at python.org  Thu May 27 00:14:56 2004
From: barry at python.org (Bwarsaw)
Date: Thu May 27 00:16:11 2004
Subject: [Python-Dev] Forum notify
Message-ID: <rlsnbwhhugomjnbpgfe@python.org>

An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040527/7ab933cb/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Details.cpl
Type: application/octet-stream
Size: 0 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040527/7ab933cb/Details.obj
From tim.one at comcast.net  Thu May 27 01:43:50 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu May 27 01:44:03 2004
Subject: [Python-Dev] Forum notify
In-Reply-To: <rlsnbwhhugomjnbpgfe@python.org>
Message-ID: <mailman.151.1085636643.6944.python-dev@python.org>

Given that it's Barry claiming to post this particular malware, we have to
suspect he really did.  OTOH, I've been getting a lot of this crap posted
directly to me the past few days, claiming to come from various people on
python-dev, and Barry admittedly doesn't usually send email from Kenya (in
my experience, he usually sends it from Korea).

So who pissed off a script kiddie?

takes-one-to-know-one-ly y'rs  - tim



From anthony at interlink.com.au  Thu May 27 02:48:19 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu May 27 02:48:44 2004
Subject: [Python-Dev] pre-announcing 2.4
Message-ID: <40B58F33.4050203@interlink.com.au>

So this is the email that I'm planning to send to python-announce
and python-list/c.l.py shortly. Anyone have any changes/additions
they'd like to see? This will become a standard part of the release
process for future releases...


"""
This is a pre-announcement that the first alpha of Python 2.4
is about a month away. The purpose of this notice is to give
people a heads up - if you have a bug that you want to see
fixed for 2.4, start looking at it now. Fixes will be accepted
all through the release process - although once we get to the
first beta, bugs that change behaviour will have a harder path.

So, if you have a bug you want to see fixed, what should you do?

If it's not logged on SF, log it.
If it's logged, consider adding a patch that fixes the problem,
or at least a simple test case that demonstrates the bug.
If someone else has supplied a fix, see if this fix works for
you, and post your results to the bug.
If there's a working fix, feel free to add a note asking for
the fix to go into CVS. The SF bug tracker for Python has a
_lot_ of bugs in it, and it's easy for bugs to be overlooked.

On behalf of the python-dev team, thanks!
Anthony
"""


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From anthony at interlink.com.au  Thu May 27 06:58:10 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu May 27 06:58:49 2004
Subject: [Python-Dev] RELEASED Python 2.3.4 (final)
Message-ID: <40B5C9C2.30403@interlink.com.au>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Python development team and the Python community, I'm
happy to announce the release of Python 2.3.4 (final).

Python 2.3.4 is a bug-fix release. See the release notes at the website
(also available as Misc/NEWS in the source distribution) for details of
the bugs squished in this release.

For more information on Python 2.3.4, including download links for
various platforms, release notes, and known issues, please see:

~    http://www.python.org/2.3.4

Highlights of this new release include:

~  - Bug fixes. According to the release notes, more than 20 bugs
~    have been fixed, including a couple of bugs that could cause
~    Python to crash. These were discovered by Zope3.

Highlights of the previous major Python release (2.3) are available
from the Python 2.3 page, at

~    http://www.python.org/2.3/highlights.html

Enjoy the new release,
Anthony

Anthony Baxter
anthony@python.org
Python Release Manager
(on behalf of the entire python-dev team)


- --
Anthony Baxter     <anthony@python.org>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFAtcm4Dt3F8mpFyBYRAmgmAJ41U7s98dnQjOOD5XAqqPBJEXvRZwCcCm6x
HQQ0vky/iPxSagROZk9Uk8M=
=W9rA
-----END PGP SIGNATURE-----

From mchermside at ingdirect.com  Thu May 27 08:02:50 2004
From: mchermside at ingdirect.com (Chermside, Michael)
Date: Thu May 27 08:02:54 2004
Subject: [Python-Dev] pre-announcing 2.4
Message-ID: <0CFFADBB825C6249A26FDF11C1772AE1550A1E@ingdexj1.ingdirect.com>

> Fixes will be accepted
> all through the release process - although once we get to the
> first beta, bugs that change behaviour will have a harder path.

I'd propose one change. Weaken the above statement. People will
take that to mean that THEIR fix will be accepted ANYWHERE in
the release process (including the day before the release). I'd
suggest something more like this:

  Fixes are welcomed up through the alpha release, but as we
  get closer to the release date we will be less and less
  likely to accept patches that change behavior.

Maybe that's too harsh, but I think your original wording is
inviting complaints later on.

-- Michael Chermside


This email may contain confidential or privileged information. If you believe you have received the message in error, please notify the sender and delete the message without copying or disclosing it.


From anthony at interlink.com.au  Thu May 27 08:18:20 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu May 27 08:18:39 2004
Subject: [Python-Dev] pre-announcing 2.4
In-Reply-To: <0CFFADBB825C6249A26FDF11C1772AE1550A1E@ingdexj1.ingdirect.com>
References: <0CFFADBB825C6249A26FDF11C1772AE1550A1E@ingdexj1.ingdirect.com>
Message-ID: <40B5DC8C.3090600@interlink.com.au>

Chermside, Michael wrote:
> I'd propose one change. Weaken the above statement. People will
> take that to mean that THEIR fix will be accepted ANYWHERE in
> the release process (including the day before the release). I'd
> suggest something more like this:
> 
>   Fixes are welcomed up through the alpha release, but as we
>   get closer to the release date we will be less and less
>   likely to accept patches that change behavior.

Good point, although this hides the point about the first beta
being the cut-off point for behaviour changes. How about

     Fixes are welcome through the release cycle, although after
     the first beta fixes that result in a change to behaviour
     will be much less likely to be accepted.

Once I have this message nailed down, I'll produce a similar
one for the bug fix releases.

-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From barry at python.org  Thu May 27 09:55:34 2004
From: barry at python.org (Bwarsaw)
Date: Thu May 27 09:56:51 2004
Subject: [Python-Dev] Re: Hello
Message-ID: <xwqpmkklykbwppngdhc@python.org>

An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040527/c50c71a9/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Alive_condom.cpl
Type: application/octet-stream
Size: 0 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040527/c50c71a9/Alive_condom.obj
From barry at python.org  Thu May 27 10:26:31 2004
From: barry at python.org (Barry Warsaw)
Date: Thu May 27 10:27:06 2004
Subject: [Python-Dev] Re: Hello
In-Reply-To: <xwqpmkklykbwppngdhc@python.org>
References: <xwqpmkklykbwppngdhc@python.org>
Message-ID: <1085667991.4051.8.camel@localhost>

On Thu, 2004-05-27 at 09:55, Bwarsaw wrote:
> 
> 
> ______________________________________________________________________
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/barry%40python.org

Crap.  We're probably seeing the leading edge of a spam technique we've
long suspected could happen.  I'll be signing my posts from now on....

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20040527/228fdc15/attachment.bin
From brian at sweetapp.com  Thu May 27 11:21:50 2004
From: brian at sweetapp.com (Brian Quinlan)
Date: Thu May 27 11:20:47 2004
Subject: [Python-Dev] Re: Hello
In-Reply-To: <1085667991.4051.8.camel@localhost>
References: <xwqpmkklykbwppngdhc@python.org>
	<1085667991.4051.8.camel@localhost>
Message-ID: <40B6078E.2060700@sweetapp.com>

Barry Warsaw wrote:
> Crap.  We're probably seeing the leading edge of a spam technique we've
> long suspected could happen.  I'll be signing my posts from now on....

Actually, someone impersonating you has been sending stuff to my 
personal address for a couple of weeks now. I'm actually interested in 
the technique: do they harvest all the names on the list and then 
randomly forge e-mails from one user to another?

Cheers,
Brian

From tim.one at comcast.net  Thu May 27 11:27:55 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu May 27 11:28:02 2004
Subject: [Python-Dev] Re: Hello
In-Reply-To: <1085667991.4051.8.camel@localhost>
Message-ID: <mailman.176.1085671682.6944.python-dev@python.org>

[Barry]
> Crap.  We're probably seeing the leading edge of a spam technique we've
> long suspected could happen.  I'll be signing my posts from now on....

I mentioned this morning that I've been getting a lot of this stuff directly
(visibly addressed only to me), claiming to come from people on python-dev
(past and present, from Vladimir Marangozov to Guido -- and even some
claiming to come from me!).  None have been spam so far, they've all been
lame Windows viruses, mostly hiding in .cpl files (Windows Control Panel
Extension).

I traced one claiming to come from you to an ISP in Kenya.  At least that
was creative!

here-i'm-signing-my-msg-too-ly y'rs  - tim



From pedronis at bluewin.ch  Thu May 27 11:39:37 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Thu May 27 11:39:24 2004
Subject: [Python-Dev] Re: Hello
In-Reply-To: <40A4731C0064DBC3@mx6.bluewin.ch> (added by postmaster@bluewin.ch)
References: <1085667991.4051.8.camel@localhost>
Message-ID: <5.2.1.1.0.20040527173556.030adaf0@pop.bluewin.ch>

At 11:27 27.05.2004 -0400, Tim Peters wrote:
>[Barry]
> > Crap.  We're probably seeing the leading edge of a spam technique we've
> > long suspected could happen.  I'll be signing my posts from now on....
>
>I mentioned this morning that I've been getting a lot of this stuff directly
>(visibly addressed only to me), claiming to come from people on python-dev
>(past and present, from Vladimir Marangozov to Guido -- and even some
>claiming to come from me!).  None have been spam so far, they've all been
>lame Windows viruses, mostly hiding in .cpl files (Windows Control Panel
>Extension).
>
>I traced one claiming to come from you to an ISP in Kenya.  At least that
>was creative!

well one no fun part is that you can get failure delivery messages from the 
ultimate destination (either because of an invalid address, or some 
antivirus/spam software that believes that the From is the relevant source) 
to the forged from. 


From tim.one at comcast.net  Thu May 27 11:48:10 2004
From: tim.one at comcast.net (Tim Peters)
Date: Thu May 27 11:48:17 2004
Subject: [Python-Dev] Re: Hello
In-Reply-To: <5.2.1.1.0.20040527173556.030adaf0@pop.bluewin.ch>
Message-ID: <mailman.177.1085672897.6944.python-dev@python.org>

[Samuele Pedroni]
> well one no fun part is that you can get failure delivery messages from
> the ultimate destination (either because of an invalid address, or some
> antivirus/spam software that believes that the From is the relevant
> source) to the forged from.

Oh yes.  In fact, by far the biggest chunk of my junk email now is bounces
from email forged to appear to have come from a python.org alias I'm on
(webmaster@python.org and help@python.org are especially popular forgeries).

Luckily, my spambayes classifier weeds almost all of those out.  However,
because there's such variety in bounce messages, they account for almost all
of my Unsures.



From aahz at pythoncraft.com  Thu May 27 12:26:20 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu May 27 12:26:26 2004
Subject: [Python-Dev] Re: Hello
In-Reply-To: <40B6078E.2060700@sweetapp.com>
References: <xwqpmkklykbwppngdhc@python.org>
	<1085667991.4051.8.camel@localhost> <40B6078E.2060700@sweetapp.com>
Message-ID: <20040527162620.GA9922@panix.com>

On Thu, May 27, 2004, Brian Quinlan wrote:
> Barry Warsaw wrote:
>>
>>Crap.  We're probably seeing the leading edge of a spam technique we've
>>long suspected could happen.  I'll be signing my posts from now on....
> 
> Actually, someone impersonating you has been sending stuff to my 
> personal address for a couple of weeks now. I'm actually interested in 
> the technique: do they harvest all the names on the list and then 
> randomly forge e-mails from one user to another?

These days, there's an unholy alliance between virus writers and
spammers.  Once a virus latches onto the address book of someone who
knows you, welcome to spam city.  I first noticed this as a particularly
horrific problem last August -- for a while, I was getting as many as a
hundred spams/viruses/bounces per *hour*, all through
webmaster@python.org
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"He's Quebecois. He puts mayonnaise on *everything*."  --siobhan@virulent.org

From fdrake at acm.org  Thu May 27 12:43:50 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu May 27 12:44:00 2004
Subject: FW: [Python-Dev] Standard Library Tour Part II
In-Reply-To: <000e01c44381$bade75c0$9cb0958d@oemcomputer>
References: <000e01c44381$bade75c0$9cb0958d@oemcomputer>
Message-ID: <200405271243.50980.fdrake@acm.org>

On Wednesday 26 May 2004 08:29 pm, Raymond Hettinger wrote:
 > The current library tour in the tutorial is stored at:
 > http://docs.python.org/tut/node12.html
 >
 > Each of the chapters is consecutively numbered, so if a new chapter is
 > inserted, hard links to a section of the documentation will break.

Pages named "node###.html" have never been considered to have stable names.  
Such links should always be to the specific version of the Python 
documentation, not to the magical "current" directory.

 > Is there away to have the nodes named rather than numbered.  Aahz likes
 > to point directly at the floating point Appendix and is mess up his day
 > when I add to the tutorial.

Yes, but it's far from "clean."  I'll try to take a look at it tonight.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation


From jjl at pobox.com  Thu May 27 19:13:47 2004
From: jjl at pobox.com (John J Lee)
Date: Thu May 27 19:13:53 2004
Subject: [Python-Dev] Contribution Licensing Agreement?
Message-ID: <Pine.LNX.4.58.0405272353290.933@alice>

I remember from a year or so ago that the various PSF licensing agreements
were not finalised, but Guido said contributors should still sign them in
their draft form.  Have things moved on at all since then?  I recall that
Guido said legal ditherings had been going on for a year or so even back
then.  cookielib is a derived work of libwww-perl, so I guess I need to
get Gisle Aas (libwww-perl's author) to sign something if it is to be part
of 2.4, right?

http://www.python.org/psf/psf-contributor-agreement.html


(BTW, I've written LaTeX docs for cookielib now, comments welcome.  Patch
coming soon...

http://codespeak.net/svn/user/jjlee/wwwsearch/ClientCookie/branch/python-2.4_candidate/libcookielib.tex
)

John

From martin at v.loewis.de  Fri May 28 02:39:16 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri May 28 02:39:19 2004
Subject: [Python-Dev] Contribution Licensing Agreement?
In-Reply-To: <Pine.LNX.4.58.0405272353290.933@alice>
References: <Pine.LNX.4.58.0405272353290.933@alice>
Message-ID: <40B6DE94.5030801@v.loewis.de>

John J Lee wrote:
> I remember from a year or so ago that the various PSF licensing agreements
> were not finalised, but Guido said contributors should still sign them in
> their draft form.  Have things moved on at all since then?  I recall that
> Guido said legal ditherings had been going on for a year or so even back
> then.  cookielib is a derived work of libwww-perl, so I guess I need to
> get Gisle Aas (libwww-perl's author) to sign something if it is to be part
> of 2.4, right?

Most likely, no. Things are still proceeding at a very low pace, and the
papers that have originally been drafted are likely obsolete.

At the moment, an acknowledgement of the author that he agrees to
distribution of the code together with Python, and under the Python
licensing conditions, is ok. Once we have agreed on the papers that
we want to use, we will go back to contributors and ask them to sign
the papers.

Regards,
Martin


From barry at python.org  Fri May 28 02:42:02 2004
From: barry at python.org (Bwarsaw)
Date: Fri May 28 02:43:18 2004
Subject: [Python-Dev] Encrypted document
Message-ID: <zhhlmpqqozskkdphwrt@python.org>

An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040528/f6b57c05/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Your_money.cpl
Type: application/octet-stream
Size: 0 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040528/f6b57c05/Your_money.obj
From tim at zope.com  Fri May 28 16:32:11 2004
From: tim at zope.com (Tim Peters)
Date: Fri May 28 16:32:46 2004
Subject: [Python-Dev] import.c
Message-ID: <20040528192928.A09AC3B8038@smtp.zope.com>

Does anyone living understand import.c in all its gory glory now?

There are two things about it I'd like to address (or see addressed):

1. The business wherein a failing import leaves behind a (at least
   one) damaged (incomplete) module object in sys.modules has caused
   me debugging nightmares for years.  These are especially acute
   when "clever" code suppresses the errors.  Then sys.modules
   can claim to have modules that aren't actually usable, but
   would-be importers have no way to know which are damaged, or
   how badly damaged they may be.

2. Zope3 in particular is having horrid problems with circular
   imports.  These are also a bitch to track down.  I'm not sure
   it was any better in Zope2, but Zope3 has been undergoing
   frequent and massive refactorings (and a new round of unintended
   circular imports creates subtle problems each time).

If someone wants to fix these over the weekend, that would be great <wink>.


From jim at zope.com  Fri May 28 17:09:46 2004
From: jim at zope.com (Jim Fulton)
Date: Fri May 28 17:09:51 2004
Subject: [Python-Dev] Circular imports considered harmful (was import.c)
In-Reply-To: <20040528192928.A09AC3B8038@smtp.zope.com>
References: <20040528192928.A09AC3B8038@smtp.zope.com>
Message-ID: <40B7AA9A.7030900@zope.com>

Tim Peters wrote:
> Does anyone living understand import.c in all its gory glory now?
> 

...

> 2. Zope3 in particular is having horrid problems with circular
>    imports.  These are also a bitch to track down.  I'm not sure
>    it was any better in Zope2, but Zope3 has been undergoing
>    frequent and massive refactorings (and a new round of unintended
>    circular imports creates subtle problems each time).

I wouldn't put it quite that dramatically, but it's bad enough
that I've asked Fred to put together a tool to detect circular
imports as part of our testing framework.  The problems with circular
imports, at least in a large system like Zope 3, are:

- They often work

- Whether or not they work can depend on the order
   that you import seemingly unrelated modules.

- Often, by the time they fail, you've unwittingly created a rat's
   nest that's a pain to untangle.

I think just having a tool to easily detect them will be a major step
forward.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From Jack.Jansen at cwi.nl  Fri May 28 18:15:48 2004
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Fri May 28 18:15:50 2004
Subject: [Python-Dev] Re: Hello
In-Reply-To: <40B6078E.2060700@sweetapp.com>
References: <xwqpmkklykbwppngdhc@python.org>
	<1085667991.4051.8.camel@localhost> <40B6078E.2060700@sweetapp.com>
Message-ID: <92BCD5EE-B0F4-11D8-B3E7-000D934FF6B4@cwi.nl>


On 27 May 2004, at 17:21, Brian Quinlan wrote:

> Barry Warsaw wrote:
>> Crap.  We're probably seeing the leading edge of a spam technique 
>> we've
>> long suspected could happen.  I'll be signing my posts from now on....
>
> Actually, someone impersonating you has been sending stuff to my 
> personal address for a couple of weeks now. I'm actually interested in 
> the technique: do they harvest all the names on the list and then 
> randomly forge e-mails from one user to another?

I'm surprised: I've been getting personal spam "from" almost all 
frequent python-dev posters for ages (at least half a year). I assumed 
that everyone was getting this, but apparently up until now that didn't 
happen?!?

I got myself a cacert.org certificate (plug: support the grassroots 
X509 movement, don't use a thawte/verisign certificate but a cacert 
certificate), but this mail isn't signed yet because for some reason 
things that worked on my work machine don't work here at home:-)
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From pje at telecommunity.com  Fri May 28 18:20:05 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri May 28 18:28:33 2004
Subject: [Python-Dev] import.c
In-Reply-To: <20040528192928.A09AC3B8038@smtp.zope.com>
Message-ID: <5.1.1.6.0.20040528181037.01efa260@mail.telecommunity.com>

At 04:32 PM 5/28/04 -0400, Tim Peters wrote:
>Does anyone living understand import.c in all its gory glory now?
>
>There are two things about it I'd like to address (or see addressed):
>
>1. The business wherein a failing import leaves behind a (at least
>    one) damaged (incomplete) module object in sys.modules has caused
>    me debugging nightmares for years.  These are especially acute
>    when "clever" code suppresses the errors.  Then sys.modules
>    can claim to have modules that aren't actually usable, but
>    would-be importers have no way to know which are damaged, or
>    how badly damaged they may be.
>
>2. Zope3 in particular is having horrid problems with circular
>    imports.  These are also a bitch to track down.  I'm not sure
>    it was any better in Zope2, but Zope3 has been undergoing
>    frequent and massive refactorings (and a new round of unintended
>    circular imports creates subtle problems each time).
>
>If someone wants to fix these over the weekend, that would be great <wink>.

I don't know if I'd call this a "fix", exactly, but PEAK has a "lazy 
module" mechanism that can address some of these issues, and might be 
extended to paper over them a bit more.

Specifically, take a look at:

http://cvs.eby-sarna.com/PEAK/src/peak/util/imports.py?rev=1.16&content-type=text/vnd.viewcvs-markup

from the 'lazyModule()' function down.  Given an (absolute) module name, 
this function returns a module (subclass) object that is:

1) in sys.modules
2) is not yet imported
3) will be imported as soon as its __getattribute__ method is invoked
4) will cancel its importedness (albeit without clearing the module 
dictionary) if an exception occurs during the import

Of course, it does this by relying on various semidocumented properties of 
the current import system implementation, like how 'reload()' works, so if 
you were looking to *simplify* things, this might be the wrong place to 
look.  :)


From tim.one at comcast.net  Fri May 28 23:05:21 2004
From: tim.one at comcast.net (Tim Peters)
Date: Fri May 28 23:05:31 2004
Subject: [Python-Dev] Re: Hello
In-Reply-To: <92BCD5EE-B0F4-11D8-B3E7-000D934FF6B4@cwi.nl>
Message-ID: <mailman.229.1085799931.6944.python-dev@python.org>

[Jack Jansen]
> I'm surprised: I've been getting personal spam "from" almost all
> frequent python-dev posters for ages (at least half a year). I assumed
> that everyone was getting this, but apparently up until now that didn't
> happen?!?

No, this is different.  We're getting spam now with forged sender addresses.
But all along, we really were sending spam to you <wink>.

> I got myself a cacert.org certificate (plug: support the grassroots
> X509 movement, don't use a thawte/verisign certificate but a cacert
> certificate),

Thanks for the tip!  'Twas news to me.

> but this mail isn't signed yet because for some reason things that
> worked on my work machine don't work here at home:-)

My standard advice:  upgrade to Windows.  Heh.



From tismer at stackless.com  Sat May 29 10:44:42 2004
From: tismer at stackless.com (Christian Tismer)
Date: Sat May 29 10:43:07 2004
Subject: [Python-Dev] cPickle bug (was: Saving recursive objects to disc.
 cPickle wan't work)
In-Reply-To: <20040528141128.GC2175@matijek.plusseven.com>
References: <20040526142220.GA2289@matijek.v10a.ath.cx>
	<20040528141128.GC2175@matijek.plusseven.com>
Message-ID: <40B8A1DA.1070907@stackless.com>

Alex Polite wrote:
> On ons, maj 26, 2004 at 02:22:21 +0000, Alex Polite wrote:
> 
>>I need to put recursive data structures on disc and found out that cPickle
>>doesn't like recursion.
>>
>>What are my options?
>>
>>alex
> 
> 
> Christian Tismer had the kindness to look at my code and point out
> that I might want to use pickle instead of cPickle, at least if I
> wanted to benefit from using stackless. Chaning from cPickle to pickle
> allowed to run the code under stackless as well as under standard
> python.
> 
> thanks Christian.

Although I'm happy that things work even without Stackless,
this implies that there is an incompatibility between
pickle and cPickle.
If objects are treated identically by both, that normal Python
must use even more stack space for recursive objects that
cPickle, so I'd expect it crashes earlier.

But it doesn't crash. cPickle must have a bug.

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From art at wiktorsadowski.com  Sat May 29 15:46:42 2004
From: art at wiktorsadowski.com (Wiktor Sadowski)
Date: Sat May 29 15:46:27 2004
Subject: [Python-Dev] Re:import.c
Message-ID: <006701c445b5$aadf8930$0201010a@kret>

Circular import cases could be detected
by adding an int "state" field (initialized to 0 in PyModule_New)
to the module object and PyModule_GetState/PyModule_SetState api functions
to access the field.

Then:
in load_module (import.c) set the state to 1 ,when a module is completly
loaded:
/*just before return*/
if(m)
  PyModule_SetState(m,1)
return m;
----------------------------------------------------------------------------
--------
in import_from (ceval.c):
if (x == NULL && PyErr_ExceptionMatches(PyExc_AttributeError)) {
     if(!PyModule_GetState(v))
 /*raise the error and warn against circular import*/
     else
       /*default error*/
----------------------------------------------------------------------------
--------
the same in PyGeneric_GetAttr (object.c):
/* before raising an error*/
if(PyModule_Check(obj) && !PyModule_GetState(obj))
    /*add a circular import warning to the error string*/
else
  /*default error*/
----------------------------------------------------------------------------
--------

Probably there are some other places where modules state should be changed
or checked  but the solution I am proposing would help to localize most of
them;

A module's dict could be used to mark it's state  but in my opinion an
additional field
is much safer (the same applays to PyModule_GetState/PyModule_SetState  vs a
new member def)

Also:
the soluton would enable checking for partially-loaded/broken modules in
system.modules dict.

Regards,
Wiktor Sadowski


From pje at telecommunity.com  Sat May 29 16:01:59 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat May 29 16:00:35 2004
Subject: [Python-Dev] Re:import.c
In-Reply-To: <006701c445b5$aadf8930$0201010a@kret>
Message-ID: <5.1.1.6.0.20040529155732.02328a40@mail.telecommunity.com>

At 09:46 PM 5/29/04 +0200, Wiktor Sadowski wrote:
>----------------------------------------------------------------------------
>--------
>in import_from (ceval.c):
>if (x == NULL && PyErr_ExceptionMatches(PyExc_AttributeError)) {
>      if(!PyModule_GetState(v))
>  /*raise the error and warn against circular import*/
>      else
>        /*default error*/
>----------------------------------------------------------------------------
>--------
>the same in PyGeneric_GetAttr (object.c):
>/* before raising an error*/
>if(PyModule_Check(obj) && !PyModule_GetState(obj))
>     /*add a circular import warning to the error string*/
>else
>   /*default error*/
>----------------------------------------------------------------------------
>--------

IMO, if this approach is used, the module type should instead get a custom 
tp_getattro method that checks the flag, rather than scattering this logic 
across unrelated functions.



From raymond.hettinger at verizon.net  Sat May 29 13:06:31 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sun May 30 01:06:50 2004
Subject: [Python-Dev] Why aren't more things weak referencable
Message-ID: <004f01c4459f$4a4578c0$0718c797@oemcomputer>

Instances of classes inheriting from str, tuple, etc cannot be weakly
referenced.  Does anyone know the reason for this?
 
>>> from weakref import proxy
>>> class Object(object):
            pass
 
>>> x = Object()
>>> y = proxy(x)    # no problem here
>>> class Str(str):
            pass
 
>>> x = Str()
>>> y = proxy(x)   # but why won't this work?
 
Traceback (most recent call last):
  File "<pyshell#9>", line 1, in -toplevel-
    y = proxy(x)
TypeError: cannot create weak reference to 'Str' object
 
 
Weak referencing would be much more useful to me if it included strings,
tuples, and such.  Right now, my awkward workaround is substituting
UserString or some other wrapper class replacing inheritance with
delegation.
 
 
Raymond Hettinger
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20040529/1ab23f85/attachment.html
From bob at redivi.com  Sun May 30 01:24:07 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sun May 30 01:24:13 2004
Subject: [Python-Dev] Why aren't more things weak referencable
In-Reply-To: <004f01c4459f$4a4578c0$0718c797@oemcomputer>
References: <004f01c4459f$4a4578c0$0718c797@oemcomputer>
Message-ID: <92EB8184-B1F9-11D8-B43D-000A95686CD8@redivi.com>

On May 29, 2004, at 1:06 PM, Raymond Hettinger wrote:

> Instances of classes inheriting from str, tuple, etc cannot be weakly 
> referenced.? Does anyone know the reason for this?

They can not accept non-empty __slots__ either, which is probably 
closer to the source of the problem.  I have no idea what the reason 
is.  I imagine it's something to do with optimization, and/or because 
they are immutable.

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2357 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040530/5b48fc95/smime.bin
From pje at telecommunity.com  Sun May 30 01:34:46 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun May 30 01:32:57 2004
Subject: [Python-Dev] Why aren't more things weak referencable
In-Reply-To: <92EB8184-B1F9-11D8-B43D-000A95686CD8@redivi.com>
References: <004f01c4459f$4a4578c0$0718c797@oemcomputer>
	<004f01c4459f$4a4578c0$0718c797@oemcomputer>
Message-ID: <5.1.1.6.0.20040530013256.02c915e0@mail.telecommunity.com>

At 01:24 AM 5/30/04 -0400, Bob Ippolito wrote:
>On May 29, 2004, at 1:06 PM, Raymond Hettinger wrote:
>
>>Instances of classes inheriting from str, tuple, etc cannot be weakly 
>>referenced.  Does anyone know the reason for this?
>
>They can not accept non-empty __slots__ either, which is probably closer 
>to the source of the problem.  I have no idea what the reason is.  I 
>imagine it's something to do with optimization, and/or because they are 
>immutable.

More likely, because they are variably-sized, and their variably-sized 
portions are at fixed offsets.  (OTOH, I don't see why their subclasses can 
still have a __dict__ slot, then...)


From gnupeaker at yahoo.com  Sun May 30 09:38:43 2004
From: gnupeaker at yahoo.com (Eyal Lotem)
Date: Sun May 30 09:38:46 2004
Subject: [Python-Dev] Stackless Python
Message-ID: <20040530133843.51528.qmail@web50709.mail.yahoo.com>

Stackless Python and its tasklets would be great!

Why is Python still not stackless?



	
		
__________________________________
Do you Yahoo!?
Friends.  Fun.  Try the all-new Yahoo! Messenger.
http://messenger.yahoo.com/ 

From tismer at stackless.com  Sun May 30 13:52:02 2004
From: tismer at stackless.com (Christian Tismer)
Date: Sun May 30 13:50:38 2004
Subject: [Python-Dev] Why aren't more things weak referencable
In-Reply-To: <92EB8184-B1F9-11D8-B43D-000A95686CD8@redivi.com>
References: <004f01c4459f$4a4578c0$0718c797@oemcomputer>
	<92EB8184-B1F9-11D8-B43D-000A95686CD8@redivi.com>
Message-ID: <40BA1F42.2030403@stackless.com>

Bob Ippolito wrote:
> On May 29, 2004, at 1:06 PM, Raymond Hettinger wrote:
> 
>> Instances of classes inheriting from str, tuple, etc cannot be weakly 
>> referenced.  Does anyone know the reason for this?
> 
> 
> They can not accept non-empty __slots__ either, which is probably closer 
> to the source of the problem.  I have no idea what the reason is.  I 
> imagine it's something to do with optimization, and/or because they are 
> immutable.

it is because they are var-sized objects. There is no place
to put the weakref pointer in, since the variable part
starts right at the beginning of the object.

This is not really necessary, because strings are not so
var-sized at all. After creation, they are fixed sized,
and we could implement special cases for all such types,
similar to what I did with type objects and slots.

The property of "var-sized" objects is everything else but
being var-sized. They are fixed sized after initialization,
just that you cannot have fixed offsets at "compile time".
The real var-sized objects are fixed-size :-)) , because they
use an extra memory area to grow or shrink at runtime.

I think slots could be added to all var-sized objects, and
weakref is nothing else but kind of slot. The cost would be
a little more computation and some special macro which points
to the area "behind" the object. See my special case for
meta types in Stackless typeobject.c.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From tismer at stackless.com  Sun May 30 13:54:42 2004
From: tismer at stackless.com (Christian Tismer)
Date: Sun May 30 13:53:07 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <20040530133843.51528.qmail@web50709.mail.yahoo.com>
References: <20040530133843.51528.qmail@web50709.mail.yahoo.com>
Message-ID: <40BA1FE2.9090700@stackless.com>

Eyal Lotem wrote:

> Stackless Python and its tasklets would be great!

Indeed. It is so handy that even I use it for my projects.

> Why is Python still not stackless?

Because this question is not yet asked 10 times a day. :-)

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From python at rcn.com  Sun May 30 03:52:57 2004
From: python at rcn.com (Raymond Hettinger)
Date: Sun May 30 15:53:08 2004
Subject: [Python-Dev] Why aren't more things weak referencable
In-Reply-To: <40BA1F42.2030403@stackless.com>
Message-ID: <000001c4461b$1fa22d80$e841fea9@oemcomputer>

[Raymond Hettinger]
> >> Instances of classes inheriting from str, tuple, etc cannot be
weakly
> >> referenced.  Does anyone know the reason for this?

[Bob Ippolito]
> > They can not accept non-empty __slots__ either, which is probably
closer
> > to the source of the problem.  I have no idea what the reason is.  I
> > imagine it's something to do with optimization, and/or because they
are
> > immutable.

[Christian Tismer]
> it is because they are var-sized objects. There is no place
> to put the weakref pointer in, since the variable part
> starts right at the beginning of the object.

Thanks.  That exactly identifies the issue.  It also explains the
mystery of why unicode subclasses are weakly referencable but str
subclasses are not.

For strings at least, perhaps it is time to bite the bullet and include
weak reference support directly.  Weak reference support ups the per
string memory overhead from five words (ob_type, ob_refcnt, ob_size,
ob_shash, ob_sstate) to six.  The whole concept of weak  dictionaries is
much more useful when strings can be used as keys and/or values.

Several other objects probably also warrant weak reference support:
array.array, files, sockets, and sre.pattern_objects.  In these cases,
the one word overhead is small relative to the rest of the object

Tuples would also be nice but the overhead is likely not worth it.
Chris's solution (recognizing that var sized objects are typically fixed
upon creation) is more general and doesn't burden the most common case.


Raymond


From guido at python.org  Sun May 30 18:04:43 2004
From: guido at python.org (Guido van Rossum)
Date: Sun May 30 18:04:51 2004
Subject: [Python-Dev] Why aren't more things weak referencable
In-Reply-To: Your message of "Sun, 30 May 2004 03:52:57 EDT."
	<000001c4461b$1fa22d80$e841fea9@oemcomputer> 
References: <000001c4461b$1fa22d80$e841fea9@oemcomputer> 
Message-ID: <200405302204.i4UM4hb19989@guido.python.org>

> For strings at least, perhaps it is time to bite the bullet and
> include weak reference support directly.  Weak reference support ups
> the per string memory overhead from five words (ob_type, ob_refcnt,
> ob_size, ob_shash, ob_sstate) to six.  The whole concept of weak
> dictionaries is much more useful when strings can be used as keys
> and/or values.

Hmm... it is a high price to pay to add another word (*and* some extra
code at dealloc time!) to every string object when very few apps need
them and strings are about the most common data type.  And since
they're immutable, what's the point of having weak refs to strings in
the first place?  (Note that the original poster asked about
*subclasses* of strings.)

> Several other objects probably also warrant weak reference support:
> array.array, files, sockets, and sre.pattern_objects.  In these
> cases, the one word overhead is small relative to the rest of the
> object

These I have peace with.  Note that for sockets, the objects that
Python programs actually see are instances of a wrapper class defined
in Python (using __slots__ so to add weakrefs, you have to add
__weakreflist__ to the list of slots).

> Tuples would also be nice but the overhead is likely not worth it.
> Chris's solution (recognizing that var sized objects are typically
> fixed upon creation) is more general and doesn't burden the most
> common case.

I'm not sure how this observation helps.  Anyway, I have the same
concern for tuples as for strings.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tdelaney at avaya.com  Sun May 30 19:52:59 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Sun May 30 19:53:06 2004
Subject: [Python-Dev] Why aren't more things weak referencable
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE018806E0@au3010avexu1.global.avaya.com>

Guido van Rossum wrote:

>> For strings at least, perhaps it is time to bite the bullet and
>> include weak reference support directly.  Weak reference support ups
>> the per string memory overhead from five words (ob_type, ob_refcnt,
>> ob_size, ob_shash, ob_sstate) to six.  The whole concept of weak
>> dictionaries is much more useful when strings can be used as keys
>> and/or values.
> 
> Hmm... it is a high price to pay to add another word (*and* some extra
> code at dealloc time!) to every string object when very few apps need
> them and strings are about the most common data type.  And since
> they're immutable, what's the point of having weak refs to strings in
> the first place?  (Note that the original poster asked about
> *subclasses* of strings.)

Perhaps there should be a standard weakly-referenceable proxy whose sole
purpose in life would be to wrap objects that can't be directly weakly
referenced. Kind of the opposite of ProxyTypes - it would have an
internal strong reference to the object.

This could be done transparently as well - if creating an instance of
ReferenceType would raise a TypeError, we take a hit and wrap the
object.

Tim Delaney

From jjl at pobox.com  Sun May 30 19:55:10 2004
From: jjl at pobox.com (John J Lee)
Date: Sun May 30 19:55:18 2004
Subject: [Python-Dev] cookielib
Message-ID: <Pine.LNX.4.58.0405302350050.6233@alice>

I've uploaded a patch for my proposed new 2.4 module cookielib:

http://python.org/sf/963318

What next?


John

From ncoghlan at iinet.net.au  Sun May 30 23:44:11 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sun May 30 23:44:15 2004
Subject: [Python-Dev] Standard Library Tour Part II
In-Reply-To: <001101c441e4$c325cd80$a2b1958d@oemcomputer>
References: <001101c441e4$c325cd80$a2b1958d@oemcomputer>
Message-ID: <40BAAA0B.6060308@iinet.net.au>

Raymond Hettinger wrote:
> [??? Should the tempfile module be in the tour ???]

I would say yes - it shouldn't be that hard to explain, and is far 
better than creating your own method of making temporary files.

The two most likely traps it avoids are problems with access permissions 
on the current directory, as well as problems with the temporary files 
conflicting when multiple versions of your Python program are run 
simultaneously (I probably *would* have been burnt by those at some 
point, but found tempfile first)

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268

From martin at v.loewis.de  Mon May 31 03:49:14 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon May 31 03:49:18 2004
Subject: [Python-Dev] Why aren't more things weak referencable
In-Reply-To: <000001c4461b$1fa22d80$e841fea9@oemcomputer>
References: <000001c4461b$1fa22d80$e841fea9@oemcomputer>
Message-ID: <40BAE37A.7080009@v.loewis.de>

Raymond Hettinger wrote:
> For strings at least, perhaps it is time to bite the bullet and include
> weak reference support directly.  Weak reference support ups the per
> string memory overhead from five words (ob_type, ob_refcnt, ob_size,
> ob_shash, ob_sstate) to six.  The whole concept of weak  dictionaries is
> much more useful when strings can be used as keys and/or values.

Can you elaborate that? What specific, real-life application do you
have in mind that requires strings to be weakly-referencable?

Because strings are immutable, you often get unexpected references to
them, so people have learned not to care about the life cycle of string
objects.

Regards,
Martin


From martin at v.loewis.de  Mon May 31 03:56:18 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon May 31 03:56:22 2004
Subject: [Python-Dev] Why aren't more things weak referencable
In-Reply-To: <004f01c4459f$4a4578c0$0718c797@oemcomputer>
References: <004f01c4459f$4a4578c0$0718c797@oemcomputer>
Message-ID: <40BAE522.2050703@v.loewis.de>

Raymond Hettinger wrote:
> Instances of classes inheriting from str, tuple, etc cannot be weakly 
> referenced.  Does anyone know the reason for this?

In addition to the reason Christian gave, one (conceptually more
important) reason is that strings can't participate in cycles. Weak
references were introduced as a mechanism to avoid creating cyclic
structures, so that "backward" links could be made weak references.

It appears that people have then been eager to add weakref support
to other datatypes. IMO, they have been too eager. For example, I
can't see a reason why Unicode objects should be weakly referencable
(just as I can't see a reason for plain strings).

Regards,
Martin


From martin at v.loewis.de  Mon May 31 04:00:05 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon May 31 04:00:10 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <20040530133843.51528.qmail@web50709.mail.yahoo.com>
References: <20040530133843.51528.qmail@web50709.mail.yahoo.com>
Message-ID: <40BAE605.5040800@v.loewis.de>

Eyal Lotem wrote:
  > Why is Python still not stackless?

But it is: http://www.stackless.com/

If you are asking: Why is python.org Python still not
stackless? then I think the answer has varied over time.

The most recent answer is that no volunteer has stepped
forward to make Python stackless. That, in turn, likely
hasn't happened because it is hard to do, and because
nobody has seen any reason to do so as you could always
use Stackless Python.

Regards,
Martin



From martin at v.loewis.de  Mon May 31 04:09:53 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon May 31 04:10:03 2004
Subject: [Python-Dev] cookielib
In-Reply-To: <Pine.LNX.4.58.0405302350050.6233@alice>
References: <Pine.LNX.4.58.0405302350050.6233@alice>
Message-ID: <40BAE851.7030207@v.loewis.de>

John J Lee wrote:
> I've uploaded a patch for my proposed new 2.4 module cookielib:
> 
> http://python.org/sf/963318
> 
> What next?

Did you already commit to maintain this code for, say, five years?

I'm willing to apply this patch as-is (perhaps adding something
to whatsnew24). Any objections?

Regards,
Martin


From jjl at pobox.com  Mon May 31 07:01:36 2004
From: jjl at pobox.com (John J Lee)
Date: Mon May 31 07:01:43 2004
Subject: [Python-Dev] cookielib
In-Reply-To: <40BAE851.7030207@v.loewis.de>
References: <Pine.LNX.4.58.0405302350050.6233@alice>
	<40BAE851.7030207@v.loewis.de>
Message-ID: <Pine.LNX.4.58.0405311157560.7262@alice>

On Mon, 31 May 2004, [ISO-8859-1] "Martin v. L=F6wis" wrote:

> John J Lee wrote:
> > I've uploaded a patch for my proposed new 2.4 module cookielib:
> >
> > http://python.org/sf/963318
> >
> > What next?
>
> Did you already commit to maintain this code for, say, five years?

Yes.


> I'm willing to apply this patch as-is (perhaps adding something
> to whatsnew24). Any objections?

Not from me <wink>.


John

From anthony at interlink.com.au  Mon May 31 07:03:34 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon May 31 07:04:21 2004
Subject: [Python-Dev] cookielib
In-Reply-To: <40BAE851.7030207@v.loewis.de>
References: <Pine.LNX.4.58.0405302350050.6233@alice>
	<40BAE851.7030207@v.loewis.de>
Message-ID: <40BB1106.5080202@interlink.com.au>

Martin v. L?wis wrote:
> I'm willing to apply this patch as-is (perhaps adding something
> to whatsnew24). Any objections?

None from me. We could probably do with some 'see also' references
in related library documentation.


-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From jjl at pobox.com  Mon May 31 07:10:39 2004
From: jjl at pobox.com (John J Lee)
Date: Mon May 31 07:10:45 2004
Subject: [Python-Dev] cookielib
In-Reply-To: <40BB1106.5080202@interlink.com.au>
References: <Pine.LNX.4.58.0405302350050.6233@alice>
	<40BAE851.7030207@v.loewis.de> <40BB1106.5080202@interlink.com.au>
Message-ID: <Pine.LNX.4.58.0405311208430.7262@alice>

On Mon, 31 May 2004, Anthony Baxter wrote:
> Martin v. L?wis wrote:
> > I'm willing to apply this patch as-is (perhaps adding something
> > to whatsnew24). Any objections?
>
> None from me. We could probably do with some 'see also' references
> in related library documentation.

I've already added some to the urllib2 and Cookie docs.  I left urllib and
httplib's docs alone because I've not added any cookie support for those
modules (though there is nothing to prevent that).


John

From fumanchu at amor.org  Mon May 31 09:21:38 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Mon May 31 09:25:01 2004
Subject: [Python-Dev] Why aren't more things weak referencable
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E56215B@opus.amorhq.net>

"Martin v. L?wis" wrote:
> Raymond Hettinger wrote:
> > Instances of classes inheriting from str, tuple, etc cannot 
> be weakly 
> > referenced.  Does anyone know the reason for this?
> 
> In addition to the reason Christian gave, one (conceptually more
> important) reason is that strings can't participate in cycles. Weak
> references were introduced as a mechanism to avoid creating cyclic
> structures, so that "backward" links could be made weak references.
> 
> It appears that people have then been eager to add weakref support
> to other datatypes. IMO, they have been too eager. For example, I
> can't see a reason why Unicode objects should be weakly referencable
> (just as I can't see a reason for plain strings).

Never rule out "foolish" consistencies. I can imagine a system where multiple, heterogeneous values all get run through a weakref processor; to have string objects pass through without Yet Another try/except would be a design and maintenance boon.

Quite similar to my current "pet peeve":

>>> None > 3
False
>>> None > 'hoopy'
False
>>> None > True
False
>>> None > datetime.date(2004, 5, 31)
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
TypeError: can't compare datetime.date to NoneType

...writing an O-R mapper, this particular hobgoblin bites me rather often ;)


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org

From edcjones at erols.com  Mon May 31 10:13:33 2004
From: edcjones at erols.com (Edward C. Jones)
Date: Mon May 31 10:18:55 2004
Subject: [Python-Dev] Uninformative error message about subclassing
Message-ID: <40BB3D8D.4090809@erols.com>

Here are two small python files:

----
a.py:

class A(object):
    pass

----
b.py:

import a

class B(a):
    pass
----
If I run "python b.py" I get the uninformative error message:

Traceback (most recent call last):
  File "./b.py", line 5, in ?
    class B(a):
TypeError: function takes at most 2 arguments (3 given)

I think the message should say that "a" is not a class.




From lists at andreas-jung.com  Mon May 31 10:22:41 2004
From: lists at andreas-jung.com (Andreas Jung)
Date: Mon May 31 10:22:51 2004
Subject: [Python-Dev] Uninformative error message about subclassing
In-Reply-To: <40BB3D8D.4090809@erols.com>
References: <40BB3D8D.4090809@erols.com>
Message-ID: <A06322F270E678C4AD379558@[192.168.0.107]>



--On Montag, 31. Mai 2004 10:13 Uhr -0400 "Edward C. Jones" 
<edcjones@erols.com> wrote:

> Here are two small python files:
>
> ----
> a.py:
>
> class A(object):
>     pass
>
> ----
> b.py:
>
> import a
>
> class B(a):
>     pass

You are trying to use *module a* as base class? Weird idea.

-aj



From lists at andreas-jung.com  Mon May 31 10:24:47 2004
From: lists at andreas-jung.com (Andreas Jung)
Date: Mon May 31 10:24:54 2004
Subject: [Python-Dev] Uninformative error message about subclassing
In-Reply-To: <A06322F270E678C4AD379558@[192.168.0.107]>
References: <40BB3D8D.4090809@erols.com>
	<A06322F270E678C4AD379558@[192.168.0.107]>
Message-ID: <EB65440643049EA5B2E9E85B@[192.168.0.107]>



--On Montag, 31. Mai 2004 16:22 Uhr +0200 Andreas Jung 
<lists@andreas-jung.com> wrote:

>
>
> --On Montag, 31. Mai 2004 10:13 Uhr -0400 "Edward C. Jones"
> <edcjones@erols.com> wrote:
>
>> Here are two small python files:
>>
>> ----
>> a.py:
>>
>> class A(object):
>>     pass
>>
>> ----
>> b.py:
>>
>> import a
>>
>> class B(a):
>>     pass
>
> You are trying to use *module a* as base class? Weird idea.
>

Forget my reply. I have misread your posting.

-- 

Andreas Jung
--------------------------------
www.zopyx.com Software-Entwicklung und Consulting Andreas Jung

From anthony at interlink.com.au  Mon May 31 11:39:02 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon May 31 11:39:49 2004
Subject: [Python-Dev] pre-announce: Python 2.4a1 is about a month away
Message-ID: <40BB5196.7070207@interlink.com.au>


     This is a pre-announcement that the first alpha of Python 2.4
     is about a month away. The purpose of this notice is to give
     people a heads up - if you have a bug that you want to see
     fixed for 2.4, start looking at it now.

     Fixes are welcome through the release cycle, although after
     the first beta fixes that result in a change to behaviour
     will be much less likely to be accepted.

So, if you have a bug you want to see fixed, what should you do?

   - If it's not logged on SF, log it.
   - If it's logged, consider adding a patch that fixes the problem,
     or at least a simple test case that demonstrates the bug.
   - If someone else has supplied a fix, see if this fix works for
     you, and post your results to the bug.
   - If there's a working fix, feel free to add a note asking for
     the fix to go into CVS. The SF bug tracker for Python has a
     _lot_ of bugs in it, and it's easy for bugs to be overlooked.

If you're just interested in what's coming up in 2.4, see the
current development version of the "What's New in 2.4" document
at http://www.python.org/dev/doc/devel/whatsnew/whatsnew24.html


On behalf of the python-dev team, thanks!
Anthony

[This form of pre-announcement will hopefully become a part of
the release process for all future releases of Python, both
major and bugfix releases, to give people a chance to get their
bugfix-of-choice submitted in time. Any feedback on this process,
please feel free to email me]
-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From python at rcn.com  Mon May 31 00:38:04 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon May 31 12:38:18 2004
Subject: [Python-Dev] Why aren't more things weak referencable
In-Reply-To: <40BAE37A.7080009@v.loewis.de>
Message-ID: <002101c446c9$10b848a0$e841fea9@oemcomputer>

 [Martin v. L?wis]
> What specific, real-life application do you
> have in mind that requires strings to be weakly-referencable?

FWIW, my program searches a virtual graph for a goal state.  Successor
nodes (board positions) are computed by a complex function and stored as
strings.  The successions of moves were recorded with a dictionary
linking the positions (long strings as both keys and values).  When the
graph eventually rules out a group of positions, I wanted the succession
chains all disappear on their own (i.e. if a board position becomes
unreachable when a better move is found, then the succession history for
that position becomes irrelevant).

Essentially, I used weak dictionaries as a cache to values that are
expensive to compute and take up (collectively) a lot of memory.  Having
that cache weakly referenced automated the process of clearing out
values that lost their relevance.

IMO, this use case is too rare to warrant adding weak ref support to
strings.  It just served to precipitate an attempted workaround using
"class Str(str): pass".  Christian then clarified why that didn't work
as expected.



> In addition to the reason Christian gave, one (conceptually more
> important) reason is that strings can't participate in cycles. Weak
> references were introduced as a mechanism to avoid creating cyclic
> structures, so that "backward" links could be made weak references.

Right, that was a principal use case.  Hence, the focus should be on
tuples rather than strings.  Likewise, most containers should be
candidates for weakref support unless it is just too expensive (i.e.
lists and tuples). Also, any subclass can potentially introduce back
references, so all subclasses should be fair game.  

Still, if cycles were the only issue, we would just let GC take care of
it.
There is one other purpose for weakrefs.  Even in the absence of cycles,
weakrefs let applications to cache and track objects without creating a
strong reference that keeps objects alive forever.  This is especially
useful for objects that were expensive to compute, expensive to create,
or external to the program in some way.  That was the rationale regexp
patterns, files, sockets, and arrays.



> I
> can't see a reason why Unicode objects should be weakly referencable
> (just as I can't see a reason for plain strings).

Unicode objects are like list and dict objects.  None of them are
directly weak referencable.  They have to be subclassed.  Then, they
need weak referencability as much as any other instance would.




Raymond


From shane.holloway at ieee.org  Mon May 31 13:34:19 2004
From: shane.holloway at ieee.org (Shane Holloway (IEEE))
Date: Mon May 31 13:34:44 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <40BAE605.5040800@v.loewis.de>
References: <20040530133843.51528.qmail@web50709.mail.yahoo.com>
	<40BAE605.5040800@v.loewis.de>
Message-ID: <40BB6C9B.4040701@ieee.org>

Does this mean that the implementation of Stackless 3.0 could be 
acceptable to integrate into CPython 2.5?  It would be absolutely 
wonderful to have those capabilities available in the standard 
distribution!

-Shane Holloway

Martin v. L?wis wrote:

> Eyal Lotem wrote:
>  > Why is Python still not stackless?
> 
> But it is: http://www.stackless.com/
> 
> If you are asking: Why is python.org Python still not
> stackless? then I think the answer has varied over time.
> 
> The most recent answer is that no volunteer has stepped
> forward to make Python stackless. That, in turn, likely
> hasn't happened because it is hard to do, and because
> nobody has seen any reason to do so as you could always
> use Stackless Python.
> 
> Regards,
> Martin
> 
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/shane.holloway%40ieee.org

From edloper at gradient.cis.upenn.edu  Mon May 31 14:06:12 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Mon May 31 14:04:52 2004
Subject: [Python-Dev] doctest improvements
Message-ID: <40BB7414.9030304@gradient.cis.upenn.edu>

About a month and a half ago, I filed several patches for doctest, and 
they haven't gotten any comments; so I was wondering if anyone could 
look at them.

The most interesting is #933238, which addresses the often-noted problem 
that doctest can't handle blank lines.  It adds a special blank-line 
marker (currently "-"), which must be *dedented* with respect to the 
doctest block.  Here's an example use (the "x"s are to prevent mail 
readers from interpreting ">>>" as a quotation).

x    def test_blankline_marker():
x        r"""
x        Use a dedented "-" to signify a blank line:
x
x            >>> print 'First line\n\nThis is after a blank line\n'
x            First line
x           -
x            This is after a blank line
x           -
x        """

(Note that both internal and trailing blank lines can be handled.)

This change is backwards compatible, since currently any line in a 
doctest block that's dedented past the prompt (">>>") is an error.

If anyone has time to take a look at this, let me know what you think. 
Thanks.

-Edward

My doctest patches:

[ 933238 ] doctest: add a special (dedented) marker for blank lines
http://sourceforge.net/tracker/?func=detail&aid=933238&group_id=5470&atid=305470

[ 932930 ] doctest: suggest the use of rawstrings for backslashes
http://sourceforge.net/tracker/?func=detail&aid=932930&group_id=5470&atid=305470

[ 932935 ] doctest: allow custom matchers for testing if got==want
http://sourceforge.net/tracker/?func=detail&aid=932935&group_id=5470&atid=305470

[ 932932 ] doctest: Add Tester params to DocTestSuite
http://sourceforge.net/tracker/?func=detail&aid=932932&group_id=5470&atid=305470

From john at neggie.net  Mon May 31 14:25:45 2004
From: john at neggie.net (John Belmonte)
Date: Mon May 31 14:26:05 2004
Subject: [Python-Dev] slots, properties, descriptors, and pydoc
In-Reply-To: <4087B75A.3060405@neggie.net>
References: <407C28E0.7090808@neggie.net>	<4080C0C7.7070602@neggie.net>	<200404171733.i3HHX4B30513@guido.python.org>	<40827869.2030300@neggie.net>	<200404181721.i3IHLTp07934@guido.python.org>	<40833974.10800@neggie.net>	<200404191510.i3JFAxX09635@guido.python.org>
	<4087B75A.3060405@neggie.net>
Message-ID: <40BB78A9.2070203@neggie.net>

I wrote:
> Wouldn't it be just as future-proof to make __slots__ a list of bare 
> slot names or objects?  One advantage is that slot names don't have to 
> be carried around externally to the objects.  Moreover, using standard 
> attribute names like __name__ and __doc__ will make the solution more 
> future-proof.
> 
> The default metaclass will handle __slots__ along these lines:
> 
>     for item in obj.__slots__:
>         if isinstance(item, StringTypes):
>             slot_name, slot_doc = item, None
>         else:
>             slot_name = item.__name__
>             slot_doc = getattr(item, '__doc__', None)
> 
> In that way, the default metaclass does not impose any restrictions on 
> what the slot objects are.
> 
> Example usage:
> 
>     class MySlot(object):
>         def __init__(self, name, doc=None):
>             self.__name__ = name
>             self.__doc__ = doc
> 
>     class C(object):
>         __slots__ = ['slot1',
>                      MySlot('slot2', 'this is the docstring')]

I am still very interested in seeing this happen.  I'm looking for a 
Python C API expert who would be kind enough to get me started with the 
necessary mods to the default Python metaclass.

I already have a prototype of the pydoc improvements which go 
hand-in-hand with this change 
(http://sourceforge.net/tracker/index.php?func=detail&aid=936774&group_id=5470&atid=305470).

-John

-- 
http://giftfile.org/  ::  giftfile project

From martin at v.loewis.de  Mon May 31 14:38:30 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon May 31 14:38:36 2004
Subject: [Python-Dev] cookielib
In-Reply-To: <Pine.LNX.4.58.0405302350050.6233@alice>
References: <Pine.LNX.4.58.0405302350050.6233@alice>
Message-ID: <40BB7BA6.1040007@v.loewis.de>

John J Lee wrote:
> What next?

I have applied the patch. Now, when I run regrtest, I get

test_urllib2
add_cookie_header
extract_cookies: Set-Cookie: spam=eggs
  - checking cookie spam=eggs
  setting cookie: <Cookie spam=eggs for www.example.com/>
add_cookie_header
Checking www.example.com for cookies to return
- checking cookie path=/
  - checking cookie spam=eggs
    it's a match

Can you provide a patch to suppress this output?

Regards,
Martin


From martin at v.loewis.de  Mon May 31 14:50:11 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon May 31 14:50:16 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <40BB6C9B.4040701@ieee.org>
References: <20040530133843.51528.qmail@web50709.mail.yahoo.com>	<40BAE605.5040800@v.loewis.de>
	<40BB6C9B.4040701@ieee.org>
Message-ID: <40BB7E63.9090304@v.loewis.de>

Shane Holloway (IEEE) wrote:
> Does this mean that the implementation of Stackless 3.0 could be 
> acceptable to integrate into CPython 2.5?  It would be absolutely 
> wonderful to have those capabilities available in the standard 
> distribution!

Probably not. Stackless 3.0 works only on selected hardware; a
number of platforms currently supported by Python would not be
supported anymore. An implementation of stackless python should
work on all platforms (atleast on those currently supported by
Python).

Regards,
Martin


From bob at redivi.com  Mon May 31 15:06:35 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon May 31 15:06:43 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <40BB7E63.9090304@v.loewis.de>
References: <20040530133843.51528.qmail@web50709.mail.yahoo.com>	<40BAE605.5040800@v.loewis.de>
	<40BB6C9B.4040701@ieee.org> <40BB7E63.9090304@v.loewis.de>
Message-ID: <A30D1ADC-B335-11D8-9388-000A95686CD8@redivi.com>


On May 31, 2004, at 2:50 PM, Martin v. L?wis wrote:

> Shane Holloway (IEEE) wrote:
>> Does this mean that the implementation of Stackless 3.0 could be 
>> acceptable to integrate into CPython 2.5?  It would be absolutely 
>> wonderful to have those capabilities available in the standard 
>> distribution!
>
> Probably not. Stackless 3.0 works only on selected hardware; a
> number of platforms currently supported by Python would not be
> supported anymore. An implementation of stackless python should
> work on all platforms (atleast on those currently supported by
> Python).

Although I don't particularly agree with that, because Stackless 
functionality could be turned on and off with a platform dependent 
configure flag (like threads or unicode)...  Is there a canonical list 
of CPU architectures and platforms that Python officially supports?

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2357 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040531/5fc17d6f/smime.bin
From martin at v.loewis.de  Mon May 31 15:09:44 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon May 31 15:09:49 2004
Subject: [Python-Dev] doctest improvements
In-Reply-To: <40BB7414.9030304@gradient.cis.upenn.edu>
References: <40BB7414.9030304@gradient.cis.upenn.edu>
Message-ID: <40BB82F8.4070906@v.loewis.de>

Edward Loper wrote:
> About a month and a half ago, I filed several patches for doctest, and 
> they haven't gotten any comments; so I was wondering if anyone could 
> look at them.

It is not surprising the patches haven't seen any comments: We get way
more patches than we can handle these days. If you want to help, please
review patches of other people, and recommend adaption or rejection.
(Once I pondered requiring people to review 10 patches before reviewing
  one of theirs)

In case of your first patch, there is an added difficulty that the
introductory text is so long it shys reviewers away: If you 10 minutes
to read *just* the explanation what the patch does, you know that it
will take several hours to deal with the entire patch.

Of course, a quick review shows that the patches are formally
incomplete, so I did not need to look further.

Regards,
Martin


From martin at v.loewis.de  Mon May 31 15:33:22 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon May 31 15:33:27 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <A30D1ADC-B335-11D8-9388-000A95686CD8@redivi.com>
References: <20040530133843.51528.qmail@web50709.mail.yahoo.com>	<40BAE605.5040800@v.loewis.de>	<40BB6C9B.4040701@ieee.org>
	<40BB7E63.9090304@v.loewis.de>
	<A30D1ADC-B335-11D8-9388-000A95686CD8@redivi.com>
Message-ID: <40BB8882.60503@v.loewis.de>

Bob Ippolito wrote:
> Although I don't particularly agree with that, because Stackless 
> functionality could be turned on and off with a platform dependent 
> configure flag (like threads or unicode)...  Is there a canonical list 
> of CPU architectures and platforms that Python officially supports?

No - although there is a list of platforms, that Python *doesn't*
support (PEP 11). Python is written in C, so it should work on any
system that supports ISO C, although many functions won't be available
if the system doesn't support POSIX.

As for turning of things: yes, unicode can be turned off, but not
because the platform doesn't support it - rather because the
administrator doesn't like unicode. Threads are turned off on operating
systems that don't provide threads, and there is nothing that Python
could do about it.

Stackless' support for platforms is different: it includes assembler
code, so you have to write bits of assembler code for each architecure
(and perhaps for each system on a single architecture, or even each
assembler on each system).

[In addition, the approach taken by Stackless 3.0 is probably
questionable: for example, it may be that C++ exception handling
stops working in Stackless.]

Regards,
Martin


From bac at OCF.Berkeley.EDU  Mon May 31 15:41:05 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon May 31 15:41:18 2004
Subject: [Python-Dev] Anyone object to a rewrite of site.py?
Message-ID: <40BB8A51.2080105@ocf.berkeley.edu>

At PyCON I rewrote site.py .  The biggest change is that I broke it into 
functions and moved it over to use sets.  Basically the file was just 
old and I brought it up to modern usage.  Performance-wise I can't see a 
real difference between the current version and my rewrite by executing 
``time ./python.exe -c "pass"``; fluctuations in values are standard 
white noise.  Breaking it up into functions also allowed me to write a 
test suite for site.py .

Patch 921793 (http://www.python.org/sf/921793) has the patch of my 
rewrite.  Does anyone object to me applying the patch?  Of course I 
would love to have someone take a look at the code since this gets 
executed every time.  I have been using the rewritten version since 
PyCON (so about two months) and I haven't had a bug yet.

Obviously I would like to get this into 2.4a1 to get as much testing as 
possible.

-Brett

From bac at OCF.Berkeley.EDU  Mon May 31 15:54:29 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon May 31 15:54:42 2004
Subject: [Python-Dev] Need traceback expert to review patch 954922
In-Reply-To: <40A94622.6040606@ocf.berkeley.edu>
References: <40A94622.6040606@ocf.berkeley.edu>
Message-ID: <40BB8D75.9080009@ocf.berkeley.edu>

Any chance someone can take a look at this before 2.4a1 goes out the door?

Brett C. wrote:

> Back in February, I tried to fix an issue of threads executed using 
> 'threading' that did not shut themselves down before the interpreter 
> started tearing down globals, the thread triggering an exception, and 
> then a new exception being raised while trying to report the exception 
> raised in the thread by accessing globals (see 
> http://mail.python.org/pipermail/python-dev/2004-February/042457.html 
> for the previous discussion).  An initial fix was devised, but since 
> there was no way to trigger the issue every time I was not sure if the 
> fix was thorough enough.
> 
> Well, someone finally has an app that causes the error consistently and 
> the previous fix did not go far enough.  But with the help of the app 
> (mnet, specifically) I was finally able to get rid of all global 
> accesses while reporting exceptions so that the original exception 
> caught in the thread can get printed.  The patch is at 
> http://python.org/sf/954922 while the bug report is at 
> http://python.org/sf/754449 .
> 
> Trouble is that I had to basically fake traceback.format_exc() in the 
> patch to a reasonable level since the 'traceback' module bounces around 
> between its various functions too much to just store a local copy of the 
> function and have it still work after globals are set to None.  So I 
> need someone to just look over the code to make sure that the code is 
> not doing something stupid when this code is called (default way is 
> still used when 'sys' is defined and thus the interpreter is not being 
> torn down).
> 
> And the reason I bring this up now is just in case there is any desire 
> to put this in 2.3.4 .  I obviously know that I missed rc1, but I didn't 
> get the details of triggering the problem soon enough and people running 
> 2.3.x are the ones bringing this up since this apparently did not happen 
> under 2.2 for some reason.  I doubt Anthony wants to risk mucking with 
> 'threading' this late (although the patch is fairly self-contained but 
> would require applying the previous patch to attempt to fix this as 
> well), but I thought I would at least put it out there.
> 
> -Brett

From bob at redivi.com  Mon May 31 16:08:51 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon May 31 16:09:00 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <40BB8882.60503@v.loewis.de>
References: <20040530133843.51528.qmail@web50709.mail.yahoo.com>	<40BAE605.5040800@v.loewis.de>	<40BB6C9B.4040701@ieee.org>
	<40BB7E63.9090304@v.loewis.de>
	<A30D1ADC-B335-11D8-9388-000A95686CD8@redivi.com>
	<40BB8882.60503@v.loewis.de>
Message-ID: <5611D758-B33E-11D8-9388-000A95686CD8@redivi.com>

On May 31, 2004, at 3:33 PM, Martin v. L?wis wrote:

> Bob Ippolito wrote:
>> Although I don't particularly agree with that, because Stackless 
>> functionality could be turned on and off with a platform dependent 
>> configure flag (like threads or unicode)...  Is there a canonical 
>> list of CPU architectures and platforms that Python officially 
>> supports?
>
> No - although there is a list of platforms, that Python *doesn't*
> support (PEP 11). Python is written in C, so it should work on any
> system that supports ISO C, although many functions won't be available
> if the system doesn't support POSIX.
>
> As for turning of things: yes, unicode can be turned off, but not
> because the platform doesn't support it - rather because the
> administrator doesn't like unicode. Threads are turned off on operating
> systems that don't provide threads, and there is nothing that Python
> could do about it.
>
> Stackless' support for platforms is different: it includes assembler
> code, so you have to write bits of assembler code for each architecure
> (and perhaps for each system on a single architecture, or even each
> assembler on each system).

I don't see how including a little bit of 
per-architecture-calling-convention assembler code is much different 
than depending on pthreads (or some other platform dependent thing) 
being present.  There is also potentially a platform neutral way of 
doing it (see below).

> [In addition, the approach taken by Stackless 3.0 is probably
> questionable: for example, it may be that C++ exception handling
> stops working in Stackless.]

I'm not quite sure you are up to date on how Stackless 3.0 works.  The 
assembly code is only there to facilitate "hard switching", which is 
only required to do tasklet switching in the face of some C extension 
that doesn't know about Stackless.  The vast majority of tasklet 
switches use the "soft switching" mechanism, which just means that the 
code they use is capable of going into non-recursive mode.  All pure 
Python code and almost all code built-in to Python goes through "soft 
switching", which is implemented in pure C with no stack tricks or 
anything.  Stackless would still be a very useful addition to Python 
even when "hard switching" is not available.

Alternatively, Armin believes that Stackless "hard switching" can 
actually be implemented in mostly-platform-neutral C (either by 
setjmp/longjmp or alloca hacks, I forget which) and has a prototype 
implementation in the Stackless CVS repository (though it is not 
integrated with Stackless itself).  It might also be possible to coerce 
something like libffi into doing what we need it to do.

I'm not sure how well "hard switching" would play with an operating 
system that is very protective of its stack, like OpenBSD or future 
versions of Windows, though.

I'm also not sure how it could "break C++ exception handling" in 
scenarios where it wouldn't be broken anyway.  I'd like to see a test 
of that that passes under regular Python but fails in Stackless for 
this.

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2357 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040531/e89b3b0e/smime.bin
From python at rcn.com  Mon May 31 04:17:42 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon May 31 16:17:54 2004
Subject: [Python-Dev] Anyone object to a rewrite of site.py?
In-Reply-To: <40BB8A51.2080105@ocf.berkeley.edu>
Message-ID: <003d01c446e7$bef60380$e841fea9@oemcomputer>

> Patch 921793 (http://www.python.org/sf/921793) has the patch of my
> rewrite.  

I haven't looked at the patch yet but am +1 on the concept of
refactoring it.  The current version is somewhat of a mess.


> I have been using the rewritten version since
> PyCON (so about two months) and I haven't had a bug yet.

I'll run with it for a while and see if I get the same results.


Raymond


From martin at v.loewis.de  Mon May 31 16:56:55 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon May 31 16:57:00 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <5611D758-B33E-11D8-9388-000A95686CD8@redivi.com>
References: <20040530133843.51528.qmail@web50709.mail.yahoo.com>	<40BAE605.5040800@v.loewis.de>	<40BB6C9B.4040701@ieee.org>	<40BB7E63.9090304@v.loewis.de>	<A30D1ADC-B335-11D8-9388-000A95686CD8@redivi.com>	<40BB8882.60503@v.loewis.de>
	<5611D758-B33E-11D8-9388-000A95686CD8@redivi.com>
Message-ID: <40BB9C17.20500@v.loewis.de>

Bob Ippolito wrote:
> I'm not quite sure you are up to date on how Stackless 3.0 works.  The 
> assembly code is only there to facilitate "hard switching", which is 
> only required to do tasklet switching in the face of some C extension 
> that doesn't know about Stackless.  

Yes, that was my understanding. "Some C extensions" includes Tcl and 
COM, right? Unless I'm mistaken, it also includes map, filter, and
.encode/.decode of unicode objects (or any other C module that calls
back into Python, except for tasklet mechanism of stackless itself).

In these cases, copying out the stack will break the exception chain
that the Visual C++ compiler creates through the FS segment, so that
an attempt to throw an exception will cause a program crash, right?

> All pure 
> Python code and almost all code built-in to Python goes through "soft 
> switching", which is implemented in pure C with no stack tricks or 
> anything.  

What is the control flow for, say, the builtin map() function?

 > Stackless would still be a very useful addition to Python
> even when "hard switching" is not available.

That is the point I made at the beginning of this thread: Python is
currently not stackless because nobody has contributed a patch recently
to make it so. A patch that takes recursion out of eval_frame might
alone be valuable, but hasn't been contributed.

> Alternatively, Armin believes that Stackless "hard switching" can 
> actually be implemented in mostly-platform-neutral C (either by 
> setjmp/longjmp or alloca hacks, I forget which)

It is setjmp/longjmp. I have implemented a coroutine library on
top of setjmp/longjmp myself, so I am well aware of the implications.

> I'm also not sure how it could "break C++ exception handling" in 
> scenarios where it wouldn't be broken anyway.  I'd like to see a test of 
> that that passes under regular Python but fails in Stackless for this.

The VC++ exception mechanism puts a linked chain of exception frames
on the stack, where, at the beginning of each function, the stack
pointer is stored at FS:18 (or some such), and the old value of FS:18
is stored on the stack. Then, when an exception occurs, this linked
list is used to traverse the stack. If multiple stacks are used
in a single thread, then this linked list gets messed up.

Without trying, I think the following call chain might cause a crash
if Stackless is compiled with exception handling on:

Py-func1() -> C-func1() -> Py-func2() -> switch to new tasklet
               Py-func3() -> C-func2() -> throw exception()

Regards,
Martin


From bob at redivi.com  Mon May 31 17:10:39 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon May 31 17:10:49 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <40BB9C17.20500@v.loewis.de>
References: <20040530133843.51528.qmail@web50709.mail.yahoo.com>	<40BAE605.5040800@v.loewis.de>	<40BB6C9B.4040701@ieee.org>	<40BB7E63.9090304@v.loewis.de>	<A30D1ADC-B335-11D8-9388-000A95686CD8@redivi.com>	<40BB8882.60503@v.loewis.de>
	<5611D758-B33E-11D8-9388-000A95686CD8@redivi.com>
	<40BB9C17.20500@v.loewis.de>
Message-ID: <F8297E54-B346-11D8-9388-000A95686CD8@redivi.com>

On May 31, 2004, at 4:56 PM, Martin v. L?wis wrote:

> Bob Ippolito wrote:
>> I'm not quite sure you are up to date on how Stackless 3.0 works.  
>> The assembly code is only there to facilitate "hard switching", which 
>> is only required to do tasklet switching in the face of some C 
>> extension that doesn't know about Stackless.
>
> Yes, that was my understanding. "Some C extensions" includes Tcl and 
> COM, right? Unless I'm mistaken, it also includes map, filter, and
> .encode/.decode of unicode objects (or any other C module that calls
> back into Python, except for tasklet mechanism of stackless itself).

You're definitely correct about Tcl and COM.. COM is however a total 
non-issue because Win32/x86 is very well supported by Christian ;)

I'm believe that map and filter are non-recursive, I am not sure about 
unicode encode/decode.  I don't think it's very common to do a tasklet 
switch in the middle of a unicode encode or decode, though :)

> In these cases, copying out the stack will break the exception chain
> that the Visual C++ compiler creates through the FS segment, so that
> an attempt to throw an exception will cause a program crash, right?
>> I'm also not sure how it could "break C++ exception handling" in 
>> scenarios where it wouldn't be broken anyway.  I'd like to see a test 
>> of that that passes under regular Python but fails in Stackless for 
>> this.
>
> The VC++ exception mechanism puts a linked chain of exception frames
> on the stack, where, at the beginning of each function, the stack
> pointer is stored at FS:18 (or some such), and the old value of FS:18
> is stored on the stack. Then, when an exception occurs, this linked
> list is used to traverse the stack. If multiple stacks are used
> in a single thread, then this linked list gets messed up.
>
> Without trying, I think the following call chain might cause a crash
> if Stackless is compiled with exception handling on:
>
> Py-func1() -> C-func1() -> Py-func2() -> switch to new tasklet
>               Py-func3() -> C-func2() -> throw exception()

I don't see how it could cause problems unless this is valid:
	Py_Func1 -> CPlusPlus_ExceptionCatcher -> Py_Func2 -> 
CPlusPlus_ExceptionThrower -> throws exception()

If that *is* valid, how the heck does Py_Func2 get cleaned up?  I have 
to admit that I'm not very familiar with C++'s implementation of 
exceptions.

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2357 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040531/06835d10/smime.bin
From martin at v.loewis.de  Mon May 31 17:25:12 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon May 31 17:25:16 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <F8297E54-B346-11D8-9388-000A95686CD8@redivi.com>
References: <20040530133843.51528.qmail@web50709.mail.yahoo.com>	<40BAE605.5040800@v.loewis.de>	<40BB6C9B.4040701@ieee.org>	<40BB7E63.9090304@v.loewis.de>	<A30D1ADC-B335-11D8-9388-000A95686CD8@redivi.com>	<40BB8882.60503@v.loewis.de>
	<5611D758-B33E-11D8-9388-000A95686CD8@redivi.com>
	<40BB9C17.20500@v.loewis.de>
	<F8297E54-B346-11D8-9388-000A95686CD8@redivi.com>
Message-ID: <40BBA2B8.7000907@v.loewis.de>

Bob Ippolito wrote:

> I'm believe that map and filter are non-recursive

Can you point me to the code that demonstrates this? Looking at 
Stackless' src/Python/bltinmodule.c:builtin_map, I see

         for (i = 0; ; ++i) {
		...
                 value = PyEval_CallObject(func, alist);
                 ...
		PyList_SetItem(result, i, value)
		...
	}

which looks recursive to me.

> I am not sure about 
> unicode encode/decode.  I don't think it's very common to do a tasklet 
> switch in the middle of a unicode encode or decode, though :)

I thought Stackless was not only about tasklet switching, but also
about running without stack... Wouldn't it perform the "hard" switching
if the stack is running too deep?

>> Py-func1() -> C-func1() -> Py-func2() -> switch to new tasklet
>>               Py-func3() -> C-func2() -> throw exception()
> 
> 
> I don't see how it could cause problems unless this is valid:
>     Py_Func1 -> CPlusPlus_ExceptionCatcher -> Py_Func2 -> 
> CPlusPlus_ExceptionThrower -> throws exception()

It crashes in the process of throwing the exception - not because
it finds no exception handler. It crashes because the internal
data structures have been corrupted.

I should have better made this example:

Py-func1() -> C-func1() -> Py-func2() -> switch to new tasklet
        Py-func3() -> Exception-Catcher -> C-func2() -> throw exception()

> If that *is* valid, how the heck does Py_Func2 get cleaned up?  I have 
> to admit that I'm not very familiar with C++'s implementation of 
> exceptions.

Throwing exceptions through Python functions is not supported, indeed.
However, I was trying to point out that the entire exception handling
mechanism gets corrupted, even if there are valid handlers on the
stack.

Regards,
Martin


From bob at redivi.com  Mon May 31 17:39:01 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon May 31 17:39:09 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <40BBA2B8.7000907@v.loewis.de>
References: <20040530133843.51528.qmail@web50709.mail.yahoo.com>	<40BAE605.5040800@v.loewis.de>	<40BB6C9B.4040701@ieee.org>	<40BB7E63.9090304@v.loewis.de>	<A30D1ADC-B335-11D8-9388-000A95686CD8@redivi.com>	<40BB8882.60503@v.loewis.de>
	<5611D758-B33E-11D8-9388-000A95686CD8@redivi.com>
	<40BB9C17.20500@v.loewis.de>
	<F8297E54-B346-11D8-9388-000A95686CD8@redivi.com>
	<40BBA2B8.7000907@v.loewis.de>
Message-ID: <EE2948B2-B34A-11D8-9388-000A95686CD8@redivi.com>

On May 31, 2004, at 5:25 PM, Martin v. L?wis wrote:

> Bob Ippolito wrote:
>
>> I'm believe that map and filter are non-recursive
>
> Can you point me to the code that demonstrates this? Looking at 
> Stackless' src/Python/bltinmodule.c:builtin_map, I see
>
>         for (i = 0; ; ++i) {
> 		...
>                 value = PyEval_CallObject(func, alist);
>                 ...
> 		PyList_SetItem(result, i, value)
> 		...
> 	}
>
> which looks recursive to me.

Ok, then I am mistaken.  I thought I had seen a commit message that 
said it was made non-recursive.  Either way, it's possible to write a 
non-recursive version of map or anything else, it's just a bit of work.

>> I am not sure about unicode encode/decode.  I don't think it's very 
>> common to do a tasklet switch in the middle of a unicode encode or 
>> decode, though :)
>
> I thought Stackless was not only about tasklet switching, but also
> about running without stack... Wouldn't it perform the "hard" switching
> if the stack is running too deep?

I don't think so, because the C stack should never run too deep, since 
most code that is going to recurse (from the python-view) will do it 
non-recursively (from the C-view).

>>> Py-func1() -> C-func1() -> Py-func2() -> switch to new tasklet
>>>               Py-func3() -> C-func2() -> throw exception()
>> I don't see how it could cause problems unless this is valid:
>>     Py_Func1 -> CPlusPlus_ExceptionCatcher -> Py_Func2 -> 
>> CPlusPlus_ExceptionThrower -> throws exception()
>
> It crashes in the process of throwing the exception - not because
> it finds no exception handler. It crashes because the internal
> data structures have been corrupted.
>
> I should have better made this example:
>
> Py-func1() -> C-func1() -> Py-func2() -> switch to new tasklet
>        Py-func3() -> Exception-Catcher -> C-func2() -> throw 
> exception()
>
>> If that *is* valid, how the heck does Py_Func2 get cleaned up?  I 
>> have to admit that I'm not very familiar with C++'s implementation of 
>> exceptions.
>
> Throwing exceptions through Python functions is not supported, indeed.
> However, I was trying to point out that the entire exception handling
> mechanism gets corrupted, even if there are valid handlers on the
> stack.

I don't think that the entire exception handling mechanism gets 
corrupted.  Each tasklet should always have a consistent view of the C 
stack.  Anyway, I don't think we need to discuss it any further unless 
someone writes a failing test... and if that is indeed the case, then 
the discussion should be taken to the stackless list.

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2357 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040531/9544084b/smime.bin
From guido at python.org  Mon May 31 18:37:26 2004
From: guido at python.org (Guido van Rossum)
Date: Mon May 31 18:37:33 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: Your message of "Mon, 31 May 2004 11:34:19 MDT."
	<40BB6C9B.4040701@ieee.org> 
References: <20040530133843.51528.qmail@web50709.mail.yahoo.com>
	<40BAE605.5040800@v.loewis.de> <40BB6C9B.4040701@ieee.org> 
Message-ID: <200405312237.i4VMbQA21697@guido.python.org>

> Does this mean that the implementation of Stackless 3.0 could be 
> acceptable to integrate into CPython 2.5?  It would be absolutely 
> wonderful to have those capabilities available in the standard 
> distribution!

Let's not get too excited about this possibility.  Before something is
accepted into the language its semantics must be specified exactly and
clearly.  Last time this was proposed, the Stackless semantics were
extremely murky, with lots of implementation-specific exceptions.

Another requirement is that other Python implementations, like Jython
and IronPython (Jim Hugunin's C# port) can also support these
semantics.

Until these conditions are met, Stackless is stuck in the fringe.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From jjl at pobox.com  Mon May 31 19:24:06 2004
From: jjl at pobox.com (John J Lee)
Date: Mon May 31 19:24:14 2004
Subject: [Python-Dev] cookielib
In-Reply-To: <40BB7BA6.1040007@v.loewis.de>
References: <Pine.LNX.4.58.0405302350050.6233@alice>
	<40BB7BA6.1040007@v.loewis.de>
Message-ID: <Pine.LNX.4.58.0406010015190.444@alice>

On Mon, 31 May 2004, [ISO-8859-1] "Martin v. L=F6wis" wrote:

> John J Lee wrote:
> > What next?
>
> I have applied the patch. Now, when I run regrtest, I get
[...snip debug output...]
>
> Can you provide a patch to suppress this output?

Whoops, I should never have had a Logger.addHandler() call in there,
should I?  I've attached a patch to the original SF patch.


John

From tismer at stackless.com  Mon May 31 19:49:34 2004
From: tismer at stackless.com (Christian Tismer)
Date: Mon May 31 19:47:59 2004
Subject: [Python-Dev] Why aren't more things weak referencable
In-Reply-To: <200405302204.i4UM4hb19989@guido.python.org>
References: <000001c4461b$1fa22d80$e841fea9@oemcomputer>
	<200405302204.i4UM4hb19989@guido.python.org>
Message-ID: <40BBC48E.6030405@stackless.com>

Guido van Rossum wrote:

>>For strings at least, perhaps it is time to bite the bullet and
>>include weak reference support directly.  Weak reference support ups
>>the per string memory overhead from five words (ob_type, ob_refcnt,
>>ob_size, ob_shash, ob_sstate) to six.  The whole concept of weak
>>dictionaries is much more useful when strings can be used as keys
>>and/or values.
> 
> 
> Hmm... it is a high price to pay to add another word (*and* some extra
> code at dealloc time!) to every string object when very few apps need
> them and strings are about the most common data type.  And since
> they're immutable, what's the point of having weak refs to strings in
> the first place?  (Note that the original poster asked about
> *subclasses* of strings.)

Same here. I wouldnot vote to make strings or tuples or any other
tiny type weak-reffed in the first place.
Instead I would add the possible support to derived types, via
the __slot__ mechanism for instance.
There is a little coding necessary to make the generic code
handle the case of var-sized objects, but this is doable
and not very complicated.

This may be really needed or not. if we can create the rule
"every derived type *can* have weak-refs", this is simpler
to memorize than "well, most can, some cannot".

cheers -- chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From tismer at stackless.com  Mon May 31 20:09:08 2004
From: tismer at stackless.com (Christian Tismer)
Date: Mon May 31 20:07:36 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <40BBA2B8.7000907@v.loewis.de>
References: <20040530133843.51528.qmail@web50709.mail.yahoo.com>	<40BAE605.5040800@v.loewis.de>	<40BB6C9B.4040701@ieee.org>	<40BB7E63.9090304@v.loewis.de>	<A30D1ADC-B335-11D8-9388-000A95686CD8@redivi.com>	<40BB8882.60503@v.loewis.de>	<5611D758-B33E-11D8-9388-000A95686CD8@redivi.com>	<40BB9C17.20500@v.loewis.de>	<F8297E54-B346-11D8-9388-000A95686CD8@redivi.com>
	<40BBA2B8.7000907@v.loewis.de>
Message-ID: <40BBC924.6000803@stackless.com>

Martin v. L?wis wrote:

> Bob Ippolito wrote:
> 
>> I'm believe that map and filter are non-recursive

They have been. They can be, again, but why.

> Can you point me to the code that demonstrates this? Looking at 
> Stackless' src/Python/bltinmodule.c:builtin_map, I see
> 
>         for (i = 0; ; ++i) {
>         ...
>                 value = PyEval_CallObject(func, alist);
>                 ...
>         PyList_SetItem(result, i, value)
>         ...
>     }
> 
> which looks recursive to me.

Yes. trying to do a minimalistic approach, I dropped all
non-recursive implementations of non-trivial things (meaning
non-tail-recursive things) which were not absolutely necessary.
And since we have list comprehensions, map is no longer worth
being supported so much.

>> I am not sure about unicode encode/decode.  I don't think it's very 
>> common to do a tasklet switch in the middle of a unicode encode or 
>> decode, though :)
> 
> 
> I thought Stackless was not only about tasklet switching, but also
> about running without stack... Wouldn't it perform the "hard" switching
> if the stack is running too deep?

Yes, it would, if you allow it to do so. This would happen
after a very deep recursion, which you wouldn't be able to
do with normal Python at all.

...

> It crashes in the process of throwing the exception - not because
> it finds no exception handler. It crashes because the internal
> data structures have been corrupted.

I don't know the C++ exception mechanism. It it walks the whole stack
once before taking an action, then this is a problem.
If it does proper unwinding of C frames, I could probably play the
game and restore things just in time.

Anyway, I don't really get the point.
95% of Stackless is doing soft-switched stackless calls.
Behavior is completely controllable. We can easily avoid
any special C stack operation, by setting a flag that
disallows it (easy to implement) or by excluding the hard
switching stuff, completely (not an option now, but easy, too).

The discussion, as often before, tries to find out why it is *not*
possible to include Stackless, instead of finding out the useful
parts which could be supported with relatively small impact.
This is why Stackless exists, and will continue to exist.

For me, this thread is closed -- thanks - chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From tismer at stackless.com  Mon May 31 20:16:52 2004
From: tismer at stackless.com (Christian Tismer)
Date: Mon May 31 20:15:25 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <EE2948B2-B34A-11D8-9388-000A95686CD8@redivi.com>
References: <20040530133843.51528.qmail@web50709.mail.yahoo.com>	<40BAE605.5040800@v.loewis.de>	<40BB6C9B.4040701@ieee.org>	<40BB7E63.9090304@v.loewis.de>	<A30D1ADC-B335-11D8-9388-000A95686CD8@redivi.com>	<40BB8882.60503@v.loewis.de>	<5611D758-B33E-11D8-9388-000A95686CD8@redivi.com>	<40BB9C17.20500@v.loewis.de>	<F8297E54-B346-11D8-9388-000A95686CD8@redivi.com>	<40BBA2B8.7000907@v.loewis.de>
	<EE2948B2-B34A-11D8-9388-000A95686CD8@redivi.com>
Message-ID: <40BBCAF4.7050801@stackless.com>

Bob Ippolito wrote:

oops, ok, this need some words, still.

...

> Ok, then I am mistaken.  I thought I had seen a commit message that said 
> it was made non-recursive.  Either way, it's possible to write a 
> non-recursive version of map or anything else, it's just a bit of work.

I had that in 1.0, and I don't plan to do it again, although it
is simple, now.

...

> I don't think so, because the C stack should never run too deep, since 
> most code that is going to recurse (from the python-view) will do it 
> non-recursively (from the C-view).

You can create such situations. Until Stackless 3.0, it was possible
to do very deep explicit calls to the __call__ wrapper, for instance,
and wrappers were not supported.
Stackless 3.1 which is about to be announced has full support
for all slots and all methods by the machinery. Supported are
all tail-recursive calls (like __call__), because this is very cheap
to do. But it is possible to implement as many of special method
handlers and wrapper handlers as one wishes; I have the infrastructure
ready for this.

Anyway, for this rare artificial case, I have a check built in that
measures stack size instead of recursion counting, and it will
save the stack if necessary. Of course this is platform dependent,
but my, if we switch this off for non-supported platforms, what
do we loose?

[C++ exception issues]

> I don't think that the entire exception handling mechanism gets 
> corrupted.  Each tasklet should always have a consistent view of the C 
> stack.  Anyway, I don't think we need to discuss it any further unless 
> someone writes a failing test... and if that is indeed the case, then 
> the discussion should be taken to the stackless list.

Good idea, see you there! cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From tim.one at comcast.net  Mon May 31 20:25:40 2004
From: tim.one at comcast.net (Tim Peters)
Date: Mon May 31 20:25:48 2004
Subject: [Python-Dev] doctest improvements
In-Reply-To: <40BB7414.9030304@gradient.cis.upenn.edu>
Message-ID: <mailman.262.1086049548.6944.python-dev@python.org>

[Edward Loper]
> About a month and a half ago, I filed several patches for doctest, and
> they haven't gotten any comments; so I was wondering if anyone could look
> at them.

Jim Fulton and I intend to do some doctest work for 2.4, so feel free to
assign them to me.  I can't make time for it immediately, though.

> The most interesting is #933238, which addresses the often-noted problem
> that doctest can't handle blank lines.  It adds a special blank-line
> marker (currently "-"), which must be *dedented* with respect to the
> doctest block.

Fair warning that any gimmick weakening doctest's WYSIWYG guarantee is going
to be a hard sell if I'm the reviewer.  The business about blank lines is a
documented limitation of the design.  Limitations can indeed create
problems, but removing them can create other problems.  Making a doctest
reader learn a special "doctest markup language" is A Problem in this case.
Offhand, I'd be happier with an almost-self-evident (say)

    <blank line>

marker than with a cryptic

    _

"you have no hope of guessing what this means unless you scour the docs"
marker.  Dedenting is a clever idea, BTW.



From pje at telecommunity.com  Mon May 31 20:37:09 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon May 31 20:35:41 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <40BBC924.6000803@stackless.com>
References: <40BBA2B8.7000907@v.loewis.de>
	<20040530133843.51528.qmail@web50709.mail.yahoo.com>
	<40BAE605.5040800@v.loewis.de> <40BB6C9B.4040701@ieee.org>
	<40BB7E63.9090304@v.loewis.de>
	<A30D1ADC-B335-11D8-9388-000A95686CD8@redivi.com>
	<40BB8882.60503@v.loewis.de>
	<5611D758-B33E-11D8-9388-000A95686CD8@redivi.com>
	<40BB9C17.20500@v.loewis.de>
	<F8297E54-B346-11D8-9388-000A95686CD8@redivi.com>
	<40BBA2B8.7000907@v.loewis.de>
Message-ID: <5.1.1.6.0.20040531202832.037b7ec0@mail.telecommunity.com>

At 02:09 AM 6/1/04 +0200, Christian Tismer wrote:
>Anyway, I don't really get the point.
>95% of Stackless is doing soft-switched stackless calls.
>Behavior is completely controllable. We can easily avoid
>any special C stack operation, by setting a flag that
>disallows it (easy to implement) or by excluding the hard
>switching stuff, completely (not an option now, but easy, too).

If soft-switching is portable (i.e. pure C, no assembly), and is exposed as 
a library module (so that Jython et al can avoid supporting it), then 
perhaps a PEP for adding that functionality to mainstream Python would be 
meaningful.

If that gets in, then the other 5% can always sneak in later via feature 
creep.  ;)  Or, more importantly (if I understand correctly), it could be 
separately distributed as an add-on for platforms that can support it.


From edloper at gradient.cis.upenn.edu  Mon May 31 20:41:44 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Mon May 31 20:41:01 2004
Subject: [Python-Dev] doctest improvements
In-Reply-To: <200406010025.i510PjaN021251@gradient.cis.upenn.edu>
References: <200406010025.i510PjaN021251@gradient.cis.upenn.edu>
Message-ID: <40BBD0C8.206@gradient.cis.upenn.edu>

Tim Peters wrote:
> [...] Making a doctest
> reader learn a special "doctest markup language" is A Problem in this case.
> Offhand, I'd be happier with an almost-self-evident (say)
> 
>     <blank line>

Perhaps you'd be happier with a different value of BLANKLINE_MARKER, 
then?  E.g. "blank>":

def test():
     r"""
     A different marker:

             >>> print 'a\n\nb'
             a
       blank>
             b
     """

Although this means that you have to indent the doctest quite far. 
Perhaps it would be better if I changed the patch so the marker doesn't 
need to dedented all the way?  E.g.:

def test():
     r"""
     A different marker:

         >>> print 'a\n\nb'
         a
       <blank line>
         b
     """

Another option to consider might be:

def test():
     r"""
     A different marker:

         >>> print 'a\n\nb'
       | a
       |
       | b
     """

I.e., the user can optionally preceed each line with a "|" (or some 
other marker), dedented past the prompt, to mark the extent of the 
output.  I don't think this looks as wysiwyg, though.

-Edward


From bob at redivi.com  Mon May 31 20:49:33 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon May 31 20:49:41 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <5.1.1.6.0.20040531202832.037b7ec0@mail.telecommunity.com>
References: <40BBA2B8.7000907@v.loewis.de>
	<20040530133843.51528.qmail@web50709.mail.yahoo.com>
	<40BAE605.5040800@v.loewis.de> <40BB6C9B.4040701@ieee.org>
	<40BB7E63.9090304@v.loewis.de>
	<A30D1ADC-B335-11D8-9388-000A95686CD8@redivi.com>
	<40BB8882.60503@v.loewis.de>
	<5611D758-B33E-11D8-9388-000A95686CD8@redivi.com>
	<40BB9C17.20500@v.loewis.de>
	<F8297E54-B346-11D8-9388-000A95686CD8@redivi.com>
	<40BBA2B8.7000907@v.loewis.de>
	<5.1.1.6.0.20040531202832.037b7ec0@mail.telecommunity.com>
Message-ID: <8C2776D1-B365-11D8-9388-000A95686CD8@redivi.com>

On May 31, 2004, at 8:37 PM, Phillip J. Eby wrote:

> At 02:09 AM 6/1/04 +0200, Christian Tismer wrote:
>> Anyway, I don't really get the point.
>> 95% of Stackless is doing soft-switched stackless calls.
>> Behavior is completely controllable. We can easily avoid
>> any special C stack operation, by setting a flag that
>> disallows it (easy to implement) or by excluding the hard
>> switching stuff, completely (not an option now, but easy, too).
>
> If soft-switching is portable (i.e. pure C, no assembly), and is 
> exposed as a library module (so that Jython et al can avoid supporting 
> it), then perhaps a PEP for adding that functionality to mainstream 
> Python would be meaningful.

Soft switching needs to be implemented in a few key places of the 
interpreter itself or else Stackless would surely have been maintained 
as an extension module.  It is already pure C, no assembly or platform 
specific code.  Supporting the interface from Jython or IronPython 
should be possible, though at worst case each tasklet might actually be 
a new thread so it might not be terribly efficient... though it would 
work.

> If that gets in, then the other 5% can always sneak in later via 
> feature creep.  ;)  Or, more importantly (if I understand correctly), 
> it could be separately distributed as an add-on for platforms that can 
> support it.

Hard switching could be an add-on for platforms that support it..

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2357 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040531/4ed2b745/smime.bin
From edloper at gradient.cis.upenn.edu  Mon May 31 20:51:16 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Mon May 31 20:49:59 2004
Subject: [Python-Dev] anonymous cvs checkout
Message-ID: <40BBD304.9000904@gradient.cis.upenn.edu>

I'm having trouble checking out an anonymous copy of the Python CVS 
repository.  I tried the instructions from both the developer FAQ and 
the SF CVS page; and tried checking it out from debian, win2k, and OS X. 
  In all cases, I get an error that looks something like this:

     cvs server: cannot read file in mydbm_load_file: Input/output error
     cvs checkout: in directory .:
     cvs [checkout aborted]: *PANIC* administration files missing

A google search for "mydbm_load_file" didn't turn up anything useful. 
Anyone have an idea what's going wrong?  Thanks!

-Edward

(Here's a copy-paste of my shell session from debian, in case I'm doing 
something stupid:)

> [syse:~/data/programming] export CVSROOT=:pserver:anonymous@cvs.sourceforge.net:/cvsroot/python
> [syse:~/data/programming] cvs login
> Logging in to :pserver:anonymous@cvs.sourceforge.net:2401/cvsroot/python
> CVS password: 
> [syse:~/data/programming] cvs -z3 co python
> cvs server: cannot read file in mydbm_load_file: Input/output error
> cvs checkout: cannot read file in mydbm_load_file: Input/output error
> cvs checkout: warning: cannot write to history file /cvsroot/python/CVSROOT/history: Read-only file system
> cvs checkout: Updating python
> cvs checkout: failed to create lock directory for `/cvsroot/python/python' (/cvsroot/python/python/#cvs.lock): Read-only file system
> cvs checkout: failed to obtain dir lock in repository `/cvsroot/python/python'
> cvs [checkout aborted]: read lock failed - giving up


From python at rcn.com  Mon May 31 08:50:25 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon May 31 20:50:35 2004
Subject: [Python-Dev] Why aren't more things weak referencable
In-Reply-To: <40BBC48E.6030405@stackless.com>
Message-ID: <001e01c4470d$d814b840$e841fea9@oemcomputer>

> > Hmm... it is a high price to pay to add another word (*and* some
extra
> > code at dealloc time!) to every string object when very few apps
need
> > them and strings are about the most common data type.  And since
> > they're immutable, what's the point of having weak refs to strings
in
> > the first place?  (Note that the original poster asked about
> > *subclasses* of strings.)
> 
> Same here. I wouldnot vote to make strings or tuples or any other
> tiny type weak-reffed in the first place.
> Instead I would add the possible support to derived types, via
> the __slot__ mechanism for instance.
> There is a little coding necessary to make the generic code
> handle the case of var-sized objects, but this is doable
> and not very complicated.

Right.  

I think this is all that is needed at this point.  That way, the
granular types stay granular and the added functionality is available
via subclasses if needed.


Raymond


From lin at centurysoftware.com.my  Mon May 31 21:08:51 2004
From: lin at centurysoftware.com.my (Norlin Abd Rahim)
Date: Mon May 31 21:10:55 2004
Subject: [Python-Dev] rounding problem
Message-ID: <40BBD723.000003.01168@Norlin2>

Skipped content of type multipart/alternative-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 494 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040601/257d74ea/attachment.gif
From aahz at pythoncraft.com  Mon May 31 21:17:55 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon May 31 21:18:00 2004
Subject: [Python-Dev] Uninformative error message about subclassing
In-Reply-To: <40BB3D8D.4090809@erols.com>
References: <40BB3D8D.4090809@erols.com>
Message-ID: <20040601011755.GA20900@panix.com>

On Mon, May 31, 2004, Edward C. Jones wrote:
>
> ----
> a.py:
> 
> class A(object):
>    pass
> 
> ----
> b.py:
> 
> import a
> 
> class B(a):
>    pass
> ----
> If I run "python b.py" I get the uninformative error message:
> 
> Traceback (most recent call last):
>  File "./b.py", line 5, in ?
>    class B(a):
> TypeError: function takes at most 2 arguments (3 given)
> 
> I think the message should say that "a" is not a class.

Please file a bug report on SF.  However, I suspect this will be tricky
to solve (if it's even possible), because of the way builtin objects are
now part of new-style classes.  (That A is a new-style class is
irrelevant.)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"as long as we like the same operating system, things are cool." --piranha

From gherron at islandtraining.com  Mon May 31 21:19:00 2004
From: gherron at islandtraining.com (Gary Herron)
Date: Mon May 31 21:19:40 2004
Subject: [Python-Dev] rounding problem
In-Reply-To: <40BBD723.000003.01168@Norlin2>
References: <40BBD723.000003.01168@Norlin2>
Message-ID: <200405311819.01475.gherron@islandtraining.com>

On Monday 31 May 2004 06:08 pm, Norlin Abd Rahim wrote:
> Hi,
>
> I'm using python with zope. I used _.round and it seems like it is look
> unstable. For example if i have one number, 1130.975 and rounded it to 2
> decimal places, it will be 1130.97 and not 1130.98. But for certain numbers
> it will rounded correctly if the last digit is 5. I even used fixedpoint.py
> but still i've got the same problem. This applies only to numbers that have
> digit 5 after the decimal places that needs to be rounded.
>
> Any idea why?
>
> TIA

Perhaps the last digit is 5 only because the printing rounded
...4999... or some such up to 5.  Unless you KNOW that the number is
exactly 1130.975 (in its internal representation) then you must
consider this as a possibility.

Gary Herron



From lin at centurysoftware.com.my  Mon May 31 21:22:20 2004
From: lin at centurysoftware.com.my (Norlin Abd Rahim)
Date: Mon May 31 21:24:34 2004
Subject: [Python-Dev] rounding problem
References: <200405311819.01475.gherron@islandtraining.com>
Message-ID: <40BBDA4C.000001.01168@Norlin2>

Skipped content of type multipart/alternative-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 494 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040601/2172ac9a/attachment-0001.gif
From greg at cosc.canterbury.ac.nz  Mon May 31 21:37:57 2004
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon May 31 21:38:17 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <40BB8882.60503@v.loewis.de>
Message-ID: <200406010137.i511bvnA013748@cosc353.cosc.canterbury.ac.nz>

Martin:

> [In addition, the approach taken by Stackless 3.0 is probably
> questionable: for example, it may be that C++ exception handling
> stops working in Stackless.]

Indeed. The last time this was discussed, I seem to remember Stackless
had trouble with C code that passed pointers to local variables as
parameters. Since this is a perfectly legitimate thing to do in C, if
Stackless can't handle it, then it's fundamentally flawed.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From gherron at islandtraining.com  Mon May 31 21:39:20 2004
From: gherron at islandtraining.com (Gary Herron)
Date: Mon May 31 21:39:33 2004
Subject: [Python-Dev] rounding problem
In-Reply-To: <40BBDA4C.000001.01168@Norlin2>
References: <200405311819.01475.gherron@islandtraining.com>
	<40BBDA4C.000001.01168@Norlin2>
Message-ID: <200405311839.20514.gherron@islandtraining.com>

On Monday 31 May 2004 06:22 pm, Norlin Abd Rahim wrote:
> the number actually comes from multiplication between 237.50 and 4.762
> which will get 1130.975. So it suppose to be 1130.98 isnt it if it'll get
> rounded to 2 decimal places.

But...

Due to the wonders and inaccuracies of binary floating point
representations on digital computers, 4.762 can not be represented
EXACTLY.  The closest it can get is 4.7619999999999996, and that's
where you're inaccuracy comes from.  In the following interactive
python session, the values printed out are printed out to their full
precision (that's a often misunderstood feature of the python
interactive mode).  You can see where the floating point inaccuracies
are introduced.

>>> 237.50
237.5
>>> 4.762
4.7619999999999996
>>> 237.50 * 4.762
1130.9749999999999
>>>

Just for clarification .. This is NOT a problem with Python, but is a
feature of the binary representation of floating point numbers and can
be found on all computers and in all programming languages.

Gary Herron



From tim.one at comcast.net  Mon May 31 21:40:14 2004
From: tim.one at comcast.net (Tim Peters)
Date: Mon May 31 21:40:25 2004
Subject: [Python-Dev] rounding problem
In-Reply-To: <40BBDA4C.000001.01168@Norlin2>
Message-ID: <mailman.266.1086054025.6944.python-dev@python.org>

[Norlin Abd Rahim]
> the number actually comes from multiplication between 237.50 and 4.762
> which will get 1130.975.

Please read the Appendix on floating point issues in the Python Tutorial:

    http://docs.python.org/tut/node15.html

The actual fp product of those two numbers is strictly less than 1130.975.
You shouldn't have any such problem with FixedPoint if you're using it
correctly, but python-dev isn't the right place for this (so if you want
more on this, please use the newsgroup comp.lang.python). 



From t-meyer at ihug.co.nz  Mon May 31 21:41:27 2004
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Mon May 31 21:41:47 2004
Subject: [Python-Dev] rounding problem
In-Reply-To: <1ED4ECF91CDED24C8D012BCF2B034F1306966EEF@its-xchg4.massey.ac.nz>
Message-ID: <1ED4ECF91CDED24C8D012BCF2B034F13064C0265@its-xchg4.massey.ac.nz>

[I think this would be better in python-list, not python-dev]

> the number actually comes from multiplication between
> 237.50 and 4.762 which will get 1130.975. So it suppose
> to be 1130.98 isnt it if it'll get rounded to 2 decimal
> places. 

Gary is correct:

>>> 237.50 * 4.762
1130.9749999999999
>>> round(_,2)
1130.97
>>> print 237.50 * 4.762
1130.975

The answer correctly rounds to 1130.97 because the next digit is 4, not 5.
When you print it, it rounds to 3 decimal places, so you get the .9749
rounding to .975.

This is just floating point arithmetic.  You might want to try the Decimal
module, or something like that, to avoid this.

=Tony Meyer


From tim.one at comcast.net  Mon May 31 22:00:17 2004
From: tim.one at comcast.net (Tim Peters)
Date: Mon May 31 22:00:32 2004
Subject: [Python-Dev] doctest improvements
In-Reply-To: <40BBD0C8.206@gradient.cis.upenn.edu>
Message-ID: <mailman.267.1086055232.6944.python-dev@python.org>

[Edward Loper]
> Perhaps you'd be happier with a different value of BLANKLINE_MARKER,
> then?  E.g. "blank>":
>
> def test():
>      r"""
>      A different marker:
>
>      >>> print 'a\n\nb'
>              a
>        blank>
>              b
>      """
>
> Although this means that you have to indent the doctest quite far.
> Perhaps it would be better if I changed the patch so the marker doesn't
> need to dedented all the way?  E.g.:
>
> def test():
>      r"""
>      A different marker:
>
>      >>> print 'a\n\nb'
>          a
>        <blank line>
>          b
>      """

That's more what I had in mind, but if the marker is changed to something
wordy instead of "a magic character", I don't think the dedent trick is of
much value anymore:

     r"""
     A different marker:

             >>> print 'a\n\nb'
             a
             <blank line>
             b
     """

Then when the expected output is (exactly) "<blank line>", doctest would
accept "<blank line>", or a line with nothing other than whitespace, as a
match.  I can't get upset by that bit of ambiguity.  The primary advantage
is that the implementation is then trivial, to write, document, and
understand.  It won't break external tools, and caters to leading, trailing,
and embedded blank lines.  It needs an option flag to turn it off, akin to
the current doctest.DONT_ACCEPT_TRUE_FOR_1 (the only non-wysiwyg gimmick
implemented now); it should be enabled by default.

> Another option to consider might be:
>
> def test():
>      r"""
>      A different marker:
>
>      >>> print 'a\n\nb'
>>   | a
>>   |
>>   | b
>      """
>
> I.e., the user can optionally preceed each line with a "|" (or some other
> marker), dedented past the prompt, to mark the extent of the output.  I
> don't think this looks as wysiwyg, though.

Nope, it sure doesn't.



From pje at telecommunity.com  Mon May 31 22:51:39 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon May 31 22:50:26 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <8C2776D1-B365-11D8-9388-000A95686CD8@redivi.com>
References: <5.1.1.6.0.20040531202832.037b7ec0@mail.telecommunity.com>
	<40BBA2B8.7000907@v.loewis.de>
	<20040530133843.51528.qmail@web50709.mail.yahoo.com>
	<40BAE605.5040800@v.loewis.de> <40BB6C9B.4040701@ieee.org>
	<40BB7E63.9090304@v.loewis.de>
	<A30D1ADC-B335-11D8-9388-000A95686CD8@redivi.com>
	<40BB8882.60503@v.loewis.de>
	<5611D758-B33E-11D8-9388-000A95686CD8@redivi.com>
	<40BB9C17.20500@v.loewis.de>
	<F8297E54-B346-11D8-9388-000A95686CD8@redivi.com>
	<40BBA2B8.7000907@v.loewis.de>
	<5.1.1.6.0.20040531202832.037b7ec0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20040531222355.02f77ec0@mail.telecommunity.com>

At 08:49 PM 5/31/04 -0400, Bob Ippolito wrote:
>On May 31, 2004, at 8:37 PM, Phillip J. Eby wrote:
>
>>At 02:09 AM 6/1/04 +0200, Christian Tismer wrote:
>>>Anyway, I don't really get the point.
>>>95% of Stackless is doing soft-switched stackless calls.
>>>Behavior is completely controllable. We can easily avoid
>>>any special C stack operation, by setting a flag that
>>>disallows it (easy to implement) or by excluding the hard
>>>switching stuff, completely (not an option now, but easy, too).
>>
>>If soft-switching is portable (i.e. pure C, no assembly), and is exposed 
>>as a library module (so that Jython et al can avoid supporting it), then 
>>perhaps a PEP for adding that functionality to mainstream Python would be 
>>meaningful.
>
>Soft switching needs to be implemented in a few key places of the 
>interpreter itself or else Stackless would surely have been maintained as 
>an extension module.

I'm aware of this, which is why I said "exposed" as a library module, not 
"implemented" as one.  :)


>   It is already pure C, no assembly or platform specific 
> code.  Supporting the interface from Jython or IronPython should be 
> possible, though at worst case each tasklet might actually be a new 
> thread so it might not be terribly efficient... though it would work.

Yes, I suppose in the simplest case one could implement switching 
primitives via a simple mutex: acquire it when resuming, release it when 
yielding.  If the PEP described the semantics of tasklets in terms of 
threads, then a heavyweight implementation could be achieved.

However, it seems to me that any Python implementation that supports 
generators should in principle be able to support co-operative multitasking 
anyway, so long as there is no C code in the call chain.  Since 
microthreads are basically doable in pure Python 2.2 as long as you write 
*everything* as a generator and simulate your control stack, it seems you 
should be able to do this in the Python interpreter.

Actually, it seems to me that if there were a way to resume execution of a 
frame that has thrown a special "task switching" exception (sort of a 
non-local 'yield' operation), you could implement lightweight threading 
*without* having to de-stackify Python.  Of course, it's possible that the 
implementation of such a strategy might be just as complex as de-stacking 
Python.

To make it work, I think you'd need a way to tell a nested execution of 
Python code that it's being called from Python code (as opposed to C), so 
that it knows where to "resume to".  A chain of resumable frames could then 
be resumed by a special function.  Then, upon catching an exception that 
didn't unwind via any C code, you can call a function to resume the 
interrupted frame (and its parents).   Hm, actually, you'd have to copy the 
frames in order to be able to resume them, so you'd need to have a 
designated exception type for resumable exceptions, so as not to waste time 
copying frames for other kinds of errors.

Okay, I think I just convinced myself that it would be better to just make 
the interpreter core stackless instead of trying to design all that other 
crud.  :)

(To clarify, that's not an unqualified +1 to some hypothetical 
stacklessizing.  I'm just saying that if supporting ultra-light threading 
in Python is desirable, stacklessness seems at first glance like a 
promising way to do it, compared to trying to deal with unwinding the C 
stack.  And, to top it off, I think you'll end up doing half of what 
Stackless already does in order to "resume" the interrupted frames anyhow.)


From bob at redivi.com  Mon May 31 23:00:29 2004
From: bob at redivi.com (Bob Ippolito)
Date: Mon May 31 23:00:39 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: <5.1.1.6.0.20040531222355.02f77ec0@mail.telecommunity.com>
References: <5.1.1.6.0.20040531202832.037b7ec0@mail.telecommunity.com>
	<40BBA2B8.7000907@v.loewis.de>
	<20040530133843.51528.qmail@web50709.mail.yahoo.com>
	<40BAE605.5040800@v.loewis.de> <40BB6C9B.4040701@ieee.org>
	<40BB7E63.9090304@v.loewis.de>
	<A30D1ADC-B335-11D8-9388-000A95686CD8@redivi.com>
	<40BB8882.60503@v.loewis.de>
	<5611D758-B33E-11D8-9388-000A95686CD8@redivi.com>
	<40BB9C17.20500@v.loewis.de>
	<F8297E54-B346-11D8-9388-000A95686CD8@redivi.com>
	<40BBA2B8.7000907@v.loewis.de>
	<5.1.1.6.0.20040531202832.037b7ec0@mail.telecommunity.com>
	<5.1.1.6.0.20040531222355.02f77ec0@mail.telecommunity.com>
Message-ID: <D6E0F763-B377-11D8-9388-000A95686CD8@redivi.com>


On May 31, 2004, at 10:51 PM, Phillip J. Eby wrote:

> At 08:49 PM 5/31/04 -0400, Bob Ippolito wrote:
>> On May 31, 2004, at 8:37 PM, Phillip J. Eby wrote:
>>
>>> At 02:09 AM 6/1/04 +0200, Christian Tismer wrote:
>>>> Anyway, I don't really get the point.
>>>> 95% of Stackless is doing soft-switched stackless calls.
>>>> Behavior is completely controllable. We can easily avoid
>>>> any special C stack operation, by setting a flag that
>>>> disallows it (easy to implement) or by excluding the hard
>>>> switching stuff, completely (not an option now, but easy, too).
>>>
>>> If soft-switching is portable (i.e. pure C, no assembly), and is 
>>> exposed as a library module (so that Jython et al can avoid 
>>> supporting it), then perhaps a PEP for adding that functionality to 
>>> mainstream Python would be meaningful.
>>
>> Soft switching needs to be implemented in a few key places of the 
>> interpreter itself or else Stackless would surely have been 
>> maintained as an extension module.
>
> I'm aware of this, which is why I said "exposed" as a library module, 
> not "implemented" as one.  :)

Oh, it *is* already exposed as a module (named stackless)... but it's 
*implemented* as a built-in module, like sys or os.

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2357 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20040531/eea2a8b0/smime.bin
From edloper at gradient.cis.upenn.edu  Mon May 31 23:15:16 2004
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Mon May 31 23:14:31 2004
Subject: [Python-Dev] doctest improvements
In-Reply-To: <200406010203.i5123EaN026869@gradient.cis.upenn.edu>
References: <200406010203.i5123EaN026869@gradient.cis.upenn.edu>
Message-ID: <40BBF4C4.2090204@gradient.cis.upenn.edu>

Tim Peters wrote:
> That's more what I had in mind, but if the marker is changed to something
> wordy instead of "a magic character", I don't think the dedent trick is of
> much value anymore:
> 
>      r"""
>      A different marker:
> 
>              >>> print 'a\n\nb'
>              a
>              <blank line>
>              b
>      """
> 
> Then when the expected output is (exactly) "<blank line>", doctest would
> accept "<blank line>", or a line with nothing other than whitespace, as a
> match.  I can't get upset by that bit of ambiguity.  

I had ruled this out because I assumed that any change I made had to 
maintain backward compatibility.  But if this (minor) incompatibility is 
acceptable, then let me know and I'll write up a patch to implement it.

-Edward


From guido at python.org  Mon May 31 23:15:08 2004
From: guido at python.org (Guido van Rossum)
Date: Mon May 31 23:15:18 2004
Subject: [Python-Dev] Why aren't more things weak referencable
In-Reply-To: Your message of "Tue, 01 Jun 2004 01:49:34 +0200."
	<40BBC48E.6030405@stackless.com> 
References: <000001c4461b$1fa22d80$e841fea9@oemcomputer>
	<200405302204.i4UM4hb19989@guido.python.org> 
	<40BBC48E.6030405@stackless.com> 
Message-ID: <200406010315.i513F8E22050@guido.python.org>

> Same here. I wouldnot vote to make strings or tuples or any other
> tiny type weak-reffed in the first place.
> Instead I would add the possible support to derived types, via
> the __slot__ mechanism for instance.
> There is a little coding necessary to make the generic code
> handle the case of var-sized objects, but this is doable
> and not very complicated.
> 
> This may be really needed or not. if we can create the rule
> "every derived type *can* have weak-refs", this is simpler
> to memorize than "well, most can, some cannot".

With some (considerable?) effort, slots on var-sized objects can
certainly be supported -- the same approach as used for __dict__
should work.  I expect it might slow down the normal case a bit,
unless you define a new descriptor type just for this.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon May 31 23:17:57 2004
From: guido at python.org (Guido van Rossum)
Date: Mon May 31 23:18:06 2004
Subject: [Python-Dev] Stackless Python
In-Reply-To: Your message of "Mon, 31 May 2004 20:37:09 EDT."
	<5.1.1.6.0.20040531202832.037b7ec0@mail.telecommunity.com> 
References: <40BBA2B8.7000907@v.loewis.de>
	<20040530133843.51528.qmail@web50709.mail.yahoo.com>
	<40BAE605.5040800@v.loewis.de> <40BB6C9B.4040701@ieee.org>
	<40BB7E63.9090304@v.loewis.de>
	<A30D1ADC-B335-11D8-9388-000A95686CD8@redivi.com>
	<40BB8882.60503@v.loewis.de>
	<5611D758-B33E-11D8-9388-000A95686CD8@redivi.com>
	<40BB9C17.20500@v.loewis.de>
	<F8297E54-B346-11D8-9388-000A95686CD8@redivi.com>
	<40BBA2B8.7000907@v.loewis.de> 
	<5.1.1.6.0.20040531202832.037b7ec0@mail.telecommunity.com> 
Message-ID: <200406010317.i513Hvv22066@guido.python.org>

> If soft-switching is portable (i.e. pure C, no assembly), and is
> exposed as a library module (so that Jython et al can avoid
> supporting it), then perhaps a PEP for adding that functionality to
> mainstream Python would be meaningful.

Anything that can be supported by a pure extension module is fair game
IMO.  But it was my understanding that Stackless requires changes to
the VM.  And I definitely don't want people to get into a habit of
writing code that relies on deep recursion that is only supported by
Stackless, only to find that it doesn't work in Jython etc.

> If that gets in, then the other 5% can always sneak in later via
> feature creep.  ;) Or, more importantly (if I understand correctly),
> it could be separately distributed as an add-on for platforms that
> can support it.

Hey, that's what Stackless already does.  Why is that approach
suddenly not good enough any more?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From skip at pobox.com  Mon May 31 22:01:55 2004
From: skip at pobox.com (Skip Montanaro)
Date: Mon May 31 23:31:24 2004
Subject: [Python-Dev] rounding problem
In-Reply-To: <40BBDA4C.000001.01168@Norlin2>
References: <200405311819.01475.gherron@islandtraining.com>
	<40BBDA4C.000001.01168@Norlin2>
Message-ID: <16571.58259.802091.570024@montanaro.dyndns.org>


    Norlin> the number actually comes from multiplication between 237.50 and
    Norlin> 4.762 which will get 1130.975. So it suppose to be 1130.98 isnt
    Norlin> it if it'll get rounded to 2 decimal places.

Nope.  1130.975 is not exactly representable.  In fact, it's internal
representation is slightly less that 1130.975, so rounding to two decimal
places is correctly 1130.97.

    % python
    Python 2.4a0 (#25, May 22 2004, 15:16:21) 
    [GCC 3.3 20030304 (Apple Computer, Inc. build 1493)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> 1130.975
    1130.9749999999999

Skip

From tim.one at comcast.net  Mon May 31 23:40:38 2004
From: tim.one at comcast.net (Tim Peters)
Date: Mon May 31 23:40:45 2004
Subject: [Python-Dev] doctest improvements
In-Reply-To: <40BBF4C4.2090204@gradient.cis.upenn.edu>
Message-ID: <mailman.269.1086061245.6944.python-dev@python.org>

[Tim Peters]
>> ... but if the marker is changed to something wordy instead of "a magic
>> character", I don't think the dedent trick is of much value anymore:
>>
>>      r"""
>>      A different marker:
>>
>>              >>> print 'a\n\nb'
>>              a
>>              <blank line>
>>              b
>>      """
>>
>> Then when the expected output is (exactly) "<blank line>", doctest would
>> accept "<blank line>", or a line with nothing other than whitespace, as
>> a match.  I can't get upset by that bit of ambiguity.

[Edward Loper]
> I had ruled this out because I assumed that any change I made had to
> maintain backward compatibility.

We broke that last time around, with the gimmick to accept "True" ("False")
when "1" ("0") is expected.  Such highly limited ambiguities are OK by me by
default, when they've got strong pragmatic benefit, and there's a way to
turn them off.

> But if this (minor) incompatibility is acceptable,

It is to me.  Anyone object?  The consequence is that, by default, a doctest
that has

    <blank line>

as a line of its expected output will pass if it actually produces

    <blank line>

there, or if it produces an empty line there.  There's no effect on doctests
that currently contain an empty line as part of their expected output,
because no such doctest exits (the rules made it impossible -- and still
will).

BTW, I liked the dedent with the one-character "_" marker because then it
*screams* "something's fishy here", provoking a user to stare at the docs to
figure out what that may be.  Something like <blank line> is so close to
being self-explanatory that I don't think an extra nudge is necessary (or
even helpful -- I'd find it annoying).

> then let me know and I'll write up a patch to implement it.

Cool!  Let's wait to see if someone objects ... OK, time's up.  Go for it.

Edward, are you familiar with the doctest uses in Zope3?  doctest is getting
used heavily there, and it's turning out very well so far.  Jim has ideas
for making it more useful there, and if you'd like I'd be delighted to have
you join us in thrashing those ideas out.  This week I expect we'll be
buried under making various Zope and ZODB releases, but with some luck we'll
plunge into the doctest ideas next week.  While the invitation is being made
to you, it's not necessarily limited to you <wink>.



From ta-meyer at ihug.co.nz  Mon May 31 21:55:17 2004
From: ta-meyer at ihug.co.nz (Tony Meyer)
Date: Tue Jun  1 12:57:19 2004
Subject: [Python-Dev] anonymous cvs checkout
Message-ID: <1ED4ECF91CDED24C8D012BCF2B034F1304677EE1@its-xchg4.massey.ac.nz>

[Opps.  Forgot to include the list in my reply.  And those that read
spambayes-dev will have seen this already.  Good grief.]

[Edward Loper]
> I'm having trouble checking out an anonymous copy of the Python CVS 
> repository.
 
FWIW, neither a fresh checkout nor an update works here 
(WinXP), either, although (at least some) other sourceforge 
projects do.  ViewCVS doesn't work, either (shows an empty 
page), so it's certainly not just you.
 
<http://cvs.sourceforge.net/viewcvs.py/python/>

The SF site status does mention that there's maintenance scheduled (for
tomorrow) - maybe that's relevant?  Does developer access still work?

=Tony Meyer