From martin@v.loewis.de  Sat Dec  1 01:07:19 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Sat, 1 Dec 2001 02:07:19 +0100
Subject: [Python-Dev] -DINET6 in Makefile
In-Reply-To: <3C07F016.AEC20158@lemburg.com> (mal@lemburg.com)
References: <3C050CAF.2A5CEDC5@lemburg.com> <200111302022.fAUKMO002089@mira.informatik.hu-berlin.de> <3C07F016.AEC20158@lemburg.com>
Message-ID: <200112010107.fB117JO03160@mira.informatik.hu-berlin.de>

> > > Shouldn't the define be placed into the pyconfig.h file or
> > > only in those extensions which need it ?
> > 
> > Wouldn't it cause the same trouble there?
> 
> No, because the pyconfig.h import is under extension control
> (e.g. you can first include the system or lib header files 
> and only then import pyconfig.h).

Of course, doing so would be really stupid. Python.h *must* be the
first include, or things may break.

Regards,
Martin



From martin@v.loewis.de  Sat Dec  1 01:11:34 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Sat, 1 Dec 2001 02:11:34 +0100
Subject: [Python-Dev] gc.garbage
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCECAMKAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCCECAMKAA.tim.one@home.com>
Message-ID: <200112010111.fB11BYZ03163@mira.informatik.hu-berlin.de>

> [Martin v. Loewis]
> > I wish the C API hadn't been changed for 2.2, rendering useless all
> > code that had been created to support GC in 2.0 and 2.1.
> 
> Would we really need more than one hand to count all that code <0.9 wink>?

Mine was among it (in pyexpat), and I really hate looking at the code
now. Not only need I support two versions of Unicode (i.e. having or
not having it), but also three versions of GC, all in a single file.

Regards,
Martin


From mark.favas@csiro.au  Sat Dec  1 01:55:49 2001
From: mark.favas@csiro.au (Mark Favas)
Date: Sat, 01 Dec 2001 09:55:49 +0800
Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
Message-ID: <3C0838A5.7B35D64A@csiro.au>

Anyone else getting test_builtin failures with current CVS, or does it
only show on 64-bit platforms? Changes in the past week seem to have
caused the failure. Isolated to following (will post bug on SF):

***** older CVS works *****
201 mark@gonzo.per.dem.csiro.au python
Python 2.2b2+ (#539, Nov 26 2001, 09:52:25) [C] on osf1V4
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> s=-1-sys.maxint
>>> d=`s`
>>> d
'-9223372036854775808'
>>> s
-9223372036854775808
>>> ^D

***** current CVS fails *****
202 mark@gonzo.per.dem.csiro.au cd dist/src
203 mark@gonzo.per.dem.csiro.au ./python
Python 2.2b2+ (#541, Dec  1 2001, 08:04:58) [C] on osf1V4
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> s=-1-sys.maxint
>>> d=`s`
>>> d
'8t\x10@\x01'
>>> s
-9223372036854775808
>>> ^D


-- 
Mark Favas  -   m.favas@per.dem.csiro.au
CSIRO, Private Bag No 5, Wembley, Western Australia 6913, AUSTRALIA


From tim.one@home.com  Sat Dec  1 02:41:56 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 30 Nov 2001 21:41:56 -0500
Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
In-Reply-To: <3C0838A5.7B35D64A@csiro.au>
Message-ID: <LNBBLJKPBEHFEDALKOLCKECLMKAA.tim.one@home.com>

[Mark Favas]
> Anyone else getting test_builtin failures with current CVS,

No.

> or does it only show on 64-bit platforms? Changes in the past week
> seem to have caused the failure. Isolated to following (will post bug
> on SF):

Thanks!

> ***** older CVS works *****
> 201 mark@gonzo.per.dem.csiro.au python
> Python 2.2b2+ (#539, Nov 26 2001, 09:52:25) [C] on osf1V4
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sys
> >>> s=-1-sys.maxint
> >>> d=`s`
> >>> d
> '-9223372036854775808'
> >>> s
> -9223372036854775808
> >>> ^D
>
> ***** current CVS fails *****
> 202 mark@gonzo.per.dem.csiro.au cd dist/src
> 203 mark@gonzo.per.dem.csiro.au ./python
> Python 2.2b2+ (#541, Dec  1 2001, 08:04:58) [C] on osf1V4
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sys
> >>> s=-1-sys.maxint
> >>> d=`s`
> >>> d
> '8t\x10@\x01'
> >>> s
> -9223372036854775808
> >>> ^D

It doesn't make sense, but it *smells* like a sprintf -> PyOS_snprintf
screwup ... OK, our int repr code has always been wrong(!):

static PyObject *
int_repr(PyIntObject *v)
{
	char buf[20];
	PyOS_snprintf(buf, sizeof(buf), "%ld", v->ob_ival);
	return PyString_FromString(buf);
}

20 bytes isn't enough to hold the result on a 64-bit box (insert rant about
the idiot practice of trying to make stack buffers as small as possible).
You have 20 characters in your result, but need 21 to hold the trailing 0
byte too.  I don't know what snprintf does when there's not enough room, but
I think you just showed us what it does on Tru64 <wink>.

Doing repr impllicitly instead works because that goes thru int_print
instead of int_repr.  Doing repr explicitly "worked" before by accident (the
trailing null overwrite some random byte on the stack).  Please change 20 to
200(*) (or 64 -- your choice) and see whether that fixes it?



From guido@python.org  Sat Dec  1 03:01:24 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 30 Nov 2001 22:01:24 -0500
Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
In-Reply-To: Your message of "Fri, 30 Nov 2001 21:41:56 EST."
 <LNBBLJKPBEHFEDALKOLCKECLMKAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCKECLMKAA.tim.one@home.com>
Message-ID: <200112010301.WAA06035@cj20424-a.reston1.va.home.com>

> I don't know what snprintf does when there's not enough room, but
> I think you just showed us what it does on Tru64 <wink>.

Hm, snprintf is *supposed* to truncate the result, but it seems that
here it refused to do anything.  Maybe PyOS_snprintf should be a
wrapper that checks the return value of snprintf?  I noticed that
*none* of the recently checked-in PyOS_snprintf calls have their
return value checked, and I think it's better to leave it that way
(since in many cases we really don't know what to do instead) -- maybe
PyOS_snprintf should even return void (or does it already?).

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@home.com  Sat Dec  1 03:42:58 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 30 Nov 2001 22:42:58 -0500
Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
In-Reply-To: <200112010301.WAA06035@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEECPMKAA.tim.one@home.com>

> Hm, snprintf is *supposed* to truncate the result,

According to C99, yes, but MS has its own pre-C99 snprintf semantics, and
glibc has had at least two different versions.

> but it seems that here it refused to do anything.  Maybe PyOS_snprintf
> should be a wrapper that checks the return value of snprintf?

Check it for what?  We can't (at least not yet) count on uniform behavior
across platform snprintf implementations.

> I noticed that *none* of the recently checked-in PyOS_snprintf calls
> have their return value checked,

They were all derived (and most mindlessly) from existing sprintf calls that
didn't check either.  A goal of this transformation was to change as little
as possible.

> and I think it's better to leave it that way (since in many cases we
> really don't know what to do instead) -- maybe PyOS_snprintf should
> even return void (or does it already?).

The comments suggest it wants to return the C99-defined value (an int, < 0
for an encoding error, else the number of characters (excluding \0) written
if the size is big enough, else the number of characters that would have
been written (excluding \0) had size been big enough.  So the output was
converted in full iff the return value is non-negative and strictly less
than the size passed in.  That's fine by me, and I'll note that MS snprintf
meets that much too (if size isn't big enough, it returns a negative
result).  So perhaps in a debug build PyOS_snprintf could assert that the
returned value is non-negative and less than the passed-in size ...



From barry@zope.com  Sat Dec  1 04:17:31 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 30 Nov 2001 23:17:31 -0500
Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
References: <3C0838A5.7B35D64A@csiro.au>
 <LNBBLJKPBEHFEDALKOLCKECLMKAA.tim.one@home.com>
Message-ID: <15368.23003.392434.202067@anthem.wooz.org>

>>>>> "TP" == Tim Peters <tim.one@home.com> writes:

    TP> It doesn't make sense, but it *smells* like a sprintf ->
    TP> PyOS_snprintf screwup ... OK, our int repr code has always
    TP> been wrong(!):

    | static PyObject *
    | int_repr(PyIntObject *v)
    | {
    | 	char buf[20];
    | 	PyOS_snprintf(buf, sizeof(buf), "%ld", v->ob_ival);
    | 	return PyString_FromString(buf);
    | }

    TP> 20 bytes isn't enough to hold the result on a 64-bit box
    TP> (insert rant about the idiot practice of trying to make stack
    TP> buffers as small as possible).  You have 20 characters in your
    TP> result, but need 21 to hold the trailing 0 byte too.  I don't
    TP> know what snprintf does when there's not enough room, but I
    TP> think you just showed us what it does on Tru64 <wink>.

Heh, I was going to suggest that this might be a good place to
substitute a call to PyString_FromFormat*() but then I read this
little nugget:

			case 'd': case 'i': case 'x':
				(void) va_arg(count, int);
				/* 20 bytes should be enough to hold a 64-bit
				   integer */
				n += 20;
				break;

love-ly y'rs,
-Barry


From tim.one@home.com  Sat Dec  1 04:31:51 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 30 Nov 2001 23:31:51 -0500
Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
In-Reply-To: <15368.23003.392434.202067@anthem.wooz.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEDCMKAA.tim.one@home.com>

[Barry]
> ...
> Heh, I was going to suggest that this might be a good place to
> substitute a call to PyString_FromFormat*() but then I read this
> little nugget:
>
> 			case 'd': case 'i': case 'x':
> 				(void) va_arg(count, int);
> 				/* 20 bytes should be enough to
> hold a 64-bit
> 				   integer */
> 				n += 20;
> 				break;

This use of 20 is fine (I checked all occurrences of "20" in the codebase,
BTW); int_repr's use of 20 was wrong because it failed to allow for the
trailing \0 byte too; the loop in PyString_FromFormatV doesn't have to worry
about that (PyString_FromStringAndSize() later adds 1 of its own for the
trailing \0 -- of course it doesn't actually add anything, but it adds 1 "in
effect" <wink -- jeez, C string handling is obscure!>).



From guido@python.org  Sat Dec  1 08:06:01 2001
From: guido@python.org (Guido van Rossum)
Date: Sat, 01 Dec 2001 03:06:01 -0500
Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
In-Reply-To: Your message of "Fri, 30 Nov 2001 22:42:58 EST."
 <LNBBLJKPBEHFEDALKOLCEECPMKAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCEECPMKAA.tim.one@home.com>
Message-ID: <200112010806.DAA06935@cj20424-a.reston1.va.home.com>

> > Hm, snprintf is *supposed* to truncate the result,
> 
> According to C99, yes, but MS has its own pre-C99 snprintf semantics, and
> glibc has had at least two different versions.
> 
> > but it seems that here it refused to do anything.  Maybe PyOS_snprintf
> > should be a wrapper that checks the return value of snprintf?
> 
> Check it for what?  We can't (at least not yet) count on uniform behavior
> across platform snprintf implementations.

See below.  On my system, the snprintf man page documents both
behaviors, with the gcc version where it switches.

> > I noticed that *none* of the recently checked-in PyOS_snprintf calls
> > have their return value checked,
> 
> They were all derived (and most mindlessly) from existing sprintf calls that
> didn't check either.  A goal of this transformation was to change as little
> as possible.
> 
> > and I think it's better to leave it that way (since in many cases we
> > really don't know what to do instead) -- maybe PyOS_snprintf should
> > even return void (or does it already?).
> 
> The comments suggest it wants to return the C99-defined value (an int, < 0
> for an encoding error, else the number of characters (excluding \0) written
> if the size is big enough, else the number of characters that would have
> been written (excluding \0) had size been big enough.  So the output was
> converted in full iff the return value is non-negative and strictly less
> than the size passed in.  That's fine by me, and I'll note that MS snprintf
> meets that much too (if size isn't big enough, it returns a negative
> result).  So perhaps in a debug build PyOS_snprintf could assert that the
> returned value is non-negative and less than the passed-in size ...

Should work for gcc too.

Note that the Tru64 output suggests that the behavior (== effect on
the buffer) may differ too: a truncated version of the desired output
may or may not be written to the buffer.  This can be addressed by
pre-filling the buffer with a useful pattern in PyOS_snprintf before
calling the system's snprintf -- e.g. "*-*-*-*" (to give a patriotic
example :-).

--Guido van Rossum (home page: http://www.python.org/~guido/)

PS: I feel like I'm writing this on borrowed time -- Comcast and
Excite@Home haven't disconnected me from the net yet. :-)


From mark.favas@csiro.au  Sat Dec  1 10:25:50 2001
From: mark.favas@csiro.au (Mark Favas)
Date: Sat, 01 Dec 2001 18:25:50 +0800
Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
References: <LNBBLJKPBEHFEDALKOLCEECPMKAA.tim.one@home.com> <200112010806.DAA06935@cj20424-a.reston1.va.home.com>
Message-ID: <3C08B02E.EC833A0@csiro.au>

The increase in buf size from 20 to 64 fixes the problem, as Tim
expected. Note that Tru64 (at least V4, unsure about V5) has no native
snprintf implementation.

Guido van Rossum wrote:
> 
> > > Hm, snprintf is *supposed* to truncate the result,
> >
> > According to C99, yes, but MS has its own pre-C99 snprintf semantics, and
> > glibc has had at least two different versions.
> >
> > > but it seems that here it refused to do anything.  Maybe PyOS_snprintf
> > > should be a wrapper that checks the return value of snprintf?
> >

-- 
Mark Favas  -   mark.favas@csiro.au


From fredrik@pythonware.com  Sat Dec  1 11:35:16 2001
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Sat, 1 Dec 2001 12:35:16 +0100
Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
References: <3C0838A5.7B35D64A@csiro.au><LNBBLJKPBEHFEDALKOLCKECLMKAA.tim.one@home.com> <15368.23003.392434.202067@anthem.wooz.org>
Message-ID: <010a01c17a5c$45c1c030$ced241d5@hagrid>

Barry A. Warsaw wrote:
>     TP> 20 bytes isn't enough to hold the result on a 64-bit box
>     TP> (insert rant about the idiot practice of trying to make stack
>     TP> buffers as small as possible).  You have 20 characters in your
>     TP> result, but need 21 to hold the trailing 0 byte too.  I don't
>     TP> know what snprintf does when there's not enough room, but I
>     TP> think you just showed us what it does on Tru64 <wink>.
> 
> Heh, I was going to suggest that this might be a good place to
> substitute a call to PyString_FromFormat*() but then I read this
> little nugget:
> 
> case 'd': case 'i': case 'x':
> (void) va_arg(count, int);
> /* 20 bytes should be enough to hold a 64-bit
>    integer */
> n += 20;
> break;

The size calculated by PyString_FromFormat is used to allocate a
Python string, not a C string.  The Python string allocator always
adds an extra 0 byte at the end.

(if it hadn't, I would have used 21 instead)

</F>



From mal@lemburg.com  Sat Dec  1 11:27:20 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 01 Dec 2001 12:27:20 +0100
Subject: [Python-Dev] Subclassing varying length types (What's a
 PyStructSequence ?)
References: <LNBBLJKPBEHFEDALKOLCAECEMKAA.tim.one@home.com>
Message-ID: <3C08BE98.79946952@lemburg.com>

Tim Peters wrote:
> 
> [M.-A. Lemburg]
> > ...
> > Hmm, this makes me wonder: perhaps we should start thinking
> > about phasing out varying length PyObjects in the interpreter...
> 
> No chance, IMO:  the memory savings is too great.
>
> > esp. the inability to subclass strings looks like a bummer for
> > future extensions of this particular type. Unicode doesn't have
> > this problem, BTW.
> 
> OTOH, I know someone at Zope Corp who could testify with force about the
> memory burden of switching to Unicode strings -- if you've got gobs of 'em,
> it's much worse than a factor of 2 blowup.  Moving to obmalloc.c should help
> that a lot (two malloc overheads per Unicode string, and obmalloc overheads
> are much lower).

Perhaps we should start thinking about optimizing at least
one of the Unicode malloc away in Python 2.3: the Unicode
object itself can well be kept on a free list with the
Py_UNICODE buffer freed and set to NULL. Doesn't save any memory
but would improve the performance.

BTW, is the memory burden really such a big argument these days ?
I can imagine this being an argument on resource restrained
platforms such as Palms (thanks to Martin, the Plam guys can now
switch off Unicode completely), but hardly on gigabyte machines
with access 100s of GBs swap-space :-)

> > Or we need to come up with a fairly nice way of making
> > subclassing varying length types a lot easier, e.g. by
> > adding a special pointer ob_ext to PyObject_VAR_HEAD
> > which then allows declaring type extensions in an malloced
> > buffer.
> >
> > Thoughts ?
> 
> Not a one <wink>.

Any idea how we could make subclassing these types less
hackish, then ?

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/



From mal@lemburg.com  Sat Dec  1 13:06:37 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 01 Dec 2001 14:06:37 +0100
Subject: [Python-Dev] -DINET6 in Makefile
References: <3C050CAF.2A5CEDC5@lemburg.com> <200111302022.fAUKMO002089@mira.informatik.hu-berlin.de> <3C07F016.AEC20158@lemburg.com> <200112010107.fB117JO03160@mira.informatik.hu-berlin.de>
Message-ID: <3C08D5DD.1F3E8B55@lemburg.com>

"Martin v. Loewis" wrote:
> 
> > > > Shouldn't the define be placed into the pyconfig.h file or
> > > > only in those extensions which need it ?
> > >
> > > Wouldn't it cause the same trouble there?
> >
> > No, because the pyconfig.h import is under extension control
> > (e.g. you can first include the system or lib header files
> > and only then import pyconfig.h).
> 
> Of course, doing so would be really stupid. Python.h *must* be the
> first include, or things may break.

Why should "things break" ? I've doing this for years in lots
of Python extensions...

Back on topic: how are we going to get -DINET6 out of the
Makefile ? 

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From martin@v.loewis.de  Sat Dec  1 13:53:36 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Sat, 1 Dec 2001 14:53:36 +0100
Subject: [Python-Dev] Subclassing varying length types (What's a
 PyStructSequence ?)
In-Reply-To: <3C08BE98.79946952@lemburg.com> (mal@lemburg.com)
References: <LNBBLJKPBEHFEDALKOLCAECEMKAA.tim.one@home.com> <3C08BE98.79946952@lemburg.com>
Message-ID: <200112011353.fB1Drab17745@mira.informatik.hu-berlin.de>

> BTW, is the memory burden really such a big argument these days ?

You should consider that malloc overhead is often 16 bytes per
object. Given that PyUnicodeObject is 24 bytes in 2.2, system malloc
will allocate 48 bytes per Unicode object on modern architectures.  I
would think 100% overhead *is* a big argument. 

If you relate this to the actual data, it gets worse: A Unicode string
of length 1 would still require 32 bytes on an allocator that aligns
to 16. Therefore, to store 2 bytes of real data, you need 80 bytes of
memory.

I don't know how much overhead pymalloc adds, though; I believe it is
significantly less expensive.

Regards,
Martin


From mal@lemburg.com  Sat Dec  1 14:49:49 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 01 Dec 2001 15:49:49 +0100
Subject: [Python-Dev] Subclassing varying length types (What's a
 PyStructSequence ?)
References: <LNBBLJKPBEHFEDALKOLCAECEMKAA.tim.one@home.com> <3C08BE98.79946952@lemburg.com> <200112011353.fB1Drab17745@mira.informatik.hu-berlin.de>
Message-ID: <3C08EE0D.DF82C41F@lemburg.com>

"Martin v. Loewis" wrote:
> 
> > BTW, is the memory burden really such a big argument these days ?
> 
> You should consider that malloc overhead is often 16 bytes per
> object. Given that PyUnicodeObject is 24 bytes in 2.2, system malloc
> will allocate 48 bytes per Unicode object on modern architectures.  I
> would think 100% overhead *is* a big argument.
> 
> If you relate this to the actual data, it gets worse: A Unicode string
> of length 1 would still require 32 bytes on an allocator that aligns
> to 16. Therefore, to store 2 bytes of real data, you need 80 bytes of
> memory.
> 
> I don't know how much overhead pymalloc adds, though; I believe it is
> significantly less expensive.

Oh, I wasn't arguing against using pymalloc :-) 

I only think that nowadays, the trade-off "more flexibility vs. 
memory consumption" leans more towards the former than the latter. 
Not only because memory is cheap, but also because more flexibility 
tends to result in use of better algorithms and these can lead to 
better overall performance and lower total memory use.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From guido@python.org  Sat Dec  1 15:46:20 2001
From: guido@python.org (Guido van Rossum)
Date: Sat, 01 Dec 2001 10:46:20 -0500
Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
In-Reply-To: Your message of "Sat, 01 Dec 2001 18:25:50 +0800."
 <3C08B02E.EC833A0@csiro.au>
References: <LNBBLJKPBEHFEDALKOLCEECPMKAA.tim.one@home.com> <200112010806.DAA06935@cj20424-a.reston1.va.home.com>
 <3C08B02E.EC833A0@csiro.au>
Message-ID: <200112011546.KAA08107@cj20424-a.reston1.va.home.com>

> The increase in buf size from 20 to 64 fixes the problem, as Tim
> expected. Note that Tru64 (at least V4, unsure about V5) has no native
> snprintf implementation.

Oops, you're right.  It seems that mysnprintf.c doesn't copy any bytes
to the buffer when there's an overflow.

--Guido van Rossum (home page: http://www.python.org/~guido/)



From guido@python.org  Sat Dec  1 15:59:18 2001
From: guido@python.org (Guido van Rossum)
Date: Sat, 01 Dec 2001 10:59:18 -0500
Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
In-Reply-To: Your message of "Sat, 01 Dec 2001 10:46:20 EST."
 <200112011546.KAA08107@cj20424-a.reston1.va.home.com>
References: <LNBBLJKPBEHFEDALKOLCEECPMKAA.tim.one@home.com> <200112010806.DAA06935@cj20424-a.reston1.va.home.com> <3C08B02E.EC833A0@csiro.au>
 <200112011546.KAA08107@cj20424-a.reston1.va.home.com>
Message-ID: <200112011559.KAA08652@cj20424-a.reston1.va.home.com>

> Oops, you're right.  It seems that mysnprintf.c doesn't copy any bytes
> to the buffer when there's an overflow.

Confirmed, and fixed in CVS.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@zope.com  Sat Dec  1 17:02:54 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Sat, 1 Dec 2001 12:02:54 -0500 (EST)
Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEECPMKAA.tim.one@home.com>
References: <200112010301.WAA06035@cj20424-a.reston1.va.home.com>
 <LNBBLJKPBEHFEDALKOLCEECPMKAA.tim.one@home.com>
Message-ID: <15369.3390.364861.781161@slothrop.digicool.com>

The uses of PyOS_snprintf() in sysmodule.c actually checks the return
value.  It always checked the sprintf() returned value and triggered a
Py_FatalError() if the return value was too big.  I changed it to
print a warning message that the output was truncated.

Jeremy



From jim@zope.com  Sat Dec  1 17:22:34 2001
From: jim@zope.com (Jim Fulton)
Date: Sat, 01 Dec 2001 12:22:34 -0500
Subject: [Python-Dev] ExtensionClass plans (was Re: [ZODB-Dev] Another ExtensionClass
 incompatibility: __pow__())
References: <20011130115328.A6028@mems-exchange.org>
Message-ID: <3C0911DA.752DD61E@zope.com>

Greg Ward wrote:
> 
> Sigh.  In addition to ignoring __r{add,sub,mul,div}__() methods,
> ExtensionClass also calls __pow__() differently.

(snip)

> So what is the future of ExtensionClass, anyways? 

I intend that Python 2.2 (or maybe 2.3) will make it irrelevent.

> Are the little
> incompatibility bugs between classic Python classes and ExtensionClass
> ever going to be fixed? 

Probably not by us. Patches are gratefully accepted.

We are going to keep supporting it for a while at it's 
current level to support projects other than ZODB
that require it.

> Or has ExtensionClass been abandoned in favour
> of a future ZODB based on new-style Python 2.2 classes? 

This is my plan. :)

> If so, when
> will this future ZODB become reality?

I'm glad you asked. As part of our Zope 3 prototyping, we've
begun moving toward a next generation of ZODB.  We currently
have an experimental ZODB that:

- Doesn't require ExtensionClass

- Does depend on new-style Python classes

- Separates the Persistence and Transaction frameworks

- Uses weak references in the cache

- Uses a pure-python Persistent implementation (hence no BTrees).
  We will be creaing a C Persistent base class.

All the fun is currently happening on the Zope-3x-branch branch
of the Zope3 checkout. You can see the relevent ZODB pieces at:

http://cvs.zope.org/Zope3/lib/python/Persistence/?only_with_tag=Zope-3x-branch

http://cvs.zope.org/Zope3/lib/python/Transaction/?only_with_tag=Zope-3x-branch

http://cvs.zope.org/Zope3/lib/python/ZODB/?only_with_tag=Zope-3x-branch

Of course all of this is highly speculative at this point.

Stay tuned. ;)

Jim

--
Jim Fulton           mailto:jim@zope.com       Python Powered!        
CTO                  (888) 344-4332            http://www.python.org  
Zope Corporation     http://www.zope.com       http://www.zope.org


From jack@oratrix.nl  Sat Dec  1 23:26:00 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Sun, 02 Dec 2001 00:26:00 +0100
Subject: [Python-Dev] Differentiating $(EXT) between build and install
Message-ID: <20011201232600.45BC4E2677@oratrix.oratrix.nl>

Folks,
there's so many bug reports and questions about needing the
--with-suffix on Mac OS X when you're building on an HFS+ filesystem
that I'd like to fix this. The warning in configure doesn't cut it
because it gets lost in all the other messages.

What I'd like to do is have a separate Makefile variable, BUILDEXT,
that gives the extension to use for python while it lives in the build
directory. It would default to EXT, except if on HFS+ when EXT is
empty.

But this might change a couple of other things as well, so my question
is: is this an acceptable change so late in the beta stage?
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.cwi.nl/~jack        | ++++ see http://www.xs4all.nl/~tank/ ++++


From MarkH@ActiveState.com  Sun Dec  2 00:52:47 2001
From: MarkH@ActiveState.com (Mark Hammond)
Date: Sun, 2 Dec 2001 11:52:47 +1100
Subject: [Python-Dev] Subclassing varying length types (What's a   PyStructSequence ?)
In-Reply-To: <3C08EE0D.DF82C41F@lemburg.com>
Message-ID: <LCEPIIGDJPKCOIHOBJEPCEEBEPAA.MarkH@ActiveState.com>

[MAL]
> I only think that nowadays, the trade-off "more flexibility vs.
> memory consumption" leans more towards the former than the latter.
> Not only because memory is cheap, but also because more flexibility
> tends to result in use of better algorithms and these can lead to
> better overall performance and lower total memory use.

This tends to sound great - until you have a large app that is consuming too
many MBs :)

Mozilla's use of strings is fascinating.  They have a very complex C++
string API - all aimed at being as space-optimized as possible.  They go to
huge lengths to avoid string copies and extra allocation at the expense of
an API nobody understands :)

The point is that for fundamental types (including Unicode), medium size
apps may use millions of these objects. Everything we can do to optimize
their speed and size is beneficial.

Mark.



From adam@isdn.net.il  Sun Dec  2 07:36:43 2001
From: adam@isdn.net.il (adam)
Date: Sun, 2 Dec 2001 09:36:43 +0200
Subject: [Python-Dev] REMOVE !!
References: <E16A1pl-0000ug-00@mail.python.org>
Message-ID: <002301c17b04$17cd4a20$0101c80a@LocalHost>

----- Original Message -----
From: <python-dev-request@python.org>
To: <python-dev@python.org>
Sent: Saturday, December 01, 2001 6:33 AM
Subject: Python-Dev digest, Vol 1 #1748 - 15 msgs


> Send Python-Dev mailing list submissions to
> python-dev@python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mail.python.org/mailman/listinfo/python-dev
> or, via email, send a message with subject or body 'help' to
> python-dev-request@python.org
>
> You can reach the person managing the list at
> python-dev-admin@python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Python-Dev digest..."
>
>
> Today's Topics:
>
>    1. ArtifactFile: I can't change my patch files (James C. Ahlstrom)
>    2. Re: problems with DBM nonuniformity (Jason R. Mastaler)
>    3. Re: -DINET6 in Makefile (Martin v. Loewis)
>    4. Re: -DINET6 in Makefile (M.-A. Lemburg)
>    5. RE: gc.garbage (Tim Peters)
>    6. RE: gc.garbage (Tim Peters)
>    7. RE: Subclassing varying length types (What's a  PyStructSequence ?)
(Tim Peters)
>    8. Re: -DINET6 in Makefile (Martin v. Loewis)
>    9. Re: gc.garbage (Martin v. Loewis)
>   10. test_builtin failing? or just 64-bit platforms (Mark Favas)
>   11. RE: test_builtin failing? or just 64-bit platforms (Tim Peters)
>   12. Re: test_builtin failing? or just 64-bit platforms (Guido van
Rossum)
>   13. RE: test_builtin failing? or just 64-bit platforms (Tim Peters)
>   14. RE: test_builtin failing? or just 64-bit platforms (Barry A. Warsaw)
>   15. RE: test_builtin failing? or just 64-bit platforms (Tim Peters)
>
> --__--__--
>
> Message: 1
> Date: Fri, 30 Nov 2001 12:07:03 -0500
> From: "James C. Ahlstrom" <jim@interet.com>
> To: python-dev@python.org
> Subject: [Python-Dev] ArtifactFile: I can't change my patch files
>
> I am trying to replace files in my patch #483466, but
> I am getting
>
>   File delete: ArtifactFile: Permission Denied
>
> on SourceForge.  If I need to be a member of the Python
> project to do this, could someone please add me?  Or
> tell me what's wrong?
>
> The new files implement the changes to sys.path[0]
> that we have been discussing.  I still need to make
> further changes to import.c so that "" and relative
> paths are handled.  So don't install this patch yet,
> but look it over for problems if you want.
>
> Jim
>
>
> --__--__--
>
> Message: 2
> To: python-list@python.org, python-dev@python.org,
> Date: Fri, 30 Nov 2001 10:17:24 -0700
> From: "Jason R. Mastaler" <jason-dated-1007831845.6c6a51@mastaler.com>
> Subject: [Python-Dev] Re: problems with DBM nonuniformity
>
> Skip Montanaro <skip@pobox.com> writes:
>
> > Seems to me the natural thing to do would be to add
> > "get_data_filename" and "get_index_filename" methods (or something
> > similar) to the underlying modules (dbhash, bsddb, dbm, etc) and
> > expose them through anydbm.
>
> I see.  So you agree that with the current implementation, there isn't
> a reliable way to do what I'm trying to do with DBM?
>
> > It's too late for 2.2, but I suspect if you implemented something
> > and method name(s) could be settled on it would make it into CVS
> > early in the 2.3 cycle.  This seems like a small enough change that
> > you just file a bug report on SourceForge with the proposal and add
> > an implementation when you have something workable.
>
> I'm not sure when I'll be able to get to this, but I'll put it on my
> TODO list.  In the meantime, I think I'll just support the auto
> regeneration feature I mentioned with CDB[1] instead of DBM since its
> file interface is consistent across platforms.
>
> Footnotes:
> 1.  python-cdb extension module (http://pilcrow.madison.wi.us/)
>
> --
> (TMDA - http://tmda.sourceforge.net)
> (Python-based SPAM reduction system)
>
>
> --__--__--
>
> Message: 3
> Date: Fri, 30 Nov 2001 21:22:24 +0100
> From: "Martin v. Loewis" <martin@v.loewis.de>
> To: mal@lemburg.com
> CC: python-dev@python.org
> Subject: Re: [Python-Dev] -DINET6 in Makefile
>
> > What's the reasoning behind putting -DINET6 into the default
> > compiler options of the generic Makefile ?
>
> I believe the sole reason is that the author of the patch didn't know
> how to get it into pyconfig.h. itojun recently confirmed that all uses
> of the INET6 can be replaced with ENABLE_IPV6, and that the define may
> go away.
>
> I hesitate to change that, though, since some of the IPv6
> implementations *may* require that INET6 is defined when processing
> the "system" headers (not all IPv6 implementations we support actually
> come with the operating system).
>
> > I'm just asking because such a define will be inherited by
> > all extensions being compiled with distutils and the Makefile.pre.in
> > setup process... sounds like trouble if you ask me.
>
> What kind of trouble?
>
> > Shouldn't the define be placed into the pyconfig.h file or
> > only in those extensions which need it ?
>
> Wouldn't it cause the same trouble there?
>
> Regards,
> Martin
>
>
> --__--__--
>
> Message: 4
> Date: Fri, 30 Nov 2001 21:46:14 +0100
> From: "M.-A. Lemburg" <mal@lemburg.com>
> Organization: eGenix.com Software GmbH
> To: "Martin v. Loewis" <martin@v.loewis.de>
> CC: python-dev@python.org
> Subject: Re: [Python-Dev] -DINET6 in Makefile
>
> "Martin v. Loewis" wrote:
> >
> > > What's the reasoning behind putting -DINET6 into the default
> > > compiler options of the generic Makefile ?
> >
> > I believe the sole reason is that the author of the patch didn't know
> > how to get it into pyconfig.h. itojun recently confirmed that all uses
> > of the INET6 can be replaced with ENABLE_IPV6, and that the define may
> > go away.
> >
> > I hesitate to change that, though, since some of the IPv6
> > implementations *may* require that INET6 is defined when processing
> > the "system" headers (not all IPv6 implementations we support actually
> > come with the operating system).
>
> For Python's own use, it should suffice defining the symbol in
> pyconfig.h.
>
> > > I'm just asking because such a define will be inherited by
> > > all extensions being compiled with distutils and the Makefile.pre.in
> > > setup process... sounds like trouble if you ask me.
> >
> > What kind of trouble?
>
> The symbol could enable some logic which may not be desired
> by the application, e.g. cause system includes to change,
> socket semantics of wrapped libs could also be affected etc.
>
> > > Shouldn't the define be placed into the pyconfig.h file or
> > > only in those extensions which need it ?
> >
> > Wouldn't it cause the same trouble there?
>
> No, because the pyconfig.h import is under extension control
> (e.g. you can first include the system or lib header files
> and only then import pyconfig.h).
>
> --
> Marc-Andre Lemburg
> CEO eGenix.com Software GmbH
> ______________________________________________________________________
> Consulting & Company:                           http://www.egenix.com/
> Python Software:                        http://www.lemburg.com/python/
>
>
> --__--__--
>
> Message: 5
> From: "Tim Peters" <tim.one@home.com>
> To: "Neil Schemenauer" <nas@python.ca>,
> "PythonDev" <python-dev@python.org>
> Subject: RE: [Python-Dev] gc.garbage
> Date: Fri, 30 Nov 2001 18:30:43 -0500
>
> [Neil Schemenauer]
> > Is there some way to prevent people from assigning to certain module
> > variables?
>
> [Tim]
> >> Not that I know of.  If you're terribly concerned, gc could look up
> >> "garbage" in its dict on each access.  That's what, e.g.,
> >> PRINT_ITEM does with sys.stdout. ...
>
> [Neil]
> > What would happen if it's not a list?  PRINT_ITEM raises RuntimeError.
> > I suppose the collector could do the same.
>
> Sure, that would be fine.
>
> >> But I'd be keener to see new words spelling out which parts of the gc
> >> interface are and aren't intended "to work" across releases ...
>
> > All of them? :-)  Seriously, there could come a time when GC can no
> > longer be disabled.  The debugging and threshold stuff is fairly
> > implementation dependent.  get_referrers() and get_objects() are highly
> > implementation dependent.  I suppose gc.collect() should always be
> > available.  Anything else is fair game, IMHO.
>
> I meant "new words" in the docs, not on Python-Dev <wink>.
>
> > Incidentally, I can't say I'm happy with GC as it stands.
>
> Well, you're young and hopeful -- you'll get over both.  I have, and am
> indeed happy with GC as it stands.
>
> > It uses too much memory now that so many objects are tracked.
>
> There I disagree, but subtly:  it always used "too much" memory.  The
> marginal memory cost in adding a gazillion new tracked types was minor, as
> very few programs have a gazillion frame objects or traceback objects or
> generator-iterator objects (etc) sitting around.  The vast bulk of the
> damage was done the instant lists, tuples, dicts and instances got
tracked.
> So it goes.
>
> > I had worked on the idea of a separate heap for GC objects for a while
> > but couldn't figure out how to make generational collection to work.
>
> Generational gimmicks are rare in non-copying collectors for this very
> reason, right?
>
> > As Don Beaudry's sig says: "so much code, so little time".  :-)
>
> Time for Don to change his sig -- his young and hopeful days should be
long
> gone by now too <wink>.
>
>
>
> --__--__--
>
> Message: 6
> From: "Tim Peters" <tim.one@home.com>
> To: <python-dev@python.org>
> Subject: RE: [Python-Dev] gc.garbage
> Date: Fri, 30 Nov 2001 18:30:44 -0500
>
> [Martin v. Loewis]
> > I wish the C API hadn't been changed for 2.2, rendering useless all
> > code that had been created to support GC in 2.0 and 2.1.
>
> Would we really need more than one hand to count all that code <0.9 wink>?
>
> not-aware-of-any-myself-ly y'rs  - tim
>
>
> --__--__--
>
> Message: 7
> From: "Tim Peters" <tim.one@home.com>
> To: <python-dev@python.org>
> Subject: RE: [Python-Dev] Subclassing varying length types (What's a
PyStructSequence ?)
> Date: Fri, 30 Nov 2001 18:35:23 -0500
>
> [M.-A. Lemburg]
> > ...
> > Hmm, this makes me wonder: perhaps we should start thinking
> > about phasing out varying length PyObjects in the interpreter...
>
> No chance, IMO:  the memory savings is too great.
>
> > esp. the inability to subclass strings looks like a bummer for
> > future extensions of this particular type. Unicode doesn't have
> > this problem, BTW.
>
> OTOH, I know someone at Zope Corp who could testify with force about the
> memory burden of switching to Unicode strings -- if you've got gobs of
'em,
> it's much worse than a factor of 2 blowup.  Moving to obmalloc.c should
help
> that a lot (two malloc overheads per Unicode string, and obmalloc
overheads
> are much lower).
>
> > Or we need to come up with a fairly nice way of making
> > subclassing varying length types a lot easier, e.g. by
> > adding a special pointer ob_ext to PyObject_VAR_HEAD
> > which then allows declaring type extensions in an malloced
> > buffer.
> >
> > Thoughts ?
>
> Not a one <wink>.
>
>
>
> --__--__--
>
> Message: 8
> Date: Sat, 1 Dec 2001 02:07:19 +0100
> From: "Martin v. Loewis" <martin@v.loewis.de>
> To: mal@lemburg.com
> CC: python-dev@python.org
> Subject: Re: [Python-Dev] -DINET6 in Makefile
>
> > > > Shouldn't the define be placed into the pyconfig.h file or
> > > > only in those extensions which need it ?
> > >
> > > Wouldn't it cause the same trouble there?
> >
> > No, because the pyconfig.h import is under extension control
> > (e.g. you can first include the system or lib header files
> > and only then import pyconfig.h).
>
> Of course, doing so would be really stupid. Python.h *must* be the
> first include, or things may break.
>
> Regards,
> Martin
>
>
>
> --__--__--
>
> Message: 9
> Date: Sat, 1 Dec 2001 02:11:34 +0100
> From: "Martin v. Loewis" <martin@v.loewis.de>
> To: tim.one@home.com
> CC: python-dev@python.org
> Subject: Re: [Python-Dev] gc.garbage
>
> > [Martin v. Loewis]
> > > I wish the C API hadn't been changed for 2.2, rendering useless all
> > > code that had been created to support GC in 2.0 and 2.1.
> >
> > Would we really need more than one hand to count all that code <0.9
wink>?
>
> Mine was among it (in pyexpat), and I really hate looking at the code
> now. Not only need I support two versions of Unicode (i.e. having or
> not having it), but also three versions of GC, all in a single file.
>
> Regards,
> Martin
>
>
> --__--__--
>
> Message: 10
> Date: Sat, 01 Dec 2001 09:55:49 +0800
> From: Mark Favas <mark.favas@csiro.au>
> Organization: CSIRO
> To: python-dev@python.org
> Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
>
> Anyone else getting test_builtin failures with current CVS, or does it
> only show on 64-bit platforms? Changes in the past week seem to have
> caused the failure. Isolated to following (will post bug on SF):
>
> ***** older CVS works *****
> 201 mark@gonzo.per.dem.csiro.au python
> Python 2.2b2+ (#539, Nov 26 2001, 09:52:25) [C] on osf1V4
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sys
> >>> s=-1-sys.maxint
> >>> d=`s`
> >>> d
> '-9223372036854775808'
> >>> s
> -9223372036854775808
> >>> ^D
>
> ***** current CVS fails *****
> 202 mark@gonzo.per.dem.csiro.au cd dist/src
> 203 mark@gonzo.per.dem.csiro.au ./python
> Python 2.2b2+ (#541, Dec  1 2001, 08:04:58) [C] on osf1V4
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sys
> >>> s=-1-sys.maxint
> >>> d=`s`
> >>> d
> '8t\x10@\x01'
> >>> s
> -9223372036854775808
> >>> ^D
>
>
> --
> Mark Favas  -   m.favas@per.dem.csiro.au
> CSIRO, Private Bag No 5, Wembley, Western Australia 6913, AUSTRALIA
>
>
> --__--__--
>
> Message: 11
> From: "Tim Peters" <tim.one@home.com>
> To: "Mark Favas" <mark.favas@csiro.au>,
> <python-dev@python.org>
> Subject: RE: [Python-Dev] test_builtin failing? or just 64-bit platforms
> Date: Fri, 30 Nov 2001 21:41:56 -0500
>
> [Mark Favas]
> > Anyone else getting test_builtin failures with current CVS,
>
> No.
>
> > or does it only show on 64-bit platforms? Changes in the past week
> > seem to have caused the failure. Isolated to following (will post bug
> > on SF):
>
> Thanks!
>
> > ***** older CVS works *****
> > 201 mark@gonzo.per.dem.csiro.au python
> > Python 2.2b2+ (#539, Nov 26 2001, 09:52:25) [C] on osf1V4
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> import sys
> > >>> s=-1-sys.maxint
> > >>> d=`s`
> > >>> d
> > '-9223372036854775808'
> > >>> s
> > -9223372036854775808
> > >>> ^D
> >
> > ***** current CVS fails *****
> > 202 mark@gonzo.per.dem.csiro.au cd dist/src
> > 203 mark@gonzo.per.dem.csiro.au ./python
> > Python 2.2b2+ (#541, Dec  1 2001, 08:04:58) [C] on osf1V4
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> import sys
> > >>> s=-1-sys.maxint
> > >>> d=`s`
> > >>> d
> > '8t\x10@\x01'
> > >>> s
> > -9223372036854775808
> > >>> ^D
>
> It doesn't make sense, but it *smells* like a sprintf -> PyOS_snprintf
> screwup ... OK, our int repr code has always been wrong(!):
>
> static PyObject *
> int_repr(PyIntObject *v)
> {
> char buf[20];
> PyOS_snprintf(buf, sizeof(buf), "%ld", v->ob_ival);
> return PyString_FromString(buf);
> }
>
> 20 bytes isn't enough to hold the result on a 64-bit box (insert rant
about
> the idiot practice of trying to make stack buffers as small as possible).
> You have 20 characters in your result, but need 21 to hold the trailing 0
> byte too.  I don't know what snprintf does when there's not enough room,
but
> I think you just showed us what it does on Tru64 <wink>.
>
> Doing repr impllicitly instead works because that goes thru int_print
> instead of int_repr.  Doing repr explicitly "worked" before by accident
(the
> trailing null overwrite some random byte on the stack).  Please change 20
to
> 200(*) (or 64 -- your choice) and see whether that fixes it?
>
>
>
> --__--__--
>
> Message: 12
> To: "Tim Peters" <tim.one@home.com>
> cc: "Mark Favas" <mark.favas@csiro.au>, python-dev@python.org
> Subject: Re: [Python-Dev] test_builtin failing? or just 64-bit platforms
> From: Guido van Rossum <guido@python.org>
> Date: Fri, 30 Nov 2001 22:01:24 -0500
>
> > I don't know what snprintf does when there's not enough room, but
> > I think you just showed us what it does on Tru64 <wink>.
>
> Hm, snprintf is *supposed* to truncate the result, but it seems that
> here it refused to do anything.  Maybe PyOS_snprintf should be a
> wrapper that checks the return value of snprintf?  I noticed that
> *none* of the recently checked-in PyOS_snprintf calls have their
> return value checked, and I think it's better to leave it that way
> (since in many cases we really don't know what to do instead) -- maybe
> PyOS_snprintf should even return void (or does it already?).
>
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>
>
> --__--__--
>
> Message: 13
> From: "Tim Peters" <tim.one@home.com>
> To: "Guido van Rossum" <guido@python.org>
> Cc: "Mark Favas" <mark.favas@csiro.au>,
> <python-dev@python.org>
> Subject: RE: [Python-Dev] test_builtin failing? or just 64-bit platforms
> Date: Fri, 30 Nov 2001 22:42:58 -0500
>
> > Hm, snprintf is *supposed* to truncate the result,
>
> According to C99, yes, but MS has its own pre-C99 snprintf semantics, and
> glibc has had at least two different versions.
>
> > but it seems that here it refused to do anything.  Maybe PyOS_snprintf
> > should be a wrapper that checks the return value of snprintf?
>
> Check it for what?  We can't (at least not yet) count on uniform behavior
> across platform snprintf implementations.
>
> > I noticed that *none* of the recently checked-in PyOS_snprintf calls
> > have their return value checked,
>
> They were all derived (and most mindlessly) from existing sprintf calls
that
> didn't check either.  A goal of this transformation was to change as
little
> as possible.
>
> > and I think it's better to leave it that way (since in many cases we
> > really don't know what to do instead) -- maybe PyOS_snprintf should
> > even return void (or does it already?).
>
> The comments suggest it wants to return the C99-defined value (an int, < 0
> for an encoding error, else the number of characters (excluding \0)
written
> if the size is big enough, else the number of characters that would have
> been written (excluding \0) had size been big enough.  So the output was
> converted in full iff the return value is non-negative and strictly less
> than the size passed in.  That's fine by me, and I'll note that MS
snprintf
> meets that much too (if size isn't big enough, it returns a negative
> result).  So perhaps in a debug build PyOS_snprintf could assert that the
> returned value is non-negative and less than the passed-in size ...
>
>
>
> --__--__--
>
> Message: 14
> Date: Fri, 30 Nov 2001 23:17:31 -0500
> To: "Tim Peters" <tim.one@home.com>
> Cc: "Mark Favas" <mark.favas@csiro.au>, <python-dev@python.org>
> Subject: RE: [Python-Dev] test_builtin failing? or just 64-bit platforms
> From: barry@zope.com (Barry A. Warsaw)
>
>
> >>>>> "TP" == Tim Peters <tim.one@home.com> writes:
>
>     TP> It doesn't make sense, but it *smells* like a sprintf ->
>     TP> PyOS_snprintf screwup ... OK, our int repr code has always
>     TP> been wrong(!):
>
>     | static PyObject *
>     | int_repr(PyIntObject *v)
>     | {
>     | char buf[20];
>     | PyOS_snprintf(buf, sizeof(buf), "%ld", v->ob_ival);
>     | return PyString_FromString(buf);
>     | }
>
>     TP> 20 bytes isn't enough to hold the result on a 64-bit box
>     TP> (insert rant about the idiot practice of trying to make stack
>     TP> buffers as small as possible).  You have 20 characters in your
>     TP> result, but need 21 to hold the trailing 0 byte too.  I don't
>     TP> know what snprintf does when there's not enough room, but I
>     TP> think you just showed us what it does on Tru64 <wink>.
>
> Heh, I was going to suggest that this might be a good place to
> substitute a call to PyString_FromFormat*() but then I read this
> little nugget:
>
> case 'd': case 'i': case 'x':
> (void) va_arg(count, int);
> /* 20 bytes should be enough to hold a 64-bit
>    integer */
> n += 20;
> break;
>
> love-ly y'rs,
> -Barry
>
>
> --__--__--
>
> Message: 15
> From: "Tim Peters" <tim.one@home.com>
> To: "Barry A. Warsaw" <barry@zope.com>
> Cc: "Mark Favas" <mark.favas@csiro.au>,
> <python-dev@python.org>
> Subject: RE: [Python-Dev] test_builtin failing? or just 64-bit platforms
> Date: Fri, 30 Nov 2001 23:31:51 -0500
>
> [Barry]
> > ...
> > Heh, I was going to suggest that this might be a good place to
> > substitute a call to PyString_FromFormat*() but then I read this
> > little nugget:
> >
> > case 'd': case 'i': case 'x':
> > (void) va_arg(count, int);
> > /* 20 bytes should be enough to
> > hold a 64-bit
> >    integer */
> > n += 20;
> > break;
>
> This use of 20 is fine (I checked all occurrences of "20" in the codebase,
> BTW); int_repr's use of 20 was wrong because it failed to allow for the
> trailing \0 byte too; the loop in PyString_FromFormatV doesn't have to
worry
> about that (PyString_FromStringAndSize() later adds 1 of its own for the
> trailing \0 -- of course it doesn't actually add anything, but it adds 1
"in
> effect" <wink -- jeez, C string handling is obscure!>).
>
>
>
>
> --__--__--
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
>
>
> End of Python-Dev Digest



From tim.one@home.com  Sun Dec  2 08:41:28 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 2 Dec 2001 03:41:28 -0500
Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
In-Reply-To: <200112010806.DAA06935@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEFNMKAA.tim.one@home.com>

[Guido]
> ...
> Note that the Tru64 output suggests that the behavior (== effect on
> the buffer) may differ too: a truncated version of the desired output
> may or may not be written to the buffer.

Which later turned out to be our fault.  However, the MS native _snprintf is
worse than that:  if the buffer isn't big enough, it fills it to the end
*without* a trailing \0.  A subsequent (e.g.) PyString_FromString() would
then pick up an arbitrarily long stretch of garbage bytes beyond the
buffer's end.  You can see the effect by, e.g., making int_repr's buffer 4
bytes, then doing repr(sys.maxint), on Windows.

> This can be addressed by pre-filling the buffer with a useful pattern in
> PyOS_snprintf before calling the system's snprintf -- e.g. "*-*-*-*" (to
> give a patriotic example :-).

I don't know why you'd want to burn time doing this -- IMO, if the buffer
isn't big enough, it should almost always be due to an internal Python bug.
The point of PyOS_snprintf isn't to let us be lazy about buffer decls or
careless with formats, it's to prevent exploits even when we screw up.

> PS: I feel like I'm writing this on borrowed time -- Comcast and
> Excite@Home haven't disconnected me from the net yet. :-)

It's quite the soap opera!  AT&T pulled out of the talks, and AT&T's @Home
customers were then cut off.  The other partners are still negotiating.
Moral of the story:  if you ever start a moderately successful ISP, think
twice before spending 6 billion dollars to acquire a portal.  If you can't
resist, at least avoid selling your services below cost to companies much
bigger than you, counting on portal advertising dollars to make up the
difference.

if-it-cost-6-billion-it-must-be-worth-at-least-12-ly y'rs  - tim



From tim.one@home.com  Sun Dec  2 08:44:28 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 2 Dec 2001 03:44:28 -0500
Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
In-Reply-To: <15369.3390.364861.781161@slothrop.digicool.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEFNMKAA.tim.one@home.com>

[Jeremy Hylton]
> The uses of PyOS_snprintf() in sysmodule.c actually checks the return
> value.  It always checked the sprintf() returned value and triggered a
> Py_FatalError() if the return value was too big.  I changed it to
> print a warning message that the output was truncated.

The test appeared off on both ends to me, so I fiddled this code.  Please
check the new version against your vision of sanity (my vision:  there's
nothing special about -1:  any return value < 0 is a problem; and there's
also a problem if the return value equals the buffer size, since the return
value is exclusive of the \0 byte).



From martin@v.loewis.de  Sun Dec  2 10:21:30 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Sun, 2 Dec 2001 11:21:30 +0100
Subject: [Python-Dev] -DINET6 in Makefile
In-Reply-To: <3C08D5DD.1F3E8B55@lemburg.com> (mal@lemburg.com)
References: <3C050CAF.2A5CEDC5@lemburg.com> <200111302022.fAUKMO002089@mira.informatik.hu-berlin.de> <3C07F016.AEC20158@lemburg.com> <200112010107.fB117JO03160@mira.informatik.hu-berlin.de> <3C08D5DD.1F3E8B55@lemburg.com>
Message-ID: <200112021021.fB2ALUl01747@mira.informatik.hu-berlin.de>

> > Of course, doing so would be really stupid. Python.h *must* be the
> > first include, or things may break.
> 
> Why should "things break" ? I've doing this for years in lots
> of Python extensions...

Python.h configures the C library, e.g. for multi-threading (by
defining _REENTRANT) or LFS (by defining _FILE_OFFSET_BITS).

If you include system headers before Python.h, you may find that
different headers are differently configured. In turn, compilation may
fail or produce bogus code.

If the module itself compiles correctly, you may still find that it is
inconsistent with the python executable that is going to load it.

> Back on topic: how are we going to get -DINET6 out of the
> Makefile ? 

I just committed a change.

Regards,
Martin


From mal@lemburg.com  Sun Dec  2 10:26:41 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sun, 02 Dec 2001 11:26:41 +0100
Subject: [Python-Dev] Differentiating $(EXT) between build and install
References: <20011201232600.45BC4E2677@oratrix.oratrix.nl>
Message-ID: <3C0A01E1.779B38DA@lemburg.com>

Jack Jansen wrote:
> 
> Folks,
> there's so many bug reports and questions about needing the
> --with-suffix on Mac OS X when you're building on an HFS+ filesystem
> that I'd like to fix this. The warning in configure doesn't cut it
> because it gets lost in all the other messages.
> 
> What I'd like to do is have a separate Makefile variable, BUILDEXT,
> that gives the extension to use for python while it lives in the build
> directory. It would default to EXT, except if on HFS+ when EXT is
> empty.
> 
> But this might change a couple of other things as well, so my question
> is: is this an acceptable change so late in the beta stage?

Wouldn't tweaking the setup.py to introduce a special case
for MacOS be much simpler ? (adding BUILDEXT would need patching
of the Makefile.pre.in process as well as distutils)

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From tim.one@home.com  Sun Dec  2 10:36:24 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 2 Dec 2001 05:36:24 -0500
Subject: [Python-Dev] Subclassing varying length types (What's a   PyStructSequence ?)
In-Reply-To: <200112011353.fB1Drab17745@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEGCMKAA.tim.one@home.com>

[Martin v. Loewis]
> You should consider that malloc overhead is often 16 bytes per
> object. Given that PyUnicodeObject is 24 bytes in 2.2, system malloc
> will allocate 48 bytes per Unicode object on modern architectures.  I
> would think 100% overhead *is* a big argument.
>
> If you relate this to the actual data, it gets worse: A Unicode string
> of length 1 would still require 32 bytes on an allocator that aligns
> to 16.

I think that's unusual -- 8-byte alignment is most common even on 64-bit
boxes.  KSR had to align to 128-byte boundaries, but there's a reason KSR
died <wink -- alas, gross alignment requirements wasn't really it>.

> Therefore, to store 2 bytes of real data, you need 80 bytes of
> memory.
>
> I don't know how much overhead pymalloc adds, though; I believe it is
> significantly less expensive.

Yes, much less.  On a 32-bit box, using the current #define's, and ignoring
"arena" overhead(*), pymalloc uses 32 bytes per 4096 for bookkeeping.  The
remaining 4064 bytes can all be user data, but subject to 8-byte alignment,
and to how many whole chunks of a given size can fit in 4064 bytes.  For the
PyUnicodeObject example, 8-byte alignment is without cost, and for the rest

>>> divmod(4096 - 32, 24)
(169, 8)
>>>

That is, pymalloc can get 169 PyUnicodeObjects out of a 4KB "page", with 32
bytes for bookkeeping, and 8 bytes left over (unused) -- total overhead is
about 1%.


(*) pymalloc gets "arenas" from the system malloc, where an arena is
currently 256KB.  Up to (worst case) 4KB of that is lost to align the start
address to a 4KB boundary, and there's also the comparatively trivial
(compared to 4KB!) overhead from the system malloc.



From loewis@informatik.hu-berlin.de  Sun Dec  2 13:53:33 2001
From: loewis@informatik.hu-berlin.de (Martin von Loewis)
Date: Sun, 2 Dec 2001 14:53:33 +0100 (MET)
Subject: [Python-Dev] Release criteria
Message-ID: <200112021353.fB2DrXZ14135@paros.informatik.hu-berlin.de>

Now that there are only ten days left for the release candidate, I
wonder whether there are any release criteria beyond "builds on
Windows".

In particular, I think that the release candidate should not be
produced until "critical" SF bugs have been resolved. In #420851,
Barry indicates that bugs with a priority higher than six will block a
release. This sounds reasonable.

Looking at the remaining bugs, we see

Priority 8: two reports (assigned to loewis and nobody)

Priority 7: 11 reports (6x akuchling, 3x gvanrossum, 
                        1x jackjansen, 1x fdrake)

Given that the oldest of these reports is from 2001-07-30, I think it
is unlikely that they will be all resolved within ten days. That seems
to suggest one of the following alternatives:

1. Move the deadline for the release candidate,
2. Lower the priority for some of the reports,
3. Move the threshold for release-critical bugs (to, say, 8),
4. Give up the notion of release-critical bugs (which I would regret,
   since I hope "my" prio-8-bug should be fixed; it'll just take
   some more days).

Regards,
Martin



From jeremy@zope.com  Sun Dec  2 17:47:28 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Sun, 2 Dec 2001 12:47:28 -0500 (EST)
Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEFNMKAA.tim.one@home.com>
References: <15369.3390.364861.781161@slothrop.digicool.com>
 <LNBBLJKPBEHFEDALKOLCEEFNMKAA.tim.one@home.com>
Message-ID: <15370.26928.239833.870381@slothrop.digicool.com>

>>>>> "TP" == Tim Peters <tim.one@home.com> writes:

  TP> [Jeremy Hylton]
  >> The uses of PyOS_snprintf() in sysmodule.c actually checks the
  >> return value.  It always checked the sprintf() returned value and
  >> triggered a Py_FatalError() if the return value was too big.  I
  >> changed it to print a warning message that the output was
  >> truncated.

  TP> The test appeared off on both ends to me, so I fiddled this
  TP> code.  Please check the new version against your vision of
  TP> sanity (my vision: there's nothing special about -1: any return
  TP> value < 0 is a problem; and there's also a problem if the return
  TP> value equals the buffer size, since the return value is
  TP> exclusive of the \0 byte).

Definitely correct on the latter.  Sensibly conservative on the
former.  The Linux man pages are silent on negative return values
other than -1.  Thanks for the fixes.

Jeremy



From fdrake@acm.org  Sun Dec  2 20:45:52 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sun, 2 Dec 2001 15:45:52 -0500
Subject: [Python-Dev] Release criteria
In-Reply-To: <200112021353.fB2DrXZ14135@paros.informatik.hu-berlin.de>
References: <200112021353.fB2DrXZ14135@paros.informatik.hu-berlin.de>
Message-ID: <15370.37632.689000.981891@grendel.zope.com>

Martin von Loewis writes:
 > In particular, I think that the release candidate should not be
 > produced until "critical" SF bugs have been resolved. In #420851,
 > Barry indicates that bugs with a priority higher than six will block a
 > release. This sounds reasonable.
 > 
 > Looking at the remaining bugs, we see
 > 
 > Priority 8: two reports (assigned to loewis and nobody)
 > 
 > Priority 7: 11 reports (6x akuchling, 3x gvanrossum, 
 >                         1x jackjansen, 1x fdrake)

  The one assigned to me really must be fixed before release.  There
are a number of people who use the indexes, and they can be the only
effective way to use the affected documents much of the time.

 > Given that the oldest of these reports is from 2001-07-30, I think it
 > is unlikely that they will be all resolved within ten days. That seems
 > to suggest one of the following alternatives:
 > 
 > 1. Move the deadline for the release candidate,
 > 2. Lower the priority for some of the reports,
 > 3. Move the threshold for release-critical bugs (to, say, 8),

  The one assigned to me is at priority 7 because that would block the
release.  Shift the threshold up and I shift the priority of that bug
to match.

 > 4. Give up the notion of release-critical bugs (which I would regret,
 >    since I hope "my" prio-8-bug should be fixed; it'll just take
 >    some more days).

  I agree we should keep the concept.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From tim.one@home.com  Sun Dec  2 20:01:03 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 2 Dec 2001 15:01:03 -0500
Subject: [Python-Dev] Subclassing varying length types (What's a   PyStructSequence ?)
In-Reply-To: <3C08BE98.79946952@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEHDMKAA.tim.one@home.com>

[MAL]
> Perhaps we should start thinking about optimizing at least
> one of the Unicode malloc away in Python 2.3: the Unicode
> object itself can well be kept on a free list with the
> Py_UNICODE buffer freed and set to NULL. Doesn't save any memory
> but would improve the performance.

pymalloc would improve both, so I'd much rather pursue that in 2.3 than yet
another type-specific free list.

> BTW, is the memory burden really such a big argument these days ?
> I can imagine this being an argument on resource restrained
> platforms such as Palms (thanks to Martin, the Plam guys can now
> switch off Unicode completely), but hardly on gigabyte machines
> with access 100s of GBs swap-space :-)

Most of us have machines between those extremes, and the difference between
100MB and 300MB can be make-or-break.  I don't see that any "flexibility" is
gained merely by wasting memory <wink>.

> ...
> Any idea how we could make subclassing these types less
> hackish, then ?

Subclassing seems easy enough to me from the Python level; I don't have time
to revisit C-level subclasssing here (and I don't know that it's hackish
there either, but do think it's in need of docs).



From tim.one@home.com  Sun Dec  2 20:06:06 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 2 Dec 2001 15:06:06 -0500
Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
In-Reply-To: <15370.26928.239833.870381@slothrop.digicool.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEHEMKAA.tim.one@home.com>

[Jeremy]
> Definitely correct on the latter.  Sensibly conservative on the
> former.  The Linux man pages are silent on negative return values
> other than -1.

My draft copy of C99 sez:

    The snprintf function returns the number of characters that would have
    been written had n been sufficiently large, not counting the
    terminating null character, or a negative value if an encoding error
    occurred.

-1 isn't mentioned at all.  What does the final C99 say?  Regardless, MS's
pre-C99 implementation says:

    _snprintf returns the number of bytes stored in buffer, not counting
    the terminating null character. If the number of bytes required to
    store the data exceeds count, then count bytes of data are stored in
    buffer and a negative value is returned.

So -1 sounds Linux-specific.

> Thanks for the fixes.

Thank *you* for the opportunity to be thorough <wink>.



From tim.one@home.com  Sun Dec  2 20:39:23 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 2 Dec 2001 15:39:23 -0500
Subject: [Python-Dev] Release criteria
In-Reply-To: <200112021353.fB2DrXZ14135@paros.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEHGMKAA.tim.one@home.com>

[Martin von Loewis]
> Now that there are only ten days left for the release candidate, I
> wonder whether there are any release criteria beyond "builds on
> Windows".

Sure.

> In particular, I think that the release candidate should not be
> produced until "critical" SF bugs have been resolved. In #420851,
> Barry indicates that bugs with a priority higher than six will block a
> release. This sounds reasonable.

Yes, that's part of The Plan.

> Looking at the remaining bugs, we see
>
> Priority 8: two reports (assigned to loewis and nobody)

The "nobody" bug is sprintf -> PyOS_snprintf conversion; it's not assigned
to anyone in particular because 3 people in PythonLabs have been working on
it.

> Priority 7: 11 reports (6x akuchling, 3x gvanrossum,
>                         1x jackjansen, 1x fdrake)
>
> Given that the oldest of these reports is from 2001-07-30, I think it
> is unlikely that they will be all resolved within ten days.

Me too, although it's the Distutil (akuchling) bugs that seem most at risk.
Note that I just boosted one of the regexp bugs to priority 7 (/F, are you
there?).  Guido returns from paternity leave tomorrow, so PythonLabs will
triple its available manpower then <0.9 wink>.

> That seems to suggest one of the following alternatives:
>
> 1. Move the deadline for the release candidate,

Possible, but at this time I think it's unlikely.

> 2. Lower the priority for some of the reports,

Most likely, for the Distutils and Mac-specific bugs (no, we're not going to
hold up the release just because the "copy" menu command on a Mac-specific
console window isn't enabled).

> 3. Move the threshold for release-critical bugs (to, say, 8),

No.

> 4. Give up the notion of release-critical bugs

No.



From fredrik@pythonware.com  Sun Dec  2 21:04:54 2001
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Sun, 2 Dec 2001 22:04:54 +0100
Subject: [Python-Dev] Release criteria
References: <LNBBLJKPBEHFEDALKOLCMEHGMKAA.tim.one@home.com>
Message-ID: <019f01c17b75$02aee150$ced241d5@hagrid>

Tim Peters wrote:

> Note that I just boosted one of the regexp bugs to priority 7 (/F, are you
> there?).

hmm.  that bug appears to be fixed in the PythonWare repository,
but for some reason, I never checked it in...

I'll investigate.

</F>



From tim.one@home.com  Sun Dec  2 20:59:00 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 2 Dec 2001 15:59:00 -0500
Subject: [Python-Dev] Release criteria
In-Reply-To: <019f01c17b75$02aee150$ced241d5@hagrid>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEHIMKAA.tim.one@home.com>

[Tim]
> Note that I just boosted one of the regexp bugs to priority 7
> (/F, are you there?).

[/F]
> hmm.  that bug appears to be fixed in the PythonWare repository,
> but for some reason, I never checked it in...
>
> I'll investigate.

Thanks!  If you can resolve this before 2.2c1 Eve, I'm sure Guido's new son
will sleep much better <wink>.



From jack@oratrix.nl  Sun Dec  2 21:03:54 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Sun, 02 Dec 2001 22:03:54 +0100
Subject: [Python-Dev] Differentiating $(EXT) between build and install
In-Reply-To: Message by "M.-A. Lemburg" <mal@lemburg.com> ,
 Sun, 02 Dec 2001 11:26:41 +0100 , <3C0A01E1.779B38DA@lemburg.com>
Message-ID: <20011202210359.E1281E2677@oratrix.oratrix.nl>

Recently, "M.-A. Lemburg" <mal@lemburg.com> said:
> > What I'd like to do is have a separate Makefile variable, BUILDEXT,
> > that gives the extension to use for python while it lives in the build
> > directory. It would default to EXT, except if on HFS+ when EXT is
> > empty.
> 
> Wouldn't tweaking the setup.py to introduce a special case
> for MacOS be much simpler ? (adding BUILDEXT would need patching
> of the Makefile.pre.in process as well as distutils)

The problem is only with building the interpreter. Because HFS+ is
case-insensitive creating an executable "python" when there's already
a subdirectory "Python" isn't going to work. So the idea is to use the
BUILDEXT to create "python.exe" in stead, but install that as the
normal "python" in the eventual install location.

I would assume that the extra variable BUILDEXT in Makefile.pre.in
woulnd't bother distutils or anything else.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.cwi.nl/~jack        | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 


From guido@python.org  Sun Dec  2 23:57:41 2001
From: guido@python.org (Guido van Rossum)
Date: Sun, 02 Dec 2001 18:57:41 -0500
Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
In-Reply-To: Your message of "Sun, 02 Dec 2001 15:06:06 EST."
 <LNBBLJKPBEHFEDALKOLCKEHEMKAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCKEHEMKAA.tim.one@home.com>
Message-ID: <200112022357.SAA16201@cj20424-a.reston1.va.home.com>

> -1 isn't mentioned at all.  What does the final C99 say?  Regardless, MS's
> pre-C99 implementation says:
[...]
> So -1 sounds Linux-specific.

Being a Unix old-timer, I cringe every time I see someone test for
errors using "== -1".  Except for anomalies like getpid() (which in
the olden days could return any negative value except -1 as a
legitimate pid), for the caller *any* negative value should be
interpreted as an error, and tested for with "< 0".  Never mind that
most functions return -1.  -1000 is just as much an error.

Unhelpfully,

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Sun Dec  2 23:59:40 2001
From: guido@python.org (Guido van Rossum)
Date: Sun, 02 Dec 2001 18:59:40 -0500
Subject: [Python-Dev] Release criteria
In-Reply-To: Your message of "Sun, 02 Dec 2001 14:53:33 +0100."
 <200112021353.fB2DrXZ14135@paros.informatik.hu-berlin.de>
References: <200112021353.fB2DrXZ14135@paros.informatik.hu-berlin.de>
Message-ID: <200112022359.SAA16213@cj20424-a.reston1.va.home.com>

> Now that there are only ten days left for the release candidate, I
> wonder whether there are any release criteria beyond "builds on
> Windows".

And on Linux. :-)

> In particular, I think that the release candidate should not be
> produced until "critical" SF bugs have been resolved. In #420851,
> Barry indicates that bugs with a priority higher than six will block a
> release. This sounds reasonable.

I still abide by this rule.

> Looking at the remaining bugs, we see
> 
> Priority 8: two reports (assigned to loewis and nobody)

Yours is for BSDI, a tiny minority platform.  You set the priority
yourself, meaning you want to fix it before the release candidate.
But if you don't fix it, I'd rather lower the priority to 6 than hold
up the release.

The other is the sprintf thing -- I believe that's been addressed, at
least to a sufficient degree to warrant lowering the priority.

> Priority 7: 11 reports (6x akuchling, 3x gvanrossum, 
>                         1x jackjansen, 1x fdrake)

I plan to address mine (I'm coming back to work Monday).  Fred already
promised to address his.  Andrew has a whole bunch of distutils and
setup.py bugs assigned.  I'm not sure that these should be allowed to
hold up the release -- there just doesn't seem to be anyone at
PythonLabs who has the expertise to fix them, and frankly, I don't see
these bugs as release showstoppers.  So maybe their priority should be
lowered -- although I'd prefer to keep them at the current priority,
in the hope that when Andrew (or someone else) finds some time, they
know which bugs to address first.  That is, assuming that all the
priorities have been assigned rationally, which I'm not so sure of.

I think Jack desperately wants to fix his bug in time -- that's why he
raised the priority.

> Given that the oldest of these reports is from 2001-07-30, I think it
> is unlikely that they will be all resolved within ten days. That seems
> to suggest one of the following alternatives:
> 
> 1. Move the deadline for the release candidate,

Unlikely, unless we find a real showstopper (none of the above count).

> 2. Lower the priority for some of the reports,

Might do.

> 3. Move the threshold for release-critical bugs (to, say, 8),

No way.

> 4. Give up the notion of release-critical bugs (which I would regret,
>    since I hope "my" prio-8-bug should be fixed; it'll just take
>    some more days).

Nah.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@zope.com  Mon Dec  3 03:59:01 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Sun, 2 Dec 2001 22:59:01 -0500
Subject: [Python-Dev] Release criteria
References: <200112021353.fB2DrXZ14135@paros.informatik.hu-berlin.de>
 <200112022359.SAA16213@cj20424-a.reston1.va.home.com>
Message-ID: <15370.63621.213117.237336@anthem.wooz.org>

Don't forget the patches too.  Although I've only just skimmed them,
it looks like all the patches are at priority 5, most likely because
they haven't been triaged.

I'll try to do a quick scan through the unassigned patches.

-Barry


From mal@lemburg.com  Mon Dec  3 09:09:38 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 03 Dec 2001 10:09:38 +0100
Subject: [Python-Dev] Subclassing varying length types (What's a
 PyStructSequence ?)
References: <LNBBLJKPBEHFEDALKOLCGEHDMKAA.tim.one@home.com>
Message-ID: <3C0B4152.3739F4EA@lemburg.com>

Tim Peters wrote:
> 
> [MAL]
> > Perhaps we should start thinking about optimizing at least
> > one of the Unicode malloc away in Python 2.3: the Unicode
> > object itself can well be kept on a free list with the
> > Py_UNICODE buffer freed and set to NULL. Doesn't save any memory
> > but would improve the performance.
> 
> pymalloc would improve both, so I'd much rather pursue that in 2.3 than yet
> another type-specific free list.

Have you tried disabling all free list and using pymalloc
instead ? If this pays off, I agree, we should get rid off
all of them.

> > BTW, is the memory burden really such a big argument these days ?
> > I can imagine this being an argument on resource restrained
> > platforms such as Palms (thanks to Martin, the Plam guys can now
> > switch off Unicode completely), but hardly on gigabyte machines
> > with access 100s of GBs swap-space :-)
> 
> Most of us have machines between those extremes, and the difference between
> 100MB and 300MB can be make-or-break.  I don't see that any "flexibility" is
> gained merely by wasting memory <wink>.

I would consider moving from 8-bit strings to Unicode an
improvement in flexibility. It also results in better algroithms
(== simpler, less error-prone, etc. in this case).

As I said, it's a tradeoff flexibility vs. memory consumption.
Whether it pays off depends on your application environment. It
certainly does for companies like Micron and pays off stock-wise
for a lot of people... uhm, getting off-topic here :-)

> > ...
> > Any idea how we could make subclassing these types less
> > hackish, then ?
> 
> Subclassing seems easy enough to me from the Python level; I don't have time
> to revisit C-level subclasssing here (and I don't know that it's hackish
> there either, but do think it's in need of docs).

It is beautifully easy for non-varying-length types. Unfortunately,
it happens that some of the basic types which would be attractive
for subclassing are varying length types (such as string and
tuples).

In my case, I'm looking for away to subclass strings, but I haven't
yet found an elegant solution to the problem of adding extra
data to the instances.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/



From mal@lemburg.com  Mon Dec  3 09:25:47 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 03 Dec 2001 10:25:47 +0100
Subject: [Python-Dev] -DINET6 in Makefile
References: <3C050CAF.2A5CEDC5@lemburg.com> <200111302022.fAUKMO002089@mira.informatik.hu-berlin.de> <3C07F016.AEC20158@lemburg.com> <200112010107.fB117JO03160@mira.informatik.hu-berlin.de> <3C08D5DD.1F3E8B55@lemburg.com> <200112021021.fB2ALUl01747@mira.informatik.hu-berlin.de>
Message-ID: <3C0B451B.18D38186@lemburg.com>

"Martin v. Loewis" wrote:
> 
> > > Of course, doing so would be really stupid. Python.h *must* be the
> > > first include, or things may break.
> >
> > Why should "things break" ? I've doing this for years in lots
> > of Python extensions...
> 
> Python.h configures the C library, e.g. for multi-threading (by
> defining _REENTRANT) or LFS (by defining _FILE_OFFSET_BITS).
> 
> If you include system headers before Python.h, you may find that
> different headers are differently configured. In turn, compilation may
> fail or produce bogus code.
> 
> If the module itself compiles correctly, you may still find that it is
> inconsistent with the python executable that is going to load it.

Sure, but the wrapped lib's headers files will expect the same 
logic (they want to configure the C lib too; and for pretty much the
same reasons Python does).

In the end I think that you can't really say whether things break
or not due to the order of the #includes (lib header or Python.h
first) because both can make assumptions which may fail on way or
another. I'd close this with "Works for me" :-)

> > Back on topic: how are we going to get -DINET6 out of the
> > Makefile ?
> 
> I just committed a change.

Saw that checkin. Thanks for fixing this.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From mal@lemburg.com  Mon Dec  3 09:30:04 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 03 Dec 2001 10:30:04 +0100
Subject: [Python-Dev] Differentiating $(EXT) between build and install
References: <20011202210359.E1281E2677@oratrix.oratrix.nl>
Message-ID: <3C0B461C.FE51188F@lemburg.com>

Jack Jansen wrote:
> 
> Recently, "M.-A. Lemburg" <mal@lemburg.com> said:
> > > What I'd like to do is have a separate Makefile variable, BUILDEXT,
> > > that gives the extension to use for python while it lives in the build
> > > directory. It would default to EXT, except if on HFS+ when EXT is
> > > empty.
> >
> > Wouldn't tweaking the setup.py to introduce a special case
> > for MacOS be much simpler ? (adding BUILDEXT would need patching
> > of the Makefile.pre.in process as well as distutils)
> 
> The problem is only with building the interpreter. Because HFS+ is
> case-insensitive creating an executable "python" when there's already
> a subdirectory "Python" isn't going to work. So the idea is to use the
> BUILDEXT to create "python.exe" in stead, but install that as the
> normal "python" in the eventual install location.
> 
> I would assume that the extra variable BUILDEXT in Makefile.pre.in
> woulnd't bother distutils or anything else.

Ah, now I understand (I thought you were talking about compiling
the extensions). No objections then for the short term... however,
for the long term I guess we'll need some better solution like
e.g. renaming Python/ to Interpreter/.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From martin@v.loewis.de  Mon Dec  3 18:02:51 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Mon, 3 Dec 2001 19:02:51 +0100
Subject: [Python-Dev] Differentiating $(EXT) between build and install
In-Reply-To: <3C0B461C.FE51188F@lemburg.com> (mal@lemburg.com)
References: <20011202210359.E1281E2677@oratrix.oratrix.nl> <3C0B461C.FE51188F@lemburg.com>
Message-ID: <200112031802.fB3I2po01345@mira.informatik.hu-berlin.de>

> Ah, now I understand (I thought you were talking about compiling
> the extensions). No objections then for the short term... however,
> for the long term I guess we'll need some better solution like
> e.g. renaming Python/ to Interpreter/.

Or rename "python" to "parrot"?

Regards,
Martin


From fdrake@acm.org  Mon Dec  3 21:25:48 2001
From: fdrake@acm.org (Fred L. Drake)
Date: Mon,  3 Dec 2001 16:25:48 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20011203212548.5590C28698@cj42289-a.reston1.va.home.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

Added documentation for the ScrolledText module.


From skip@pobox.com (Skip Montanaro)  Tue Dec  4 00:44:03 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Mon, 3 Dec 2001 18:44:03 -0600
Subject: [Python-Dev] Going off-list for awhile
Message-ID: <15372.7251.365887.715297@beluga.mojam.com>

Folks,

I've set my various python-related subscriptions to nomail for the time
being.  Both of our web servers (www.musi-cal.com and www.mojam.com) were
recently broken into and then used as launching points for attacks on other
machines.  They are both offline (probably for good).  I am scrambling to
try and get a new server set up and secured.  Hopefully I'll be back in
business in a few days.

-- 
Skip Montanaro (skip@pobox.com - http://www.mojam.com/)


From neal@metaslash.com  Tue Dec  4 02:18:06 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Mon, 03 Dec 2001 21:18:06 -0500
Subject: [Python-Dev] pychecker & python 2.2
Message-ID: <3C0C325E.BD592F56@metaslash.com>

Guido had mentioned awhile ago about possibly adding pychecker to Tools.
If people want it in, what needs to be done?

There are 2 differences between the last released version of pychecker
and the current CVS.  1 is a bug fix for inconsistent return types and 
the other is a new warning for local variables which shadow globals.

I will run the current pychecker on the std library soon and report
any problems.

I have been running purify and all of the "major" problems have bugs
reports.  These "major" problems are memory leaks, the biggest is
about 5k from test_parser which leaks when doing recursion in build_node_children() parsermodule.c:728.

Neal


From neal@metaslash.com  Tue Dec  4 02:42:14 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Mon, 03 Dec 2001 21:42:14 -0500
Subject: [Python-Dev] pychecker on 2.2 current
Message-ID: <3C0C3806.B7325138@metaslash.com>

Only found one potential problem:

mimetypes.py:405: No global (sys) found

This is inside usage() which presumably should only be used internally.  
sys is imported if __name__ == '__main__'.
So the problem can only be triggered if usage() is called by external source.

Possible fixes are:
	sys could be locally imported in usage()
	sys could be imported at the module scope
	usage could be made _usage() to denote it's private

Neal


From ping@lfw.org  Tue Dec  4 12:54:25 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Tue, 4 Dec 2001 04:54:25 -0800 (PST)
Subject: [Python-Dev] __getattribute__ should require a string?
Message-ID: <Pine.LNX.4.10.10112040445190.21231-100000@server1.lfw.org>

In the current Python, __getattribute__ can be called directly
with a non-string, causing much weirdness:

    >>> print ''.__getattribute__(1)
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    AttributeError: 'str' object has no attribute ' =E4=B7*'
    >>>=20

To avoid this, i've added a simple check to Objects/object.c:

    diff -c -r2.161 object.c
    *** object.c    2001/11/04 07:29:31     2.161
    --- object.c    2001/12/04 12:46:13
    ***************
    *** 1210,1215 ****
    --- 1210,1221 ----
            descrgetfunc f;
            PyObject **dictptr;
     =20
    +       if (!PyString_Check(name)) {
    +               PyErr_SetString(PyExc_TypeError,
    +                               "attribute name must be string");
    +               return NULL;
    +       }
    +=20
            if (tp->tp_dict =3D=3D NULL) {
                    if (PyType_Ready(tp) < 0)
                            return NULL;

This produces the better behaviour:

    >>> print ''.__getattribute__(1)
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    TypeError: attribute name must be string
    >>>=20

I was about to check in this simple fix -- but then i discovered that
 it is currently possible to assign attributes with non-string keys:

    >>> def f(): pass
    ...=20
    >>> f.__getattribute__(1)
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    AttributeError: 'function' object has no attribute ' =E4=B7*'
    >>> f.__setattr__(1, 1)
    >>> f.__getattribute__(1)
    1
    >>>=20

Adding the above check prevents this usage.

Is there any reason why people should be allowed to assign and
 retrieve attributes with non-string names?


-- ?!ng



From guido@python.org  Tue Dec  4 13:31:32 2001
From: guido@python.org (Guido van Rossum)
Date: Tue, 04 Dec 2001 08:31:32 -0500
Subject: [Python-Dev] __getattribute__ should require a string?
In-Reply-To: Your message of "Tue, 04 Dec 2001 04:54:25 PST."
 <Pine.LNX.4.10.10112040445190.21231-100000@server1.lfw.org>
References: <Pine.LNX.4.10.10112040445190.21231-100000@server1.lfw.org>
Message-ID: <200112041331.IAA06968@cj20424-a.reston1.va.home.com>

> Is there any reason why people should be allowed to assign and
>  retrieve attributes with non-string names?

Not really, but why should we actively try to prevent it when it's
easy to do self.__dict__[1] = 'a'?  This means you can't rely on all
attribute names being strings, and I'm not going to prevent *that*.

The proper fix IMO for the bug you found is to be more careful when
formatting the error message.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Tue Dec  4 15:32:27 2001
From: guido@python.org (Guido van Rossum)
Date: Tue, 04 Dec 2001 10:32:27 -0500
Subject: [Python-Dev] __getattribute__ should require a string?
In-Reply-To: Your message of "Tue, 04 Dec 2001 08:31:32 EST."
 <200112041331.IAA06968@cj20424-a.reston1.va.home.com>
References: <Pine.LNX.4.10.10112040445190.21231-100000@server1.lfw.org>
 <200112041331.IAA06968@cj20424-a.reston1.va.home.com>
Message-ID: <200112041532.fB4FWR011720@odiug.zope.com>

> > Is there any reason why people should be allowed to assign and
> >  retrieve attributes with non-string names?
> 
> Not really, but why should we actively try to prevent it when it's
> easy to do self.__dict__[1] = 'a'?  This means you can't rely on all
> attribute names being strings, and I'm not going to prevent *that*.
> 
> The proper fix IMO for the bug you found is to be more careful when
> formatting the error message.

I take it back.  All the other functions in object.c require the
getattr/setattr argument to be a string or Unicode object.  I'll fix
object.__getattribute__ and object.__setattr__ (which are really
PyObject_Generic{Get,Set}Attr) to do the same.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From arigo@ulb.ac.be  Tue Dec  4 16:26:34 2001
From: arigo@ulb.ac.be (Armin Rigo)
Date: Tue, 4 Dec 2001 17:26:34 +0100 (MET)
Subject: [Python-Dev] Psyco to SourceForge
Message-ID: <Pine.OSF.3.96.1011204171506.18805B-100000@aster2.ulb.ac.be>

Hello everybody,

I have opened a "Psyco" project at SourceForge:

  http://sourceforge.net/projects/psyco/

I suggest that people interested in Psyco post a welcome and start
discussion in the mailing list that I am currently opening there. This
will allow me to stop (mis)using python-dev@python.org for announces.

The source code I have uploaded at SourceForge (and in a .tgz archive at
the Psyco homepage) is version 0.3.4, which is an important rewrite over
the previous version. I have tried to address the two 'bad design' issues
I mentionned in ISSUES.txt:

* pycompiler.c-and-friends are much clearer now, thanks to a clean
interface in vcompiler.h and processor.h to all the gory details;

* the structure of the Python-dependent part Psyco now very closely
follows that of the Python interpreter. Anyone used to the Objects/xxx.c
files of Python should feel at home with Psyco's objects/pxxx.c files. In
fact, they are just a rewrite of the Python interpreter into some "meta-C"
code useable by the specializer of Psyco. If you gave up while trying to
read the code of 0.3.3 please have another look now :-)


Thanks,

Armin Rigo.



From fdrake@acm.org  Tue Dec  4 17:03:32 2001
From: fdrake@acm.org (Fred L. Drake)
Date: Tue,  4 Dec 2001 12:03:32 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20011204170332.0B76A28698@beowolf.digicool.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

The indexes are now working for the Python/C API manual and the Reference
Manual.



From thomas.heller@ion-tof.com  Tue Dec  4 19:35:19 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Tue, 4 Dec 2001 20:35:19 +0100
Subject: [Python-Dev] nested scopes confusion
Message-ID: <083a01c17cfa$e13f6120$e000a8c0@thomasnotebook>

I would have expected the following code to print
1, 2, ..., 9. Instead it prints 25, 25, 25, .. 25.

def functions():
    result = []
    for i in range(10):
        def mth(*args): return i
        result.append(mth)
    i = 25
    return result

for mth in functions():
    print mth()

Reading PEP227, I can (barely) understand why it behaves this way.

How do I achieve the desired effect? Note that the default argument
trick (def mth(i=i): ...) does not work because *args is present.

Thomas



From thomas.heller@ion-tof.com  Tue Dec  4 19:47:59 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Tue, 4 Dec 2001 20:47:59 +0100
Subject: [Python-Dev] nested scopes confusion
References: <083a01c17cfa$e13f6120$e000a8c0@thomasnotebook>
Message-ID: <08dc01c17cfc$a5ef22c0$e000a8c0@thomasnotebook>

From: "Thomas Heller" <thomas.heller@ion-tof.com>
> How do I achieve the desired effect? Note that the default argument
> trick (def mth(i=i): ...) does not work because *args is present.
Answering my own question (sorry), this works:


def functions():
    result = []
    for i in range(10):
        def make_function(index):
            def mth():
                return index
            return mth
        mth = make_function(i)
        result.append(mth)
    i = 25
    return result

for mth in functions():
    print mth()

But how will I understand this 3 months from now?

Thomas



From guido@python.org  Tue Dec  4 19:47:23 2001
From: guido@python.org (Guido van Rossum)
Date: Tue, 04 Dec 2001 14:47:23 -0500
Subject: [Python-Dev] nested scopes confusion
In-Reply-To: Your message of "Tue, 04 Dec 2001 20:35:19 +0100."
 <083a01c17cfa$e13f6120$e000a8c0@thomasnotebook>
References: <083a01c17cfa$e13f6120$e000a8c0@thomasnotebook>
Message-ID: <200112041947.fB4JlWR24198@odiug.zope.com>

> I would have expected the following code to print
> 1, 2, ..., 9. Instead it prints 25, 25, 25, .. 25.
> 
> def functions():
>     result = []
>     for i in range(10):
>         def mth(*args): return i
>         result.append(mth)
>     i = 25
>     return result
> 
> for mth in functions():
>     print mth()
> 
> Reading PEP227, I can (barely) understand why it behaves this way.

Yes, you're out of luck.  Like in all decent languages with nested
scope, Python binds to the variable, not to its value at the time the
inner function is defined.

> How do I achieve the desired effect? Note that the default argument
> trick (def mth(i=i): ...) does not work because *args is present.

Use a class with a __call__ method:

    class mth:
	def __init__(self, i):
	    self.i = i
	def __call__(self, *args):
	    return self.i

and use:

    result.append(mth(i))

--Guido van Rossum (home page: http://www.python.org/~guido/)


From thomas.heller@ion-tof.com  Tue Dec  4 19:58:59 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Tue, 4 Dec 2001 20:58:59 +0100
Subject: [Python-Dev] nested scopes confusion
References: <083a01c17cfa$e13f6120$e000a8c0@thomasnotebook>  <200112041947.fB4JlWR24198@odiug.zope.com>
Message-ID: <090401c17cfe$2ef12ea0$e000a8c0@thomasnotebook>

> Yes, you're out of luck.  Like in all decent languages with nested
> scope, Python binds to the variable, not to its value at the time the
> inner function is defined.

So nested scopes are not really an alternative (at least in this case)
to the 'lambda self, i=i: i' trick.

> 
> > How do I achieve the desired effect? Note that the default argument
> > trick (def mth(i=i): ...) does not work because *args is present.
> 
> Use a class with a __call__ method:

That's what I was trying to avoid, because it is too slow.

Thanks,

Thomas



From jeremy@zope.com  Tue Dec  4 19:57:23 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Tue, 4 Dec 2001 14:57:23 -0500 (EST)
Subject: [Python-Dev] nested scopes confusion
In-Reply-To: <083a01c17cfa$e13f6120$e000a8c0@thomasnotebook>
References: <083a01c17cfa$e13f6120$e000a8c0@thomasnotebook>
Message-ID: <15373.10915.984130.453517@slothrop.digicool.com>

> def functions():
>     result = []
>     for i in range(10):
>         def mth(*args): return i
>         result.append(mth)
>     i = 25
>     return result
> 
> for mth in functions():
>     print mth()

>>>>> "TH" == Thomas Heller <thomas.heller@ion-tof.com> writes:

  TH> Reading PEP227, I can (barely) understand why it behaves this
  TH> way.

It behaves this way because the reference to i in mth() uses the
binding for i introduced in functions().  The important point here is
that the binding is used, but i can be bound to different values at
different times.  The function mth() sees the value that i is bound to
when it is called.  In your example, mth() isn't called until after
the loop finishes executing.

  TH> How do I achieve the desired effect? Note that the default
  TH> argument trick (def mth(i=i): ...) does not work because *args
  TH> is present.

All sorts of ways, I imagine.  You could use a class:

class MthClass:

      def __init__(self, val):
	  self.val = val

      def mth(self, *args):
	  return self.val

This version is probably immediately obvious to the reader.

You could add an extra layer of functions, but this looks like a
pretty obscure use of nested scopes to me.

>>> def functions():
... 	result = []
...	for i in range(10):
...		def wrap(i):
...			def mth(*args):
...				return i
...			return mth
...		result.append(wrap(i))
...	i = 25
...	return result
... 
>>> l = functions()
>>> l[0]()
0
>>> l[1]()
1

Jeremy



From guido@python.org  Tue Dec  4 19:59:39 2001
From: guido@python.org (Guido van Rossum)
Date: Tue, 04 Dec 2001 14:59:39 -0500
Subject: [Python-Dev] nested scopes confusion
In-Reply-To: Your message of "Tue, 04 Dec 2001 20:47:59 +0100."
 <08dc01c17cfc$a5ef22c0$e000a8c0@thomasnotebook>
References: <083a01c17cfa$e13f6120$e000a8c0@thomasnotebook>
 <08dc01c17cfc$a5ef22c0$e000a8c0@thomasnotebook>
Message-ID: <200112041959.fB4Jxdn24268@odiug.zope.com>

> Answering my own question (sorry), this works:
> 
> 
> def functions():
>     result = []
>     for i in range(10):
>         def make_function(index):
>             def mth():
>                 return index
>             return mth
>         mth = make_function(i)
>         result.append(mth)
>     i = 25
>     return result
> 
> for mth in functions():
>     print mth()
> 
> But how will I understand this 3 months from now?

What's the problem with understanding this code?  It looks totally
clear to me.  You could add a few comments or use more fancyful
function names.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@zope.com  Tue Dec  4 20:06:53 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Tue, 4 Dec 2001 15:06:53 -0500 (EST)
Subject: [Python-Dev] nested scopes confusion
In-Reply-To: <090401c17cfe$2ef12ea0$e000a8c0@thomasnotebook>
References: <083a01c17cfa$e13f6120$e000a8c0@thomasnotebook>
 <200112041947.fB4JlWR24198@odiug.zope.com>
 <090401c17cfe$2ef12ea0$e000a8c0@thomasnotebook>
Message-ID: <15373.11485.440768.572641@slothrop.digicool.com>

>>>>> "TH" == Thomas Heller <thomas.heller@ion-tof.com> writes:

  >> > How do I achieve the desired effect? Note that the default
  >> > argument trick (def mth(i=i): ...) does not work because *args
  >> > is present.
  >>
  >> Use a class with a __call__ method:

  TH> That's what I was trying to avoid, because it is too slow.

Calling a regular method on an instance is about 5x faster than
calling an instance that defines an __call__().  

But the performance characteristics of class vs. function is really
the bottleneck of your application?  Are you sure that nested fuctions
are really cheaper?

Jeremy



From thomas.heller@ion-tof.com  Tue Dec  4 20:28:17 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Tue, 4 Dec 2001 21:28:17 +0100
Subject: [Python-Dev] nested scopes confusion
References: <083a01c17cfa$e13f6120$e000a8c0@thomasnotebook><200112041947.fB4JlWR24198@odiug.zope.com><090401c17cfe$2ef12ea0$e000a8c0@thomasnotebook> <15373.11485.440768.572641@slothrop.digicool.com>
Message-ID: <097901c17d02$46f6a3a0$e000a8c0@thomasnotebook>

> But the performance characteristics of class vs. function is really
> the bottleneck of your application?  Are you sure that nested fuctions
> are really cheaper?
Is seems do.

Here's a simple test:

def Method(index):
    return lambda self, *args: index

class A:
    def test(self, *args):
        return 5

class C:
    test = Method(42)

a = A()
c = C()

Calling a.test() takes 3.3 microseconds, and c.test() takes 3.16 microseconds.

Thomas



From thomas.heller@ion-tof.com  Tue Dec  4 20:37:41 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Tue, 4 Dec 2001 21:37:41 +0100
Subject: [Python-Dev] nested scopes confusion
References: <083a01c17cfa$e13f6120$e000a8c0@thomasnotebook><200112041947.fB4JlWR24198@odiug.zope.com><090401c17cfe$2ef12ea0$e000a8c0@thomasnotebook> <15373.11485.440768.572641@slothrop.digicool.com> <097901c17d02$46f6a3a0$e000a8c0@thomasnotebook>
Message-ID: <099f01c17d03$97d83f30$e000a8c0@thomasnotebook>

From: "Thomas Heller" <thomas.heller@ion-tof.com>
> > But the performance characteristics of class vs. function is really
> > the bottleneck of your application?  Are you sure that nested fuctions
> > are really cheaper?
[...]
What were I talking about ;-) ?

These are functions using nested scopes, not nested functions.

Sorry, Thomas



From jeremy@zope.com  Tue Dec  4 20:41:29 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Tue, 4 Dec 2001 15:41:29 -0500 (EST)
Subject: [Python-Dev] nested scopes confusion
In-Reply-To: <097901c17d02$46f6a3a0$e000a8c0@thomasnotebook>
References: <083a01c17cfa$e13f6120$e000a8c0@thomasnotebook>
 <200112041947.fB4JlWR24198@odiug.zope.com>
 <090401c17cfe$2ef12ea0$e000a8c0@thomasnotebook>
 <15373.11485.440768.572641@slothrop.digicool.com>
 <097901c17d02$46f6a3a0$e000a8c0@thomasnotebook>
Message-ID: <15373.13561.525007.203007@slothrop.digicool.com>

>>>>> "TH" == Thomas Heller <thomas.heller@ion-tof.com> writes:

  >> But the performance characteristics of class vs. function is
  >> really the bottleneck of your application?  Are you sure that
  >> nested fuctions are really cheaper?

  TH> Calling a.test() takes 3.3 microseconds, and c.test() takes 3.16
  TH> microseconds.

So I 5% difference, based on a single set of measurements with one
compiler/platform/etc., is enough to go out of your way to avoid
classes?  That seems conceivable for some very small set of
applications, but not in general.

Jeremy




From thomas.heller@ion-tof.com  Tue Dec  4 21:04:31 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Tue, 4 Dec 2001 22:04:31 +0100
Subject: [Python-Dev] nested scopes confusion
References: <083a01c17cfa$e13f6120$e000a8c0@thomasnotebook><200112041947.fB4JlWR24198@odiug.zope.com><090401c17cfe$2ef12ea0$e000a8c0@thomasnotebook><15373.11485.440768.572641@slothrop.digicool.com><097901c17d02$46f6a3a0$e000a8c0@thomasnotebook> <15373.13561.525007.203007@slothrop.digicool.com>
Message-ID: <0a2901c17d07$570bf8d0$e000a8c0@thomasnotebook>

From: "Jeremy Hylton" <jeremy@zope.com>
> >>>>> "TH" == Thomas Heller <thomas.heller@ion-tof.com> writes:
> 
>   >> But the performance characteristics of class vs. function is
>   >> really the bottleneck of your application?  Are you sure that
>   >> nested fuctions are really cheaper?
> 
>   TH> Calling a.test() takes 3.3 microseconds, and c.test() takes 3.16
>   TH> microseconds.
> 
> So I 5% difference, based on a single set of measurements with one
> compiler/platform/etc., is enough to go out of your way to avoid
> classes?
No. The test only shows (or suggests for this particular compiler/platform)
that calling methods (which are using nested scopes) has the same
speed than normal methods and not a factor of 5 or whatever
slowdown of callable classes.

>   That seems conceivable for some very small set of
> applications, but not in general.
True.

Thomas



From martin@v.loewis.de  Tue Dec  4 20:17:46 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Tue, 4 Dec 2001 21:17:46 +0100
Subject: [Python-Dev] nested scopes confusion
In-Reply-To: <090401c17cfe$2ef12ea0$e000a8c0@thomasnotebook>
 (thomas.heller@ion-tof.com)
References: <083a01c17cfa$e13f6120$e000a8c0@thomasnotebook>  <200112041947.fB4JlWR24198@odiug.zope.com> <090401c17cfe$2ef12ea0$e000a8c0@thomasnotebook>
Message-ID: <200112042017.fB4KHkU01849@mira.informatik.hu-berlin.de>

> So nested scopes are not really an alternative (at least in this case)
> to the 'lambda self, i=i: i' trick.

In this case, they aren't. In many other cases (e.g. when the variable
from the outer scope is self), nobody will assign to this variable, in
which case this does exactly the same thing as the i=i trick.

Regards,
Martin


From nas@python.ca  Tue Dec  4 22:36:06 2001
From: nas@python.ca (Neil Schemenauer)
Date: Tue, 4 Dec 2001 14:36:06 -0800
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure,1.274,1.275 configure.in,1.283,1.284
In-Reply-To: <E16BMbV-0007gP-00@usw-pr-cvs1.sourceforge.net>; from fdrake@users.sourceforge.net on Tue, Dec 04, 2001 at 12:55:49PM -0800
References: <E16BMbV-0007gP-00@usw-pr-cvs1.sourceforge.net>
Message-ID: <20011204143606.A19738@glacier.arctrix.com>

> + if test "$Py_DEBUG" = 'true'; then
> +   :
> + else
> +   OPT="-DNDEBUG $OPT"
> + fi
> + 

Shouldn't this go into DEFS?

  Neil


From fdrake@acm.org  Tue Dec  4 23:26:27 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 4 Dec 2001 18:26:27 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure,1.274,1.275 configure.in,1.283,1.284
In-Reply-To: <20011204143606.A19738@glacier.arctrix.com>
References: <E16BMbV-0007gP-00@usw-pr-cvs1.sourceforge.net>
 <20011204143606.A19738@glacier.arctrix.com>
Message-ID: <15373.23459.612323.606576@grendel.zope.com>

Neil Schemenauer writes:
 > > + if test "$Py_DEBUG" = 'true'; then
 > > +   :
 > > + else
 > > +   OPT="-DNDEBUG $OPT"
 > > + fi
 > > + 
 > 
 > Shouldn't this go into DEFS?

  I didn't see DEFS getting set anywhere else, and other places in
configure.in add -D options to OPT, CC, or CCSHARED.  I've no personal
preference though; if someone with more autokarma than I changes it, I
won't scream.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From guido@python.org  Tue Dec  4 23:12:51 2001
From: guido@python.org (Guido van Rossum)
Date: Tue, 04 Dec 2001 18:12:51 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src configure,1.274,1.275 configure.in,1.283,1.284
In-Reply-To: Your message of "Tue, 04 Dec 2001 18:26:27 EST."
 <15373.23459.612323.606576@grendel.zope.com>
References: <E16BMbV-0007gP-00@usw-pr-cvs1.sourceforge.net> <20011204143606.A19738@glacier.arctrix.com>
 <15373.23459.612323.606576@grendel.zope.com>
Message-ID: <200112042312.SAA07545@cj20424-a.reston1.va.home.com>

> Neil Schemenauer writes:
>  > > + if test "$Py_DEBUG" = 'true'; then
>  > > +   :
>  > > + else
>  > > +   OPT="-DNDEBUG $OPT"
>  > > + fi
>  > > + 
>  > 
>  > Shouldn't this go into DEFS?
> 
>   I didn't see DEFS getting set anywhere else, and other places in
> configure.in add -D options to OPT, CC, or CCSHARED.  I've no personal
> preference though; if someone with more autokarma than I changes it, I
> won't scream.  ;-)
> 
>   -Fred

I think DEFS gets put in pyconfig.h, which gets included by Python.h,
defeating the whole purpose.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Wed Dec  5 05:49:19 2001
From: fdrake@acm.org (Fred L. Drake)
Date: Wed,  5 Dec 2001 00:49:19 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20011205054919.A1E7B28698@beowolf.digicool.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

A wide variety of minor additions and clarifications.


From naliniprakash@earthlink.net  Wed Dec  5 06:09:43 2001
From: naliniprakash@earthlink.net (Nalini Prakash)
Date: Tue, 4 Dec 2001 22:09:43 -0800
Subject: [Python-Dev] Building PyXPCOM from Mozilla source/ImportError: cannot import name _xpcom
Message-ID: <000801c17d53$6fd608a0$3953afd8@popcorn>

This is a multi-part message in MIME format.

------=_NextPart_000_0005_01C17D10.60B38110
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Apologize if this is not the right group for this question. I am trying =
to build PyXPCOM from Mozilla 0.9.6 source tree on Win 2000. I have =
managed to do make successfully and I can import xpcom. When I import =
xpcom._xpcom, I get "ImportError: cannot import name _xpcom" and the =
path includes Mozilla bin directory. The configure.html file hints at =
something being wrong with _xpcom(_d).pyd.=20

How do I fix this problem?

TIA, Praki

------=_NextPart_000_0005_01C17D10.60B38110
Content-Type: text/html;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content=3D"text/html; charset=3Diso-8859-1" =
http-equiv=3DContent-Type>
<META content=3D"MSHTML 5.00.2920.0" name=3DGENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY bgColor=3D#ffffff>
<DIV><FONT face=3DArial size=3D2>Apologize if this is not the right =
group for this=20
question. I am trying to build PyXPCOM from Mozilla 0.9.6 source tree on =
Win=20
2000. I have managed to do make successfully and I can import xpcom. =
When I=20
import xpcom._xpcom, I get "ImportError: cannot import name _xpcom" and =
the path=20
includes Mozilla bin directory. The configure.html file hints at =
something being=20
wrong with <EM>_xpcom(_d).pyd. </EM></FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2>How do I fix this problem?</FONT></DIV>
<DIV>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2>TIA, Praki</FONT></DIV></BODY></HTML>

------=_NextPart_000_0005_01C17D10.60B38110--



From MarkH@ActiveState.com  Wed Dec  5 06:38:38 2001
From: MarkH@ActiveState.com (Mark Hammond)
Date: Wed, 5 Dec 2001 17:38:38 +1100
Subject: [Python-Dev] Building PyXPCOM from Mozilla source/ImportError: cannot import name _xpcom
In-Reply-To: <000801c17d53$6fd608a0$3953afd8@popcorn>
Message-ID: <LCEPIIGDJPKCOIHOBJEPOEJJEPAA.MarkH@ActiveState.com>

The correct forum for this is the Python XPCOM mailing list - details at
http://listserv.ActiveState.com/mailman/listinfo/pyxpcom

Please direct all followups there.

Your PYTHONPATH needs to be set so that the main "xpcom" Python package is
set on it.  This directory should have an "_xpcom.pyd" (or xpcom_d.pyd for
debug builds).  I am guessing that this .pyd simply does not exist in that
directory.

If you receive another message that you omitted mentioning, regarding a
required .dll file not being found, then the problem is likely that the PATH
is not set correctly - but it appears from your message that it is OK.

Hope this helps,

Mark.
-----Original Message-----
From: python-dev-admin@python.org [mailto:python-dev-admin@python.org]On
Behalf Of Nalini Prakash
Sent: Wednesday, 5 December 2001 5:10 PM
To: python-dev@python.org
Subject: [Python-Dev] Building PyXPCOM from Mozilla source/ImportError:
cannot import name _xpcom


Apologize if this is not the right group for this question. I am trying to
build PyXPCOM from Mozilla 0.9.6 source tree on Win 2000. I have managed to
do make successfully and I can import xpcom. When I import xpcom._xpcom, I
get "ImportError: cannot import name _xpcom" and the path includes Mozilla
bin directory. The configure.html file hints at something being wrong with
_xpcom(_d).pyd.

How do I fix this problem?

TIA, Praki



From anthony@interlink.com.au  Wed Dec  5 07:05:29 2001
From: anthony@interlink.com.au (Anthony Baxter)
Date: Wed, 05 Dec 2001 18:05:29 +1100
Subject: [Python-Dev] python2.1.2 plans, plus the curse of PEP-006.
Message-ID: <200112050705.fB575UN09520@mbuna.arbhome.com.au>

Ok, after a bit of an unfortunate break I'm back into the backporting.
My current plan is to have a release candidate for 2.1.2 due a couple of
days before Christmas, and then try to get it tested as much as possible
before a release either in the week between christmas/new year (not optimal)
or shortly after. I'd hoped to get this stuff done much sooner, but 
unfortunately a combination of various personal matters has knocked me out
for a couple of weeks.

(I'm not sure if it's just me, but it's possible the reason no-one's 
done two bugfix releases in a row is that PEP-006 is accursed, and
brings down the wrath of gods on anyone foolish enough to get involved.
I'm hoping that the last few weeks horror is over and done now...)

Anthony

--
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.



From mal@lemburg.com  Wed Dec  5 08:28:18 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 05 Dec 2001 09:28:18 +0100
Subject: [Python-Dev] nested scopes confusion
References: <083a01c17cfa$e13f6120$e000a8c0@thomasnotebook> <08dc01c17cfc$a5ef22c0$e000a8c0@thomasnotebook>
Message-ID: <3C0DDAA2.F12F365C@lemburg.com>

Thomas Heller wrote:
> 
> From: "Thomas Heller" <thomas.heller@ion-tof.com>
> > How do I achieve the desired effect? Note that the default argument
> > trick (def mth(i=i): ...) does not work because *args is present.
> Answering my own question (sorry), this works:
> 
> def functions():
>     result = []
>     for i in range(10):
>         def make_function(index):
>             def mth():
>                 return index
>             return mth
>         mth = make_function(i)
>         result.append(mth)
>     i = 25
>     return result
> 
> for mth in functions():
>     print mth()
> 
> But how will I understand this 3 months from now?

Why do you put the function definitions *inside*
the functions() constructor ? AFAICTL the confusion is caused
by this overly complicated nesting which doesn't seem
necessary.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From guido@python.org  Wed Dec  5 13:43:02 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Dec 2001 08:43:02 -0500
Subject: [Python-Dev] python2.1.2 plans, plus the curse of PEP-006.
In-Reply-To: Your message of "Wed, 05 Dec 2001 18:05:29 +1100."
 <200112050705.fB575UN09520@mbuna.arbhome.com.au>
References: <200112050705.fB575UN09520@mbuna.arbhome.com.au>
Message-ID: <200112051343.IAA13534@cj20424-a.reston1.va.home.com>

Welcome back, Anthony!  Glad to hear you haven't given up.  If there's
anything we can do to alleviate the curse, let us know...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@zope.com  Wed Dec  5 14:49:04 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Wed, 5 Dec 2001 09:49:04 -0500
Subject: [Python-Dev] python2.1.2 plans, plus the curse of PEP-006.
References: <200112050705.fB575UN09520@mbuna.arbhome.com.au>
Message-ID: <15374.13280.512488.799375@anthem.wooz.org>

>>>>> "AB" == Anthony Baxter <anthony@interlink.com.au> writes:

    AB> (I'm not sure if it's just me, but it's possible the reason
    AB> no-one's done two bugfix releases in a row is that PEP-006 is
    AB> accursed, and brings down the wrath of gods on anyone foolish
    AB> enough to get involved.  I'm hoping that the last few weeks
    AB> horror is over and done now...)

Anthony,

First, thanks for taking on the responsibility, and glad you're back!

Second, remember that PEP 6 is a work in progress, especially since
you're breaking new ground.  If PEP 6 actually hinders patch releases
rather than helps them, then IMHO the PEP is probably broken.  Is your
problem with it the prohibitions?  Or is it the procedure?  Is the PEP
too incomplete to be helpful?

Please do suggest changes!  If you want to co-own it so you can update
it yourself, we can do that too.

Patch releases should be easy(er).

-Barry


From fdrake@acm.org  Wed Dec  5 16:36:19 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 5 Dec 2001 11:36:19 -0500
Subject: [Python-Dev] pychecker on 2.2 current
In-Reply-To: <3C0C3806.B7325138@metaslash.com>
References: <3C0C3806.B7325138@metaslash.com>
Message-ID: <15374.19715.930571.37939@grendel.zope.com>

Neal Norwitz writes:
 > mimetypes.py:405: No global (sys) found
 > 
 > This is inside usage() which presumably should only be used internally.  
 > sys is imported if __name__ == '__main__'.
 > So the problem can only be triggered if usage() is called by external source.
 > 
 > Possible fixes are:
 > 	sys could be locally imported in usage()
 > 	sys could be imported at the module scope
 > 	usage could be made _usage() to denote it's private

Neal,
  I'll propose another fix, but I don't know what pychecker would
say.  ;-)  I'd like to move the definition of usage() into the if
__name__ == "__main__" block, in which case the problem goes away
since there wouldn't be a usage() unless sys has been imported.
  I'll check it in that way.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From jack@oratrix.nl  Wed Dec  5 15:59:39 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Wed, 05 Dec 2001 16:59:39 +0100
Subject: [Python-Dev] Mac OS X sys.platform value
Message-ID: <20011205155939.8936A303183@snelboot.oratrix.nl>

Folks,
Apple "wisely" changed their uname version numbering scheme with a 
micro-release (10.1.1). Of course, nobody cares about this, except Python, 
which stuffs the version into sys.platform, from where it is used by a 
gazillion things.

Anyway, where sys.platform used to be "darwin1" upto and including 10.1.0 it 
is now "darwin5" for 10.1.1 (and it will go up to "darwin6" for 10.2, etc).

I am of a mind to take the "1" out of sys.platform, so that it becomes 
"darwin".

Good idea or not?
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.cwi.nl/~jack        | ++++ see http://www.xs4all.nl/~tank/ ++++




From guido@python.org  Wed Dec  5 16:07:19 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Dec 2001 11:07:19 -0500
Subject: [Python-Dev] Mac OS X sys.platform value
In-Reply-To: Your message of "Wed, 05 Dec 2001 16:59:39 +0100."
 <20011205155939.8936A303183@snelboot.oratrix.nl>
References: <20011205155939.8936A303183@snelboot.oratrix.nl>
Message-ID: <200112051607.fB5G7Jw28901@odiug.zope.com>

> Anyway, where sys.platform used to be "darwin1" upto and including 10.1.0 it 
> is now "darwin5" for 10.1.1 (and it will go up to "darwin6" for 10.2, etc).
> 
> I am of a mind to take the "1" out of sys.platform, so that it becomes 
> "darwin".
> 
> Good idea or not?

Fine with me as long as it doesn't affect other platforms. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz@rahul.net  Wed Dec  5 17:43:38 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Wed, 5 Dec 2001 09:43:38 -0800 (PST)
Subject: [Python-Dev] python2.1.2 plans, plus the curse of PEP-006.
In-Reply-To: <200112050705.fB575UN09520@mbuna.arbhome.com.au> from "Anthony Baxter" at Dec 05, 2001 06:05:29 PM
Message-ID: <20011205174339.3EC2CE8C1@waltz.rahul.net>

Anthony Baxter wrote:
> 
> (I'm not sure if it's just me, but it's possible the reason no-one's 
> done two bugfix releases in a row is that PEP-006 is accursed, and
> brings down the wrath of gods on anyone foolish enough to get involved.
> I'm hoping that the last few weeks horror is over and done now...)

Hmmmm....  I think the fact that three different people have stepped up
to the plate to handle bugfix releases is a *feature*.  In addition,
three releases in six months does not a long-term trend make.  As Barry
said, if there's a problem with PEP6, I'd love to know about it; other
than that, there haven't IMO been enough Python releases and patches to
warrant more bugfix releases than have occurred or are on the plate.

Good work so far!
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

We must not let the evil of a few trample the freedoms of the many.


From Anthony Baxter <anthony@interlink.com.au>  Wed Dec  5 22:09:40 2001
From: Anthony Baxter <anthony@interlink.com.au> (Anthony Baxter)
Date: Thu, 06 Dec 2001 09:09:40 +1100
Subject: [Python-Dev] python2.1.2 plans, plus the curse of PEP-006.
In-Reply-To: Message from barry@zope.com (Barry A. Warsaw)
 of "Wed, 05 Dec 2001 09:49:04 CDT." <15374.13280.512488.799375@anthem.wooz.org>
Message-ID: <200112052209.fB5M9eZ14675@mbuna.arbhome.com.au>

>>> Barry A. Warsaw wrote
> Please do suggest changes!  If you want to co-own it so you can update
> it yourself, we can do that too.
> Patch releases should be easy(er).

I had a think about this, and I think the major issue is that there
was quite a gap to backfill between 2.1.1 and me starting 2.1.2. There
was a bit of backporting (thank you twouters!), but not that much - and 
not that much done as far as marking bugfixes in the changelogs.

Given the rate of change in the python core, I'd say that it's more
likely that people are going to want the ability to switch versions 
when it suits them, and thus keeping up the version before the current
bleeding edge is appropriate.

I'd say that something like the following would make life easier

  The branch for 2.N-maint should be maintained until 2.(N+1) gets 
  a first patch release. That is, 2.1-maint gets bugfixes until 2.2.1
  gets out the door. Alternately, waiting X months after 2.(N+1).0
  would be another approach.

In any case, marking bugfix candidates in CVS logs should be a standard
thing to do. 

In terms of concrete things I'm looking for right now: 
. someone who's either a HP/UX victim^W user who's prepared to try and 
work out what bits need backporting for the threading fix, or someone who's
got a HP/UX box and is prepared to let me ssh into the box and try and do
it myself.
. people to test the 2.1.2 release candidate when it's ready on a bunch
of different platforms - I only have access to solaris, linux, and whatever's
in the sf compile farm.

Anthony


From martin@v.loewis.de  Wed Dec  5 20:00:18 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Wed, 5 Dec 2001 21:00:18 +0100
Subject: [Python-Dev] Mac OS X sys.platform value
In-Reply-To: <20011205155939.8936A303183@snelboot.oratrix.nl> (message from
 Jack Jansen on Wed, 05 Dec 2001 16:59:39 +0100)
References: <20011205155939.8936A303183@snelboot.oratrix.nl>
Message-ID: <200112052000.fB5K0Ip01526@mira.informatik.hu-berlin.de>

> Anyway, where sys.platform used to be "darwin1" upto and including 10.1.0 it 
> is now "darwin5" for 10.1.1 (and it will go up to "darwin6" for 10.2, etc).
> 
> I am of a mind to take the "1" out of sys.platform, so that it becomes 
> "darwin".
> 
> Good idea or not?

Bad idea. Anybody comparing to "darwin1" will have to update the
code. When they do so, they could just as well write
sys.platform.startswith("darwin").

The same happens on Linux, which happens to be "linux2" since Linux
kernel version 2.x is in use. That says little about the system,
though, since glibc version or name of the distribution may be
relevant.

On OS X, it seems sys.platform could be used to distinguish minor
differences. We should not take away that opportunity.

Regards,
Martin



From aahz@rahul.net  Wed Dec  5 22:43:00 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Wed, 5 Dec 2001 14:43:00 -0800 (PST)
Subject: [Python-Dev] python2.1.2 plans, plus the curse of PEP-006.
In-Reply-To: <200112052209.fB5M9eZ14675@mbuna.arbhome.com.au> from "Anthony Baxter" at Dec 06, 2001 09:09:40 AM
Message-ID: <20011205224300.9EFFEE8C7@waltz.rahul.net>

Anthony Baxter wrote:
> 
> I'd say that something like the following would make life easier
> 
>   The branch for 2.N-maint should be maintained until 2.(N+1) gets 
>   a first patch release. That is, 2.1-maint gets bugfixes until 2.2.1
>   gets out the door. Alternately, waiting X months after 2.(N+1).0
>   would be another approach.

What does "should be maintained" mean?  With CVS, IIUC, there's no
reason not to keep branches around forever.  If instead what you're
talking about is policy for work to be assigned, I won't let that into
PEP6; the entire point of PEP6 is that it only prescribes procedure for
someone who is already working, it does not specify any work to be
accomplished in the abstract.  

The original form of PEP6 was looser in this respect (in terms of
specifying when patch releases would occur and so on), but I was
convinced by Guido et al to stick strictly to "how" and not "when" or
"what".  At the very least, I think we should stick with the current
base plan for one or two more release cycles before making any
structured attempts to integrate bugfixes into the standard Python
release cycle.

> In any case, marking bugfix candidates in CVS logs should be a standard
> thing to do. 

The problem lies in creating the social structure to make that happen.
People who monitor CVS need to get into the habit of reminding each
other to mark bugfix candidates.  But that's beyond the scope of PEP6.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

We must not let the evil of a few trample the freedoms of the many.


From neal@metaslash.com  Thu Dec  6 01:28:43 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Wed, 05 Dec 2001 20:28:43 -0500
Subject: [Python-Dev] pychecker on 2.2 current
References: <3C0C3806.B7325138@metaslash.com> <15374.19715.930571.37939@grendel.zope.com>
Message-ID: <3C0EC9CB.48BBD0F0@metaslash.com>

"Fred L. Drake, Jr." wrote:
> 
> Neal Norwitz writes:
>  > mimetypes.py:405: No global (sys) found
>  >
>  > This is inside usage() which presumably should only be used internally.
>  > sys is imported if __name__ == '__main__'.
>  > So the problem can only be triggered if usage() is called 
>  > by external source.
>  >
>  > Possible fixes are:
>  >      sys could be locally imported in usage()
>  >      sys could be imported at the module scope
>  >      usage could be made _usage() to denote it's private
> 
>   I'll propose another fix, but I don't know what pychecker would
> say.  ;-)  I'd like to move the definition of usage() into the if
> __name__ == "__main__" block, in which case the problem goes away
> since there wouldn't be a usage() unless sys has been imported.
>   I'll check it in that way.  ;-)

I like your fix better than my suggestions. :-)
I thought pychecker would complain, but it's happy too. :-)

Neal


From guido@python.org  Thu Dec  6 02:15:15 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Dec 2001 21:15:15 -0500
Subject: [Python-Dev] pychecker on 2.2 current
In-Reply-To: Your message of "Wed, 05 Dec 2001 20:28:43 EST."
 <3C0EC9CB.48BBD0F0@metaslash.com>
References: <3C0C3806.B7325138@metaslash.com> <15374.19715.930571.37939@grendel.zope.com>
 <3C0EC9CB.48BBD0F0@metaslash.com>
Message-ID: <200112060215.VAA18132@cj20424-a.reston1.va.home.com>

> I thought pychecker would complain, but it's happy too. :-)

But isn't that because the new code is all at the module-global level,
which pychecker doesn't "see"?  Or have you fixed that yet?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Thu Dec  6 02:26:15 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Wed, 05 Dec 2001 21:26:15 -0500
Subject: [Python-Dev] pychecker on 2.2 current
References: <3C0C3806.B7325138@metaslash.com> <15374.19715.930571.37939@grendel.zope.com>
 <3C0EC9CB.48BBD0F0@metaslash.com> <200112060215.VAA18132@cj20424-a.reston1.va.home.com>
Message-ID: <3C0ED747.8E6E519C@metaslash.com>

Guido van Rossum wrote:
> 
> > I thought pychecker would complain, but it's happy too. :-)
> 
> But isn't that because the new code is all at the module-global level,
> which pychecker doesn't "see"?  Or have you fixed that yet?

pychecker does "see" all code at the module level, but it isn't perfect.
If there is code like:

	import sys
	if sys.platform == 'win32':
		def x(a, b): pass
	else:
		def x(a): pass

The following incorrect warning is produced:
	t2.py:5: Redefining attribute (x) original line (3)

Because both versions of the function x() are seen.

Neal


From guido@python.org  Thu Dec  6 02:35:25 2001
From: guido@python.org (Guido van Rossum)
Date: Wed, 05 Dec 2001 21:35:25 -0500
Subject: [Python-Dev] pychecker on 2.2 current
In-Reply-To: Your message of "Wed, 05 Dec 2001 21:26:15 EST."
 <3C0ED747.8E6E519C@metaslash.com>
References: <3C0C3806.B7325138@metaslash.com> <15374.19715.930571.37939@grendel.zope.com> <3C0EC9CB.48BBD0F0@metaslash.com> <200112060215.VAA18132@cj20424-a.reston1.va.home.com>
 <3C0ED747.8E6E519C@metaslash.com>
Message-ID: <200112060235.VAA18752@cj20424-a.reston1.va.home.com>

> > But isn't that because the new code is all at the module-global level,
> > which pychecker doesn't "see"?  Or have you fixed that yet?
> 
> pychecker does "see" all code at the module level,

I stand corrected.  I think this was the case for an older version?

> but it isn't perfect.  If there is code like:
> 
> 	import sys
> 	if sys.platform == 'win32':
> 		def x(a, b): pass
> 	else:
> 		def x(a): pass
> 
> The following incorrect warning is produced:
> 	t2.py:5: Redefining attribute (x) original line (3)
> 
> Because both versions of the function x() are seen.

Hm, I would call that warning correct. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Thu Dec  6 02:50:24 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Wed, 05 Dec 2001 21:50:24 -0500
Subject: [Python-Dev] pychecker on 2.2 current
References: <3C0C3806.B7325138@metaslash.com> <15374.19715.930571.37939@grendel.zope.com> <3C0EC9CB.48BBD0F0@metaslash.com> <200112060215.VAA18132@cj20424-a.reston1.va.home.com>
 <3C0ED747.8E6E519C@metaslash.com> <200112060235.VAA18752@cj20424-a.reston1.va.home.com>
Message-ID: <3C0EDCF0.94AB3B88@metaslash.com>

Guido van Rossum wrote:
> 
> > > But isn't that because the new code is all at the module-global level,
> > > which pychecker doesn't "see"?  Or have you fixed that yet?
> >
> > pychecker does "see" all code at the module level,
> 
> I stand corrected.  I think this was the case for an older version?

Yes, you are correct it was the case a while ago.
BTW, same is true for code at the class scope, 
ie, it used to be missed, but it is checked now.

> > but it isn't perfect.  If there is code like:
> >
> >       import sys
> >       if sys.platform == 'win32':
> >               def x(a, b): pass
> >       else:
> >               def x(a): pass
> >
> > The following incorrect warning is produced:
> >       t2.py:5: Redefining attribute (x) original line (3)
> >
> > Because both versions of the function x() are seen.
> 
> Hm, I would call that warning correct. :-)

:-)


From neal@metaslash.com  Thu Dec  6 02:50:38 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Wed, 05 Dec 2001 21:50:38 -0500
Subject: [Python-Dev] memory leaks in 2.2
Message-ID: <3C0EDCFE.D01E6EA3@metaslash.com>

Here is a list of all the outstanding memory leaks from the regression tests.
All of these leaks have a bug report on SF.

test_descr:	117 bytes (0.0407%)
test_email:	 30 bytes (0.0149%)
test_re:	 30 bytes (0.0149%)
test_richcmp:	 16 bytes (0.0106%)
test_scope:	 39 bytes (0.031%)
test_sre:	188 bytes (0.137%)
test_symtable:	 12 bytes (0.00991%)
test_unicode:	 66 bytes (0.0224%)

syntax error in interpreter: 38 bytes (0.05%)


From Anthony Baxter <anthony@interlink.com.au>  Thu Dec  6 02:59:11 2001
From: Anthony Baxter <anthony@interlink.com.au> (Anthony Baxter)
Date: Thu, 06 Dec 2001 13:59:11 +1100
Subject: [Python-Dev] python2.1.2 plans, plus the curse of PEP-006.
In-Reply-To: Message from aahz@rahul.net (Aahz Maruch)
 of "Wed, 05 Dec 2001 14:43:00 -0800." <20011205224300.9EFFEE8C7@waltz.rahul.net>
Message-ID: <200112060259.fB62xBi16480@mbuna.arbhome.com.au>

>>> Aahz Maruch wrote
> What does "should be maintained" mean?  With CVS, IIUC, there's no
> reason not to keep branches around forever.  If instead what you're

I mean "should continually have backported patches applied".

> talking about is policy for work to be assigned, I won't let that into
> PEP6; the entire point of PEP6 is that it only prescribes procedure for
> someone who is already working, it does not specify any work to be
> accomplished in the abstract.  

Oh, sure - I'm not talking about adding it to PEP006 - it's not appropriate
for it. More talking about the abstract here.
 
> The problem lies in creating the social structure to make that happen.
> People who monitor CVS need to get into the habit of reminding each
> other to mark bugfix candidates.  But that's beyond the scope of PEP6.

*nod* absolutely.


From tim.one@home.com  Thu Dec  6 04:14:00 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 5 Dec 2001 23:14:00 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <3C0EDCFE.D01E6EA3@metaslash.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEFMMLAA.tim.one@home.com>

[Neal Norwitz]
> Here is a list of all the outstanding memory leaks from the
> regression tests.
> All of these leaks have a bug report on SF.
>
> test_descr:	117 bytes (0.0407%)
> test_email:	 30 bytes (0.0149%)
> test_re:	 30 bytes (0.0149%)
> test_richcmp:	 16 bytes (0.0106%)
> test_scope:	 39 bytes (0.031%)
> test_sre:	188 bytes (0.137%)
> test_symtable:	 12 bytes (0.00991%)
> test_unicode:	 66 bytes (0.0224%)
>
> syntax error in interpreter: 38 bytes (0.05%)

Can anyone lend a hand here?  Take a test, stick it in an infinite loop,
watch your process size blow off the roof, then whittle the test down until
the trigger becomes clear.  We found a very serious 2.2 leak today doing
this to test_binop.py, and not even Guido (let alone God ...) knows how many
more may be lurking.  It's not a Good Sign that test_binop has vanished from
the list now but everything else is still there.

have-some-fun-plug-a-leak-ly y'rs  - tim



From fdrake@acm.org  Thu Dec  6 05:17:58 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 6 Dec 2001 00:17:58 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEFMMLAA.tim.one@home.com>
References: <3C0EDCFE.D01E6EA3@metaslash.com>
 <LNBBLJKPBEHFEDALKOLCCEFMMLAA.tim.one@home.com>
Message-ID: <15374.65414.640831.895781@grendel.zope.com>

Tim Peters writes:
 > more may be lurking.  It's not a Good Sign that test_binop has vanished from
 > the list now but everything else is still there.

  Err, test_parser is no longer on the list!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From tim.one@home.com  Thu Dec  6 04:36:51 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 5 Dec 2001 23:36:51 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <15374.65414.640831.895781@grendel.zope.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEFOMLAA.tim.one@home.com>

[Tim]
> more may be lurking.  It's not a Good Sign that test_binop has
> vanished from the list now but everything else is still there.

[Fred L. Drake, Jr.]
>   Err, test_parser is no longer on the list!

Which is bad news:  it means that test_binop and test_parser had leaks for
reasons independent of each other, and of all the others on the list:  we're
not making swift progress on these.  You only looked at test_parser because
Guido assigned it to you.  I only looked at test_binop because I was
coughing too hard to get any difficult work done <wink>.



From fdrake@acm.org  Thu Dec  6 06:58:38 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 6 Dec 2001 01:58:38 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEFOMLAA.tim.one@home.com>
References: <15374.65414.640831.895781@grendel.zope.com>
 <LNBBLJKPBEHFEDALKOLCGEFOMLAA.tim.one@home.com>
Message-ID: <15375.5918.776405.747630@grendel.zope.com>

Tim Peters writes:
 > Which is bad news:  it means that test_binop and test_parser had leaks for
 > reasons independent of each other, and of all the others on the list:  we're
 > not making swift progress on these.  You only looked at test_parser because
 > Guido assigned it to you.  I only looked at test_binop because I was
 > coughing too hard to get any difficult work done <wink>.

  Aside from these specific memory-leak reports, we've generally
fallen down on bug triage.  I'm not sure what we can do about this in
the short term, but I really think we're not doing as good a job at
this as we should be.
  I wonder if we could take a lesson from the Mozilla community and
organize Bug Days (group hack days where the goal is to resolve bugs)
or have a "BugAThon" where we call for smaller/simpler test cases that
exercise reported bugs.  I suspect a page about how to narrow down a
bug to a smaller test case would be very helpful, especially if it
also discusses how to use the various available tools for things like
pinpointing leaks, etc.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From tim.one@home.com  Thu Dec  6 07:00:28 2001
From: tim.one@home.com (Tim Peters)
Date: Thu, 6 Dec 2001 02:00:28 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <15375.5918.776405.747630@grendel.zope.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEGEMLAA.tim.one@home.com>

[Fred L. Drake, Jr.]
>   Aside from these specific memory-leak reports, we've generally
> fallen down on bug triage.

Look on the bright side:  we're doing a much better job on bugs than on
patch review, and our patch review work in turn is stellar compared to what
we've accomplished in the PSF <wink>.

> I'm not sure what we can do about this in the short term, but I really
> think we're not doing as good a job at this as we should be.

I think it's just one of many areas where we "should be" doing a better job.

> I wonder if we could take a lesson from the Mozilla community and
> organize Bug Days (group hack days where the goal is to resolve bugs)
> or have a "BugAThon" where we call for smaller/simpler test cases that
> exercise reported bugs.  I suspect a page about how to narrow down a
> bug to a smaller test case would be very helpful, especially if it
> also discusses how to use the various available tools for things like
> pinpointing leaks, etc.

Frankly, I don't think it will help.  If you look at the bug statistics,
we've made steady and dramatic improvement in bug turnaround time over the
last six months, and we can't sustain even that improvement.  Fact is we
have very few people fixing bugs or reviewing patches.  It's not like the
majority of Python developers work on this stuff in spurts, it's that most
don't work on it at all, and almost nobody works on it routinely.  We
couldn't even meet our modest internal (admittedly imposed by me <wink>)
"close one bug or patch per day per person" goal last week (you'll never see
that idea again either ...).

Maybe we should resume offering a free copy of the next Python release to
contributors working beyond the bounds of sanity?  Na -- even Thomas
eventually figured out he could get a free copy anyway -- let's just pray he
doesn't tell Martin <wink>.

better-at-eating-kung-pao-chicken-than-motivating-the-overburdened-
    to-spread-themselves-thinner-than-gas-ly y'rs  - tim



From DavidA@ActiveState.com  Thu Dec  6 07:12:45 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Wed, 05 Dec 2001 23:12:45 -0800
Subject: [Python-Dev] memory leaks in 2.2
References: <15374.65414.640831.895781@grendel.zope.com>
 <LNBBLJKPBEHFEDALKOLCGEFOMLAA.tim.one@home.com> <15375.5918.776405.747630@grendel.zope.com>
Message-ID: <3C0F1A6D.417B603C@ActiveState.com>

"Fred L. Drake, Jr." wrote:

>   Aside from these specific memory-leak reports, we've generally
> fallen down on bug triage.  I'm not sure what we can do about this in
> the short term, but I really think we're not doing as good a job at
> this as we should be.

It's hard.  Bugzilla has some good features there w.r.t. "unverified"
vs. "new" vs. "accepted" and "fixed" vs "resolved", QA contacts, keyword
handling for milestone/release targets, etc. I'm not sure how the SF
bugtracker compares in practice but from a distance it seems a little
weak there.

[BTW -- I know this isn't the right channel, but we (the Komodo team)
have found a couple of bugs in popen recently which y'all may not have
seen yet and you might want to tackle for 2.2 -- we'll get them in SF in
the next few days hopefully, but this is better than nothing:
  - popen on Windows XP seems problematic (needs a repro test there --
not analyzed yet)
  - popen on Posix has a nasty failure mode if the cmd argument is
Unicode (see the first line in Popen3._run_child for a hint =)]

>   I wonder if we could take a lesson from the Mozilla community and
> organize Bug Days (group hack days where the goal is to resolve bugs)
> or have a "BugAThon" where we call for smaller/simpler test cases that
> exercise reported bugs.  I suspect a page about how to narrow down a
> bug to a smaller test case would be very helpful, especially if it
> also discusses how to use the various available tools for things like
> pinpointing leaks, etc.

One thing the Mozilla community has going for it is really dedicated QA
folks.  One in particular, Asa Dotzler, does an amazing job at bug
triage, categorization, etc.  He's now a Netscape employee, but used to
do this for fun.  It would be a good way for someone who'se 'on the
periphery' to get more deeply involved and learn a lot.  In general,
that kind of QA job can be a gateway to fame and fortune much like
documentation writing can be, and you really get to learn the
product/technology.

I don't have first-hand experience of the success of bug days, but
they're doing more of them, so they must get some value out of them. 

--david guessing-from-the-"it hurts! then don't do that!"-corrolary
ascher


From martin@v.loewis.de  Thu Dec  6 07:27:33 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Thu, 6 Dec 2001 08:27:33 +0100
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEGEMLAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCIEGEMLAA.tim.one@home.com>
Message-ID: <200112060727.fB67RXZ08417@mira.informatik.hu-berlin.de>

> Maybe we should resume offering a free copy of the next Python release to
> contributors working beyond the bounds of sanity?  Na -- even Thomas
> eventually figured out he could get a free copy anyway -- let's just pray he
> doesn't tell Martin <wink>.

:-) I personally don't see it as a problem that bug reports are piling
up. The GCC GNATS has 1341 "open" bug reports (meaning either "open"
or "analyzed") at the moment. bugzilla.mozilla.org reports 19611 bugs
in the "open" categories. Microsoft reportedly has over 60000 open
reports for Windows 2000. Neither of these groups despair :-)

As for the things we should change: I believe it is pointless to sit
on a report for a long time without doing anything about this. If
nothing happens for a year, the person that was assigned this report
likely won't do anything about it for the next months, either.
Unassigning things you know you won't do is particularly important for
patches, but to a degree also important for bug reports. It signals
that somebody else may need to step forward, or that some deity should
just decide on the fate of this report.

Regards,
Martin


From DavidA@ActiveState.com  Thu Dec  6 07:39:08 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Wed, 05 Dec 2001 23:39:08 -0800
Subject: [Python-Dev] memory leaks in 2.2
References: <LNBBLJKPBEHFEDALKOLCIEGEMLAA.tim.one@home.com>
Message-ID: <3C0F209C.9FAE0599@ActiveState.com>

Tim Peters wrote:
> 
> [Fred L. Drake, Jr.]
> >   Aside from these specific memory-leak reports, we've generally
> > fallen down on bug triage.
> 
> Look on the bright side:  we're doing a much better job on bugs than on
> patch review, and our patch review work in turn is stellar compared to what
> we've accomplished in the PSF <wink>.

I'll pipe in with our experience with Mozilla & Komodo, which have
related but different processes.

Mozilla has a _very strict_ review process.  Up to two reviews and one
super-review for each checkin
(http://www.mozilla.org/hacking/reviewers.html).  99% of all
communication is done through comments & patches attached to bugs.  It
works for them, in that the quality of the code checkins tends to be
high, but it's a fairly imposing process (which is limiting the number
of contributors, I expect -- not because the process is _that_ hard to
follow, but it's hard to discover and learn and adopt, especially as
part of a side project).  

In the Komodo team, the number and type of reviews we require depends on
the stage of the product (how close we are to release).  We used to have
periods when no review was needed, but I think we're going to keep the
minimal one-review requirement on indefinitely.  At critical times we've
even had a "change review board" process in place.  Having tools that
generate good patches with 'proposed changelogs' and apply them safely,
good integration between the source code control system and the bug
tracking system are all good things which have made our code quality go
up as well.  Bugzilla is a good patch manager as well.

It takes time to learn/teach how to be a good reviewer -- it's not the
same kind of skill as writing a patch or logging a good bug.  Review
processes also affect 'superficial development speed' -- the patch
author has to sit on a patch until the process is complete, and that can
be frustrating.  Also, to the extent that patches uncover other bugs, it
can slow down the bug discovery process.  Of course we wouldn't do it if
we didn't feel it increased the long-term effective rate of quality
increase.

Note that in both of the above cases, _everyone_ must go through the
review process, including the top dogs, for any change to the code base,
build system, etc. (the documentation and test suite subtree are
excluded usually).   Note also that I don't mean to imply that the above
process is the right one for Python.  I'm just speaking on the processes
I know something about =).

--david


From martin@v.loewis.de  Thu Dec  6 07:40:22 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Thu, 6 Dec 2001 08:40:22 +0100
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <3C0F1A6D.417B603C@ActiveState.com> (message from David Ascher on
 Wed, 05 Dec 2001 23:12:45 -0800)
References: <15374.65414.640831.895781@grendel.zope.com>
 <LNBBLJKPBEHFEDALKOLCGEFOMLAA.tim.one@home.com> <15375.5918.776405.747630@grendel.zope.com> <3C0F1A6D.417B603C@ActiveState.com>
Message-ID: <200112060740.fB67eMV08466@mira.informatik.hu-berlin.de>

> It's hard.  Bugzilla has some good features there w.r.t. "unverified"
> vs. "new" vs. "accepted" and "fixed" vs "resolved", QA contacts, keyword
> handling for milestone/release targets, etc. I'm not sure how the SF
> bugtracker compares in practice but from a distance it seems a little
> weak there.

SF recommends to use the Group for "unsupported",
"unverified". Unfortunately, the filtering capabilities leave a lot of
wishes.

>   - popen on Posix has a nasty failure mode if the cmd argument is
> Unicode (see the first line in Popen3._run_child for a hint =)]

You mean,

    def _run_child(self, cmd):
        if isinstance(cmd, types.StringTypes):
            cmd = ['/bin/sh', '-c', cmd]

:-? Fixed since 2001/12/02.

> It would be a good way for someone who'se 'on the periphery' to get
> more deeply involved and learn a lot.

This is free software. Most contributors work on it for fun, or
because they need it for their job.

Posting "volunteers needed" may be a good idea, but then, perhaps not
so shortly before 2.2.

Regards,
Martin


From DavidA@ActiveState.com  Thu Dec  6 07:49:39 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Wed, 05 Dec 2001 23:49:39 -0800
Subject: [Python-Dev] memory leaks in 2.2
References: <15374.65414.640831.895781@grendel.zope.com>
 <LNBBLJKPBEHFEDALKOLCGEFOMLAA.tim.one@home.com> <15375.5918.776405.747630@grendel.zope.com> <3C0F1A6D.417B603C@ActiveState.com> <200112060740.fB67eMV08466@mira.informatik.hu-berlin.de>
Message-ID: <3C0F2313.C9B7824F@ActiveState.com>

>         if isinstance(cmd, types.StringTypes):
>             cmd = ['/bin/sh', '-c', cmd]
> 
> :-? Fixed since 2001/12/02.

Ha!  Good.  One more resolved bug then =).

> > It would be a good way for someone who'se 'on the periphery' to get
> > more deeply involved and learn a lot.
> 
> This is free software. Most contributors work on it for fun, or
> because they need it for their job.

Sure.  Are you disagreeing with something I said?

--david


From anthony@interlink.com.au  Thu Dec  6 07:45:04 2001
From: anthony@interlink.com.au (Anthony Baxter)
Date: Thu, 06 Dec 2001 18:45:04 +1100
Subject: [Python-Dev] on the subject of SF...
Message-ID: <200112060745.fB67j4e18189@mbuna.arbhome.com.au>

can someone with the appropriate level of magickness add a 'Python 2.1.2'
group to the bug tracker and patch tracker? I've been meaning to get this
done for a while... ta.
Anthony


From martin@v.loewis.de  Thu Dec  6 07:47:31 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Thu, 6 Dec 2001 08:47:31 +0100
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <3C0F209C.9FAE0599@ActiveState.com> (message from David Ascher on
 Wed, 05 Dec 2001 23:39:08 -0800)
References: <LNBBLJKPBEHFEDALKOLCIEGEMLAA.tim.one@home.com> <3C0F209C.9FAE0599@ActiveState.com>
Message-ID: <200112060747.fB67lV108506@mira.informatik.hu-berlin.de>

> Note that in both of the above cases, _everyone_ must go through the
> review process, including the top dogs, for any change to the code base,
> build system, etc. (the documentation and test suite subtree are
> excluded usually).   Note also that I don't mean to imply that the above
> process is the right one for Python.  I'm just speaking on the processes
> I know something about =).

To report on the processes of another project; GCC organizes people in
"blanket write priviledge", "maintainers" (CPU port, OS, misc), and
"write after approval". The blanket priviledge allows people to commit
contributions they have reviewed from contributors without any write
access; those are used sparingly. Maintainers can change "their" files
any time they like, except after a freeze. Everybody else must wait
for a patch review.

Regards,
Martin



From barry@zope.com  Thu Dec  6 07:49:02 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 6 Dec 2001 02:49:02 -0500
Subject: [Python-Dev] on the subject of SF...
References: <200112060745.fB67j4e18189@mbuna.arbhome.com.au>
Message-ID: <15375.8942.722195.533597@anthem.wooz.org>

>>>>> "AB" == Anthony Baxter <anthony@interlink.com.au> writes:

    AB> can someone with the appropriate level of magickness add a
    AB> 'Python 2.1.2' group to the bug tracker and patch tracker?
    AB> I've been meaning to get this done for a while... ta.  Anthony

Done!
-Barry


From martin@v.loewis.de  Thu Dec  6 07:52:38 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Thu, 6 Dec 2001 08:52:38 +0100
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <3C0F2313.C9B7824F@ActiveState.com> (message from David Ascher on
 Wed, 05 Dec 2001 23:49:39 -0800)
References: <15374.65414.640831.895781@grendel.zope.com>
 <LNBBLJKPBEHFEDALKOLCGEFOMLAA.tim.one@home.com> <15375.5918.776405.747630@grendel.zope.com> <3C0F1A6D.417B603C@ActiveState.com> <200112060740.fB67eMV08466@mira.informatik.hu-berlin.de> <3C0F2313.C9B7824F@ActiveState.com>
Message-ID: <200112060752.fB67qcw08535@mira.informatik.hu-berlin.de>

> > > It would be a good way for someone who'se 'on the periphery' to get
> > > more deeply involved and learn a lot.
> > 
> > This is free software. Most contributors work on it for fun, or
> > because they need it for their job.
> 
> Sure.  Are you disagreeing with something I said?

No. I just think this won't help much. Just because it would be good
for somebody doesn't mean that this somebody will actually invest time
into the project. People normally get involved by other means than
starting as a bug/patch reviewer. Typically, they get involved because
they need something fixed that apparently nobody else has fixed for
them while they were waiting for it (atleast that's how I got involved
in all free software projects I was working on).

Regards,
Martin



From mal@lemburg.com  Thu Dec  6 08:40:00 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 06 Dec 2001 09:40:00 +0100
Subject: [Python-Dev] on the subject of SF...
References: <200112060745.fB67j4e18189@mbuna.arbhome.com.au> <15375.8942.722195.533597@anthem.wooz.org>
Message-ID: <3C0F2EE0.BE1D2980@lemburg.com>


"Barry A. Warsaw" wrote:
> 
> >>>>> "AB" == Anthony Baxter <anthony@interlink.com.au> writes:
> 
>     AB> can someone with the appropriate level of magickness add a
>     AB> 'Python 2.1.2' group to the bug tracker and patch tracker?
>     AB> I've been meaning to get this done for a while... ta.  Anthony
> 
> Done!

And please also a Group "Python 2.3" so that feature requests
for that version can be assigned to it ?!

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From mal@lemburg.com  Thu Dec  6 09:08:59 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 06 Dec 2001 10:08:59 +0100
Subject: [Python-Dev] Mac OS X sys.platform value
References: <20011205155939.8936A303183@snelboot.oratrix.nl> <200112052000.fB5K0Ip01526@mira.informatik.hu-berlin.de>
Message-ID: <3C0F35AB.2CF9D067@lemburg.com>

"Martin v. Loewis" wrote:
> 
> > Anyway, where sys.platform used to be "darwin1" upto and including 10.1.0 it
> > is now "darwin5" for 10.1.1 (and it will go up to "darwin6" for 10.2, etc).
> >
> > I am of a mind to take the "1" out of sys.platform, so that it becomes
> > "darwin".
> >
> > Good idea or not?
> 
> Bad idea. Anybody comparing to "darwin1" will have to update the
> code. When they do so, they could just as well write
> sys.platform.startswith("darwin").
>
> The same happens on Linux, which happens to be "linux2" since Linux
> kernel version 2.x is in use. That says little about the system,
> though, since glibc version or name of the distribution may be
> relevant.

Exactly and that's why I think that version numbers in sys.platform
are plain wrong. 

There are better tools for finding out details
about the system, e.g. platform.py which you can download from my
Python pages.

> On OS X, it seems sys.platform could be used to distinguish minor
> differences. We should not take away that opportunity.

As you already noted, the information given in the current
version number is too inaccurate to be of any real value. 

Why not add platform.py to the std lib in Python 2.3 and remove
all version info from sys.platform ?!

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From ping@lfw.org  Thu Dec  6 09:57:41 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Thu, 6 Dec 2001 01:57:41 -0800 (PST)
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <3C0EDCFE.D01E6EA3@metaslash.com>
Message-ID: <Pine.LNX.4.10.10112060155330.8657-100000@server1.lfw.org>

On Wed, 5 Dec 2001, Neal Norwitz wrote:
> Here is a list of all the outstanding memory leaks from the regression tests.
> All of these leaks have a bug report on SF.
> 
> test_descr:	117 bytes (0.0407%)

Okay, i have tested this case and narrowed it down to:

    while 1:
        class C:
            __metaclass__ = type

Try it on a Python interpreter near you.  With mine, it grows
without bound.

As far as i can tell, of all the calls in test_descr.test_main,
only metaclass() leaks memory.


-- ?!ng



From mal@lemburg.com  Thu Dec  6 12:12:46 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 06 Dec 2001 13:12:46 +0100
Subject: [Python-Dev] Comparing sequences ?!
Message-ID: <3C0F60BE.5616F544@lemburg.com>

I just tried (1,2,3) == [1,2,3] and found that the compare
returns false. Is that intended ?

Background: 1.0 == 1 == 1L also works, so it seems natural
that comparing sequences of different types should work in the
same way. I would even expect a two iterators to compare equal
if they always return the same values (this could cause an endless
loop for some endless iterators ;-).

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From fdrake@acm.org  Thu Dec  6 13:25:15 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 6 Dec 2001 08:25:15 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEGEMLAA.tim.one@home.com>
References: <15375.5918.776405.747630@grendel.zope.com>
 <LNBBLJKPBEHFEDALKOLCIEGEMLAA.tim.one@home.com>
Message-ID: <15375.29115.974845.732239@grendel.zope.com>

Tim Peters writes:
 > better-at-eating-kung-pao-chicken-than-motivating-the-overburdened-
 >     to-spread-themselves-thinner-than-gas-ly y'rs  - tim

  Ah, but gas can be spread infinately thin -- something that can't be
said of most developers!  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From guido@python.org  Thu Dec  6 13:30:58 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Dec 2001 08:30:58 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: Your message of "Wed, 05 Dec 2001 23:14:00 EST."
 <LNBBLJKPBEHFEDALKOLCCEFMMLAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCCEFMMLAA.tim.one@home.com>
Message-ID: <200112061330.IAA21262@cj20424-a.reston1.va.home.com>

> Can anyone lend a hand here?  Take a test, stick it in an infinite loop,
> watch your process size blow off the roof, then whittle the test down until
> the trigger becomes clear.  We found a very serious 2.2 leak today doing
> this to test_binop.py, and not even Guido (let alone God ...) knows how many
> more may be lurking.  It's not a Good Sign that test_binop has vanished from
> the list now but everything else is still there.

I'll take test_descr.py right now.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Dec  6 13:38:54 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Dec 2001 08:38:54 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: Your message of "Wed, 05 Dec 2001 23:36:51 EST."
 <LNBBLJKPBEHFEDALKOLCGEFOMLAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCGEFOMLAA.tim.one@home.com>
Message-ID: <200112061338.IAA21671@cj20424-a.reston1.va.home.com>

I can't find a leak in test_descr.py when I simply call test_main() in
an infinite loop.  (I can when I use reload() in the loop, but that's
an import issue I believe.)

Neil, did you test this before or after I fixed the __slots__ leak?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh@python.net  Thu Dec  6 13:42:23 2001
From: mwh@python.net (Michael Hudson)
Date: 06 Dec 2001 08:42:23 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: "Martin v. Loewis"'s message of "Thu, 6 Dec 2001 08:40:22 +0100"
References: <15374.65414.640831.895781@grendel.zope.com> <LNBBLJKPBEHFEDALKOLCGEFOMLAA.tim.one@home.com> <15375.5918.776405.747630@grendel.zope.com> <3C0F1A6D.417B603C@ActiveState.com> <200112060740.fB67eMV08466@mira.informatik.hu-berlin.de>
Message-ID: <2my9kgcvnk.fsf@starship.python.net>

"Martin v. Loewis" <martin@v.loewis.de> writes:

> > It's hard.  Bugzilla has some good features there w.r.t. "unverified"
> > vs. "new" vs. "accepted" and "fixed" vs "resolved", QA contacts, keyword
> > handling for milestone/release targets, etc. I'm not sure how the SF
> > bugtracker compares in practice but from a distance it seems a little
> > weak there.
> 
> SF recommends to use the Group for "unsupported",
> "unverified". Unfortunately, the filtering capabilities leave a lot of
> wishes.

Is roundup usable yet?  I played with it a bit yesterday, but I'm not
really sure what's needed.  It would have the advantage that if it
didn't do something we needed, we could probably club it into doing it
easily enough.

Cheers,
M.

-- 
  I never disputed the Perl hacking skill of the Slashdot creators. 
  My objections are to the editors' taste, the site's ugly visual 
  design, and the Slashdot community's raging stupidity.
     -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq


From guido@python.org  Thu Dec  6 13:48:25 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Dec 2001 08:48:25 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: Your message of "Thu, 06 Dec 2001 08:27:33 +0100."
 <200112060727.fB67RXZ08417@mira.informatik.hu-berlin.de>
References: <LNBBLJKPBEHFEDALKOLCIEGEMLAA.tim.one@home.com>
 <200112060727.fB67RXZ08417@mira.informatik.hu-berlin.de>
Message-ID: <200112061348.IAA22096@cj20424-a.reston1.va.home.com>

> As for the things we should change: I believe it is pointless to sit
> on a report for a long time without doing anything about this. If
> nothing happens for a year, the person that was assigned this report
> likely won't do anything about it for the next months, either.
> Unassigning things you know you won't do is particularly important for
> patches, but to a degree also important for bug reports. It signals
> that somebody else may need to step forward, or that some deity should
> just decide on the fate of this report.

Yup.  We've missed important bugs in the past because they happened to
be assigned to someone who didn't care.  (I noticed that Jack
recommended some bug reporter to assign bugs to him rather than let
them sit anonymously.  That may work in his case, but in general it's
the wrong thing to do.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fredrik@pythonware.com  Thu Dec  6 13:50:51 2001
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Thu, 6 Dec 2001 14:50:51 +0100
Subject: [Python-Dev] memory leaks in 2.2
References: <15374.65414.640831.895781@grendel.zope.com> <LNBBLJKPBEHFEDALKOLCGEFOMLAA.tim.one@home.com> <15375.5918.776405.747630@grendel.zope.com> <3C0F1A6D.417B603C@ActiveState.com> <200112060740.fB67eMV08466@mira.informatik.hu-berlin.de> <2my9kgcvnk.fsf@starship.python.net>
Message-ID: <01ad01c17e5d$051fe840$0900a8c0@spiff>

Michael Hudson wrote:

> Is roundup usable yet?  I played with it a bit yesterday, but I'm not
> really sure what's needed.  It would have the advantage that if it
> didn't do something we needed, we could probably club it into doing it
> easily enough.

the drawback with "easily enough" is that you cannot resist
modifying it -- so instead of learning how to use the tool you
have in the best way, you end up hacking on the tool instead
of doing more important work.

(fwiw, we're using a very slightly modified version of Ping's
original one-day hack.  took a couple of months to find the
right "use patterns", but now that we know how to use it in
a way that fits our development process, I doubt you can
find a "more usable" tool...)

</F>




From guido@python.org  Thu Dec  6 13:59:53 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Dec 2001 08:59:53 -0500
Subject: [Python-Dev] Comparing sequences ?!
In-Reply-To: Your message of "Thu, 06 Dec 2001 13:12:46 +0100."
 <3C0F60BE.5616F544@lemburg.com>
References: <3C0F60BE.5616F544@lemburg.com>
Message-ID: <200112061359.IAA22524@cj20424-a.reston1.va.home.com>

> I just tried (1,2,3) == [1,2,3] and found that the compare
> returns false. Is that intended ?

Yes.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Thu Dec  6 14:01:16 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Thu, 06 Dec 2001 09:01:16 -0500
Subject: [Python-Dev] memory leaks in 2.2
References: <LNBBLJKPBEHFEDALKOLCGEFOMLAA.tim.one@home.com> <200112061338.IAA21671@cj20424-a.reston1.va.home.com>
Message-ID: <3C0F7A2C.1B416FF0@metaslash.com>

Guido van Rossum wrote:
> 
> I can't find a leak in test_descr.py when I simply call test_main() in
> an infinite loop.  (I can when I use reload() in the loop, but that's
> an import issue I believe.)
> 
> Neil, did you test this before or after I fixed the __slots__ leak?

I believe I tested after your initial checkin, but before the most recent.
I just updated and re-ran.  117 bytes were still leaked.  I'll try to narrow
down the test to get a smaller test case.

Neal


From guido@python.org  Thu Dec  6 14:01:06 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Dec 2001 09:01:06 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: Your message of "Thu, 06 Dec 2001 08:38:54 EST."
 <200112061338.IAA21671@cj20424-a.reston1.va.home.com>
References: <LNBBLJKPBEHFEDALKOLCGEFOMLAA.tim.one@home.com>
 <200112061338.IAA21671@cj20424-a.reston1.va.home.com>
Message-ID: <200112061401.JAA22566@cj20424-a.reston1.va.home.com>

> I can't find a leak in test_descr.py when I simply call test_main() in
> an infinite loop.  (I can when I use reload() in the loop, but that's
> an import issue I believe.)
> 
> Neil, did you test this before or after I fixed the __slots__ leak?

Never mind, with Ping's hint I see a leak.  I'll fixit momentarily.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From jeremy@zope.com  Thu Dec  6 14:15:34 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Thu, 6 Dec 2001 09:15:34 -0500 (EST)
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <3C0EDCFE.D01E6EA3@metaslash.com>
References: <3C0EDCFE.D01E6EA3@metaslash.com>
Message-ID: <15375.32134.907220.808367@slothrop.digicool.com>

>>>>> "NN" == Neal Norwitz <neal@metaslash.com> writes:

  NN> Here is a list of all the outstanding memory leaks from the
  NN> regression tests.  All of these leaks have a bug report on SF.

I don't see an SF bug report for test_scope.  If you have any data
from Purify, that would be helpful.  It certainly looks like it's
leaking.

Jeremy





From neal@metaslash.com  Thu Dec  6 14:18:47 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Thu, 06 Dec 2001 09:18:47 -0500
Subject: [Python-Dev] memory leaks in 2.2
References: <3C0EDCFE.D01E6EA3@metaslash.com> <15375.32134.907220.808367@slothrop.digicool.com>
Message-ID: <3C0F7E47.BF969740@metaslash.com>

Jeremy Hylton wrote:
> 
> >>>>> "NN" == Neal Norwitz <neal@metaslash.com> writes:
> 
>   NN> Here is a list of all the outstanding memory leaks from the
>   NN> regression tests.  All of these leaks have a bug report on SF.
> 
> I don't see an SF bug report for test_scope.  If you have any data
> from Purify, that would be helpful.  It certainly looks like it's
> leaking.

https://sourceforge.net/tracker/index.php?func=detail&aid=485152&group_id=5470&atid=105470


From neal@metaslash.com  Thu Dec  6 14:24:19 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Thu, 06 Dec 2001 09:24:19 -0500
Subject: [Python-Dev] memory leaks in 2.2
References: <LNBBLJKPBEHFEDALKOLCGEFOMLAA.tim.one@home.com>
 <200112061338.IAA21671@cj20424-a.reston1.va.home.com> <200112061401.JAA22566@cj20424-a.reston1.va.home.com>
Message-ID: <3C0F7F93.5555623B@metaslash.com>

Guido van Rossum wrote:
> 
> > I can't find a leak in test_descr.py when I simply call test_main() in
> > an infinite loop.  (I can when I use reload() in the loop, but that's
> > an import issue I believe.)
> >
> > Neil, did you test this before or after I fixed the __slots__ leak?
> 
> Never mind, with Ping's hint I see a leak.  I'll fixit momentarily.

In test_descr, buffer_inherit() leaks 35 bytes which may indicate
the problem.

Neal


From mal@lemburg.com  Thu Dec  6 14:26:28 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 06 Dec 2001 15:26:28 +0100
Subject: [Python-Dev] Comparing sequences ?!
References: <3C0F60BE.5616F544@lemburg.com> <200112061359.IAA22524@cj20424-a.reston1.va.home.com>
Message-ID: <3C0F8014.FA2ECF58@lemburg.com>

Guido van Rossum wrote:
> 
> > I just tried (1,2,3) == [1,2,3] and found that the compare
> > returns false. Is that intended ?
> 
> Yes.

I presume that means "...and we're not going to change it." ;-)

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From barry@zope.com  Thu Dec  6 14:36:11 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 6 Dec 2001 09:36:11 -0500
Subject: [Python-Dev] on the subject of SF...
References: <200112060745.fB67j4e18189@mbuna.arbhome.com.au>
 <15375.8942.722195.533597@anthem.wooz.org>
 <3C0F2EE0.BE1D2980@lemburg.com>
Message-ID: <15375.33371.540885.525907@anthem.wooz.org>

>>>>> "M" == M  <mal@lemburg.com> writes:

    M> And please also a Group "Python 2.3" so that feature requests
    M> for that version can be assigned to it ?!

So far we have no groups defined for the feature request tracker.  I
wonder if that's intentional.

-Barry


From guido@python.org  Thu Dec  6 14:47:18 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Dec 2001 09:47:18 -0500
Subject: [Python-Dev] Comparing sequences ?!
In-Reply-To: Your message of "Thu, 06 Dec 2001 15:26:28 +0100."
 <3C0F8014.FA2ECF58@lemburg.com>
References: <3C0F60BE.5616F544@lemburg.com> <200112061359.IAA22524@cj20424-a.reston1.va.home.com>
 <3C0F8014.FA2ECF58@lemburg.com>
Message-ID: <200112061447.JAA23518@cj20424-a.reston1.va.home.com>

> > > I just tried (1,2,3) == [1,2,3] and found that the compare
> > > returns false. Is that intended ?
> > 
> > Yes.
> 
> I presume that means "...and we're not going to change it." ;-)

Not any time soon.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Thu Dec  6 21:52:16 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Dec 2001 16:52:16 -0500
Subject: [Python-Dev] Need help with distutils
Message-ID: <200112062152.fB6LqGT11265@odiug.zope.com>

Andrew Kuchling just unassigned from himself a whole slew of
distutils-related bug reports.  It appears that neither Andrew nor
Greg Ward (the original maintainer) has the time to maintain this code
any more.  No-one at PythonLabs understands it or has the time to
learn about it.  So it's effectively orphaned.

Does anybody want to volunteer?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Thu Dec  6 22:39:05 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 06 Dec 2001 23:39:05 +0100
Subject: [Python-Dev] Need help with distutils
References: <200112062152.fB6LqGT11265@odiug.zope.com>
Message-ID: <3C0FF389.9C5FF5A5@lemburg.com>

Guido van Rossum wrote:
> 
> Andrew Kuchling just unassigned from himself a whole slew of
> distutils-related bug reports.  It appears that neither Andrew nor
> Greg Ward (the original maintainer) has the time to maintain this code
> any more.  No-one at PythonLabs understands it or has the time to
> learn about it.  So it's effectively orphaned.
> 
> Does anybody want to volunteer?

The distutils list can take over, I guess. I will certainly want
to see distutils have a bright future and so do many other
users like e.g. Thomas Heller, Jack Jansen, etc.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From DavidA@ActiveState.com  Fri Dec  7 00:56:46 2001
From: DavidA@ActiveState.com (David Ascher)
Date: Thu, 06 Dec 2001 16:56:46 -0800
Subject: [Python-Dev] memory leaks in 2.2
References: <15374.65414.640831.895781@grendel.zope.com> <LNBBLJKPBEHFEDALKOLCGEFOMLAA.tim.one@home.com> <15375.5918.776405.747630@grendel.zope.com> <3C0F1A6D.417B603C@ActiveState.com> <200112060740.fB67eMV08466@mira.informatik.hu-berlin.de> <2my9kgcvnk.fsf@starship.python.net> <01ad01c17e5d$051fe840$0900a8c0@spiff>
Message-ID: <3C1013CE.B1AE8D82@ActiveState.com>

Fredrik Lundh wrote:
> 
> the drawback with "easily enough" is that you cannot resist
> modifying it -- so instead of learning how to use the tool you
> have in the best way, you end up hacking on the tool instead
> of doing more important work.

Indeed =)

> (fwiw, we're using a very slightly modified version of Ping's
> original one-day hack.  took a couple of months to find the
> right "use patterns", but now that we know how to use it in
> a way that fits our development process, I doubt you can
> find a "more usable" tool...)

FWIW, we used to use a fairly seriously modified version of Ping's
original one-day hack, and gave it up.  It was generating too much email
and the 'nosylist' concept wasn't working for us.  The more explicit
list management that bugzilla uses seems to work better for us.  YMMV.

--david


From Keila - Curitiba"<arosadesaron@zipmail.com.br  Fri Dec  7 02:07:57 2001
From: Keila - Curitiba"<arosadesaron@zipmail.com.br (Keila - Curitiba)
Date: Fri, 7 Dec 2001 00:07:57 -0200
Subject: [Python-Dev] CONVITE HP ALERTA
Message-ID: <200112070203.fB723rD51272@smtp.ieg.com.br>

Ol=E1!
Veja meu site pessoal. Basta clicar no endere=E7o
abaixo. GARANTO SER SUI-GENERIS  -  CLIQUE ABAIXO:
http://www.pastorinha.atfreeweb.com
Mais de 162.000 internautas visitaram a PG., existe 6 =C1lbuns:
Se voc=EA quiser, por favor, indique minha Home Page, a outros
Internautas.  Mais detalhes, se comunique, passe um e-mail, que
responderei brevemente. Dentro da Home Page, ao lado das fotos,
voc=EA poder=E1 saber muito mais sobre mim!
Obrigada.
e-mail: arosadesaron@zipmail.com.br
Beijos:-     Keila - Curitiba - Pr
- Podes falar comigo, direto dela. Brevemente uma Carta Aberta.
- Embora derrubem a p=E1gina eu a subo em 3 horas novamente.

 "Esta mensagem =E9 enviada com a complac=EAncia da nova legisla=E7=E3o=20
sobre
correio eletr=F4nico, Se=E7=E3o 301, Par=E1grafo (a) (2) (c) Decreto S. 1=
618,
T=EDtulo Terceiro aprovado pelo "105=BA Congresso Base das Normativas
Internacionais sobre o SPAM". Este E-mail n=E3o poder=E1 ser considerado
SPAM quando incluir uma forma de ser removido. Para ser removido
de futuros correios, simplesmente responda indicando
no Assunto: REMOVER"


From gward@python.net  Fri Dec  7 02:27:57 2001
From: gward@python.net (Greg Ward)
Date: Thu, 6 Dec 2001 21:27:57 -0500
Subject: [Python-Dev] Mac OS X sys.platform value
In-Reply-To: <20011205155939.8936A303183@snelboot.oratrix.nl>
References: <20011205155939.8936A303183@snelboot.oratrix.nl>
Message-ID: <20011206212757.A9167@gerg.ca>

On 05 December 2001, Jack Jansen said:
> Anyway, where sys.platform used to be "darwin1" upto and including 10.1.0 it 
> is now "darwin5" for 10.1.1 (and it will go up to "darwin6" for 10.2, etc).
> 
> I am of a mind to take the "1" out of sys.platform, so that it becomes 
> "darwin".

+1 from me.  If we could turn the clock back 5 or 6 years, I would also
do the same on Linux.

Note that I share Marc-André's dim view of tacking the first digit of
the kernel version number (or whatever) onto sys.platform -- the
Distutils has Yet Another way of creating platform descriptions.  It's
more complex than sys.platform, but quite a lot less fancy than
Marc-André's platform.py.

See distutils.util.get_platform(), if you're curious.

        Greg
-- 
Greg Ward - Linux weenie                                gward@python.net
http://starship.python.net/~gward/
Never put off till tomorrow what you can put off till the day after tomorrow.


From anthony@interlink.com.au  Fri Dec  7 03:58:41 2001
From: anthony@interlink.com.au (Anthony Baxter)
Date: Fri, 07 Dec 2001 14:58:41 +1100
Subject: [Python-Dev] dumbdbm.py
Message-ID: <200112070358.fB73wf425864@mbuna.arbhome.com.au>

just had reason to be looking at dumbdbm.py - perhaps
"terminallybrokendbm.py" would be a better name. There's a whole host of
"oh. my. god." type things in there that are just broken, with massive
dataloss problems if you end up (innocently) getting it from anydbm.py.

I'm going to fix it either this arvo/evening or tomorrow, but it really 
shouldn't go out as part of 2.2 in it's current form. Is it too late to
consider either fixing it or removing it from anydbm's list?

Anthony


From guido@python.org  Fri Dec  7 04:03:29 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 06 Dec 2001 23:03:29 -0500
Subject: [Python-Dev] dumbdbm.py
In-Reply-To: Your message of "Fri, 07 Dec 2001 14:58:41 +1100."
 <200112070358.fB73wf425864@mbuna.arbhome.com.au>
References: <200112070358.fB73wf425864@mbuna.arbhome.com.au>
Message-ID: <200112070403.XAA07071@cj20424-a.reston1.va.home.com>

> just had reason to be looking at dumbdbm.py - perhaps
> "terminallybrokendbm.py" would be a better name. There's a whole host of
> "oh. my. god." type things in there that are just broken, with massive
> dataloss problems if you end up (innocently) getting it from anydbm.py.
> 
> I'm going to fix it either this arvo/evening or tomorrow, but it really 
> shouldn't go out as part of 2.2 in it's current form. Is it too late to
> consider either fixing it or removing it from anydbm's list?

Please fix it.  It was never intended to be production quality.  But
it has found more use than it should.

Then again, dbhash.py isn't much better...[*] :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)

[*] http://www.sleepycat.com/historic.html


From mwh@python.net  Fri Dec  7 15:59:13 2001
From: mwh@python.net (Michael Hudson)
Date: 07 Dec 2001 10:59:13 -0500
Subject: [Python-Dev] on the subject of SF...
In-Reply-To: barry@zope.com's message of "Thu, 6 Dec 2001 09:36:11 -0500"
References: <200112060745.fB67j4e18189@mbuna.arbhome.com.au> <15375.8942.722195.533597@anthem.wooz.org> <3C0F2EE0.BE1D2980@lemburg.com> <15375.33371.540885.525907@anthem.wooz.org>
Message-ID: <2mu1v33tta.fsf@starship.python.net>

barry@zope.com (Barry A. Warsaw) writes:

> >>>>> "M" == M  <mal@lemburg.com> writes:
> 
>     M> And please also a Group "Python 2.3" so that feature requests
>     M> for that version can be assigned to it ?!
> 
> So far we have no groups defined for the feature request tracker.  I
> wonder if that's intentional.

Does anybody look at the feature request tracker ever?  I notice I'm
one of four possible assignees (presumably 'cause I'm new).  Does
email from it go anywhere?

Cheers,
M.

-- 
  Slim Shady is fed up with your shit, and he's going to kill you.
                         -- Eminem, "Public Service Announcement 2000"


From guido@python.org  Fri Dec  7 16:13:49 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Dec 2001 11:13:49 -0500
Subject: [Python-Dev] on the subject of SF...
In-Reply-To: Your message of "07 Dec 2001 10:59:13 EST."
 <2mu1v33tta.fsf@starship.python.net>
References: <200112060745.fB67j4e18189@mbuna.arbhome.com.au> <15375.8942.722195.533597@anthem.wooz.org> <3C0F2EE0.BE1D2980@lemburg.com> <15375.33371.540885.525907@anthem.wooz.org>
 <2mu1v33tta.fsf@starship.python.net>
Message-ID: <200112071613.LAA09963@cj20424-a.reston1.va.home.com>

> Does anybody look at the feature request tracker ever?  I notice I'm
> one of four possible assignees (presumably 'cause I'm new).  Does
> email from it go anywhere?

New features were supposed to be sent to python-bugs-list@python.org,
but somehow that wasn't set, so no, nobody has looked at this in a
long time... :-(  I'll have a look now, and I've set the email address.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Fri Dec  7 16:15:24 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 07 Dec 2001 11:15:24 -0500
Subject: [Python-Dev] update on memory leaks in 2.2
Message-ID: <3C10EB1C.9302CA8@metaslash.com>

Command line: (longexp crashes purify for some reason)
	./python -E -tt ./Lib/test/regrtest.py -u network -x test_longexp.py

Results:
	168 tests OK.
	1 test failed:
	    test_socket_ssl
	15 tests skipped:
	    test_al test_bsddb test_cd test_cl test_curses test_dl test_gl
	    test_imgfile test_linuxaudiodev test_nis test_ntpath test_openpty
	    test_sundry test_winreg test_winsound

test test_socket_ssl crashed -- socket.sslerror: SSL_CTX_new error

When running test_curses:

	test test_curses crashed -- _curses.error: curs_set() returned ERR


Uninitialized Memory Reads (don't think these are python's fault):

	gdbm_open()	gdbmmodule.c:68 
	getaddrinfo()	socketmodule.c:636

Memory Leaks:

	NONE

Potential Memory Leaks:

	1012 from 4 different allocating points -- all in WeakRef

Things look great!

Let me know if any of the skipped tests ought to have been run.
Most likely possibility seems to be bsddb, sundry also needs bsddb.

Neal


From guido@python.org  Fri Dec  7 16:26:58 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Dec 2001 11:26:58 -0500
Subject: [Python-Dev] tracker admin
Message-ID: <200112071626.LAA09993@cj20424-a.reston1.va.home.com>

I just discovered a few new (?) features of the bug/patch tracker
admin page.

- You can disable anonymous submission.  I've done this for all our
  trackers (bugs, patches, features), since we have way too many
  problems with anonymous submissions where the user doesn't provide
  enough info and we have no way to get back to them.

- You can provide custom text for the "submit new" page.  I've added a
  warning about the annoying upload checkbox, and some additional
  suggestions for patches.  Please feel free to provide additional
  suggestions.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Dec  7 16:38:35 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Dec 2001 11:38:35 -0500
Subject: [Python-Dev] update on memory leaks in 2.2
In-Reply-To: Your message of "Fri, 07 Dec 2001 11:15:24 EST."
 <3C10EB1C.9302CA8@metaslash.com>
References: <3C10EB1C.9302CA8@metaslash.com>
Message-ID: <200112071638.LAA12876@cj20424-a.reston1.va.home.com>

> Command line: (longexp crashes purify for some reason)
> 	./python -E -tt ./Lib/test/regrtest.py -u network -x test_longexp.py
> 
> Results:
> 	168 tests OK.
> 	1 test failed:
> 	    test_socket_ssl
> 	15 tests skipped:
> 	    test_al test_bsddb test_cd test_cl test_curses test_dl test_gl
> 	    test_imgfile test_linuxaudiodev test_nis test_ntpath test_openpty
> 	    test_sundry test_winreg test_winsound

A few of these skips weren't expected: test_sundry, test_openpty,
test_bsddb.  What's missing to run those?

> test test_socket_ssl crashed -- socket.sslerror: SSL_CTX_new error

Works for me.  Probably your platform.  The SSL code sucks anyway.

> When running test_curses:
> 
> 	test test_curses crashed -- _curses.error: curs_set() returned ERR

Works for me.
> 
> Uninitialized Memory Reads (don't think these are python's fault):
> 
> 	gdbm_open()	gdbmmodule.c:68 
> 	getaddrinfo()	socketmodule.c:636
> 
> Memory Leaks:
> 
> 	NONE

Woohoo!  So I'll close the remaining leak report!

> Potential Memory Leaks:
> 
> 	1012 from 4 different allocating points -- all in WeakRef

Any more info on these, or are these just red herrings?

> Things look great!

Agreed.

> Let me know if any of the skipped tests ought to have been run.
> Most likely possibility seems to be bsddb, sundry also needs bsddb.

Ah, that's why sundry fails.  I think it shouldn't depend on bsddb;
I'll look into that.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From neal@metaslash.com  Fri Dec  7 16:54:54 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 07 Dec 2001 11:54:54 -0500
Subject: [Python-Dev] update on memory leaks in 2.2
References: <3C10EB1C.9302CA8@metaslash.com> <200112071638.LAA12876@cj20424-a.reston1.va.home.com>
Message-ID: <3C10F45E.6476A376@metaslash.com>

Guido van Rossum wrote:
> 
> > Command line: (longexp crashes purify for some reason)
> >       ./python -E -tt ./Lib/test/regrtest.py -u network -x test_longexp.py
> >
> > Results:
> >       168 tests OK.
> >       1 test failed:
> >           test_socket_ssl
> >       15 tests skipped:
> >           test_al test_bsddb test_cd test_cl test_curses test_dl test_gl
> >           test_imgfile test_linuxaudiodev test_nis test_ntpath test_openpty
> >           test_sundry test_winreg test_winsound
> 
> A few of these skips weren't expected: test_sundry, test_openpty,
> test_bsddb.  What's missing to run those?

The platform is Sun Solaris 8 (2.8).  Purify 2002.

For bsddb, do I need to install something that isn't "standard"?

test test_openpty skipped -- No openpty() available.

> > Potential Memory Leaks:
> >
> >       1012 from 4 different allocating points -- all in WeakRef
> 
> Any more info on these, or are these just red herrings?

The details are below.

Neal
--
         PLK: 704 bytes potentially leaked in 16 blocks
         This memory was allocated from:
               malloc         [rtlib.o]
               _PyObject_GC_New [gcmodule.c:832]
               PyWeakref_NewRef [weakrefobject.c:37]
               weakref_ref    [_weakref.c:73]
               PyCFunction_Call [methodobject.c:101]
               eval_frame     [ceval.c:1989]
               PyEval_EvalCodeEx [ceval.c:2570]
               function_call  [funcobject.c:374]
               PyObject_Call  [abstract.c:1665]
               instancemethod_call [classobject.c:2276]
               PyObject_Call  [abstract.c:1665]
               PyEval_CallObjectWithKeywords [ceval.c:3034]
         Block of 44 bytes (16 times); last block at 0x18bf0a8


--

         PLK: 220 bytes potentially leaked in 5 blocks
         This memory was allocated from:
               malloc         [rtlib.o]
               _PyObject_GC_New [gcmodule.c:832]
               PyWeakref_NewRef [weakrefobject.c:37]
               add_subclass   [typeobject.c:2054]
               PyType_Ready   [typeobject.c:2026]
               type_new       [typeobject.c:1142]
               type_call      [typeobject.c:148]
               PyObject_Call  [abstract.c:1665]
               PyEval_CallObjectWithKeywords [ceval.c:3034]
               PyObject_CallFunction [abstract.c:1656]
               build_class    [ceval.c:3583]
               eval_frame     [ceval.c:1507]
         Block of 44 bytes (5 times); last block at 0xced9b8

--

         PLK: 44 bytes potentially leaked at 0xcb2ba8
         This memory was allocated from:
               malloc         [rtlib.o]
               _PyObject_GC_New [gcmodule.c:832]
               PyWeakref_NewRef [weakrefobject.c:37]
               add_subclass   [typeobject.c:2054]
               PyType_Ready   [typeobject.c:2026]
               type_new       [typeobject.c:1142]
               tp_new_wrapper [typeobject.c:2586]
               PyCFunction_Call [methodobject.c:69]
               PyObject_Call  [abstract.c:1665]
               do_call        [ceval.c:3247]
               eval_frame     [ceval.c:1984]
               PyEval_EvalCodeEx [ceval.c:2570]

--

         PLK: 44 bytes potentially leaked at 0xcb3470
         This memory was allocated from:
               malloc         [rtlib.o]
               _PyObject_GC_New [gcmodule.c:832]
               PyWeakref_NewRef [weakrefobject.c:37]
               add_subclass   [typeobject.c:2054]
               PyType_Ready   [typeobject.c:2026]
               type_new       [typeobject.c:1142]
               tp_new_wrapper [typeobject.c:2586]
               PyCFunction_Call [methodobject.c:69]
               PyObject_Call  [abstract.c:1665]
               slot_tp_new    [typeobject.c:3355]
               type_call      [typeobject.c:148]
               PyObject_Call  [abstract.c:1665]


From neal@metaslash.com  Fri Dec  7 16:58:53 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Fri, 07 Dec 2001 11:58:53 -0500
Subject: [Python-Dev] update on memory leaks in 2.2
References: <3C10EB1C.9302CA8@metaslash.com> <200112071638.LAA12876@cj20424-a.reston1.va.home.com>
Message-ID: <3C10F54D.F20C2C71@metaslash.com>

Guido van Rossum wrote:

> Ah, that's why sundry fails.  I think it shouldn't depend on bsddb;
> I'll look into that.

test_sundry passes with no leaks now.

Neal


From fdrake@acm.org  Fri Dec  7 18:01:36 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 7 Dec 2001 13:01:36 -0500
Subject: [Python-Dev] update on memory leaks in 2.2
In-Reply-To: <3C10F45E.6476A376@metaslash.com>
References: <3C10EB1C.9302CA8@metaslash.com>
 <200112071638.LAA12876@cj20424-a.reston1.va.home.com>
 <3C10F45E.6476A376@metaslash.com>
Message-ID: <15377.1024.829156.22274@grendel.zope.com>

Neal Norwitz's Purify writes:
 >          PLK: 704 bytes potentially leaked in 16 blocks
 >          This memory was allocated from:
 >                malloc         [rtlib.o]
 >                _PyObject_GC_New [gcmodule.c:832]
 >                PyWeakref_NewRef [weakrefobject.c:37]
 >                weakref_ref    [_weakref.c:73]
 >                PyCFunction_Call [methodobject.c:101]
 >                eval_frame     [ceval.c:1989]
 >                PyEval_EvalCodeEx [ceval.c:2570]
 >                function_call  [funcobject.c:374]
 >                PyObject_Call  [abstract.c:1665]
 >                instancemethod_call [classobject.c:2276]
 >                PyObject_Call  [abstract.c:1665]
 >                PyEval_CallObjectWithKeywords [ceval.c:3034]
 >          Block of 44 bytes (16 times); last block at 0x18bf0a8

  Free'd weakref objects are stored in a free list, so it makes sense
that these are labelled potential leaks.
  We could either give up using a free list for these, or we could
allocate blocks of these rather than allocating them individually.
The latter would allow still better performance and would reduce the
malloc overhead.  That change would be a higher risk than tossing the
free list, and tossing it would be a higher risk than keeping it this
close to the release.  But we're probably far enough away that any of
the options (no free list, blocking allocation, and leaving it alone)
are manageable.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From barry@zope.com  Fri Dec  7 17:20:22 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 7 Dec 2001 12:20:22 -0500
Subject: [Python-Dev] update on memory leaks in 2.2
References: <3C10EB1C.9302CA8@metaslash.com>
 <200112071638.LAA12876@cj20424-a.reston1.va.home.com>
 <3C10F45E.6476A376@metaslash.com>
 <15377.1024.829156.22274@grendel.zope.com>
Message-ID: <15376.64086.678564.584731@anthem.wooz.org>

Note that there are tons of memory that is validly still alive when
Python exists.  As we've seen, trying to clean all that up /can/ have
a big negative impact on shutdown.  So I wouldn't worry about them.

Insure labels them as (paraphrasing) "memory still allocated at exit",
distinct from memory leaked.  I think this is equivalent to Purify's
"potentially leaked" and "leaked" memory respectively.

Note that at least stringobject.c has a hook for clearing out all
those interned strings at program shutdown time, which I added to
clean up Insure's output.  When INTERN_STRINGS is defined, the extra
function _Py_ReleaseInternedStrings() is defined.  Then, in
Modules/main.c, in Py_Main(), if __INSURE__ is defined that function
is called, so this memory doesn't show up in the report.

It may be worthwhile generalizing this approach, and adding it to
other static long-lived data structures, simply as an aid to memory
debugging.  As long as it isn't enabled by default, that should be
fine.

-Barry


From guido@python.org  Fri Dec  7 17:28:15 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Dec 2001 12:28:15 -0500
Subject: [Python-Dev] update on memory leaks in 2.2
In-Reply-To: Your message of "Fri, 07 Dec 2001 13:01:36 EST."
 <15377.1024.829156.22274@grendel.zope.com>
References: <3C10EB1C.9302CA8@metaslash.com> <200112071638.LAA12876@cj20424-a.reston1.va.home.com> <3C10F45E.6476A376@metaslash.com>
 <15377.1024.829156.22274@grendel.zope.com>
Message-ID: <200112071728.MAA13868@cj20424-a.reston1.va.home.com>

> Neal Norwitz's Purify writes:
>  >          PLK: 704 bytes potentially leaked in 16 blocks
>  >          This memory was allocated from:
>  >                malloc         [rtlib.o]
>  >                _PyObject_GC_New [gcmodule.c:832]
>  >                PyWeakref_NewRef [weakrefobject.c:37]
>  >                weakref_ref    [_weakref.c:73]
>  >                PyCFunction_Call [methodobject.c:101]
>  >                eval_frame     [ceval.c:1989]
>  >                PyEval_EvalCodeEx [ceval.c:2570]
>  >                function_call  [funcobject.c:374]
>  >                PyObject_Call  [abstract.c:1665]
>  >                instancemethod_call [classobject.c:2276]
>  >                PyObject_Call  [abstract.c:1665]
>  >                PyEval_CallObjectWithKeywords [ceval.c:3034]
>  >          Block of 44 bytes (16 times); last block at 0x18bf0a8
> 
>   Free'd weakref objects are stored in a free list, so it makes sense
> that these are labelled potential leaks.
>   We could either give up using a free list for these, or we could
> allocate blocks of these rather than allocating them individually.
> The latter would allow still better performance and would reduce the
> malloc overhead.  That change would be a higher risk than tossing the
> free list, and tossing it would be a higher risk than keeping it this
> close to the release.  But we're probably far enough away that any of
> the options (no free list, blocking allocation, and leaving it alone)
> are manageable.

Thanks for the explanation!  The simplest way to avoid the Purify
"potential leak" complaints then might be to add a way to explicitly
zap this free list at Py_Finalize() time, like we do with other custom
free lists.  That should be a post-2.2 feature.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@home.com  Fri Dec  7 17:35:21 2001
From: tim.one@home.com (Tim Peters)
Date: Fri, 7 Dec 2001 12:35:21 -0500
Subject: [Python-Dev] update on memory leaks in 2.2
In-Reply-To: <15377.1024.829156.22274@grendel.zope.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEMHMLAA.tim.one@home.com>

[Fred L. Drake, Jr.]
>   Free'd weakref objects are stored in a free list, so it makes sense
> that these are labelled potential leaks.
>   We could either give up using a free list for these, or we could
> allocate blocks of these rather than allocating them individually.
> The latter would allow still better performance and would reduce the
> malloc overhead.

-0.  When we turn on pymalloc in 2.3, blocks will get allocated by magic,
and more space-efficiently than you're going to do by hand (it bites one
malloc overhead per 256KB).  pymalloc will likely be a little slower than a
wholly customized free list, but it will get all the memory benefit, and
unlike type-specific free lists allows memory to be recycled across types as
programs progress.

>  That change would be a higher risk than tossing the free list, and
> tossing it would be a higher risk than keeping it this close to the
> release.  But we're probably far enough away that any of the options
> (no free list, blocking allocation, and leaving it alone) are
> manageable.

There's another possibility:  as float objects do, add a module xxx_fini()
routine to clear the free list at shutdown.



From fdrake@acm.org  Fri Dec  7 18:33:06 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 7 Dec 2001 13:33:06 -0500
Subject: [Python-Dev] update on memory leaks in 2.2
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEMHMLAA.tim.one@home.com>
References: <15377.1024.829156.22274@grendel.zope.com>
 <LNBBLJKPBEHFEDALKOLCGEMHMLAA.tim.one@home.com>
Message-ID: <15377.2914.145010.889503@grendel.zope.com>

Tim Peters writes:
 > -0.  When we turn on pymalloc in 2.3, blocks will get allocated by magic,
 > and more space-efficiently than you're going to do by hand (it bites one
 > malloc overhead per 256KB).  pymalloc will likely be a little slower than a

  That's good enough for me!  I'll look forward to ripping the free
list out completely.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From guido@python.org  Fri Dec  7 17:44:04 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Dec 2001 12:44:04 -0500
Subject: [Python-Dev] update on memory leaks in 2.2
In-Reply-To: Your message of "Fri, 07 Dec 2001 12:20:22 EST."
 <15376.64086.678564.584731@anthem.wooz.org>
References: <3C10EB1C.9302CA8@metaslash.com> <200112071638.LAA12876@cj20424-a.reston1.va.home.com> <3C10F45E.6476A376@metaslash.com> <15377.1024.829156.22274@grendel.zope.com>
 <15376.64086.678564.584731@anthem.wooz.org>
Message-ID: <200112071744.MAA13971@cj20424-a.reston1.va.home.com>

> Insure labels them as (paraphrasing) "memory still allocated at exit",
> distinct from memory leaked.  I think this is equivalent to Purify's
> "potentially leaked" and "leaked" memory respectively.

I don't think so.  Insure (uselessly) lists *all* memory that's still
allocated and reachable at exit.  Purify only reports certain blocks
(in this case they were a bunch of weak refs) as potentially leaked.
I'm not sure what makes them potentially leaked, but it must be a
stronger condition than "still exists at exit".  As Fred mentioned,
this may have to do with the weaklist free list implementation.

> Note that at least stringobject.c has a hook for clearing out all
> those interned strings at program shutdown time, which I added to
> clean up Insure's output.  When INTERN_STRINGS is defined, the extra
> function _Py_ReleaseInternedStrings() is defined.  Then, in
> Modules/main.c, in Py_Main(), if __INSURE__ is defined that function
> is called, so this memory doesn't show up in the report.
> 
> It may be worthwhile generalizing this approach, and adding it to
> other static long-lived data structures, simply as an aid to memory
> debugging.  As long as it isn't enabled by default, that should be
> fine.

Most types that have a custom free list already have a cleanup routine
that's always called by Py_Finalize(); the weakref free list is
lacking one.  The interned list is special because it's generally
unsafe to clear it out when Py_Finalize() might be followed by
Py_Initialize() again -- that's why it's only done when running
Insure++.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Fri Dec  7 18:02:30 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 07 Dec 2001 13:02:30 -0500
Subject: [Python-Dev] update on memory leaks in 2.2
In-Reply-To: Your message of "Fri, 07 Dec 2001 12:56:51 EST."
 <3C1102E3.447B6D8B@metaslash.com>
References: <3C10EB1C.9302CA8@metaslash.com> <200112071638.LAA12876@cj20424-a.reston1.va.home.com> <3C10F45E.6476A376@metaslash.com> <200112071713.MAA13808@cj20424-a.reston1.va.home.com>
 <3C1102E3.447B6D8B@metaslash.com>
Message-ID: <200112071802.NAA14180@cj20424-a.reston1.va.home.com>

> Guido van Rossum wrote:
> 
> > What's a potential leak?  Could it be that these are in cycles?  The
> > reported blocks are all weak reference objects; most of then are
> > allocated by add_subclass(), which uses weak refs for the list of
> > subclasses of a given (new-style) class.  I suppose most weak refs are
> > in cycles (else there would be no need for weak refs).

Neal replied (in private email):

> From page 157 of this doc:
> 
> http://www.rational.com/support/documentation/manuals/unixsuites/pdf/purify/purify.pdf
> 
> A PLK message describes heap memory that you might have
> leaked. You have pointers only to the middle of the region. PLK is a
> warning message.
> 
> Memory in use can sometimes appear as a PLK if the pointer
> returned by malloc is offset. A common cause is referencing a
> substring within a large string. Another example is when a
> pointer to a C++ object is cast to the second or later base class of a
> multiply-inherited object. It is offset past the other base class
> objects.
> 
> Truly leaked memory can sometimes appear as a PLK, if some
> non-pointer integer within the program space, when interpreted
> as a pointer, points within an otherwise leaked block of memory.
> This is rather rare. Inspect the code to differentiate between these
> causes of PLK reports.

Thanks, that's very helpful!  Here's how I now understand what's
happened.  Weak references are garbage collected, which means that the
malloc pointer points 12 bytes *before* the start of the object as the
rest of Python knows about it.  Objects in the free list are linked
together via their object pointer, so they are only known via a
pointer into their "middle".  We don't see this warning for other
GC'ed objects, because they are in one of the collector's linked
lists, which point to the start of the GC header which is the
malloc'ed pointer.  But freed weak references are not known to the
collector!

So I'm confident that these are not leaks.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Sat Dec  8 06:30:53 2001
From: fdrake@acm.org (Fred L. Drake)
Date: Sat,  8 Dec 2001 01:30:53 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20011208063053.2367D286BC@beowolf.digicool.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

Another day of updates; various fixes and clarifications.


From fredrik@pythonware.com  Sun Dec  9 10:04:17 2001
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Sun, 9 Dec 2001 11:04:17 +0100
Subject: [Python-Dev] update on memory leaks in 2.2
References: <3C10EB1C.9302CA8@metaslash.com> <200112071638.LAA12876@cj20424-a.reston1.va.home.com> <3C10F45E.6476A376@metaslash.com> <15377.1024.829156.22274@grendel.zope.com>              <15376.64086.678564.584731@anthem.wooz.org>  <200112071744.MAA13971@cj20424-a.reston1.va.home.com>
Message-ID: <05ab01c18098$e2380130$ced241d5@hagrid>

guido wrote:
> I don't think so.  Insure (uselessly) lists *all* memory that's still
> allocated and reachable at exit.  Purify only reports certain blocks
> (in this case they were a bunch of weak refs) as potentially leaked.
> I'm not sure what makes them potentially leaked, but it must be a
> stronger condition than "still exists at exit".

purify runs a garbage collection algorithm on your program
to figure out if something has leaked.

iirc, a potential memory leak is when you no longer have a
pointer to the beginning of an allocated block, but there's a
valid pointer to somewhere inside the block.

a true leak is when you no longer point to the block (i.e when
even a conservative garbage collector could safely destroy it).

</F>




From tim.one@home.com  Mon Dec 10 00:06:13 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 9 Dec 2001 19:06:13 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <3C0F1A6D.417B603C@ActiveState.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEDDMMAA.tim.one@home.com>

[David Ascher]
>   - popen on Windows XP seems problematic (needs a repro test there --
> not analyzed yet)

We (PythonLabs) don't have XP, so you're on your own for tracking it down.
popen() fails in more than one bizarre way on Win9X, and always has, and
likely always will.



From tim.one@home.com  Mon Dec 10 00:11:43 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 9 Dec 2001 19:11:43 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <200112060727.fB67RXZ08417@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEDEMMAA.tim.one@home.com>

[Martin v. Loewis]
> :-) I personally don't see it as a problem that bug reports are piling
> up. The GCC GNATS has 1341 "open" bug reports (meaning either "open"
> or "analyzed") at the moment. bugzilla.mozilla.org reports 19611 bugs
> in the "open" categories. Microsoft reportedly has over 60000 open
> reports for Windows 2000. Neither of these groups despair :-)

If any language I worked on previously had a few hundred open bug reports
(not counting feature requests), we would have shut the group down and
pursued a new implementation.  If you can get the open bugs to fit on one
screen, it tends to stay that way; once it spills over a screen, it appears
there's no bound on how many additional screens are tolerated (0, 1,
infinity).  I'm doing my part by reducing my font size and buying a much
larger monitor <wink>.



From jason@crash.org  Mon Dec 10 00:30:00 2001
From: jason@crash.org (Jason L. Asbahr)
Date: Sun, 9 Dec 2001 18:30:00 -0600
Subject: [Python-Dev] win32all
Message-ID: <EIEFLCFECLLBKGPNJJIMAEJHHLAA.jason@crash.org>

Any thoughts on making this package more prominent on the Python.org
download
pages?  Like under the primary Python 2.1.1 and Python 2.2 download links?



______________________________________________________________________
Jason Asbahr
jason@asbahr.com





From guido@python.org  Mon Dec 10 01:14:26 2001
From: guido@python.org (Guido van Rossum)
Date: Sun, 09 Dec 2001 20:14:26 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: Your message of "Sun, 09 Dec 2001 19:11:43 EST."
 <LNBBLJKPBEHFEDALKOLCIEDEMMAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCIEDEMMAA.tim.one@home.com>
Message-ID: <200112100114.UAA11756@cj20424-a.reston1.va.home.com>

> [Martin v. Loewis]
> > :-) I personally don't see it as a problem that bug reports are piling
> > up. The GCC GNATS has 1341 "open" bug reports (meaning either "open"
> > or "analyzed") at the moment. bugzilla.mozilla.org reports 19611 bugs
> > in the "open" categories. Microsoft reportedly has over 60000 open
> > reports for Windows 2000. Neither of these groups despair :-)

[Tim]
> If any language I worked on previously had a few hundred open bug reports
> (not counting feature requests), we would have shut the group down and
> pursued a new implementation.

It depends on what the bugs are.  Many bugs in our tracker are
intractable because we don't have the platform where it occurs or the
user didn't supply enough information to reproduce it.  I imagine the
GCC bugs are much of the same, just more (because GCC is more complex
and wider used than Python -- most GCC users are even more clueless
about GCC's implementation than Python users are about Python's
implementation).

I try to close bugs from clueless users if there's no response to a
request for more info, but often something that might be a bug but
isn't easily reproduced stays open for many many months.

So I'm not sure that it's realistic to keep all bugs on a page.

I did my part against bugs without someone to ask for more info by
forbidding anonymous bugs, recently.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido@python.org  Mon Dec 10 03:05:03 2001
From: guido@python.org (Guido van Rossum)
Date: Sun, 09 Dec 2001 22:05:03 -0500
Subject: [Python-Dev] win32all
In-Reply-To: Your message of "Sun, 09 Dec 2001 18:30:00 CST."
 <EIEFLCFECLLBKGPNJJIMAEJHHLAA.jason@crash.org>
References: <EIEFLCFECLLBKGPNJJIMAEJHHLAA.jason@crash.org>
Message-ID: <200112100305.WAA12125@cj20424-a.reston1.va.home.com>

> Any thoughts on making this package more prominent on the Python.org
> download
> pages?  Like under the primary Python 2.1.1 and Python 2.2 download links?

Good idea.  Done for 1.5.2, 2.0.1 and 2.1.1.  Not for 2.2 yet, since
there's no version of win32all for 2.2 available AFAIK.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@home.com  Mon Dec 10 03:44:31 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 9 Dec 2001 22:44:31 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <200112100114.UAA11756@cj20424-a.reston1.va.home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEDMMMAA.tim.one@home.com>

[Tim]
> If any language I worked on previously had a few hundred open
> bug reports (not counting feature requests), we would have shut the
> group down and pursued a new implementation.

[Guido]
> It depends on what the bugs are.

Actually, it would not have:  a backlog of that many bugs (regardless of
nature) would mean they'd never get fixed -- it's hopeless.  Time then to
cut losses.  The compilers I worked on previously sold multi-million dollar
machines, though, and had to work only with our own OS(es) and libraries.
Python's life is more complicated than that -- and Python doesn't have a few
hundred open bugs anyway (there are only 183 right now <wink>).  The Python
bug base also has gripes about everything from docs to python-mode.el in it.

> Many bugs in our tracker are intractable because we don't have the
> platform where it occurs or the user didn't supply enough information
> to reproduce it.

Those should be closed after a reasonable time:  nobody is going to work on
them.  OTOH, I don't think there are many *open* bugs of this ilk (I
routinely close such after a month, with "closed for lack of requested
followup" -- and nobody has complained about that).

> I imagine the GCC bugs are much of the same, just more (because GCC is
> more complex and wider used than Python -- most GCC users are even more
> clueless about GCC's implementation than Python users are about Python's
> implementation).

My only interaction with the gcc bug base was helping to whittle down oodles
of bad codegen cases on new ports.  I have no idea what else may be in
there.

> I try to close bugs from clueless users if there's no response to a
> request for more info, but often something that might be a bug but
> isn't easily reproduced stays open for many many months.

I don't think there are many open bugs of that ilk either.

> So I'm not sure that it's realistic to keep all bugs on a page.

We were close once.  Now the open patches don't even fit on a page.

> I did my part against bugs without someone to ask for more info by
> forbidding anonymous bugs, recently.

Appreciated, but those were the easiest to close <0.9 wink>.

the-things-that-get-done-are-the-things-that-get-worked-on-ly y'rs  - tim



From tim.one@home.com  Mon Dec 10 05:22:15 2001
From: tim.one@home.com (Tim Peters)
Date: Mon, 10 Dec 2001 00:22:15 -0500
Subject: [Python-Dev] Subclassing varying length types (What's a    PyStructSequence ?)
In-Reply-To: <3C0B4152.3739F4EA@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEECMMAA.tim.one@home.com>

[MAL]
> Have you tried disabling all free list and using pymalloc
> instead ?

No, but I haven't tried anything -- it's a 2.3 issue.

> If this pays off, I agree, we should get rid off all of them.

When I do try it <wink>, it will be slower but more memory-efficient (both
data and code) than the type-specific free lists, and faster and much more
memory-efficient than using malloc().

> ...
> I would consider moving from 8-bit strings to Unicode an
> improvement in flexibility.

Sure.  Moving from one malloc to two is orthogonal.

> It also results in better algroithms (== simpler, less error-prone,
> etc. in this case).

Unclear what "it" means; assuming it means using two mallocs instead of one
for a Unicode string object, the 8-bit string algorithms haven't been a
particular source of bugs.  People mutating strings at the C level has been.

> As I said, it's a tradeoff flexibility vs. memory consumption.
> Whether it pays off depends on your application environment. It
> certainly does for companies like Micron and pays off stock-wise
> for a lot of people... uhm, getting off-topic here :-)

I've got nothing against Unicode (apart from the larger issue that the whole
world would obviously be a lot better off if they switched to American
English <wink>).

>> Subclassing seems easy enough to me from the Python level; I
>> don't have time to revisit C-level subclasssing here (and I don't
>> know that it's hackish there either, but do think it's in need of
>> docs).

> It is beautifully easy for non-varying-length types. Unfortunately,
> it happens that some of the basic types which would be attractive
> for subclassing are varying length types (such as string and
> tuples).

It's easy to subclass from str and tuple in Python -- even to add your own
instance data.

> In my case, I'm looking for away to subclass strings, but I haven't
> yet found an elegant solution to the problem of adding extra
> data to the instances.

It's easy if you're willing to use a dict:

class STR(str):
     def __new__(cls, strguts, n):
         self = str.__new__(cls, strguts)
         self.n = n
         return self

s = STR('abc', 42)
print s    # abc
print s.n  # 42

__slots__ doesn't work here, though.

I admit I personally don't see much attraction to subclassing from str and
tuple, apart from adding additional *methods*.  I suppose someone could code
up two-malloc variants ...



From fredrik@pythonware.com  Mon Dec 10 08:26:52 2001
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Mon, 10 Dec 2001 09:26:52 +0100
Subject: [Python-Dev] memory leaks in 2.2
References: <LNBBLJKPBEHFEDALKOLCIEDEMMAA.tim.one@home.com>
Message-ID: <00b701c18155$9b4bbd80$ced241d5@hagrid>

tim wrote:
> If any language I worked on previously had a few hundred open bug reports
> (not counting feature requests).

btw, what's the right way to flag something as a feature
request?

both the data type and the group fields can be set to "feature
request", but I'm not allowed to change the former.

(and all my feature requests are listed under "Python - Bugs")

</F>



From martin@v.loewis.de  Mon Dec 10 08:24:29 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Mon, 10 Dec 2001 09:24:29 +0100
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEDMMMAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCOEDMMMAA.tim.one@home.com>
Message-ID: <200112100824.fBA8OTa01801@mira.informatik.hu-berlin.de>

> My only interaction with the gcc bug base was helping to whittle down oodles
> of bad codegen cases on new ports.  I have no idea what else may be in
> there.

Many of them are of this kind. As a C++ person, I notice also (with
regrets) that a significant portion is of the kind "C++ compiler
rejects valid code, or crashes". They are easy to reproduce, so they
probably would have caused your former companies to throw away the C++
front-end and rewrite it from scratch. In g++, this won't happen:
nobody has the resources to rewrite it from scratch. And even with
these bugs, g++ still is much better than the competition :-)

> > So I'm not sure that it's realistic to keep all bugs on a page.
> 
> We were close once.  Now the open patches don't even fit on a page.

I think the only time we were close was when the database was new.

Regards,
Martin


From martin@v.loewis.de  Mon Dec 10 08:34:19 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Mon, 10 Dec 2001 09:34:19 +0100
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <00b701c18155$9b4bbd80$ced241d5@hagrid> (fredrik@pythonware.com)
References: <LNBBLJKPBEHFEDALKOLCIEDEMMAA.tim.one@home.com> <00b701c18155$9b4bbd80$ced241d5@hagrid>
Message-ID: <200112100834.fBA8YJ901851@mira.informatik.hu-berlin.de>

> btw, what's the right way to flag something as a feature
> request?

I'd prefer to see them in the Feature Requests tracker. The Feature
Requests group predates the separate feature requests tracker; if they
are in a separate tracker, people won't have to go over them again and
again

> both the data type and the group fields can be set to "feature
> request", but I'm not allowed to change the former.

At the moment, it might be fastest if you post the numbers, so I can
reclassify them. Alternatively, someone with appropriate powers could
give you write access to the feature requests tracker, and the data
type field (I think you need to be tracker admin for that).

Regards,
Martin


From mal@lemburg.com  Mon Dec 10 09:42:59 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 10 Dec 2001 10:42:59 +0100
Subject: [Python-Dev] on the subject of SF...
References: <200112060745.fB67j4e18189@mbuna.arbhome.com.au> <15375.8942.722195.533597@anthem.wooz.org> <3C0F2EE0.BE1D2980@lemburg.com> <15375.33371.540885.525907@anthem.wooz.org> <2mu1v33tta.fsf@starship.python.net>
Message-ID: <3C1483A3.AF10032F@lemburg.com>

Michael Hudson wrote:
> 
> barry@zope.com (Barry A. Warsaw) writes:
> 
> > >>>>> "M" == M  <mal@lemburg.com> writes:
> >
> >     M> And please also a Group "Python 2.3" so that feature requests
> >     M> for that version can be assigned to it ?!
> >
> > So far we have no groups defined for the feature request tracker.  I
> > wonder if that's intentional.
> 
> Does anybody look at the feature request tracker ever?  I notice I'm
> one of four possible assignees (presumably 'cause I'm new).  Does
> email from it go anywhere?

I was actually thinking of the bug tracker. I saw two group entries
there for specific Python versions and thought it would be nice
to use the group as feature request target specifier.

Nevermind,
-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From mal@lemburg.com  Mon Dec 10 09:44:50 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 10 Dec 2001 10:44:50 +0100
Subject: [Python-Dev] tracker admin
References: <200112071626.LAA09993@cj20424-a.reston1.va.home.com>
Message-ID: <3C148412.3FA26450@lemburg.com>

Guido van Rossum wrote:
> 
> I just discovered a few new (?) features of the bug/patch tracker
> admin page.
> 
> - You can disable anonymous submission.  I've done this for all our
>   trackers (bugs, patches, features), since we have way too many
>   problems with anonymous submissions where the user doesn't provide
>   enough info and we have no way to get back to them.
> 
> - You can provide custom text for the "submit new" page.  I've added a
>   warning about the annoying upload checkbox, and some additional
>   suggestions for patches.  Please feel free to provide additional
>   suggestions.

That's great news !

Both of these will help a lot with the PSF discussions we recently
had about how to legally deal with submissions.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From fredrik@pythonware.com  Mon Dec 10 10:16:38 2001
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Mon, 10 Dec 2001 11:16:38 +0100
Subject: [Python-Dev] memory leaks in 2.2
References: <LNBBLJKPBEHFEDALKOLCOEDMMMAA.tim.one@home.com> <200112100824.fBA8OTa01801@mira.informatik.hu-berlin.de>
Message-ID: <011e01c18163$d61f3780$ced241d5@hagrid>

martin wrote:
> ...rewrite it from scratch.

more likely, buy one from EDG:

    http://www.edg.com/customers.html

</F>



From mal@lemburg.com  Mon Dec 10 10:57:10 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 10 Dec 2001 11:57:10 +0100
Subject: [Python-Dev] Subclassing varying length types (What's a
 PyStructSequence ?)
References: <LNBBLJKPBEHFEDALKOLCOEECMMAA.tim.one@home.com>
Message-ID: <3C149506.A5689065@lemburg.com>

Tim Peters wrote:
> 
> [MAL]
> > Have you tried disabling all free list and using pymalloc
> > instead ?
> 
> No, but I haven't tried anything -- it's a 2.3 issue.
> 
> > If this pays off, I agree, we should get rid off all of them.
> 
> When I do try it <wink>, it will be slower but more memory-efficient (both
> data and code) than the type-specific free lists, and faster and much more
> memory-efficient than using malloc().

Well, let's do some pybench runs next year and see what the results
look like.
 
> > ...
> > I would consider moving from 8-bit strings to Unicode an
> > improvement in flexibility.
> 
> Sure.  Moving from one malloc to two is orthogonal.

You know that I know that you knew what I was talking about :-)
 
> > It also results in better algroithms (== simpler, less error-prone,
> > etc. in this case).
> 
> Unclear what "it" means; assuming it means using two mallocs instead of one
> for a Unicode string object, the 8-bit string algorithms haven't been a
> particular source of bugs.  People mutating strings at the C level has been.

If you ever try to support more than ASCII text in a user program,
you'll find that having to deal with only one encoding safes you
a whole lot of trouble. I won't even start talking about variable
length encodings, encodings with builtin shift state and other
goodies which are a complete nightmare to handle (e.g. various
character properties such as title case, upper/lower mappings,
different ways to encode a single character, collation,...).
 
> > As I said, it's a tradeoff flexibility vs. memory consumption.
> > Whether it pays off depends on your application environment. It
> > certainly does for companies like Micron and pays off stock-wise
> > for a lot of people... uhm, getting off-topic here :-)
> 
> I've got nothing against Unicode (apart from the larger issue that the whole
> world would obviously be a lot better off if they switched to American
> English <wink>).

I suppose Mandarin would reach a larger share in world 
population ... and they *need* Unicode :-)
 
> >> Subclassing seems easy enough to me from the Python level; I
> >> don't have time to revisit C-level subclasssing here (and I don't
> >> know that it's hackish there either, but do think it's in need of
> >> docs).
> 
> > It is beautifully easy for non-varying-length types. Unfortunately,
> > it happens that some of the basic types which would be attractive
> > for subclassing are varying length types (such as string and
> > tuples).
> 
> It's easy to subclass from str and tuple in Python -- even to add your own
> instance data.

Yeah, but that's not the point. I want to do this in C...
 
> > In my case, I'm looking for away to subclass strings, but I haven't
> > yet found an elegant solution to the problem of adding extra
> > data to the instances.
> 
> It's easy if you're willing to use a dict:

I would be willing to use a dictionary. It's only that even the
dictionary trick doesn't seem to work at C level.
 
> class STR(str):
>      def __new__(cls, strguts, n):
>          self = str.__new__(cls, strguts)
>          self.n = n
>          return self
> 
> s = STR('abc', 42)
> print s    # abc
> print s.n  # 42
> 
> __slots__ doesn't work here, though.
> 
> I admit I personally don't see much attraction to subclassing from str and
> tuple, apart from adding additional *methods*.  I suppose someone could code
> up two-malloc variants ...

If you look at mxURL you'll find an extension type which tries
to play nice with strings -- it would be a good candidate for
a string subtype.

A string type which carries along an encoding attribute would be
another good candidate for a string subtype.

Both need extra attributes/data fields.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From fdrake@acm.org  Mon Dec 10 15:14:27 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 10 Dec 2001 10:14:27 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <200112100824.fBA8OTa01801@mira.informatik.hu-berlin.de>
References: <LNBBLJKPBEHFEDALKOLCOEDMMMAA.tim.one@home.com>
 <200112100824.fBA8OTa01801@mira.informatik.hu-berlin.de>
Message-ID: <15380.53587.201337.1803@grendel.zope.com>

Martin v. Loewis writes:
 > I think the only time we were close was when the database was new.

  No; when the DB was new (but old enough to notice), it contained an
import of all the old JiterBug reports.  I think we had it pretty well
whittled down around the 2.0 release.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From jeremy@zope.com  Mon Dec 10 10:20:01 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Mon, 10 Dec 2001 05:20:01 -0500 (EST)
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <15380.53587.201337.1803@grendel.zope.com>
References: <LNBBLJKPBEHFEDALKOLCOEDMMMAA.tim.one@home.com>
 <200112100824.fBA8OTa01801@mira.informatik.hu-berlin.de>
 <15380.53587.201337.1803@grendel.zope.com>
Message-ID: <15380.35921.588329.420402@slothrop.digicool.com>

>>>>> "FLD" == Fred L Drake, <fdrake@acm.org> writes:

  FLD> Martin v. Loewis writes:
  >> I think the only time we were close was when the database was
  >> new.

  FLD>   No; when the DB was new (but old enough to notice), it
  FLD>   contained an
  FLD> import of all the old JiterBug reports.  I think we had it
  FLD> pretty well whittled down around the 2.0 release.

That's right.  We spent a lot of time fixing bugs prior to the 2.0
release.

Jeremy



From tim.one@home.com  Mon Dec 10 17:30:37 2001
From: tim.one@home.com (Tim Peters)
Date: Mon, 10 Dec 2001 12:30:37 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <00b701c18155$9b4bbd80$ced241d5@hagrid>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEGCMMAA.tim.one@home.com>

[Fredrik Lundh]
> btw, what's the right way to flag something as a feature
> request?
>
> both the data type and the group fields can be set to "feature
> request", but I'm not allowed to change the former.

After literally hundreds of mouse clicks, just about everyone should have
permission to do that now.  The "data type" field determines the tracker a
thing lives in, so that's the more important of the two to change.

Unfortunately, when you move a report, old URLs to the report become
invalid, and bring up a bizarre "invalide artifact ID" (something like that)
page.

> (and all my feature requests are listed under "Python - Bugs")

Then I'm going to raise them all to priority 9 <wink>.



From tim.one@home.com  Mon Dec 10 17:44:22 2001
From: tim.one@home.com (Tim Peters)
Date: Mon, 10 Dec 2001 12:44:22 -0500
Subject: Feature Requests (RE: [Python-Dev] memory leaks in 2.2)
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEGCMMAA.tim.one@home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEGDMMAA.tim.one@home.com>

The SF Feature Request tracker didn't have any Categories defined.  I've now
added the same Categories as are available in the Bug tracker:

Build
Demos and Tools
Distutils
Documentation
Extension Modules
IDLE
Installation
Macintosh
Parser/Compiler
Python Interpreter Core
Python Library
Regular Expressions
Threads
Tkinter
Type/class unification
Unicode
Windows
XML


Caution:  it *could* be that when you change trackers, the Category is lost;
you may have to set it again after it's moved (this was certainly the case
when the FR tracker didn't have any Categories).



From tim.one@home.com  Tue Dec 11 04:31:39 2001
From: tim.one@home.com (Tim Peters)
Date: Mon, 10 Dec 2001 23:31:39 -0500
Subject: [Python-Dev] memory leaks in 2.2
In-Reply-To: <200112100824.fBA8OTa01801@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEJGMMAA.tim.one@home.com>

[Tim]
>> My only interaction with the gcc bug base was helping to
>> whittle down oodles of bad codegen cases on new ports. ...

[Martin v. Loewis]
> Many of them are of this kind. As a C++ person, I notice also (with
> regrets) that a significant portion is of the kind "C++ compiler
> rejects valid code, or crashes". They are easy to reproduce, so they
> probably would have caused your former companies to throw away the C++
> front-end and rewrite it from scratch.

As Fredrik suggested, we bought front ends whenever possible.  When a single
machine sells for millions, and you can buy a highly regarded front end for
a few percent of that, you'd be crazy to write your own.

> In g++, this won't happen:  nobody has the resources to rewrite it from
> scratch. And even with these bugs, g++ still is much better than the
> competition :-)

I'd be more upset if anyone had proved that C++ is compilable <wink>.

[Guido]
>>> So I'm not sure that it's realistic to keep all bugs on a page.

>> We were close once.  Now the open patches don't even fit on a page.

> I think the only time we were close was when the database was new.

As Fred and Jeremy said, when we (PythonLabs) worked intensely on closing
bugs for 1.6 and 2.0, we were close to being in phenomenal shape here.  We
don't have a fraction of the internal resource to throw at that anymore,
though, and volunteers haven't taken up the slack.  So I figure the only
reason Python stays in exceptionally good bug shape is that we also ignore
the contributed patches <0.9 wink>.



From mal@lemburg.com  Tue Dec 11 10:04:55 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 11 Dec 2001 11:04:55 +0100
Subject: Feature Requests (RE: [Python-Dev] memory leaks in 2.2)
References: <LNBBLJKPBEHFEDALKOLCOEGDMMAA.tim.one@home.com>
Message-ID: <3C15DA47.FA5C39DA@lemburg.com>

Tim Peters wrote:
> 
> The SF Feature Request tracker didn't have any Categories defined.  I've now
> added the same Categories as are available in the Bug tracker:
> 
> Build
> Demos and Tools
> Distutils
> Documentation
> Extension Modules
> IDLE
> Installation
> Macintosh
> Parser/Compiler
> Python Interpreter Core
> Python Library
> Regular Expressions
> Threads
> Tkinter
> Type/class unification
> Unicode
> Windows
> XML

Would be nice if we could also have Groups "Python 2.3", "Python 3k",
etc.
 
> Caution:  it *could* be that when you change trackers, the Category is lost;
> you may have to set it again after it's moved (this was certainly the case
> when the FR tracker didn't have any Categories).

Changing the data type pretty much reset all fields for me (including 
the assigned field which is particularly annoying) and the URL 
becomes invalid too.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From martin@v.loewis.de  Tue Dec 11 10:12:00 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Tue, 11 Dec 2001 11:12:00 +0100
Subject: Feature Requests (RE: [Python-Dev] memory leaks in 2.2)
In-Reply-To: <3C15DA47.FA5C39DA@lemburg.com> (mal@lemburg.com)
References: <LNBBLJKPBEHFEDALKOLCOEGDMMAA.tim.one@home.com> <3C15DA47.FA5C39DA@lemburg.com>
Message-ID: <200112111012.fBBAC0I01773@mira.informatik.hu-berlin.de>

> Would be nice if we could also have Groups "Python 2.3", "Python 3k",
> etc.

I'm not sure. Feature requests are supposed to stay open until they
are either implemented, withdrawn, or rejected for good. If you want
to express that some requests are more important than others (so that
they ought to go into 2.3, instead of P3k), you should give them a
higher priority.

Regards,
Martin



From mal@lemburg.com  Tue Dec 11 11:00:14 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 11 Dec 2001 12:00:14 +0100
Subject: Feature Requests (RE: [Python-Dev] memory leaks in 2.2)
References: <LNBBLJKPBEHFEDALKOLCOEGDMMAA.tim.one@home.com> <3C15DA47.FA5C39DA@lemburg.com> <200112111012.fBBAC0I01773@mira.informatik.hu-berlin.de>
Message-ID: <3C15E73E.985DB93@lemburg.com>

"Martin v. Loewis" wrote:
> 
> > Would be nice if we could also have Groups "Python 2.3", "Python 3k",
> > etc.
> 
> I'm not sure. Feature requests are supposed to stay open until they
> are either implemented, withdrawn, or rejected for good. If you want
> to express that some requests are more important than others (so that
> they ought to go into 2.3, instead of P3k), you should give them a
> higher priority.

If you look at it this way, yes. It's just that setting the Group
to e.g. P3k would tell the submitter that the request is being 
considered, but won't get implemented until a few releases 
from now or that a specific request won't make it into e.g. 
Python 2.2 because of a feature freeze. Anyway, it's just an idea.

BTW, I also think that we could use the feature request tracker
to track all the PEP 42 requests and redirect PEP 42 to the
tracker.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/


From fdrake@acm.org  Tue Dec 11 22:16:01 2001
From: fdrake@acm.org (Fred L. Drake)
Date: Tue, 11 Dec 2001 17:16:01 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20011211221601.DB780286BC@cj42289-a.reston1.va.home.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

Many various updates, including documentation for generators and extended
call syntax.



From skip@pobox.com (Skip Montanaro)  Tue Dec 11 23:48:58 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Tue, 11 Dec 2001 17:48:58 -0600
Subject: [Python-Dev] new pubkey for SF
Message-ID: <15382.39786.533044.951881@beluga.mojam.com>

Folks,

Where do I send a new SSH pubkey to regain non-anonymous access to the CVS
repository?

Thx,

Skip



From tim.one@home.com  Wed Dec 12 00:31:57 2001
From: tim.one@home.com (Tim Peters)
Date: Tue, 11 Dec 2001 19:31:57 -0500
Subject: [Python-Dev] new pubkey for SF
In-Reply-To: <15382.39786.533044.951881@beluga.mojam.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEMCMMAA.tim.one@home.com>

> Where do I send a new SSH pubkey to regain non-anonymous access to the
> CVS repository?

Log in to SF.  From your "personal page", click on Account Options, then
down to Shell Account Information, then click Edit Keys.  It should be
obvious from there.  Beware the 2 (4? 8? 512?) hour synchronization delay.



From bizopp2002us@yahoo.com  Wed Dec 12 20:52:46 2001
From: bizopp2002us@yahoo.com (John McCuullen)
Date: Wed, 12 Dec 2001 21:52:46 +0100
Subject: [Python-Dev] The Secret to Supercharge your MLM!
Message-ID: <2191817-220011231220524660@yahoo.com>

Discover

"The Secret to Supercharge your MLM!"=20

A must read=2E=2E=2E=20


Hi there,=20
The secret is out!=20
Here's the mail we've all been waiting for! Read carefully
and take immediate action on it!=20
I've discovered an amazing formula that will give your
MLM an enormous enrollers explosion=2E You'll benefit
hugely if you use it with YOUR primary MLM!=20
-THE ULTIMATE RECRUITMENT FORMULA-=20
So if you are already in MLM or about to get into one
this will be the most important mail you will ever read=2E=20
Right now I know people who enroll dozens in their
downlines WEEKLY using the formula I'm about to
reveal=2E=20
Fred Stege, Founder of the company enrolled
127 people within 7 day's after launching! One of his
enrollees from this batch has become top recruiter in
his company within 90 days after using this incredible
easy-to-use concept=2E=20

The NEW Formula is meant as a Front End Program to
enhance "YOUR" MLM=2E I'm NOT pitching you in any
MLM=2E
____________________________________________
In fact you may try it for absolutely FREE and you'll
get an electrifying stack of tools straight away!
____________________________________________=20

We've discovered that the majority of Qualified MLM
Leads (Prospects) all have 1 thing in common=2E=2E=2E=20
They all are looking for 3 things they desperately want
and can't get in any one 100% generic System=2E=2E=2E=20
I just know that you'll agree with these 3 things and
they are all to your advantage!=20

Let me reveal you the 3 most sought after topics first:=20
=B7 Cutting Edge MLM Information=2E
=B7 A Legitimate Proven Formula=2E
=B7 Pre Qualified Leads=2E (And how to get them for FREE)=20

Isn't that what everybody is looking for?=2E=2E
Will these topics ever go out of style?=2E=2E
Doesn't THAT Attract the Masses?=2E=2E=20
Does 'OLD SCHOOL MLM' attract the masses?=2E=2E=20

Discover "How you can EARN RESIDUAL INCOME
While Recruiting on Auto Pilot whether they join your
MLM or not!"=20

Welcome to 21st Century Massive Recruitment=2E=20
I encourage you to check the details and sign up
immediately=2E That means NOW!=20

For more info please mail to mccuullen47@zonnet=2Enl
and write your First Name + 'MORE INFO' in the subject line=2E
You'll receive further instructions immediately=2E=20

Sincerely,=20
John McCuullen

P=2ES=2E
Don't forget to write your FIRST NAME + 'MORE INFO'
In Subject line!=20

P=2EP=2ES=2E
On the given site you'll find a secret that every PRO
in our industry knows and uses=2E=2E=2E
It's called: S-L-L-G: Self Liquidating Lead Generation=2E
It will Create a Huge, Electrifying Surge in Your Cash Flow=2E=20

P=2EP=2EP=2ES=2E
Find out the NEW 3 foot rule in MLM=2E "Why Old School
MLM is nearly dead=2E"=20



From fredrik@pythonware.com  Thu Dec 13 02:23:56 2001
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Thu, 13 Dec 2001 03:23:56 +0100
Subject: [Python-Dev] what's the current plan for 2.2c1?
Message-ID: <005101c1837d$3d0fc980$ced241d5@hagrid>

the pep (and python.org) says december 12.

just curious /F



From fdrake@acm.org  Thu Dec 13 19:08:41 2001
From: fdrake@acm.org (Fred L. Drake)
Date: Thu, 13 Dec 2001 14:08:41 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20011213190841.2EF95286BC@cj42289-a.reston1.va.home.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

Updated the build instructions for extensions on Windows.


From barry@zope.com  Thu Dec 13 20:01:36 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 13 Dec 2001 15:01:36 -0500
Subject: [Python-Dev] Python 2.2rc1 - release branch created
Message-ID: <15385.2336.521466.6255@anthem.wooz.org>

I've created the Python 2.2rc1 release branch.  The CVS tag is
`r22rc1-branch'.  There's also a tag `r22rc1-fork' -- that's the point
on the trunk where the branch forks off.  So far the only change in
the branch is the version number.

As usual, no one except Guido and myself (or our authorized bots)
should make checkins to the branch.  Checkins to the trunk are okay
but discouraged, and generally will /not/ be merged into the release
branch unless you notify us (a comment in the checkin message would
suffice).

I'll be cutting the release tomorrow morning local time (Friday
December 14).  After the release is done, the branch will be merged
back into the trunk.

Please note that as this is a release candidate, only essential or
showstopper bugs should be fixed.  Definitely, no new features should
be added.  According to PEP 251, Python 2.2 final is currently
scheduled for 19-Dec-2001.  If you're unsure whether to make a change
or not, please contact us first.

Enjoy,
-Barry


From fdrake@acm.org  Thu Dec 13 20:02:45 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 13 Dec 2001 15:02:45 -0500
Subject: [Python-Dev] Doc/ tree closed for release
Message-ID: <15385.2405.102160.574585@grendel.zope.com>

  The Doc/ tree is now closed so I can prepare for the release.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From fdrake@acm.org  Fri Dec 14 04:25:14 2001
From: fdrake@acm.org (Fred L. Drake)
Date: Thu, 13 Dec 2001 23:25:14 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20011214042514.2204B286BC@beowolf.digicool.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

Final 2.2c1 documentation.


From barry@zope.com  Fri Dec 14 20:06:13 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 14 Dec 2001 15:06:13 -0500
Subject: [Python-Dev] RELEASED Python 2.2 release candidate 1 is out!
Message-ID: <15386.23477.137871.205510@anthem.wooz.org>

Today we release Python 2.2c1, the first and hopefully only release
candidate for Python 2.2, for your extravagation, eyestrain, and
elucidation.

Our thanks to everyone who helped test Python 2.2 during its alpha and
beta phases, and to everyone who contributed patches, feedback, and
suggestions!  Barring any showstopper bugs, we plan on releasing
Python 2.2 final in one week.  Please continue to report any bugs you
find to the bug tracker:

    http://sourceforge.net/bugs/?group_id=5470

Highlights of what's new for this release are outlined below.  For a
more complete list, please see:

    http://sf.net/project/shownotes.php?release_id=65583

Andrew Kuchling is writing a gentle introduction to the most important
changes, titled "What's New in Python 2.2":

  http://www.amk.ca/python/2.2/

Guido's written his own introduction to the type/class unification at:

  http://www.python.org/2.2/descrintro.html

Enjoy,
-Barry

-------------------- snip snip --------------------
What's New in Python 2.2c1?
Release date: 14-Dec-2001

- The "delete attribute" method of descriptor objects is called
  __delete__, not __del__.

- Some subtle issues with the super built-in were fixed.

- hash() of an instance of a subclass of mutable type now properly
  raises TypeError.

- New style objects now support deleting their __dict__.

- -Qnew now works as documented in PEP 238.

- complex() now only allows the first argument to be a string
  argument.

- gc.get_referents was renamed to gc.get_referrers.

- It is no longer necessary to use --with-suffix when building on a
  case-insensitive file system (such as Mac OS X HFS+).

- In unix-Python on Mac OS X (and darwin) sys.platform is now "darwin",
  without any trailing digits.


From barry@zope.com  Fri Dec 14 20:19:56 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 14 Dec 2001 15:19:56 -0500
Subject: [Python-Dev] RELEASED Python 2.2 release candidate 1 is out!
References: <15386.23477.137871.205510@anthem.wooz.org>
Message-ID: <15386.24300.351531.84724@anthem.wooz.org>

Sigh.  A cut and paste error omitted the download link from the
announcement.  As usual, it is

    http://www.python.org/2.2/

Apologies, and thanks MAL!

-Barry


From guido@python.org  Sat Dec 15 16:23:44 2001
From: guido@python.org (Guido van Rossum)
Date: Sat, 15 Dec 2001 11:23:44 -0500
Subject: [Python-Dev] mpz module broken with GMP 4.0
Message-ID: <200112151623.LAA32158@cj20424-a.reston1.va.home.com>

See SF bug #493631:

http://sourceforge.net/tracker/index.php?func=detail&aid=493631&group_id=5470&atid=105470

The mpz modules doesn't work with GMP 4.0 (which is brand new --
http://www.swox.com/gmp/ lists Dec 1, 2001 as the release date).

It looks like the mpz module is a mess.  The setup.py file contains a
reference to pympz.sourceforge.net, but that doesn't exist.

There is, however, a SF project gmpy, which seems to be a Python
wrapper for GMP 3.0 (does it work with GMP 4.0?).  I'm cc'ing the
project admins, maybe they can shed light on this.

Does anybody know how to fix mpzmodule.c?  Or should we just get rid
of it?  Should we at least fix the reference in setup.py to point to
gmpy?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Sat Dec 15 16:43:45 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sat, 15 Dec 2001 11:43:45 -0500
Subject: [Python-Dev] regression tests with regrtest.py -r
Message-ID: <15387.32193.854351.323371@grendel.zope.com>

  On a lark, I decided to try running the regression tests with -r,
which tells regrtest.py to run the tests in random order.  With this,
I get the following warnings from test_long.py:

../Lib/test/test_long.py:266: OverflowWarning: integer addition
  special = [0, 1, 2, 3, sys.maxint-1, sys.maxint, sys.maxint+1]
../Lib/test/test_long.py:287: OverflowWarning: integer addition
  got = x + y
../Lib/test/test_long.py:291: OverflowWarning: integer subtraction
  got = x - y
../Lib/test/test_long.py:295: OverflowWarning: integer multiplication
  got = x * y
../Lib/test/test_long.py:313: OverflowWarning: integer exponentiation
  got = x ** y
../Lib/test/test_long.py:320: OverflowWarning: integer exponentiation
  got = pow(x, y, z)
../Lib/test/test_long.py:377: OverflowWarning: integer exponentiation
  value = 10 ** exp

  This was with the CVS trunk, which is still virtually identical to
the 2.2c1 release.
  I won't have time to look at them this weekend, but this should be
fixed.  It may be possible there are other order-dependent behaviors,
so it would be really good if people who have time can run the tests
with -r on whatever platforms are available and report any other
anomolies.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From guido@python.org  Sat Dec 15 18:01:07 2001
From: guido@python.org (Guido van Rossum)
Date: Sat, 15 Dec 2001 13:01:07 -0500
Subject: [Python-Dev] regression tests with regrtest.py -r
In-Reply-To: Your message of "Sat, 15 Dec 2001 11:43:45 EST."
 <15387.32193.854351.323371@grendel.zope.com>
References: <15387.32193.854351.323371@grendel.zope.com>
Message-ID: <200112151801.NAA02181@cj20424-a.reston1.va.home.com>

>   On a lark, I decided to try running the regression tests with -r,

Good idea!

> which tells regrtest.py to run the tests in random order.  With this,
> I get the following warnings from test_long.py:
> 
> ../Lib/test/test_long.py:266: OverflowWarning: integer addition
>   special = [0, 1, 2, 3, sys.maxint-1, sys.maxint, sys.maxint+1]
[...]
> 
>   This was with the CVS trunk, which is still virtually identical to
> the 2.2c1 release.
>   I won't have time to look at them this weekend, but this should be
> fixed.  It may be possible there are other order-dependent behaviors,
> so it would be really good if people who have time can run the tests
> with -r on whatever platforms are available and report any other
> anomolies.

I've found the cause: test_scope.py contains a call to
warnings.resetwarnings(), which removes *all* warning filters, not
just those made by the calling module.  Because of the way
OverflowWarning is suppressed, this removes the "preset" filter that
suppresses OverflowWarning.

Since an API redesign before the 2.2 release is out of the question,
I'm fixing this by removing the resetwarnings() call from
test_scope.py and making the filterwarnings() call there a little more
specific.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aleax@aleax.it  Sat Dec 15 18:23:26 2001
From: aleax@aleax.it (Alex Martelli)
Date: Sat, 15 Dec 2001 19:23:26 +0100
Subject: [Python-Dev] Re: mpz module broken with GMP 4.0
In-Reply-To: <200112151623.LAA32158@cj20424-a.reston1.va.home.com>
References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com>
Message-ID: <0112151923261C.01556@arthur>

On Saturday 15 December 2001 17:23, Guido van Rossum wrote:
> See SF bug #493631:
>
> http://sourceforge.net/tracker/index.php?func=detail&aid=493631&group_id=
>5470&atid=105470
>
> The mpz modules doesn't work with GMP 4.0 (which is brand new --
> http://www.swox.com/gmp/ lists Dec 1, 2001 as the release date).
>
> It looks like the mpz module is a mess.  The setup.py file contains a
> reference to pympz.sourceforge.net, but that doesn't exist.
>
> There is, however, a SF project gmpy, which seems to be a Python
> wrapper for GMP 3.0 (does it work with GMP 4.0?).  I'm cc'ing the
> project admins, maybe they can shed light on this.

Thanks for the indication -- I was not aware of the new GMP 4.0.  A rapid 
test shows the current gmpy.c, release 0.8, breaks in exactly one place 
(structure gmpstate_t used to have a member named seed, now renamed to 
_mp_seed).  Once that single line is fixed, gmpy 0.8 builds fine (on a 
Linux box) with the new GMP 4.0 and passes all of its 973 unit-tests (with 
Python 2.2b1).  I think we should rapidly release a 0.8.1 with the one-line 
fix (arranged so it keeps building with GMP 3.1 while also building with 
4.0, of course), and plan the modest gmpy extensions needed to exploit 
4.0's novelties later (it's likely that little or no work is needed for 
that, since 4.0 seems to have mostly relaxed previous argument constraints, 
and gmpy generally doesn't duplicate GMP's own argument validity tests).

Having gmpy fully exploit Python 2.2 is a longer-term objective (expose all 
types and make them inheritable).


Alex


From guido@python.org  Sat Dec 15 18:31:45 2001
From: guido@python.org (Guido van Rossum)
Date: Sat, 15 Dec 2001 13:31:45 -0500
Subject: [Python-Dev] Re: mpz module broken with GMP 4.0
In-Reply-To: Your message of "Sat, 15 Dec 2001 19:23:26 +0100."
 <0112151923261C.01556@arthur>
References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com>
 <0112151923261C.01556@arthur>
Message-ID: <200112151831.NAA02998@cj20424-a.reston1.va.home.com>

Alex, do you have a suggestion for what to do with mpzmodule.c in the
2.2 distribution?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Sat Dec 15 18:34:26 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sat, 15 Dec 2001 13:34:26 -0500
Subject: [Python-Dev] Re: mpz module broken with GMP 4.0
In-Reply-To: <0112151923261C.01556@arthur>
References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com>
 <0112151923261C.01556@arthur>
Message-ID: <15387.38834.552687.60342@grendel.zope.com>

Alex Martelli writes:
 > Thanks for the indication -- I was not aware of the new GMP 4.0.  A rapid 
 > test shows the current gmpy.c, release 0.8, breaks in exactly one place 

  I'm adding a link to the gmpy website from the mpz module
documentation.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From tim.one@home.com  Sat Dec 15 18:43:26 2001
From: tim.one@home.com (Tim Peters)
Date: Sat, 15 Dec 2001 13:43:26 -0500
Subject: [Python-Dev] regression tests with regrtest.py -r
In-Reply-To: <15387.32193.854351.323371@grendel.zope.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEHFMNAA.tim.one@home.com>

[Fred L. Drake, Jr.]
> ...
> It may be possible there are other order-dependent behaviors,

There are, but ...

> so it would be really good if people who have time can run the tests
> with -r on whatever platforms are available and report any other
> anomolies.

By far the most frequent cause of errors when using -r is bogus module
objects left behind in sys.modules on non-Unix platforms that can't import
some Unix-specific things.  A later attempt to import the same module again
doesn't fail, but since the module object is incomplete it fails for some
other reason when a test attempts to use the module.

I spent countless hours worming around this in the past, but lost heart for
it; on my Win9X box, the failing test has usually scrolled off the (50-line)
DOS box before regrtest completes, and so also the history of the test
ordering that caused it.  Heh -- -r failed in test___all__ while I was
typing this, but I've no idea why.  I'll pursue it until I get sick to death
of it <wink>.



From fdrake@acm.org  Sat Dec 15 18:46:16 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sat, 15 Dec 2001 13:46:16 -0500
Subject: [Python-Dev] regression tests with regrtest.py -r
In-Reply-To: <200112151801.NAA02181@cj20424-a.reston1.va.home.com>
References: <15387.32193.854351.323371@grendel.zope.com>
 <200112151801.NAA02181@cj20424-a.reston1.va.home.com>
Message-ID: <15387.39544.377965.395044@grendel.zope.com>

Guido van Rossum writes:
 > I've found the cause: test_scope.py contains a call to
 > warnings.resetwarnings(), which removes *all* warning filters, not
 > just those made by the calling module.  Because of the way

  This is the danger of running the tests the way we do (all in one
process), but that also makes us be more careful with the tests.
  I'll have to run with -r more often!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From mal@lemburg.com  Sat Dec 15 19:03:21 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 15 Dec 2001 20:03:21 +0100
Subject: [Python-Dev] Re: mpz module broken with GMP 4.0
References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com>	<0112151923261C.01556@arthur> <15387.38834.552687.60342@grendel.zope.com>
Message-ID: <3C1B9E79.1020202@lemburg.com>

Fred L. Drake, Jr. wrote:

> Alex Martelli writes:
>  > Thanks for the indication -- I was not aware of the new GMP 4.0.  A rapid 
>  > test shows the current gmpy.c, release 0.8, breaks in exactly one place 
> 
>   I'm adding a link to the gmpy website from the mpz module
> documentation.


You might also want to add a link to mxNumber which is another
wrapper for GMP. Haven't tried it with GMP 4.0 yet, but it works
great with 3.1.1 -- even on Windows.

	http://www.egenix.com/files/python/mxNumber.html

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/



From tim.one@home.com  Sat Dec 15 20:29:15 2001
From: tim.one@home.com (Tim Peters)
Date: Sat, 15 Dec 2001 15:29:15 -0500
Subject: [Python-Dev] regression tests with regrtest.py -r
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEHFMNAA.tim.one@home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEHJMNAA.tim.one@home.com>

Here's one cause of current -r failures on Windows:

C:\Code\python\PCbuild>python ../lib/test/regrtest.py \
                              test_pty test___all__
test_pty
test test_pty skipped -- No module named termios
test___all__
test test___all__ failed -- pty has no __all__ attribute
1 test failed:
    test___all__
1 test skipped:
    test_pty
Those skips are all expected on win32.


It's "the usual":  importing pty fails on Windows, but test_pty then leaves
behind a bogus pty module object in sys.modules, which fools test___all__.
Note that pty.py already has this atrocity from a previous round of -r
wrestling:

"""
# Absurd:  import termios and then delete it.  This is to force an attempt
# to import pty to raise an ImportError on platforms that lack termios.
# Without this explicit import of termios here, some other module may
# import tty first, which in turn imports termios and dies with an
# ImportError then.  But since tty *does* exist across platforms, that
# leaves a damaged module object for tty in sys.modules, and the import
# of tty here then appears to work despite that the tty imported is junk.
import termios
del termios

import tty
"""

test___all__ also tries to compensate for pty (among others) insanity:

"""
def check_all(modname):
    names = {}
    try:
        exec "import %s" % modname in names
    except ImportError:
        # Silent fail here seems the best route since some modules
        # may not be available in all environments.
        # Since an ImportError may leave a partial module object in
        # sys.modules, get rid of that first.  Here's what happens if
        # you don't:  importing pty fails on Windows because pty tries to
        # import FCNTL, which doesn't exist.  That raises an ImportError,
        # caught here.  It also leaves a partial pty module in sys.modules.
        # So when test_pty is called later, the import of pty succeeds,
        # but shouldn't.  As a result, test_pty crashes with an
        # AttributeError instead of an ImportError, and regrtest interprets
        # the latter as a test failure (ImportError is treated as "test
        # skipped" -- which is what test_pty should say on Windows).
        try:
            del sys.modules[modname]
        except KeyError:
            pass
        return
"""

I've added another hack to regrtest.py to worm around the test_pty followed
by test___all__ case (more generally, any test_xxx followed by test___all__
case, when test_xxx fails due to leaving behind a damaged module object for
xxx).  However, that still doesn't fix

    test_rlcompleter

followed by

    test___all__

on Windows.  In that case, test_rlcompleter leaves behind a damaged
rlcompleter module, because rlcompleter.py does exist on Windows (it fails
due to the lack of the readline module).

leaving-behind-incomplete-module-objects-still-seems-like-a-bad-idea-
    to-me-ly y'rs  - tim



From fdrake@acm.org  Sat Dec 15 20:36:42 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sat, 15 Dec 2001 15:36:42 -0500
Subject: [Python-Dev] Re: mpz module broken with GMP 4.0
In-Reply-To: <3C1B9E79.1020202@lemburg.com>
References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com>
 <0112151923261C.01556@arthur>
 <15387.38834.552687.60342@grendel.zope.com>
 <3C1B9E79.1020202@lemburg.com>
Message-ID: <15387.46170.620220.286507@grendel.zope.com>

M.-A. Lemburg writes:
 > You might also want to add a link to mxNumber which is another
 > wrapper for GMP. Haven't tried it with GMP 4.0 yet, but it works
 > great with 3.1.1 -- even on Windows.

  I'll have this checked in shortly -- thanks!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From fdrake@beowolf.digicool.com  Sat Dec 15 20:44:16 2001
From: fdrake@beowolf.digicool.com (Fred Drake)
Date: Sat, 15 Dec 2001 15:44:16 -0500 (EST)
Subject: [Python-Dev] [maintenance doc updates]
Message-ID: <20011215204416.B421F286BC@beowolf.digicool.com>

The development version of the documentation has been updated:

	http://python.sourceforge.net/maint-docs/

Miscellaneous minor updates/fixes.


From fdrake@acm.org  Sat Dec 15 20:46:40 2001
From: fdrake@acm.org (Fred L. Drake)
Date: Sat, 15 Dec 2001 15:46:40 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20011215204640.8FF5E286BC@beowolf.digicool.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

Miscellaneous minor updates/fixes post 2.2c1.


From aleax@aleax.it  Sat Dec 15 21:17:12 2001
From: aleax@aleax.it (Alex Martelli)
Date: Sat, 15 Dec 2001 22:17:12 +0100
Subject: [Python-Dev] Re: mpz module broken with GMP 4.0
In-Reply-To: <200112151831.NAA02998@cj20424-a.reston1.va.home.com>
References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com> <0112151923261C.01556@arthur> <200112151831.NAA02998@cj20424-a.reston1.va.home.com>
Message-ID: <0112152217121F.01556@arthur>

On Saturday 15 December 2001 19:31, Guido van Rossum wrote:
> Alex, do you have a suggestion for what to do with mpzmodule.c in the
> 2.2 distribution?

Remove it?  It appears to me that there's no maintenance of it, and either 
Lemburg's module, or gmpy, would be a better bet for Pythonistas needing 
GMP access, anyway.


Alex


From tim.one@home.com  Sat Dec 15 21:20:32 2001
From: tim.one@home.com (Tim Peters)
Date: Sat, 15 Dec 2001 16:20:32 -0500
Subject: [Python-Dev] regrtest.py mystery
Message-ID: <LNBBLJKPBEHFEDALKOLCAEHNMNAA.tim.one@home.com>

We do this after running every test:

        # Unload the newly imported modules (best effort finalization)
        for module in sys.modules.keys():
            if module not in save_modules and module.startswith("test."):
                test_support.unload(module)

Unfortunately, that doesn't nuke whatever damaged non-test module objects
may have been left behind.  If I change it to make a truly <wink> best
effort:

            if module not in save_modules:
                test_support.unload(module)

then about a dozen std tests fail in mysterious ways, but only when running
more than one test.  Like just running

    test_email test_string

causes test_string to die like so:

python ../lib/test/regrtest.py -v test_email test_string

...

test test_string crashed -- exceptions.AttributeError: 'NoneType' object
                            has no attribute 'get'
Traceback (most recent call last):
  File "../lib/test/regrtest.py", line 305, in runtest
    the_module = __import__(test, globals(), locals(), [])
  File "../lib/test\test_string.py", line 34, in ?
    string_tests.run_method_tests(test)
  File "../lib/test\string_tests.py", line 233, in run_method_tests
    verify('hello world'.encode('zlib') == data)
  File "C:\CODE\PYTHON\lib\encodings\__init__.py", line 43, in
search_function
    entry = _cache.get(encoding,_unknown)
AttributeError: 'NoneType' object has no attribute 'get'

It seems generally the case that these failures are due to various
module-level names getting rebound to None.  A simpler example:

python  ../lib/test/regrtest.py -v test_string test_string

...

test test_string crashed -- exceptions.AttributeError: 'NoneType' object
                            has no attribute 'compress'
Traceback (most recent call last):
  File "../lib/test/regrtest.py", line 305, in runtest
    the_module = __import__(test, globals(), locals(), [])
  File "../lib/test\test_string.py", line 34, in ?
    string_tests.run_method_tests(test)
  File "../lib/test\string_tests.py", line 233, in run_method_tests
    verify('hello world'.encode('zlib') == data)
  File "C:\CODE\PYTHON\lib\encodings\zlib_codec.py", line 25, in zlib_encode
    output = zlib.compress(input)
AttributeError: 'NoneType' object has no attribute 'compress'

What's up with that?  So far, they all seem to involve the encodings
directory ...



From martin@v.loewis.de  Sat Dec 15 22:36:39 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Sat, 15 Dec 2001 23:36:39 +0100
Subject: [Python-Dev] Re: mpz module broken with GMP 4.0
In-Reply-To: <0112152217121F.01556@arthur> (message from Alex Martelli on Sat,
 15 Dec 2001 22:17:12 +0100)
References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com> <0112151923261C.01556@arthur> <200112151831.NAA02998@cj20424-a.reston1.va.home.com> <0112152217121F.01556@arthur>
Message-ID: <200112152236.fBFMadK05615@mira.informatik.hu-berlin.de>

> Remove it?

I think this is unacceptable for 2.2, now that 2.2c1 carries the
module. It seems that it still works fine as before with older
versions of GMP, and it also seems that there are a few users out
there of the module.

If users care enough, somebody will contribute a patch. If they don't
care, we can deprecate it for 2.3, and remove it in 2.4 (following the
guidelines for language evolution).

Regards,
Martin


From martin@v.loewis.de  Sat Dec 15 22:52:17 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Sat, 15 Dec 2001 23:52:17 +0100
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEHNMNAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCAEHNMNAA.tim.one@home.com>
Message-ID: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de>

> What's up with that?  So far, they all seem to involve the encodings
> directory ...

Python/codecs.c holds search functions in _PyCodec_SearchPath, among
them encodings.__init__.search_function. It also imports "encodings"
the first time somebody invokes .encode/.decode on some kind of
string, but doesn't hold onto the module.

When the last reference to encodings.__init__ goes away, Python will
clear all globals - but codecs still holds a reference to
search_function, so that won't go away. Invoking search_function later
will cause the problem you see.

One solution would be to bind all globals used inside
encodings.__init__.search_function to the search_function, e.g. by
means of a class:

class GlobalSearchFunction:
  _cache =
  _unknown = 
  class CodecRegistryError(...):
  def search_function(self,encoding):

codecs.register(GlobalSearchFunction().search_function)

Of course, if this gets cleaned from sys.modules, nobody will be able
to catch CodecRegistryError anymore (since re-importing the module
will produce a different class).

Regards,
Martin



From fdrake@acm.org  Sat Dec 15 23:53:57 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sat, 15 Dec 2001 18:53:57 -0500
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de>
References: <LNBBLJKPBEHFEDALKOLCAEHNMNAA.tim.one@home.com>
 <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de>
Message-ID: <15387.58005.329451.71260@grendel.zope.com>

Martin v. Loewis writes:
 > Python/codecs.c holds search functions in _PyCodec_SearchPath, among
 > them encodings.__init__.search_function. It also imports "encodings"
 > the first time somebody invokes .encode/.decode on some kind of
 > string, but doesn't hold onto the module.

  Perhaps it should hold onto the module as well?  That would avoid it
getting cleaned up before possibly dependent modules.
  For the test case problem, perhaps there should be a list of
exceptional modules that don't get cleaned up by regrtest once they
appear.  encodings and warnings would probably both need to be on the
list.  This list should probably be in the sys module; it would also
be useful for the RollbackImporter in the unittestgui.py script that
comes with the separate PyUnit package.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From guido@python.org  Sun Dec 16 01:01:35 2001
From: guido@python.org (Guido van Rossum)
Date: Sat, 15 Dec 2001 20:01:35 -0500
Subject: [Python-Dev] Re: mpz module broken with GMP 4.0
In-Reply-To: Your message of "Sat, 15 Dec 2001 23:36:39 +0100."
 <200112152236.fBFMadK05615@mira.informatik.hu-berlin.de>
References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com> <0112151923261C.01556@arthur> <200112151831.NAA02998@cj20424-a.reston1.va.home.com> <0112152217121F.01556@arthur>
 <200112152236.fBFMadK05615@mira.informatik.hu-berlin.de>
Message-ID: <200112160101.UAA03490@cj20424-a.reston1.va.home.com>

> > Remove it?
> 
> I think this is unacceptable for 2.2, now that 2.2c1 carries the
> module. It seems that it still works fine as before with older
> versions of GMP, and it also seems that there are a few users out
> there of the module.
> 
> If users care enough, somebody will contribute a patch. If they don't
> care, we can deprecate it for 2.3, and remove it in 2.4 (following the
> guidelines for language evolution).

Let's mark it as deprecated in 2.2, and remove it in 2.3.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@home.com  Sun Dec 16 05:23:43 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 16 Dec 2001 00:23:43 -0500
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: <15387.58005.329451.71260@grendel.zope.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEIIMNAA.tim.one@home.com>

[Fred]
>   Perhaps it should hold onto the module as well?  That would avoid it
> getting cleaned up before possibly dependent modules.
>   For the test case problem, perhaps there should be a list of
> exceptional modules that don't get cleaned up by regrtest once they
> appear.  encodings and warnings would probably both need to be on the
> list.  This list should probably be in the sys module; it would also
> be useful for the RollbackImporter in the unittestgui.py script that
> comes with the separate PyUnit package.

I expect the problem is intractable.  Here's a revealing experiment:  add
this block to the start of test___all__.check_all():

    if modname in sys.modules:
        del sys.modules[modname]

That is, just prior to test___all__ trying to import a module, remove *only*
that module from sys.modules if it's already there.  Sounds pretty harmless,
right?  But on Windows, it causes 28 standard tests to fail (possibly more
on Linux):

28 tests failed:
    test___all__ test_difflib test_email test_fileinput test_fnmatch
    test_future test_gettext test_glob test_global test_htmllib
    test_htmlparser test_inspect test_mimetypes test_mmap test_os
    test_pyclbr test_re test_regex test_repr test_scope test_sre
    test_strftime test_strop test_sundry test_tokenize test_urllib2
    test_xmllib test_xmlrpc

Trying to figure which modules "hurt" turns out to be a nightmare.  Here's
the minimal set that allows the test suite to pass again on Windows, using
the default test order:

    if modname in sys.modules:
        if modname not in (
"copy_reg",
"os",
"sgmllib",
"sre",
"tempfile",
"tokenize",
"warnings",
):
            del sys.modules[modname]

Am I gonna check that in?  Fat chance <wink>.



From tim.one@home.com  Sun Dec 16 05:42:33 2001
From: tim.one@home.com (Tim Peters)
Date: Sun, 16 Dec 2001 00:42:33 -0500
Subject: [Python-Dev] regression tests with regrtest.py -r
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEHFMNAA.tim.one@home.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEIIMNAA.tim.one@home.com>

[Tim]
> ordering that caused it.  Heh -- -r failed in test___all__ while I was
> typing this, but I've no idea why.  I'll pursue it until I get
> sick to death of it <wink>.

In case anyone's waiting, stop -- I got sick to death of it, and won't be
checking in anything related.  (BTW, there is no test_rlcompleter, of
course; it's the combo of test_sundry and test___all__ that screws up (both
of those import rlcompleter))



From martin@v.loewis.de  Sun Dec 16 09:38:34 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Sun, 16 Dec 2001 10:38:34 +0100
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: <15387.58005.329451.71260@grendel.zope.com> (fdrake@acm.org)
References: <LNBBLJKPBEHFEDALKOLCAEHNMNAA.tim.one@home.com>
 <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com>
Message-ID: <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de>

>  > Python/codecs.c holds search functions in _PyCodec_SearchPath, among
>  > them encodings.__init__.search_function. It also imports "encodings"
>  > the first time somebody invokes .encode/.decode on some kind of
>  > string, but doesn't hold onto the module.
> 
>   Perhaps it should hold onto the module as well?  That would avoid it
> getting cleaned up before possibly dependent modules.

Re-importing encodings would still do funny things, then: It would
register itself once more, resulting in two search functions being
registered. codecs would hold one of the encodings modules, but not
the other. So if the second search function is ever invoked, you get
the same error.

Regards,
Martin


From skip@pobox.com (Skip Montanaro)  Sun Dec 16 15:11:31 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Sun, 16 Dec 2001 09:11:31 -0600
Subject: [Python-Dev] regression tests with regrtest.py -r
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEIIMNAA.tim.one@home.com>
References: <LNBBLJKPBEHFEDALKOLCCEHFMNAA.tim.one@home.com>
 <LNBBLJKPBEHFEDALKOLCOEIIMNAA.tim.one@home.com>
Message-ID: <15388.47523.871195.694764@12-248-41-177.client.attbi.com>

    Tim> In case anyone's waiting, stop -- I got sick to death of it, and
    Tim> won't be checking in anything related.  (BTW, there is no
    Tim> test_rlcompleter, of course; it's the combo of test_sundry and
    Tim> test___all__ that screws up (both of those import rlcompleter))

A couple wild-ass-guess ideas:

    * Perhaps more test___all__ smarts are needed?  You added a "known
      skips" list to regrtest.py.  test___all__ could perhaps avoid
      importing any module that's not supposed to work on the current
      platform.

    * How about comparing sys.modules.keys() before and after a test and
      then remove any that were added that are Python modules?  (It's not
      safe to remove extension modules, right?)

calling-it-intractable-means-someone-will-solve-it-ly, y'rs,

Skip


From simon@simon-cozens.org  Sun Dec 16 21:14:05 2001
From: simon@simon-cozens.org (Simon Cozens)
Date: Sun, 16 Dec 2001 21:14:05 +0000
Subject: [Python-Dev] First cut of pyc2pasm released
Message-ID: <20011216211405.GA16141@netthink.co.uk>

I hope you do not mind another brief Parroty interlude. If you're sick
of that kind of thing, I apologise, I'm not trying to poach your time, so
excuse me.

But! With the 0.0.3 release of Parrot list week, we've got sufficiently
powerful that it's sensible to think about running other languages'
bytecode on the Parrot VM. (Converting the bytecode alone saves the
hassle of writing a compiler, which is the difficult bit.)

So naturally, I started with Perl. I think that lasted for about half an
hour before I ran away screaming. Don't get me wrong, I like Perl, I
just wouldn't like to turn its op tree into anything sensible. So I
thought I'd show the buggers.

Python's bytecode is nice and straightforward, and dis.py turns it into
relatively simple assembly. The operations that it needs, Parrot
provides, even things like set_lineno and rot_four. It was only a matter
of time before I could disassemble Python bytecode and put it together
into Parrot assembly. 

http://archive.develooper.com/perl6-internals@perl.org/msg07040.html is
about as far as I got. It works. It runs "Hello World" and things
slightly more complex, and it groks simple variable manipulation.
It wouldn't be much hassle to add control structures.

As a slight disclaimer, the variable classes [1] it uses - PythonInteger
and so on - don't actually exist, but you can fake it by changing the
outputted assembler code to use the ones that Parrot provides. I'm going
to work on providing Python types in the core Real Soon Now, tuit
permitting.

Anyway, there it is. Enjoy it. If you want to work on it, feel free;
if you want to take it and rewrite it in Python, then I'd be honored.
Whatever, it's out there, and we're serious about making Parrot work
for Python. (Or is it the other way around..? ;)

Simon

[1] One of Parrot's internal data types is the Parrot Magic Cookie,
something which is not a million miles distant from a PyObject. You
define PMCs by filling in a standard structure of manipulator methods,
just like in Objects/*object.c. 

-- 
Hubris is when you really do have it, enough so only the gods slap you
down. Pretentiousness is when you don't have it, and everyone slaps
you down. Arrogance is somewhere in between. 
    - Thorfinn


From mal@lemburg.com  Mon Dec 17 10:56:14 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 17 Dec 2001 11:56:14 +0100
Subject: [Python-Dev] regrtest.py mystery
References: <LNBBLJKPBEHFEDALKOLCAEHNMNAA.tim.one@home.com>
 <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de>
Message-ID: <3C1DCF4E.193D27AB@lemburg.com>

"Martin v. Loewis" wrote:
> 
> >  > Python/codecs.c holds search functions in _PyCodec_SearchPath, among
> >  > them encodings.__init__.search_function. It also imports "encodings"
> >  > the first time somebody invokes .encode/.decode on some kind of
> >  > string, but doesn't hold onto the module.
> >
> >   Perhaps it should hold onto the module as well?  That would avoid it
> > getting cleaned up before possibly dependent modules.
> 
> Re-importing encodings would still do funny things, then: It would
> register itself once more, resulting in two search functions being
> registered. codecs would hold one of the encodings modules, but not
> the other. So if the second search function is ever invoked, you get
> the same error.

Rather than "fixing" the encodings package, I'd rather suggest to
fix the unload procedure in the test suite. Some modules simply
should not be unloaded because it is known that the interpreter 
needs them for internal purposes. In the case of the encodings
package it is not only the search function that gets kept alive, it's
also the codecs themselves which are cached by the implementation
in _codecs.c.

How about following Fred's suggestion to run each test in a separate
process (possibly as an option to regrtest.py) ?!

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From guido@python.org  Mon Dec 17 14:08:31 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 17 Dec 2001 09:08:31 -0500
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Modules xxsubtype.c,2.9,2.10
In-Reply-To: Your message of "Mon, 17 Dec 2001 10:39:31 +0100."
 <3C1DBD53.2672A1C@lemburg.com>
References: <E16FmYZ-0008R4-00@usw-pr-cvs1.sourceforge.net>
 <3C1DBD53.2672A1C@lemburg.com>
Message-ID: <200112171408.JAA16669@cj20424-a.reston1.va.home.com>

[Tim]
> > Log Message:
> > David Abrahams tried to compile this as a separate DLL under MSVC, and
> > got a barrage of compile errors that didn't make sense to the C++ brain:
> > MSVC does not allow C (but does allow C++) initializers to contain
> > data addresses supplied by other DLLs.  So changed the initializers here
> > to use dummy nulls, and changed module init to plug in the foreign
> > addresses at runtime (manually simulating what C++ does by magic).  Tested
> > on Windows, and Guido tested on Linux (thanks!).  BTW, the *point* is that
> > people are going to use this module as a template for writing their own
> > subtypes, and it's unusual for extension authors to build their extensions
> > into Python directly (separate DLLs are the norm on Windows); so it's
> > better if we give them a template that works <wink>.

[MAL]
> This is a common problem when compiling Windows extensions. Perhaps
> we should provide some more generic way to help the extension
> writers here ?! FWIW, I'm using this macro for doing the deferred
> type init:
> 
> #define PyType_Init(x)                                          \
> {                                                               \
>     x.ob_type = &PyType_Type;                                   \
> }

Hm, alternatively, we could put a statement in PyType_Ready() that
initializes ob_type if it's NULL -- then we'd encourage people to call
PyType_Ready(), which is a good idea anyway.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From thomas.heller@ion-tof.com  Mon Dec 17 14:23:06 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Mon, 17 Dec 2001 15:23:06 +0100
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Modules xxsubtype.c,2.9,2.10
References: <E16FmYZ-0008R4-00@usw-pr-cvs1.sourceforge.net>              <3C1DBD53.2672A1C@lemburg.com>  <200112171408.JAA16669@cj20424-a.reston1.va.home.com>
Message-ID: <021401c18706$6aacaf20$e000a8c0@thomasnotebook>

> Hm, alternatively, we could put a statement in PyType_Ready() that
> initializes ob_type if it's NULL -- then we'd encourage people to call
> PyType_Ready(), which is a good idea anyway.
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)

+2 ;-)

Thomas



From fdrake@acm.org  Mon Dec 17 14:26:49 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 17 Dec 2001 09:26:49 -0500 (EST)
Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Modules xxsubtype.c,2.9,2.10
In-Reply-To: <200112171408.JAA16669@cj20424-a.reston1.va.home.com>
References: <E16FmYZ-0008R4-00@usw-pr-cvs1.sourceforge.net>
 <3C1DBD53.2672A1C@lemburg.com>
 <200112171408.JAA16669@cj20424-a.reston1.va.home.com>
Message-ID: <15390.169.62909.589215@cj42289-a.reston1.va.home.com>

Guido van Rossum writes:
 > Hm, alternatively, we could put a statement in PyType_Ready() that
 > initializes ob_type if it's NULL -- then we'd encourage people to call
 > PyType_Ready(), which is a good idea anyway.

  This would be very good.  I'll second Thomas's +2.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From fdrake@acm.org  Mon Dec 17 14:32:00 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 17 Dec 2001 09:32:00 -0500 (EST)
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: <3C1DCF4E.193D27AB@lemburg.com>
References: <LNBBLJKPBEHFEDALKOLCAEHNMNAA.tim.one@home.com>
 <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de>
 <15387.58005.329451.71260@grendel.zope.com>
 <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de>
 <3C1DCF4E.193D27AB@lemburg.com>
Message-ID: <15390.480.513981.132090@cj42289-a.reston1.va.home.com>

M.-A. Lemburg writes:
 > How about following Fred's suggestion to run each test in a separate
 > process (possibly as an option to regrtest.py) ?!

  I wasn't really suggesting it, but it would solve this particular
problem.  Note that the effectiveness of this to solve this particular
problem depends on the set of modules regrtest itself imports; if it
imports something and catches an ImportError, using a simple fork() to
create the child is not sufficient.  Using fork()/exec() would be
extremely slow and should be avoided if we can.
  Frankly, I like that we find interactions between the tests, so I'm
not convinced that there are any real problems with the test
framework.  I do use to a C unit test package that uses fork() to make
sure individual tests can't core the framework, but that's not what
we're protecting against here.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From mwh@python.net  Mon Dec 17 14:36:59 2001
From: mwh@python.net (Michael Hudson)
Date: 17 Dec 2001 14:36:59 +0000
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: "Fred L. Drake, Jr."'s message of "Mon, 17 Dec 2001 09:32:00 -0500 (EST)"
References: <LNBBLJKPBEHFEDALKOLCAEHNMNAA.tim.one@home.com> <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> <3C1DCF4E.193D27AB@lemburg.com> <15390.480.513981.132090@cj42289-a.reston1.va.home.com>
Message-ID: <2mbsgxlxpw.fsf@starship.python.net>

"Fred L. Drake, Jr." <fdrake@acm.org> writes:

> M.-A. Lemburg writes:
>  > How about following Fred's suggestion to run each test in a separate
>  > process (possibly as an option to regrtest.py) ?!
> 
>   I wasn't really suggesting it, but it would solve this particular
> problem.  Note that the effectiveness of this to solve this particular
> problem depends on the set of modules regrtest itself imports; if it
> imports something and catches an ImportError, using a simple fork() to
> create the child is not sufficient.  Using fork()/exec() would be
> extremely slow and should be avoided if we can.

And I think you'd have fun getting anything like that to run on MacOS
before OSX, and probably other oddball platforms (Windows at least has
the os.spawn* functions).

Making the test suite less portable is /unlikely/ to be a win, I'd
have thought.

Cheers,
M.

-- 
  People think I'm a nice guy, and the fact is that I'm a scheming,
  conniving bastard who doesn't care for any hurt feelings or lost
  hours of work if it just results in what I consider to be a better
  system.                                            -- Linus Torvalds


From barry@zope.com  Mon Dec 17 15:45:36 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 17 Dec 2001 10:45:36 -0500
Subject: [Python-Dev] regrtest.py mystery
References: <LNBBLJKPBEHFEDALKOLCAEHNMNAA.tim.one@home.com>
 <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de>
 <15387.58005.329451.71260@grendel.zope.com>
 <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de>
 <3C1DCF4E.193D27AB@lemburg.com>
Message-ID: <15390.4896.825987.759927@anthem.wooz.org>

>>>>> "M" == M  <mal@lemburg.com> writes:

    M> Rather than "fixing" the encodings package, I'd rather suggest
    M> to fix the unload procedure in the test suite.

Featurette request for Python 2.3: a builtin unload() which Does The
Right Thing to unload a module, or raise an exception if the module
cannot be unloaded.  Lots and lots of applications would find this
useful.

-Barry


From fdrake@acm.org  Mon Dec 17 15:54:51 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 17 Dec 2001 10:54:51 -0500 (EST)
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: <15390.4896.825987.759927@anthem.wooz.org>
References: <LNBBLJKPBEHFEDALKOLCAEHNMNAA.tim.one@home.com>
 <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de>
 <15387.58005.329451.71260@grendel.zope.com>
 <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de>
 <3C1DCF4E.193D27AB@lemburg.com>
 <15390.4896.825987.759927@anthem.wooz.org>
Message-ID: <15390.5451.781522.718432@cj42289-a.reston1.va.home.com>

Barry A. Warsaw writes:
 > Featurette request for Python 2.3: a builtin unload() which Does The
 > Right Thing to unload a module, or raise an exception if the module
 > cannot be unloaded.  Lots and lots of applications would find this
 > useful.

  Useful or not, isn't the basic problem that we haven't been able to
define The Right Thing?  I'd certainly expect a module to be able to
declare in some way that it could not be unloaded, or control how it
gets unloaded.  Perhaps allow Python modules to define an __unload__()
function; when unload() is called on a module, the __unload__() is
called; if that raises an exception, it gets propogated, otherwise the
unload continues if the module still exists in sys.modules.  Not clear
that this would be useful for extensions.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From paul@pfdubois.com  Mon Dec 17 16:09:59 2001
From: paul@pfdubois.com (Paul F. Dubois)
Date: Mon, 17 Dec 2001 08:09:59 -0800
Subject: [Python-Dev] RE: [Numpy-discussion] Python 2.2
In-Reply-To: <006201c186e7$da0d32f0$df13000a@RDREWXP>
Message-ID: <000001c18715$477816c0$0c01a8c0@freedom>

I downloaded and installed 2.2c1 and built the Numeric installers.
However, when I run them they all fail when the installation begins
(after one clicks the final click to install) with an access violation.
I removed any previous Numeric and it still happened.

Building and installing with setup.py install works ok.

-----Original Message-----
From: numpy-discussion-admin@lists.sourceforge.net
[mailto:numpy-discussion-admin@lists.sourceforge.net] On Behalf Of Ray
Drew
Sent: Monday, December 17, 2001 2:45 AM
To: numpy-discussion@lists.sourceforge.net
Subject: [Numpy-discussion] Python 2.2


Does anyone know when a release for Python 2.2 on Windows will be
available?

regards,

Ray Drew


_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com


_______________________________________________
Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion



From guido@python.org  Mon Dec 17 16:12:54 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 17 Dec 2001 11:12:54 -0500
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: Your message of "Mon, 17 Dec 2001 10:45:36 EST."
 <15390.4896.825987.759927@anthem.wooz.org>
References: <LNBBLJKPBEHFEDALKOLCAEHNMNAA.tim.one@home.com> <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> <3C1DCF4E.193D27AB@lemburg.com>
 <15390.4896.825987.759927@anthem.wooz.org>
Message-ID: <200112171612.LAA01013@cj20424-a.reston1.va.home.com>

> Featurette request for Python 2.3: a builtin unload() which Does The
> Right Thing to unload a module, or raise an exception if the module
> cannot be unloaded.  Lots and lots of applications would find this
> useful.

I haven't been following this discussion.  Can you give a clear
definition of The Right Thing?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@zope.com  Mon Dec 17 16:18:42 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 17 Dec 2001 11:18:42 -0500
Subject: [Python-Dev] regrtest.py mystery
References: <LNBBLJKPBEHFEDALKOLCAEHNMNAA.tim.one@home.com>
 <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de>
 <15387.58005.329451.71260@grendel.zope.com>
 <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de>
 <3C1DCF4E.193D27AB@lemburg.com>
 <15390.4896.825987.759927@anthem.wooz.org>
 <15390.5451.781522.718432@cj42289-a.reston1.va.home.com>
Message-ID: <15390.6882.514689.285195@anthem.wooz.org>

>>>>> "Fred" == Fred L Drake, Jr <fdrake@acm.org> writes:

    Fred>   Useful or not, isn't the basic problem that we haven't
    Fred> been able to define The Right Thing?

Probably so.
    
    Fred> I'd certainly expect a module to be able to declare in some
    Fred> way that it could not be unloaded, or control how it gets
    Fred> unloaded.  Perhaps allow Python modules to define an
    Fred> __unload__() function; when unload() is called on a module,
    Fred> the __unload__() is called; if that raises an exception, it
    Fred> gets propogated, otherwise the unload continues if the
    Fred> module still exists in sys.modules.  Not clear that this
    Fred> would be useful for extensions.

The "unload protocol" would be I think.  I.e. extension modules ought
to be unloadable if they can cooperate.  From the Linux dl*()
manpages:

       dlclose decrements the  reference  count  on  the  dynamic
       library  handle  handle.   If the reference count drops to
       zero and no other loaded libraries use symbols in it, then
       the  dynamic  library is unloaded.  If the dynamic library
       exports a routine named _fini, then that routine is called
       just before the library is unloaded.

season-to-tasste-on-your-favorite-OS-bread-l y'rs,
-Barry


From guido@python.org  Mon Dec 17 16:16:27 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 17 Dec 2001 11:16:27 -0500
Subject: [Python-Dev] RE: [Numpy-discussion] Python 2.2
In-Reply-To: Your message of "Mon, 17 Dec 2001 08:09:59 PST."
 <000001c18715$477816c0$0c01a8c0@freedom>
References: <000001c18715$477816c0$0c01a8c0@freedom>
Message-ID: <200112171616.LAA04194@cj20424-a.reston1.va.home.com>

> I downloaded and installed 2.2c1 and built the Numeric installers.
> However, when I run them they all fail when the installation begins
> (after one clicks the final click to install) with an access violation.
> I removed any previous Numeric and it still happened.
> 
> Building and installing with setup.py install works ok.

I don't know anything about your installers, but could it be that you
were trying to install without Administrator permissions?  That used
to crash the previous Python installer too.  (The new one doesn't, but
it's a commercial product so we can't share it.)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@zope.com  Mon Dec 17 16:21:41 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 17 Dec 2001 11:21:41 -0500
Subject: [Python-Dev] regrtest.py mystery
References: <LNBBLJKPBEHFEDALKOLCAEHNMNAA.tim.one@home.com>
 <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de>
 <15387.58005.329451.71260@grendel.zope.com>
 <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de>
 <3C1DCF4E.193D27AB@lemburg.com>
 <15390.4896.825987.759927@anthem.wooz.org>
 <200112171612.LAA01013@cj20424-a.reston1.va.home.com>
Message-ID: <15390.7061.122528.163470@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> I haven't been following this discussion.  Can you give a
    GvR> clear definition of The Right Thing?

I think we need to define the goals of unloading.


From mwh@python.net  Mon Dec 17 16:25:54 2001
From: mwh@python.net (Michael Hudson)
Date: 17 Dec 2001 16:25:54 +0000
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: barry@zope.com's message of "Mon, 17 Dec 2001 11:21:41 -0500"
References: <LNBBLJKPBEHFEDALKOLCAEHNMNAA.tim.one@home.com> <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> <3C1DCF4E.193D27AB@lemburg.com> <15390.4896.825987.759927@anthem.wooz.org> <200112171612.LAA01013@cj20424-a.reston1.va.home.com> <15390.7061.122528.163470@anthem.wooz.org>
Message-ID: <2mheqpke3x.fsf@starship.python.net>

barry@zope.com (Barry A. Warsaw) writes:

> >>>>> "GvR" == Guido van Rossum <guido@python.org> writes:
> 
>     GvR> I haven't been following this discussion.  Can you give a
>     GvR> clear definition of The Right Thing?
> 
> I think we need to define the goals of unloading.

Doesn't that make it into a solution in search of a problem?

When is unloading useful?  I'm not being facetious, I've just never
been hit by the need for it.

Cheers,
M.

-- 
  Q: Isn't it okay to just read Slashdot for the links?
  A: No. Reading Slashdot for the links is like having "just one hit"
     off the crack pipe.
     -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq


From guido@python.org  Mon Dec 17 16:31:17 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 17 Dec 2001 11:31:17 -0500
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: Your message of "Mon, 17 Dec 2001 11:21:41 EST."
 <15390.7061.122528.163470@anthem.wooz.org>
References: <LNBBLJKPBEHFEDALKOLCAEHNMNAA.tim.one@home.com> <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> <3C1DCF4E.193D27AB@lemburg.com> <15390.4896.825987.759927@anthem.wooz.org> <200112171612.LAA01013@cj20424-a.reston1.va.home.com>
 <15390.7061.122528.163470@anthem.wooz.org>
Message-ID: <200112171631.LAA15524@cj20424-a.reston1.va.home.com>

> >>>>> "GvR" == Guido van Rossum <guido@python.org> writes:
> 
>     GvR> I haven't been following this discussion.  Can you give a
>     GvR> clear definition of The Right Thing?
> 
> I think we need to define the goals of unloading.

Then how can you call it a featurette?  Who knows if it's even a
tractable problem?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry@zope.com  Mon Dec 17 16:46:02 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Mon, 17 Dec 2001 11:46:02 -0500
Subject: [Python-Dev] regrtest.py mystery
References: <LNBBLJKPBEHFEDALKOLCAEHNMNAA.tim.one@home.com>
 <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de>
 <15387.58005.329451.71260@grendel.zope.com>
 <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de>
 <3C1DCF4E.193D27AB@lemburg.com>
 <15390.4896.825987.759927@anthem.wooz.org>
 <200112171612.LAA01013@cj20424-a.reston1.va.home.com>
 <15390.7061.122528.163470@anthem.wooz.org>
 <200112171631.LAA15524@cj20424-a.reston1.va.home.com>
Message-ID: <15390.8522.971549.671038@anthem.wooz.org>

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> Then how can you call it a featurette?  Who knows if it's
    GvR> even a tractable problem?

Okay, so it's a PEPable feature.

-Barry


From thomas.heller@ion-tof.com  Mon Dec 17 16:49:54 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Mon, 17 Dec 2001 17:49:54 +0100
Subject: [Python-Dev] RE: [Numpy-discussion] Python 2.2
References: <000001c18715$477816c0$0c01a8c0@freedom>
Message-ID: <028501c1871a$ec399260$e000a8c0@thomasnotebook>

> I downloaded and installed 2.2c1 and built the Numeric installers.
> However, when I run them they all fail when the installation begins
> (after one clicks the final click to install) with an access violation.
> I removed any previous Numeric and it still happened.
> 
> Building and installing with setup.py install works ok.
> 
Are you talking about bdist_wininst installers?
I just could reproduce a problem with them (after renaming
away all my zip.exe programs in the PATH).

Thomas



From guido@python.org  Mon Dec 17 16:52:21 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 17 Dec 2001 11:52:21 -0500
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: Your message of "Mon, 17 Dec 2001 11:46:02 EST."
 <15390.8522.971549.671038@anthem.wooz.org>
References: <LNBBLJKPBEHFEDALKOLCAEHNMNAA.tim.one@home.com> <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> <3C1DCF4E.193D27AB@lemburg.com> <15390.4896.825987.759927@anthem.wooz.org> <200112171612.LAA01013@cj20424-a.reston1.va.home.com> <15390.7061.122528.163470@anthem.wooz.org> <200112171631.LAA15524@cj20424-a.reston1.va.home.com>
 <15390.8522.971549.671038@anthem.wooz.org>
Message-ID: <200112171652.LAA15707@cj20424-a.reston1.va.home.com>

> Okay, so it's a PEPable feature.

Not until you've thought about what it should do.  I bet you lunch
that you can't come up with a working definition.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From paul@pfdubois.com  Mon Dec 17 17:24:14 2001
From: paul@pfdubois.com (Paul F. Dubois)
Date: Mon, 17 Dec 2001 09:24:14 -0800
Subject: [Python-Dev] RE: [Numpy-discussion] Python 2.2
In-Reply-To: <200112171616.LAA04194@cj20424-a.reston1.va.home.com>
Message-ID: <000201c1871f$a6f514e0$0c01a8c0@freedom>

I am on Windows Me, so there is no concept of administrator.

-----Original Message-----
From: numpy-discussion-admin@lists.sourceforge.net
[mailto:numpy-discussion-admin@lists.sourceforge.net] On Behalf Of Guido
van Rossum
Sent: Monday, December 17, 2001 8:16 AM
To: Paul F. Dubois
Cc: 'Ray Drew'; numpy-discussion@lists.sourceforge.net;
python-dev@python.org
Subject: Re: [Python-Dev] RE: [Numpy-discussion] Python 2.2


> I downloaded and installed 2.2c1 and built the Numeric installers. 
> However, when I run them they all fail when the installation begins 
> (after one clicks the final click to install) with an access 
> violation. I removed any previous Numeric and it still happened.
> 
> Building and installing with setup.py install works ok.

I don't know anything about your installers, but could it be that you
were trying to install without Administrator permissions?  That used to
crash the previous Python installer too.  (The new one doesn't, but it's
a commercial product so we can't share it.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

_______________________________________________
Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion



From jeremy@zope.com  Mon Dec 17 18:24:26 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Mon, 17 Dec 2001 13:24:26 -0500 (EST)
Subject: [Python-Dev] compiler package udpate
Message-ID: <15390.14426.296516.102494@slothrop.digicool.com>

I'd like to backport changes from the 2.2 compiler package
(Lib/compiler) to the 2.1.2 version (Tools/compiler/compiler).  The
changes are bug fixes, but include some non-trivial changes because
the old code has some non-trivial bugs.  The chief problem is that it
underestimates the stack depth, which leads to core dumps.

Any objections to me making these changes today?

Jeremy



From tim.one@home.com  Mon Dec 17 19:38:48 2001
From: tim.one@home.com (Tim Peters)
Date: Mon, 17 Dec 2001 14:38:48 -0500
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: <3C1DCF4E.193D27AB@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGENEMNAA.tim.one@home.com>

Without diving into the unloading pit, the endless problems with -r on
non-Unix boxes would go away if a module that failed to import didn't leave
behind a damaged module object in sys.modules ("damaged" == it may not
fulfill some or any of the module's contracts, and it's impossible to guess
which ones it won't, or to guess how the unfufilled ones may fail).

I'm tempted to add a pile of Windows-specific modules that import each other
in incestuous ways, so everyone can enjoy this chronic pain <wink>.

"A fix" seems to amount to treating

import M

as if it were

try:
    import M
except:
    if M in sys.modules:
        del sys.modules[M]
    raise

How would that hurt?



From ping@lfw.org  Mon Dec 17 21:05:12 2001
From: ping@lfw.org (Ka-Ping Yee)
Date: Mon, 17 Dec 2001 15:05:12 -0600 (CST)
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGENEMNAA.tim.one@home.com>
Message-ID: <Pine.LNX.4.33.0112171502420.6192-100000@server1.lfw.org>

On Mon, 17 Dec 2001, Tim Peters wrote:
> "A fix" seems to amount to treating
>
> import M
>
> as if it were
>
> try:
>     import M
> except:
>     if M in sys.modules:
>         del sys.modules[M]
>     raise

I remember suggesting exactly that some time ago (i think it was
motivated at the time by the extreme pain that broken modules were
causing for webserver-mode pydoc in its attempt to update loaded
modules if the source files had changed on disk).

Guido rejected it because you can't guarantee that the refcount on M
is 1 at the point where you attempt to 'del sys.modules[M]' above.
(For example, some other module imported by M could have imported M
again, and so hold a reference to it while M is running its startup
commands.  This is why the entry is added to sys.modules before the
body of M starts to run.)

He deemed the situation where M is loaded-but-missing-from-sys.modules
to be even worse than for M to be left loaded-but-broken therein.  If
you allow M to stay in sys.modules, then you can at least maintain the
guarantee that there is a one-to-one association between loaded module
names and loaded module objects.  If you remove M from sys.modules but
it lingers in memory, referenced elsewhere, you lose even that -- there
can be many modules loaded all with the same name and it's a nightmare.

The argument is compelling and i am forced to agree, but i still think
that we should look for a better solution.  I'll ponder this and post
any ideas i come up with.


-- ?!ng



From mal@lemburg.com  Mon Dec 17 21:25:56 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Mon, 17 Dec 2001 22:25:56 +0100
Subject: [Python-Dev] Unloading modules (regrtest.py mystery)
References: <Pine.LNX.4.33.0112171502420.6192-100000@server1.lfw.org>
Message-ID: <3C1E62E4.C4E9A7A0@lemburg.com>

Ka-Ping Yee wrote:
> 
> On Mon, 17 Dec 2001, Tim Peters wrote:
> > "A fix" seems to amount to treating
> >
> > import M
> >
> > as if it were
> >
> > try:
> >     import M
> > except:
> >     if M in sys.modules:
> >         del sys.modules[M]
> >     raise
> 
> I remember suggesting exactly that some time ago (i think it was
> motivated at the time by the extreme pain that broken modules were
> causing for webserver-mode pydoc in its attempt to update loaded
> modules if the source files had changed on disk).
> 
> Guido rejected it because you can't guarantee that the refcount on M
> is 1 at the point where you attempt to 'del sys.modules[M]' above.
> (For example, some other module imported by M could have imported M
> again, and so hold a reference to it while M is running its startup
> commands.  This is why the entry is added to sys.modules before the
> body of M starts to run.)
> 
> He deemed the situation where M is loaded-but-missing-from-sys.modules
> to be even worse than for M to be left loaded-but-broken therein.  If
> you allow M to stay in sys.modules, then you can at least maintain the
> guarantee that there is a one-to-one association between loaded module
> names and loaded module objects.  If you remove M from sys.modules but
> it lingers in memory, referenced elsewhere, you lose even that -- there
> can be many modules loaded all with the same name and it's a nightmare.
> 
> The argument is compelling and i am forced to agree, but i still think
> that we should look for a better solution.  I'll ponder this and post
> any ideas i come up with.

How about only deleting it from sys.modules iff the ref count 
is 1 ?!

Note that you cannot currently remove extension modules that have 
failed to import correctly. Would be nice if we could come 
up with a reasonable way of unloading modules, but I guess this 
requires some hard PEP work...

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From sdm7g@Virginia.EDU  Mon Dec 17 21:40:01 2001
From: sdm7g@Virginia.EDU (Steven Majewski)
Date: Mon, 17 Dec 2001 16:40:01 -0500 (EST)
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: <2mheqpke3x.fsf@starship.python.net>
Message-ID: <Pine.OSX.4.43.0112171628020.458-100000@d-128-61-180.bootp.virginia.edu>


On 17 Dec 2001, Michael Hudson wrote:

> barry@zope.com (Barry A. Warsaw) writes:
>
> > >>>>> "GvR" == Guido van Rossum <guido@python.org> writes:
> >
> >     GvR> I haven't been following this discussion.  Can you give a
> >     GvR> clear definition of The Right Thing?
> >
> > I think we need to define the goals of unloading.
>
> Doesn't that make it into a solution in search of a problem?
>
> When is unloading useful?  I'm not being facetious, I've just never
> been hit by the need for it.

Shared libraries in XlispStat support unloading.
But all of the times I've ever used it, I've used it as a 'reload' :
usually while debugging shared libs -- it's handy to keep the
interpreter window running while you fix and recompile a bug and
load it again. The only case I can thing of where I might use it
in production code would also be for a reload, where you might
have several alternate 'plugin' modules.

 When you do an unload, references to objects that were in the
shared library are replaced by references to a stub that generates
an error/exception. ( That stub routine is part of the builtin
dynamic load "module" . ) It doesn't involve traceing all the
references -- the pointers are unchanged, but the object is
changed.

 I have no idea what 'the Right Thing' for Python would be here --
I'm not quite clear on what you're trying to fix. I'm just
volunteering an example.

-- Steve Majewski




From sdm7g@Virginia.EDU  Mon Dec 17 22:19:13 2001
From: sdm7g@Virginia.EDU (Steven Majewski)
Date: Mon, 17 Dec 2001 17:19:13 -0500 (EST)
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: <Pine.OSX.4.43.0112171628020.458-100000@d-128-61-180.bootp.virginia.edu>
Message-ID: <Pine.OSX.4.43.0112171709500.458-100000@d-128-61-180.bootp.virginia.edu>


On Mon, 17 Dec 2001, Steven Majewski wrote:

>  I have no idea what 'the Right Thing' for Python would be here --
> I'm not quite clear on what you're trying to fix. I'm just
> volunteering an example.

However, I will add that if I *did* need an unload for anything,
I would think the XlispStat semantics: unloading the module
immediately and making any dangling references into exceptions,
or replacing them with references to a reloaded module,
would seem to be more useful than not really unloading it until
all of the references are decremented (if ever) and then
unloading.

I can't think of a case where I would want to unload a module,
where I would not want a dangling reference to something in that
module to generate an exception.

-- Steve




From guido@python.org  Mon Dec 17 22:24:31 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 17 Dec 2001 17:24:31 -0500
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: Your message of "Mon, 17 Dec 2001 17:19:13 EST."
 <Pine.OSX.4.43.0112171709500.458-100000@d-128-61-180.bootp.virginia.edu>
References: <Pine.OSX.4.43.0112171709500.458-100000@d-128-61-180.bootp.virginia.edu>
Message-ID: <200112172224.RAA19426@cj20424-a.reston1.va.home.com>

> However, I will add that if I *did* need an unload for anything,
> I would think the XlispStat semantics: unloading the module
> immediately and making any dangling references into exceptions,
> or replacing them with references to a reloaded module,
> would seem to be more useful than not really unloading it until
> all of the references are decremented (if ever) and then
> unloading.
> 
> I can't think of a case where I would want to unload a module,
> where I would not want a dangling reference to something in that
> module to generate an exception.

Python doesn't let you replace a reference to an object with something
else -- except in special cases (e.g. with explicit weak refs) there's
no way to know where references to an object might exist.

So that's not a useful idea, alas.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fdrake@acm.org  Mon Dec 17 22:37:53 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 17 Dec 2001 17:37:53 -0500 (EST)
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: <200112172224.RAA19426@cj20424-a.reston1.va.home.com>
References: <Pine.OSX.4.43.0112171709500.458-100000@d-128-61-180.bootp.virginia.edu>
 <200112172224.RAA19426@cj20424-a.reston1.va.home.com>
Message-ID: <15390.29633.527008.217751@cj42289-a.reston1.va.home.com>

Guido van Rossum writes:
 > So that's not a useful idea, alas.

  Well, it may be a useful idea, but difficult & impractical to
implement in Python.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From greg@cosc.canterbury.ac.nz  Mon Dec 17 22:47:06 2001
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 18 Dec 2001 11:47:06 +1300 (NZDT)
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: <Pine.LNX.4.33.0112171502420.6192-100000@server1.lfw.org>
Message-ID: <200112172247.LAA17572@s454.cosc.canterbury.ac.nz>

Ka-Ping Yee <ping@lfw.org>:

> He deemed the situation where M is loaded-but-missing-from-sys.modules
> to be even worse than for M to be left loaded-but-broken therein.

Maybe modules should have a this-module-is-broken flag which
is set when the module fails to import properly, and which
causes an exception on any further attempt to reference a
name in it.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From fredrik@pythonware.com  Mon Dec 17 23:14:43 2001
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Tue, 18 Dec 2001 00:14:43 +0100
Subject: [Python-Dev] regrtest.py mystery
References: <200112172247.LAA17572@s454.cosc.canterbury.ac.nz>
Message-ID: <164201c18750$9ef89830$ced241d5@hagrid>

greg wrote:
> 
> > He deemed the situation where M is loaded-but-missing-from-sys.modules
> > to be even worse than for M to be left loaded-but-broken therein.
> 
> Maybe modules should have a this-module-is-broken flag which
> is set when the module fails to import properly, and which
> causes an exception on any further attempt to reference a
> name in it.

how do you detect that a module is successfully imported?

the only way I can think of is "execution reached the end of
the module", which would probably break considerable amounts
of code that rely on "temporarily recursive" imports.

</F>



From greg@cosc.canterbury.ac.nz  Mon Dec 17 23:23:53 2001
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 18 Dec 2001 12:23:53 +1300 (NZDT)
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: <164201c18750$9ef89830$ced241d5@hagrid>
Message-ID: <200112172323.MAA17577@s454.cosc.canterbury.ac.nz>

Fredrik Lundh <fredrik@pythonware.com>:

> how do you detect that a module is successfully imported?
> 
> the only way I can think of is "execution reached the end of
> the module", which would probably break considerable amounts
> of code that rely on "temporarily recursive" imports.

If an exception propagates back through an import
statement, the module that it's importing is marked as
broken.

This may result in a single error causing more than one
module to be marked as broken, but I think that's the right
thing to do anyway. E.g. if you import module A which imports
module B which fails, you can't be sure that either A or B
is fit to use, so both should be marked as broken.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From tim.one@home.com  Mon Dec 17 23:36:22 2001
From: tim.one@home.com (Tim Peters)
Date: Mon, 17 Dec 2001 18:36:22 -0500
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: <164201c18750$9ef89830$ced241d5@hagrid>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEOIMNAA.tim.one@home.com>

[/F]
> how do you detect that a module is successfully imported?
>
> the only way I can think of is "execution reached the end of
> the module", which would probably break considerable amounts
> of code that rely on "temporarily recursive" imports.

Presumably Greg's flag would be initialized to "not broken", and set to
"broken" when and only when an uncaught exception is raised.  So in

    import A
    A imports B
        which imports A
        back to B
        B completes
    back to A
    A blows up before completing

A's "broken" flag isn't set until the end, so B's recursive import of A
isn't harmed.



From sdm7g@Virginia.EDU  Mon Dec 17 23:43:48 2001
From: sdm7g@Virginia.EDU (Steven Majewski)
Date: Mon, 17 Dec 2001 18:43:48 -0500 (EST)
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: <200112172224.RAA19426@cj20424-a.reston1.va.home.com>
Message-ID: <Pine.OSX.4.43.0112171811170.458-100000@d-128-61-180.bootp.virginia.edu>


On Mon, 17 Dec 2001, Guido van Rossum wrote:

> Python doesn't let you replace a reference to an object with something
> else -- except in special cases (e.g. with explicit weak refs) there's
> no way to know where references to an object might exist.

I was too fuzzy in my choice of words. What I meant was not
finding and replacing the references, but replacing the object to
which all of the 'dangling references' point.

But that won't work either: Python objects share the same head, but
total size of the object structs are too variable, and I don't know
it's possible to make the minimal size object show the required
behaviour.

 But even if that is possible, another problem with implementing it in a
Python extension module is that Python objects are not (except for maybe
functions and things that drag along a reference to global module
namespace) "owned" by their modules in anything like the association
between Lisp symbols and their packages: python modules can import other
modules or can dynamically add things to their namespaces. We can't just
iterate through a module dict and 'zero' out all of the objects there.

> So that's not a useful idea, alas.

If the problem was important enough, I think a way could be found
around those problems above, but I'm still not sure that module
unloading isn't a solution looking  for a problem!

-- Steve



From martin@v.loewis.de  Mon Dec 17 23:47:04 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Tue, 18 Dec 2001 00:47:04 +0100
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: <164201c18750$9ef89830$ced241d5@hagrid> (fredrik@pythonware.com)
References: <200112172247.LAA17572@s454.cosc.canterbury.ac.nz> <164201c18750$9ef89830$ced241d5@hagrid>
Message-ID: <200112172347.fBHNl4m11257@mira.informatik.hu-berlin.de>

> the only way I can think of is "execution reached the end of
> the module", which would probably break considerable amounts
> of code that rely on "temporarily recursive" imports.

I would have said "does not throw ImportError".

Regards,
Martin


From jeremy@zope.com  Tue Dec 18 00:06:52 2001
From: jeremy@zope.com (Jeremy Hylton)
Date: Mon, 17 Dec 2001 19:06:52 -0500 (EST)
Subject: [Python-Dev] Re: compiler package udpate
In-Reply-To: <200112172209.fBHM9Xx02804@mbuna.arbhome.com.au>
References: <jeremy@zope.com>
 <15390.14426.296516.102494@slothrop.digicool.com>
 <200112172209.fBHM9Xx02804@mbuna.arbhome.com.au>
Message-ID: <15390.34972.767112.259073@slothrop.digicool.com>

I've made some good progress on it-- backported all the obvious stuff.
I need to downgrade some of my testing tools so that they work with
the old interface and re-run them.  Once I've finished with these
tests, I'll be satisfied.

Jeremy



From greg@cosc.canterbury.ac.nz  Tue Dec 18 00:13:57 2001
From: greg@cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue, 18 Dec 2001 13:13:57 +1300 (NZDT)
Subject: Broken module detection (Re: [Python-Dev] regrtest.py mystery)
In-Reply-To: <200112172347.fBHNl4m11257@mira.informatik.hu-berlin.de>
Message-ID: <200112180013.NAA17623@s454.cosc.canterbury.ac.nz>

"Martin v. Loewis" <martin@v.loewis.de>:

> I would have said "does not throw ImportError".

Or any other error. If a module fails to complete all
its initialisation code for any reason, it can't be
trusted.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+


From guido@python.org  Tue Dec 18 00:15:59 2001
From: guido@python.org (Guido van Rossum)
Date: Mon, 17 Dec 2001 19:15:59 -0500
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: Your message of "Mon, 17 Dec 2001 18:43:48 EST."
 <Pine.OSX.4.43.0112171811170.458-100000@d-128-61-180.bootp.virginia.edu>
References: <Pine.OSX.4.43.0112171811170.458-100000@d-128-61-180.bootp.virginia.edu>
Message-ID: <200112180015.TAA19857@cj20424-a.reston1.va.home.com>

> > Python doesn't let you replace a reference to an object with something
> > else -- except in special cases (e.g. with explicit weak refs) there's
> > no way to know where references to an object might exist.
> 
> I was too fuzzy in my choice of words. What I meant was not
> finding and replacing the references, but replacing the object to
> which all of the 'dangling references' point.
> 
> But that won't work either: Python objects share the same head, but
> total size of the object structs are too variable, and I don't know
> it's possible to make the minimal size object show the required
> behaviour.

In general it's impossible.  You can't even replace the type pointer
with a pointer to a dummy type, because it would leak any objects
referenced from the original object, and it would break if the
original object lived in a special free list.

> If the problem was important enough, I think a way could be found
> around those problems above, but I'm still not sure that module
> unloading isn't a solution looking  for a problem!

Well, there are a bunch of people who want to undo a partial import,
and that boils down to about the same thing.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one@home.com  Tue Dec 18 02:41:23 2001
From: tim.one@home.com (Tim Peters)
Date: Mon, 17 Dec 2001 21:41:23 -0500
Subject: [Python-Dev] regrtest.py mystery
In-Reply-To: <Pine.LNX.4.33.0112171502420.6192-100000@server1.lfw.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEPAMNAA.tim.one@home.com>

[Ka-Ping Yee]

Hi, Ping!  If we don't hear from you again, Merry Christmas too <wink>.

> I remember suggesting exactly that some time ago (i think it was
> motivated at the time by the extreme pain that broken modules were
> causing for webserver-mode pydoc in its attempt to update loaded
> modules if the source files had changed on disk).
>
> Guido rejected it because you can't guarantee that the refcount on M
> is 1 at the point where you attempt to 'del sys.modules[M]' above.

If somebody wants to hold on to a reference to a damaged module object,
that's OK by me.  That's an unusual case, though, so shouldn't wag the dog.

> (For example, some other module imported by M could have imported M
> again, and so hold a reference to it while M is running its startup
> commands.  This is why the entry is added to sys.modules before the
> body of M starts to run.)

Understood, but it's rare.

> He deemed the situation where M is loaded-but-missing-from-sys.modules
> to be even worse than for M to be left loaded-but-broken therein.

Why?  The module is broken no mater what.  I'm much keener on seeing that
the *next* attempt to import the module doesn't falsely appear to succeed.

> If you allow M to stay in sys.modules, then you can at least maintain the
> guarantee that there is a one-to-one association between loaded module
> names and loaded module objects.  If you remove M from sys.modules but
> it lingers in memory, referenced elsewhere, you lose even that -- there
> can be many modules loaded all with the same name and it's a nightmare.

Whether there's one copy or a million, it's broken.  Given that, I don't see
value in maintaining the one_name<->one_module invariant:  of what practical
use is it if the module is an insane state regardless?  If there is some
value to it I'm missing, it would be easy to give it a unique random
(re)name <wink>.

> The argument is compelling and i am forced to agree, but i still think
> that we should look for a better solution.  I'll ponder this and post
> any ideas i come up with.

Greg's "broken module" flag would do it for me too, if an attempt to import
a  known-to-be-broken module raised ImportError (in addition to Greg's
suggestion that attribute references complain).  That scheme has holes too,
but it gets increasingly harder to fall into them.  The current scheme has
so many pits it's proved impossible not to fall into them when using
regrtest -r, just doing utterly vanilla non-recursive imports.

compelling-my-foot<wink>-ly y'rs  - tim



From tim.one@home.com  Tue Dec 18 02:42:51 2001
From: tim.one@home.com (Tim Peters)
Date: Mon, 17 Dec 2001 21:42:51 -0500
Subject: [Python-Dev] Unloading modules (regrtest.py mystery)
In-Reply-To: <3C1E62E4.C4E9A7A0@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEPBMNAA.tim.one@home.com>

[MAL]
> ...
> Would be nice if we could come up with a reasonable way of unloading
> modules, but I guess this requires some hard PEP work...

Yes, I believe it would.  There are different kinds of unloading too:  a
module may well want to do different things depending on whether, e.g., it's
really a reload, or it's because Py_Finalize was called ... hmm!  That
reminds me:  Guido sent me this msg in March; it's still sitting in my inbox
waiting for a rainy day:

"""
OK, so we need to define a way for modules to specify a function to be
called at cleanup time.  I propose that we bring this out in the open,
at least in python-dev.  Note that there are three or four potential
cleanup times:

- when the module object is deleted from sys.modules

- when reload() is used on an extension

- when an interpreter created with PyInterpreter_New() is finalized

- when Py_Finalize() is called

- when Python exits

- when the Python DLL is unloaded (Windows only)
"""

There.  Somebody double the word count and call it a PEP <wink>.

Note that this is orthogonal to regrtest -r's woes.



From mal@lemburg.com  Tue Dec 18 09:22:25 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 18 Dec 2001 10:22:25 +0100
Subject: [Python-Dev] Unloading modules (regrtest.py mystery)
References: <LNBBLJKPBEHFEDALKOLCAEPBMNAA.tim.one@home.com>
Message-ID: <3C1F0AD1.8D580A4@lemburg.com>

Tim Peters wrote:
> 
> [MAL]
> > ...
> > Would be nice if we could come up with a reasonable way of unloading
> > modules, but I guess this requires some hard PEP work...
> 
> Yes, I believe it would.  There are different kinds of unloading too:  a
> module may well want to do different things depending on whether, e.g., it's
> really a reload, or it's because Py_Finalize was called ... hmm!  That
> reminds me:  Guido sent me this msg in March; it's still sitting in my inbox
> waiting for a rainy day:
> 
> """
> OK, so we need to define a way for modules to specify a function to be
> called at cleanup time.  I propose that we bring this out in the open,
> at least in python-dev.  Note that there are three or four potential
> cleanup times:
> 
> - when the module object is deleted from sys.modules
> 
> - when reload() is used on an extension
> 
> - when an interpreter created with PyInterpreter_New() is finalized
> 
> - when Py_Finalize() is called

I think we already have this one: I usually register a Py_AtExit (or
whatever the name is) function with Python to implement this.
 
> - when Python exits
> 
> - when the Python DLL is unloaded (Windows only)

Why Windows only ? I believe we could support this on other platforms
as well (e.g. Linux).

> """
> 
> There.  Somebody double the word count and call it a PEP <wink>.

Since Barry brought this up, I guess we already have a 
volunteer ;-)
 
> Note that this is orthogonal to regrtest -r's woes.
 
True and it won't help the suite with the encodings package 
either (how would you know what to cleanup given the fact that
the codec search functions decide how encoding names are mapped to
codecs ?).

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From niemeyer@conectiva.com  Tue Dec 18 11:50:04 2001
From: niemeyer@conectiva.com (Gustavo Niemeyer)
Date: Tue, 18 Dec 2001 09:50:04 -0200
Subject: [Python-Dev] Re: mpz module broken with GMP 4.0
In-Reply-To: <0112151923261C.01556@arthur>
References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com> <0112151923261C.01556@arthur>
Message-ID: <20011218095004.B1599@ibook.distro.conectiva>

--0eh6TmSyL6TZE2Uz
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

> Thanks for the indication -- I was not aware of the new GMP 4.0.  A rapid=
=20
> test shows the current gmpy.c, release 0.8, breaks in exactly one place=
=20
> (structure gmpstate_t used to have a member named seed, now renamed to=20
> _mp_seed).  Once that single line is fixed, gmpy 0.8 builds fine (on a=20

Have you commited to the CVS? It'd be nice to have it compiling with
both, the new and the old gmp.

> Linux box) with the new GMP 4.0 and passes all of its 973 unit-tests (wit=
h=20
> Python 2.2b1).  I think we should rapidly release a 0.8.1 with the one-li=
ne=20
> fix (arranged so it keeps building with GMP 3.1 while also building with=
=20
> 4.0, of course), and plan the modest gmpy extensions needed to exploit=20

Ohh.. that's it! Nice.. :-)

I'll commit the multiprecision pi code, so the next version already
includes it.

Thanks!!

--=20
Gustavo Niemeyer

[ 2AAC 7928 0FBF 0299 5EB5  60E2 2253 B29A 6664 3A0C ]

--0eh6TmSyL6TZE2Uz
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE8Hy1sIlOymmZkOgwRAs8dAJ9UgFEYGeLQY/gjz1OMM4FGCNcPFQCg0uuv
Uvq90IZdQTWRWUXTJjjgtn0=
=KsOJ
-----END PGP SIGNATURE-----

--0eh6TmSyL6TZE2Uz--


From barry@zope.com  Tue Dec 18 14:40:55 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Tue, 18 Dec 2001 09:40:55 -0500
Subject: [Python-Dev] Unloading modules (regrtest.py mystery)
References: <LNBBLJKPBEHFEDALKOLCAEPBMNAA.tim.one@home.com>
 <3C1F0AD1.8D580A4@lemburg.com>
Message-ID: <15391.21879.762287.430662@anthem.wooz.org>

>>>>> "M" == M  <mal@lemburg.com> writes:

    >> """ There.  Somebody double the word count and call it a PEP
    >> <wink>.

    M> Since Barry brought this up, I guess we already have a 
    M> volunteer ;-)
 
Sure MAL, I'll assign you a PEP number! <wink>

-Barry


From fdrake@acm.org  Tue Dec 18 17:26:36 2001
From: fdrake@acm.org (Fred L. Drake)
Date: Tue, 18 Dec 2001 12:26:36 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20011218172636.9E823286BC@cj42289-a.reston1.va.home.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

Added documentation for pydoc.  Misc. other fixes.


From thomas.heller@ion-tof.com  Tue Dec 18 21:25:06 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Tue, 18 Dec 2001 22:25:06 +0100
Subject: Need help with bdist_wininst, was Re: [Python-Dev] RE: [Numpy-discussion] Python 2.2
References: <000001c18715$477816c0$0c01a8c0@freedom> <028501c1871a$ec399260$e000a8c0@thomasnotebook>
Message-ID: <09bc01c1880a$8f0a4530$e000a8c0@thomasnotebook>

I've just fixed the bdist_wininst command in CVS.
There is not much time left before the release of Python 2.2 (how much exactly?).

To make sure that the current version works correctly, I would
kindly request some testers to build windows installers from some
of their package distributions, and check them to make sure they work
correctly.

The full story is in bug [#483982]: Python 2.2b2 bdist_wininst crashes:
http://sourceforge.net/tracker/index.php?func=detail&aid=483982&group_id=5470&atid=105470

To test the new version, you only have to replace the file
Lib/distutils/command/bdist_wininst.py in your Python 2.2 distribution
with the new one from CVS (rev 1.27):

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Lib/distutils/command/bdist_wininst.py

Thanks,

Thomas




From mal@lemburg.com  Tue Dec 18 21:47:24 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Tue, 18 Dec 2001 22:47:24 +0100
Subject: Need help with bdist_wininst, was Re: [Python-Dev] RE:
 [Numpy-discussion] Python 2.2
References: <000001c18715$477816c0$0c01a8c0@freedom> <028501c1871a$ec399260$e000a8c0@thomasnotebook> <09bc01c1880a$8f0a4530$e000a8c0@thomasnotebook>
Message-ID: <3C1FB96C.E3EE8DCB@lemburg.com>

Thomas Heller wrote:
> 
> I've just fixed the bdist_wininst command in CVS.
> There is not much time left before the release of Python 2.2 (how much exactly?).
> 
> To make sure that the current version works correctly, I would
> kindly request some testers to build windows installers from some
> of their package distributions, and check them to make sure they work
> correctly.
> 
> The full story is in bug [#483982]: Python 2.2b2 bdist_wininst crashes:
> http://sourceforge.net/tracker/index.php?func=detail&aid=483982&group_id=5470&atid=105470
> 
> To test the new version, you only have to replace the file
> Lib/distutils/command/bdist_wininst.py in your Python 2.2 distribution
> with the new one from CVS (rev 1.27):
> 
> http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Lib/distutils/command/bdist_wininst.py

FYI, I just tested it with egenix-mx-base and egenix-mx-commercial
and both seem to work just fine.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From tim@zope.com  Tue Dec 18 22:17:22 2001
From: tim@zope.com (Tim Peters)
Date: Tue, 18 Dec 2001 17:17:22 -0500
Subject: Need help with bdist_wininst, was Re: [Python-Dev] RE: [Numpy-discussion] Python 2.2
In-Reply-To: <09bc01c1880a$8f0a4530$e000a8c0@thomasnotebook>
Message-ID: <BIEJKCLHCIOIHAGOKOLHEENICMAA.tim@zope.com>

[Thomas Heller]
> I've just fixed the bdist_wininst command in CVS.

Thank you!
> There is not much time left before the release of Python 2.2 (how 
> much exactly?).

2.2 final should be released this Friday.



From barry@zope.com  Tue Dec 18 22:19:03 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Tue, 18 Dec 2001 17:19:03 -0500
Subject: Need help with bdist_wininst, was Re: [Python-Dev] RE: [Numpy-discussion] Python 2.2
References: <000001c18715$477816c0$0c01a8c0@freedom>
 <028501c1871a$ec399260$e000a8c0@thomasnotebook>
 <09bc01c1880a$8f0a4530$e000a8c0@thomasnotebook>
Message-ID: <15391.49367.712590.185830@anthem.wooz.org>

>>>>> "TH" == Thomas Heller <thomas.heller@ion-tof.com> writes:

    TH> I've just fixed the bdist_wininst command in CVS.  There is
    TH> not much time left before the release of Python 2.2 (how much
    TH> exactly?).

About 3 days. :)

-Barry


From paul@pfdubois.com  Wed Dec 19 00:06:57 2001
From: paul@pfdubois.com (Paul F. Dubois)
Date: Tue, 18 Dec 2001 16:06:57 -0800
Subject: Need help with bdist_wininst, was Re: [Python-Dev] RE: [Numpy-discussion] Python 2.2
In-Reply-To: <09bc01c1880a$8f0a4530$e000a8c0@thomasnotebook>
Message-ID: <000001c18821$20455ec0$0c01a8c0@freedom>

Our hero! Numeric installs ok now.

-----Original Message-----
From: numpy-discussion-admin@lists.sourceforge.net
[mailto:numpy-discussion-admin@lists.sourceforge.net] On Behalf Of
Thomas Heller
Sent: Tuesday, December 18, 2001 1:25 PM
To: Paul F. Dubois; numpy-discussion@lists.sourceforge.net
Cc: python-dev@python.org
Subject: Need help with bdist_wininst, was Re: [Python-Dev] RE:
[Numpy-discussion] Python 2.2


I've just fixed the bdist_wininst command in CVS.
There is not much time left before the release of Python 2.2 (how much
exactly?).

To make sure that the current version works correctly, I would kindly
request some testers to build windows installers from some of their
package distributions, and check them to make sure they work correctly.

The full story is in bug [#483982]: Python 2.2b2 bdist_wininst crashes:
http://sourceforge.net/tracker/index.php?func=detail&aid=483982&group_id
=5470&atid=105470

To test the new version, you only have to replace the file
Lib/distutils/command/bdist_wininst.py in your Python 2.2 distribution
with the new one from CVS (rev 1.27):

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Li
b/distutils/command/bdist_wininst.py

Thanks,

Thomas



_______________________________________________
Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion



From mhammond@skippinet.com.au  Wed Dec 19 01:57:31 2001
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Wed, 19 Dec 2001 12:57:31 +1100
Subject: [Python-Dev] No longer at ActiveState.
Message-ID: <LCEPIIGDJPKCOIHOBJEPOEJLFAAA.mhammond@skippinet.com.au>

FYI, I am no longer emplyed by ActiveState (Merry Christmas! :)

ActiveState have pretty much stopped all Komodo and Python work for the
forseeable future.  This left me without an immediate project and therefore
without an immediate job.  It was very dissapointing to hear this just after
slaving to get the 1.2 release of Komodo out the door and just a few days
before Christmas.

So, please don't use my ActiveState.com email address.  Further, I will
probably be moving back to being self-employed (in the short term anyway),
so if anyone hears of suitable opportunities, then feel free to pass them my
way :)

On the bright side, it will free my time to finish up some pending Python
work I have been meaning to do.

Happy holidays,

Mark.



From neal@metaslash.com  Wed Dec 19 02:45:03 2001
From: neal@metaslash.com (Neal Norwitz)
Date: Tue, 18 Dec 2001 21:45:03 -0500
Subject: [Python-Dev] test coverage
Message-ID: <3C1FFF2F.A69CDCE@metaslash.com>

I've been doing a bunch of testing using the regression tests
and have tried to improve the test coverage.  I've made some
progress.  Overall, the regression tests do a pretty good job.
Below are all the functions that are not currently covered by
the regression tests in Objects/ Parser/ and Python/ (Modules/
has too many at this point).  This list may not be perfect,
but should be pretty close (some tests failed:  test_new,
test_funcattrs, and test_symtable).

I don't know how many of these can be reached from within 
the python (interpreter), some may only be callable from C code 
which embeds python.

After improving test coverage, I'll go back and run purify
to see if there are any new memory leaks.

Sorry, the list is kinda long.  Let me know if there's a better way 
to distribute this info.

Neal
--
Objects/abstract.c
        PyObject_CallMethodObjArgs
        PyObject_CallFunctionObjArgs
        PyObject_Type
        PyObject_Length
        PyObject_AsWriteBuffer
        PyNumber_InPlaceTrueDivide
        PySequence_Length
        PySequence_In
        PySequence_InPlaceConcat
        PySequence_InPlaceRepeat
        PyMapping_Length
        PyMapping_HasKeyString
        PyMapping_HasKey
        PyMapping_GetItemString
        PyMapping_SetItemString
        null_error
        sliceobj_from_intint
        objargs_mktuple

Objects/bufferobject.c
        PyBuffer_FromReadWriteObject
        PyBuffer_FromReadWriteMemory
        PyBuffer_New
        buffer_compare
        buffer_getwritebuf

Objects/cellobject.c
        cell_compare
        cell_repr

Objects/classobject.c
        instance_ass_item
        PyMethod_Function
        PyMethod_Self
        PyMethod_Class
        set_slot
        set_attr_slots
        set_dict
        set_bases
        set_name
        instancemethod_hash

Objects/cobject.c
        PyCObject_FromVoidPtrAndDesc
        PyCObject_GetDesc
        PyCObject_Import

Objects/complexobject.c
        PyComplex_FromDoubles
        complex_print
        complex_div

Objects/descrobject.c
        member_repr
        getset_repr
        wrapper_repr
        method_get_doc
        wrapper_get_doc
        proxy_len
        proxy_has_key
        proxy_get
        proxy_values
        proxy_items
        proxy_copy
        proxy_getiter
        proxy_str
        proxy_traverse
        wrapper_name

Objects/dictobject.c
        PyDict_Values
        PyDict_Items
        dictiter_next

Objects/fileobject.c
        PyFile_FromString
        PyFile_Name

Objects/floatobject.c
        PyFloat_AsString
        PyFloat_AsStringEx
        float_print

Objects/funcobject.c
        PyFunction_GetCode
        PyFunction_GetGlobals
        PyFunction_GetDefaults
        PyFunction_GetClosure
        PyClassMethod_New
        func_set_code
        func_set_defaults

Objects/iterobject.c
        iter_traverse
        iter_next
        calliter_traverse
        calliter_next

Objects/listobject.c
        list_print
        immutable_list_op
        immutable_list_ass

Objects/longobject.c
        PyLong_AsVoidPtr

Objects/methodobject.c
        listmethodchain
        PyCFunction_GetFunction
        PyCFunction_GetSelf
        PyCFunction_GetFlags
        meth_traverse
        meth_get__self__
        meth_compare

Objects/object.c
        _PyObject_Dump
        PyObject_Not
        PyObject_Malloc
        PyObject_Realloc
        PyObject_Free
        none_dealloc
        NotImplemented_repr

Objects/rangeobject.c
        range_compare
        range_slice
        range_tolist
        range_getattr

Objects/sliceobject.c
        PySlice_GetIndices
        slice_compare

Objects/stringobject.c
        _Py_ReleaseInternedStrings
        PyString_Decode
        PyString_Encode
        PyString_AsEncodedString
        PyString_AsDecodedString
        string_getbuffer
        string_buffer_getwritebuf

Objects/structseq.c
        structseq_repr
        structseq_concat
        structseq_repeat
        structseq_contains
        structseq_hash

Objects/tupleobject.c
        tupleprint

Objects/typeobject.c
        slot_nb_coerce
        type_set_module
        type_subclasses
        wrap_binaryfunc_r
        wrap_coercefunc
        wrap_ternaryfunc_r
        wrap_sq_delitem
        wrap_delslice
        wrap_delitem
        wrap_hashfunc
        slot_sq_length
        slot_sq_ass_item
        slot_tp_iter
        slot_tp_iternext
        super_repr

Objects/unicodeobject.c
        PyUnicodeUCS2_Replace
        PyUnicodeUCS2_AsUnicode
        PyUnicodeUCS2_GetSize
        PyUnicodeUCS2_FromWideChar
        PyUnicodeUCS2_AsWideChar
        PyUnicodeUCS2_SetDefaultEncoding
        PyUnicodeUCS2_Encode
        PyUnicodeUCS2_AsUTF16String
        PyUnicodeUCS2_AsUnicodeEscapeString
        PyUnicodeUCS2_AsRawUnicodeEscapeString
        PyUnicodeUCS2_AsCharmapString
        PyUnicodeUCS2_Translate
        PyUnicodeUCS2_Tailmatch
        PyUnicodeUCS2_Find
        utf8_decoding_error
        utf16_decoding_error
        latin1_encoding_error
        charmap_encoding_error
        unicode_index
        unicode_rindex
        unicode_buffer_getreadbuf
        unicode_buffer_getwritebuf

Objects/weakrefobject.c
        PyWeakref_GetObject
        gc_clear
        weakref_repr
        proxy_print
        proxy_repr
        proxy_compare
        proxy_nonzero
        proxy_slice
        proxy_ass_slice
        proxy_contains
        proxy_length
        proxy_setitem


Parser/bitset.c
        _Py_delbitset

Parser/grammar1.c
        PyGrammar_LabelRepr

Parser/listnode.c
        list1node
        PyNode_ListTree
        listnode

Parser/myreadline.c
        PyOS_StdioReadline
        PyOS_Readline
        my_fgets

Parser/parsetok.c
        PyParser_ParseString

Parser/pgen.c
        dumpstate
        dumpnfa
        printssdfa

Parser/pgenmain.c
        PySys_WriteStderr
        Py_FatalError
        PyOS_Readline

Parser/tokenizer.c
        indenterror



Python/ceval.c
        call_exc_trace
        PyEval_CallObject
        Py_SetRecursionLimit
        Py_GetRecursionLimit
        PyEval_AcquireLock
        PyEval_ReleaseThread
        loop_subscript
        gen_traverse
        gen_next

Python/codecs.c
        PyCodec_StreamReader
        PyCodec_StreamWriter
        build_stream_codec

Python/compile.c
        PyCode_Addr2Line
        PyNode_CompileSymtable
        code_repr
        do_pad
        dump

Python/errors.c
        PyErr_WriteUnraisable
        PyErr_BadArgument
        PyErr_NoMemory
        PyErr_BadInternalCall
        _PyErr_BadInternalCall

Python/frozenmain.c
        Py_FrozenMain

Python/getargs.c
        PyArg_VaParse

Python/import.c
        PyImport_ExtendInittab
        PyImport_ExecCodeModule
        PyImport_AppendInittab
        get_frozen_object
        imp_init_builtin
        imp_init_frozen
        imp_get_frozen_object
        imp_is_builtin
        imp_is_frozen
        imp_load_compiled
        imp_load_dynamic
        imp_load_module
        imp_load_package

Python/marshal.c
        PyMarshal_ReadShortFromFile
        PyMarshal_ReadObjectFromFile
        w_more
        marshal_dump
        marshal_load

Python/modsupport.c
        PyEval_CallFunction
        _ustrlen

Python/pyfpe.c
        PyFPE_dummy

Python/pystate.c
        PyInterpreterState_Head
        PyInterpreterState_Next
        PyInterpreterState_ThreadHead
        PyThreadState_Next

Python/pythonrun.c
        Py_NewInterpreter
        PyRun_InteractiveLoopFlags
        PyRun_InteractiveOneFlags
        parse_syntax_error
        print_error_text
        PyErr_Display
        Py_FatalError
        Py_SetPythonHome
        Py_EndInterpreter
        PyRun_AnyFile
        PyRun_AnyFileEx
        PyRun_AnyFileFlags
        PyRun_SimpleString
        PyRun_SimpleStringFlags
        PyRun_SimpleFile
        PyRun_SimpleFileEx
        PyRun_InteractiveOne
        PyRun_InteractiveLoop
        PyRun_File
        PyRun_FileFlags
        Py_CompileString
        Py_SymtableString
        PyErr_Print
        Py_AtExit
        run_pyc_file
        add_flag

Python/structmember.c
        PyMember_Set
        listmembers

Python/symtable.c
        ste_repr

Python/sysmodule.c
        mywrite
        PySys_WriteStdout
        PySys_WriteStderr
        PySys_GetFile
        PySys_AddWarnOption
        sys_excepthook
        sys_getdefaultencoding
        sys_setdefaultencoding
        sys_setcheckinterval
        sys_setrecursionlimit
        sys_getrecursionlimit
        sys_setdlopenflags
        sys_getdlopenflags

Python/thread.c
        PyThread_allocate_sema
        PyThread_free_sema
        PyThread_down_sema
        PyThread_up_sema
        PyThread__exit_thread

Python/traceback.c
        tb_displayline
        PyTraceBack_Print
        tb_clear
        tb_printinternal


From guido@python.org  Wed Dec 19 02:51:25 2001
From: guido@python.org (Guido van Rossum)
Date: Tue, 18 Dec 2001 21:51:25 -0500
Subject: [Python-Dev] No longer at ActiveState.
In-Reply-To: Your message of "Wed, 19 Dec 2001 12:57:31 +1100."
 <LCEPIIGDJPKCOIHOBJEPOEJLFAAA.mhammond@skippinet.com.au>
References: <LCEPIIGDJPKCOIHOBJEPOEJLFAAA.mhammond@skippinet.com.au>
Message-ID: <200112190251.VAA23676@cj20424-a.reston1.va.home.com>

> FYI, I am no longer emplyed by ActiveState (Merry Christmas! :)

Au, that hurts. :(

> ActiveState have pretty much stopped all Komodo and Python work for
> the forseeable future.  This left me without an immediate project
> and therefore without an immediate job.  It was very dissapointing
> to hear this just after slaving to get the 1.2 release of Komodo out
> the door and just a few days before Christmas.

I'll say.  I'm guessing we might as well drop ActiveState from the
list of PSF sponsors now.

> So, please don't use my ActiveState.com email address.  Further, I
> will probably be moving back to being self-employed (in the short
> term anyway), so if anyone hears of suitable opportunities, then
> feel free to pass them my way :)

Hmm...  The economy is going through a rough time on this side of the
pond / border as well. :(

> On the bright side, it will free my time to finish up some pending
> Python work I have been meaning to do.

Yay!

Will we see you at the next (last) Python conference?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz@rahul.net  Wed Dec 19 03:38:52 2001
From: aahz@rahul.net (Aahz Maruch)
Date: Tue, 18 Dec 2001 19:38:52 -0800 (PST)
Subject: [Python-Dev] IPC10
In-Reply-To: <200112190251.VAA23676@cj20424-a.reston1.va.home.com> from "Guido van Rossum" at Dec 18, 2001 09:51:25 PM
Message-ID: <20011219033853.67370E8C3@waltz.rahul.net>

Guido van Rossum wrote:
> 
> Will we see you at the next (last) Python conference?

Am I misreading this, or is IPC10 going to be the final Python
conference?

(I'm still way behind on c.l.py, but I figure something like this ought
to have been posted to c.l.py.announce if true.)
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

We must not let the evil of a few trample the freedoms of the many.


From tim.one@home.com  Wed Dec 19 06:13:44 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 19 Dec 2001 01:13:44 -0500
Subject: [Python-Dev] No longer at ActiveState.
In-Reply-To: <LCEPIIGDJPKCOIHOBJEPOEJLFAAA.mhammond@skippinet.com.au>
Message-ID: <LNBBLJKPBEHFEDALKOLCMECKMOAA.tim.one@home.com>

[Mark Hammond]
> FYI, I am no longer emplyed by ActiveState (Merry Christmas! :)

Indeed!  So you'll be using the PythonLabs installer again?  Pay close
attention to its "Installation Finished" screen -- I changed that months ago
in anticipation of your coming home to us <wink>.

> ActiveState have pretty much stopped all Komodo and Python work for the
> forseeable future.  This left me without an immediate project and
> therefore without an immediate job.

Bummer.  It's times like this I'm glad Aussies don't have feelings.

> It was very dissapointing to hear this just after slaving to get the 1.2
> release of Komodo out the door and just a few days before Christmas.

Yup.  If it's any consolation, BeOpen.com stopped paying us the week before
Python 2.0 was scheduled for release.  We all wanted to go on a month-long
bender, but Guido didn't let us leave his office until the release was
finished.  See?  You *could* have been working for Guido, and *then* what a
living hell your life would have been ...

> So, please don't use my ActiveState.com email address.  Further, I will
> probably be moving back to being self-employed (in the short term anyway),
> so if anyone hears of suitable opportunities, then feel free to
> pass them my way :)

Will do.

> On the bright side, it will free my time to finish up some pending Python
> work I have been meaning to do.

If you find a way to move Armin Rigo's Psyco along while getting paid for
it, I'll join you.  Else I'll just reassign every bug and patch with the
string "win" in it to you -- heh heh.

Have a relaxed holiday, Mark!  We'll still be here when you get out of
detox.

or-at-least-guido-will-be-and-that's-enough-ly y'rs  - tim



From tim.one@home.com  Wed Dec 19 06:20:09 2001
From: tim.one@home.com (Tim Peters)
Date: Wed, 19 Dec 2001 01:20:09 -0500
Subject: [Python-Dev] IPC10
In-Reply-To: <20011219033853.67370E8C3@waltz.rahul.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGECLMOAA.tim.one@home.com>

[Guido]
> Will we see you at the next (last) Python conference?

[Aahz]
> Am I misreading this, or is IPC10 going to be the final Python
> conference?

I expect Guido was confusing tenses due to time machine travel to fix the
worst 2.2 bugs before they got reported -- they were (or would have been --
or will be had they not been fixed beforehand in the past now) really
embarrassing.

Whether there will be another Python conference is always a matter for keen
speculation.  The organizers claim to lose money on it, despite the craters
it leaves in attendees' wallets.  Ask again about a year from now ...



From thomas.heller@ion-tof.com  Wed Dec 19 07:48:23 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Wed, 19 Dec 2001 08:48:23 +0100
Subject: Need help with bdist_wininst, was Re: [Python-Dev] RE: [Numpy-discussion] Python 2.2
References: <000001c18715$477816c0$0c01a8c0@freedom> <028501c1871a$ec399260$e000a8c0@thomasnotebook> <09bc01c1880a$8f0a4530$e000a8c0@thomasnotebook>
Message-ID: <0c8101c18861$a101a790$e000a8c0@thomasnotebook>

> I've just fixed the bdist_wininst command in CVS.
> There is not much time left before the release of Python 2.2 (how much exactly?).
> 
> To make sure that the current version works correctly, I would
> kindly request some testers to build windows installers from some
> of their package distributions, and check them to make sure they work
> correctly.

Thanks for the tests, Paul and Marc-Andre.

Thomas



From fdrake@acm.org  Wed Dec 19 16:52:38 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 19 Dec 2001 11:52:38 -0500 (EST)
Subject: [Python-Dev] test coverage
In-Reply-To: <3C1FFF2F.A69CDCE@metaslash.com>
References: <3C1FFF2F.A69CDCE@metaslash.com>
Message-ID: <15392.50646.688242.432369@cj42289-a.reston1.va.home.com>

Neal Norwitz writes:
 > I don't know how many of these can be reached from within 
 > the python (interpreter), some may only be callable from C code 
 > which embeds python.

  I've added a few tests for the weakref support that should at least
improve the coverage there.  I don't think I'll have any more time to
help in this effort before 2.2 goes out, but I think this is really
important.

 > After improving test coverage, I'll go back and run purify
 > to see if there are any new memory leaks.
 > 
 > Sorry, the list is kinda long.  Let me know if there's a better way 
 > to distribute this info.

  I can live with this distribution method; if it gets us to actually
improve coverage, we all get to see the benefit of a shorter list.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From mhammond@skippinet.com.au  Thu Dec 20 06:06:04 2001
From: mhammond@skippinet.com.au (Mark Hammond)
Date: Thu, 20 Dec 2001 17:06:04 +1100
Subject: [Python-Dev] No longer at ActiveState.
In-Reply-To: <200112190251.VAA23676@cj20424-a.reston1.va.home.com>
Message-ID: <LCEPIIGDJPKCOIHOBJEPAEKLFAAA.mhammond@skippinet.com.au>

> > ActiveState have pretty much stopped all Komodo and Python work for
> > the forseeable future.  This left me without an immediate project
> > and therefore without an immediate job.  It was very dissapointing
> > to hear this just after slaving to get the 1.2 release of Komodo out
> > the door and just a few days before Christmas.
>
> I'll say.  I'm guessing we might as well drop ActiveState from the
> list of PSF sponsors now.

I obviously can't speak for ActiveState ;-)  I really have no idea exactly
what "pretty much stopped ... Python work" means in reality, nor what the
longer term plans are.

> > On the bright side, it will free my time to finish up some pending
> > Python work I have been meaning to do.
>
> Yay!

Indeed :)

> Will we see you at the next (last) Python conference?

Unfortunately not :(  It would be irresponsible of me to commit to that much
money without knowing where my next dollar was coming from :)  And if you
had ever met my girlfriend, you would know that I am very aware exactly
where my next dollar is *going* :)

Mark.



From barry@zope.com  Thu Dec 20 06:18:02 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 20 Dec 2001 01:18:02 -0500
Subject: [Python-Dev] No longer at ActiveState.
References: <200112190251.VAA23676@cj20424-a.reston1.va.home.com>
 <LCEPIIGDJPKCOIHOBJEPAEKLFAAA.mhammond@skippinet.com.au>
Message-ID: <15393.33434.171378.132917@anthem.wooz.org>

>>>>> "MH" == Mark Hammond <mhammond@skippinet.com.au> writes:

    MH> Unfortunately not :( It would be irresponsible of me to commit
    MH> to that much money without knowing where my next dollar was
    MH> coming from :) And if you had ever met my girlfriend, you
    MH> would know that I am very aware exactly where my next dollar
    MH> is *going* :)

LOL, COL.

We'll raise a pint for ya Mark.

purple-ly y'rs,
-Barry


From barry@zope.com  Thu Dec 20 19:28:32 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Thu, 20 Dec 2001 14:28:32 -0500
Subject: [Python-Dev] Python 2.2 final - release branch created
Message-ID: <15394.15328.321075.820271@anthem.wooz.org>

I've created the Python 2.2 final release branch.  The CVS tag is
different than with the alpha and beta release branches.  Here, it is
called `release22-branch'.  There's also a tag `release22-fork' --
that's the point on the trunk where the branch forks off.  So far the
only change in the branch is the version number.

As usual, no one except Guido and myself (or our authorized bots)
should make checkins to the branch.  Checkins to the trunk and branch
are discouraged unless they fix showstopper bugs in Python 2.2.
Nothing will be merged from the trunk to the branch unless you notify
us, e.g. with a comment in the checkin message.

I'm planning to cut the release tomorrow morning local time, Friday
December 21.

Enjoy,
-Barry


From skip@pobox.com (Skip Montanaro)  Thu Dec 20 21:02:01 2001
From: skip@pobox.com (Skip Montanaro) (Skip Montanaro)
Date: Thu, 20 Dec 2001 15:02:01 -0600
Subject: [Python-Dev] Can we checkin on the main trunk?
Message-ID: <15394.20937.842175.101270@beluga.mojam.com>

I have a trivial change for asynchat to shut up pychecker:

    /usr/local/lib/python2.2/asynchat.py:83: Local variable (why) not used
    /usr/local/lib/python2.2/asynchat.py:217: Local variable (why) not used

(delete the "why" variable in both cases).

Can I check in on the main trunk and have it go into 2.3 without disturbing
the 2.2 release?  I'm never sure during these alpha/beta/release candidate
situations what is and isn't allowed (other than that Barry's new branch is
clearly off-limits).  This is such a trivial change that it seems like
overkill to me to submit a bug report or a patch, have it assigned to
someone who two weeks later says, "sure, looks fine to me - check it in".

(Hint: For those of us with rotting neurons, whenever development enters a
release phase such as we are in right now, it would be helpful to remind us
what we can do as well as what we can't.)

Skip


From jack@oratrix.nl  Thu Dec 20 23:20:48 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Fri, 21 Dec 2001 00:20:48 +0100
Subject: [Python-Dev] test_cpickle failing in MacPython - ideas needed
Message-ID: <20011220232048.430FDE8453@oratrix.oratrix.nl>

Folks,
test_cpickle suddenly started failing (often with a hang) in MacPython
(CVS tree, main branch). As I'm going to be offline all day tomorrow
and I'd like to get MacPython out the door simultaneously with unix
and windows-Python: does anyone know what could have changed over the
last couple of days to cause this?
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.cwi.nl/~jack        | ++++ see http://www.xs4all.nl/~tank/ ++++


From guido@python.org  Thu Dec 20 23:19:42 2001
From: guido@python.org (Guido van Rossum)
Date: Thu, 20 Dec 2001 18:19:42 -0500
Subject: [Python-Dev] Can we checkin on the main trunk?
In-Reply-To: Your message of "Thu, 20 Dec 2001 15:02:01 CST."
 <15394.20937.842175.101270@beluga.mojam.com>
References: <15394.20937.842175.101270@beluga.mojam.com>
Message-ID: <200112202319.fBKNJjR24505@odiug.zope.com>

> I have a trivial change for asynchat to shut up pychecker:
> 
>     /usr/local/lib/python2.2/asynchat.py:83: Local variable (why) not used
>     /usr/local/lib/python2.2/asynchat.py:217: Local variable (why) not used
> 
> (delete the "why" variable in both cases).
> 
> Can I check in on the main trunk and have it go into 2.3 without disturbing
> the 2.2 release?

Yes, the release branch has been made.  The trunk is all yours.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim@zope.com  Fri Dec 21 00:19:22 2001
From: tim@zope.com (Tim Peters)
Date: Thu, 20 Dec 2001 19:19:22 -0500
Subject: [Python-Dev] test_cpickle failing in MacPython - ideas needed
In-Reply-To: <20011220232048.430FDE8453@oratrix.oratrix.nl>
Message-ID: <BIEJKCLHCIOIHAGOKOLHAEDOCNAA.tim@zope.com>

[Jack Jansen]
> test_cpickle suddenly started failing (often with a hang) in MacPython
> (CVS tree, main branch).

Ouch!

> As I'm going to be offline all day tomorrow and I'd like to get MacPython
> out the door simultaneously with unix and windows-Python: does anyone
> know what could have changed over the last couple of days to cause this?

Two very recent changes:

1. Low probability.  Guido changed the way pickle works for instances
   of new-style classes with custom metatypes.

2. High probability.  Since 2001/10/15, test_pickle and test_cpickle
   actually didn't run any tests at all.  Guido discovered this when
   working on #1 and not seeing an expected error.  When they were
   converted to unittests, an inappropriate piece of
       if __name__ == '__main__':'
   boilerplate prevented them from running any tests unless run
   directly (when run via regrtest.py, they didn't do anything).
   Fred fixed that about 24 hours ago.

Unfortunately, #2 means "the real" problem may have been introduced anytime
within the last 2 months.

Please try to whittle the failure down -- I'll be happy to help, but there
are no failures here.  It would also help to know once again *exactly* how
you run the test suite (previous problems unique to you have been pinned on
something unique about the way you run tests, no matter how much you protest
in advance that it isn't possible <wink>).



From tim.one@home.com  Fri Dec 21 02:10:47 2001
From: tim.one@home.com (Tim Peters)
Date: Thu, 20 Dec 2001 21:10:47 -0500
Subject: [Python-Dev] Can we checkin on the main trunk?
In-Reply-To: <15394.20937.842175.101270@beluga.mojam.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEKAMOAA.tim.one@home.com>

[Skip Montanaro]
> ...
> (Hint: For those of us with rotting neurons, whenever development enters a
> release phase such as we are in right now, it would be helpful to remind us
> what we can do as well as what we can't.)

The things you can do are those you don't get publicly humiliated for doing.
Sure, it's a two-pass algorithm, and even then it makes the set of things you
can do at best recursively enumerable (there's no bound on how long it may take
before you're attacked, so you can never know for sure that you're safe, only
that if you're damned you'll *eventually* find out) -- but in the absence of a
PEP, that's the best we can do for you <wink>.

To a good first approximation, don't check in anything on the release *branch*
without Barry's explicit advance blessing, and I expect you'll live to be
appreciated in 2002.



From tim.one@home.com  Fri Dec 21 02:48:11 2001
From: tim.one@home.com (Tim Peters)
Date: Thu, 20 Dec 2001 21:48:11 -0500
Subject: [Python-Dev] For Jack Jansen
Message-ID: <LNBBLJKPBEHFEDALKOLCEEKEMOAA.tim.one@home.com>

Mail to you from me bounces, Jack:

Reporting-MTA: dns; femail4.home.com
Arrival-Date: Thu, 20 Dec 2001 18:43:54 -0800
Received-From-MTA: dns; CJ569191B (65.1.136.53)

Final-Recipient: RFC822; <jack@cwi.nl>
Action: failed
Status: 5.1.1
Remote-MTA: dns; hera.cwi.nl (192.16.191.8)
Diagnostic-Code: smtp; 550 <tim.one@home.com>... Blocked relay
femail4.sdc1.sfba.home.com


I'm not sure CWI wants to cut off the 4 million remaining @Home customers
via one dumb spam trick -- although I'm sure it's an effective one <wink>.



From akuchlin@mems-exchange.org  Fri Dec 21 04:53:04 2001
From: akuchlin@mems-exchange.org (akuchlin@mems-exchange.org)
Date: Thu, 20 Dec 2001 23:53:04 -0500
Subject: [Python-Dev] 2.2 "What's New" article finished
Message-ID: <20011220235304.A18909@mozart.mems-exchange.org>

I've made a few last updates and called it complete for 2.2final.
(Bug reports and suggestions are still welcome, though.)

Note for the PythonLabs gang: if you want to make a copy on python.org
or zope.com (for press release purposes, consistent page design, or
whatever), feel free to do so; the LaTeX source is in the pyhowto CVS
tree at sourceforge.net.

Oh, yeah, the URL: still http://www.amk.ca/python/2.2/ .

--amk                                                  (www.amk.ca)
The Sony Walkman will become obsolete thanks to the Sony rnman. The
instructions will be on a separate device, the Sony manman.
    -- Kibo, in the Happynet Manifesto



From anthony@interlink.com.au  Fri Dec 21 05:57:14 2001
From: anthony@interlink.com.au (Anthony Baxter)
Date: Fri, 21 Dec 2001 16:57:14 +1100
Subject: [Python-Dev] 2.1.2 status.
Message-ID: <200112210557.fBL5vE908871@mbuna.arbhome.com.au>

I'm churning through the list of "things that I want to get fixed" for 2.1.2.
I've had to abandon my initial plan of examining pretty much all patches for
suitability - the backlog since 2.1.1 is just too huge. If there's something
you know of that you think really should be in there, and you're not sure if
it is actually there yet, please let me know ASAP! 

Anthony.


From barry@zope.com  Fri Dec 21 06:05:23 2001
From: barry@zope.com (Barry A. Warsaw)
Date: Fri, 21 Dec 2001 01:05:23 -0500
Subject: [Python-Dev] Can we checkin on the main trunk?
References: <15394.20937.842175.101270@beluga.mojam.com>
Message-ID: <15394.53539.912400.804116@anthem.wooz.org>

>>>>> "SM" == Skip Montanaro <skip@pobox.com> writes:

    SM> Can I check in on the main trunk and have it go into 2.3
    SM> without disturbing the 2.2 release?  I'm never sure during
    SM> these alpha/beta/release candidate situations what is and
    SM> isn't allowed (other than that Barry's new branch is clearly
    SM> off-limits).

Trunk checkins are always safe.  We create a release branch
specifically so that no strict freeze on checkins need be imposed on
CVS committers.

The release branch is always off-limits to everyone except those of us
who are making the releases (or our designated bots), as per PEP 101.
This way we can ensure a clean, controlled release branch but not hold
up ongoing Python development and maintenance on the trunk.

BTW, this policy is fairly new with Python 2.2 and, speaking as
Release Manager, I think it has worked exceedingly well.  We actually
got the idea from the way Zope's CVS is managed, though we've tweaked
the process a bit to suite Python's differing needs.  aIMO, the most
important adjustment was in finding the right balance between the need
for a branch and the reality that the inevitable merge from branch
back to trunk can be a royal PITA (ask Tim some time about is
descr-branch merge experience. ;)

Currently we try to branch at about noon (local time 'natch) the day
before the release.  This seems like a pretty good balance between the
conflicting goals of release isolation and a pain-free trunk-merge.

I know my announcement earlier today discouraged trunk checkins until
we spin the Python 2.2 final release.  I think that makes sense given
that this is a "final" release, so there are a few special
considerations.  I'm willing to accept that my discouraging trunk
checkins is just paranoia on my part ;).

-Barry


From davida@ActiveState.com  Fri Dec 21 18:06:25 2001
From: davida@ActiveState.com (David Ascher)
Date: Fri, 21 Dec 2001 10:06:25 -0800 (PST)
Subject: [Python-Dev] ActiveState & Python
Message-ID: <Pine.LNX.4.30.0112210958450.16835-100000@latte.ActiveState.com>

As Mark mentioned in his email to python-dev, he is no longer employed
by ActiveState.  While I'm dealing with my grief about that unfortunate
event in private, I think it's worth clarifying a few things in public:

1) Over the last two years, Mark has become my personal god of
programming.  ActiveState as a whole has benefited tremendously from
having Mark on staff, I have learned a tremendous amount from him, and
the entire ActiveState team hope to be able to bring him back on board
as soon as possible.

2) The reason for Mark's departure is simple -- ActiveState has had to
make some painful cuts in staff in order to ensure our continued
survival in these difficult economic times.  We hope these cuts will be
short-lived and that we will be able to bring back the excellent
personel that had to be let go soon.

3) ActiveState is still committed to supporting its Python initiatives and
current product line. For example, a release of ActivePython based on
Python 2.2 is forthcoming.  As regards Komodo, we do have plans for future
releases.  The features we are looking at adding in future releases
include:

	* Integrated version control support (right now it's done
	  through the "external command" interface)
	* Perl autocompletion (not sure that python-dev'ers care about
          that one =)
	* Improved web services consumption (1.2 has support
	  for this already)
	* Macro support
	* Key-mappings (like emacs and vi key modes)

As usual, we look forward to feedback from customers as to what the next
version of Komodo should look like.

If you have any questions about any of the above, feel free to contact me.
I'm on vacation but am reading email periodically.

--David Ascher
  Friend of Mark's
  Director, Programming Tools
  ActiveState




From guido@python.org  Fri Dec 21 19:54:36 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 21 Dec 2001 14:54:36 -0500
Subject: [Python-Dev] RELEASED - Python 2.2 final
Message-ID: <200112211954.OAA16972@cj20424-a.reston1.va.home.com>

On December 21, just in time to be placed under the Christmas tree,
we're issuing the final release of Python 2.2.  We're proud of this
release, and expect that you'll like it.  Please check it out here:

    http://www.python.org/2.2/

Our thanks to everyone who helped test Python 2.2 during the alpha,
beta and release candidate phases, and to everyone who contributed
patches, feedback, and suggestions!  Please continue to report any
bugs you find to the bug tracker:

    http://sourceforge.net/bugs/?group_id=5470

Highlights of what's new for this release are outlined below.  For a
more complete list, please see:

    http://sf.net/project/shownotes.php?release_id=??????


What's new since 2.1
--------------------

In brief (for PEPs, see http://www.python.org/peps/):

- type/class unification (PEP 252 and 253)
- iterators (PEP 234) and generators (PEP 255)
- nested scopes standard (PEP 227)
- int overflows return longs (PEP 237)
- new operator // to future-proof int division (PEP 238)

Andrew Kuchling has written a gentle introduction to the most important
changes, titled "What's New in Python 2.2":

  http://www.amk.ca/python/2.2/

A thorough introduction to the type/class unification is at:

  http://www.python.org/2.2/descrintro.html


Status of the type/class unification
------------------------------------

With a set of changes as large as this, it's unavoidable that after
the release we'll find that there are certain things that would work
better if we did them a little differently.  Also, it is clear that
some of the new features (e.g. class and static methods, properties,
super, and slots, to mention a few) could use additional syntactic
sugar to make their use easier and more intuitive.

This will almost certainly require us to make changes in future
releases.  However, with the release of these features as part of the
Python 2.2 release, we are committed to the same standard of
release-to-release compatibility that we have used in the past, as
explained in PEP 5.

When we feel the need to change things, we'll use the PEP system to
propose changes, and we'll provide warnings and backwards compatiblity
for at least a year -- or more in cases where a year is deemed too
short by the user community.

In the meantime, we'll work on completing the documentation for these
features (both in Python and in the C API) as they are found in Python
2.2.  This documentation will clarify the status of individual
features.


What's new since 2.2c1
----------------------

(Adapted from the Misc/NEWS file.)

Type/class unification and new-style classes

- pickle.py, cPickle: allow pickling instances of new-style classes
  with a custom metaclass.

Core and builtins

- weakref proxy object: when comparing, unwrap both arguments if both
  are proxies.

Extension modules

- binascii.b2a_base64(): fix a potential buffer overrun when encoding
  very short strings.

- cPickle: the obscure "fast" mode caused stack overflows on the Mac.
  We fixed this by setting the recursion limit much smaller.  If the
  limit is too low (it only affects performance), you can change it by
  defining PY_CPICKLE_FAST_LIMIT when compiling cPickle.c (or in
  pyconfig.h).

Library

- dumbdbm.py: fixed a dumb old bug (the file didn't get synched at
  close or delete time).

- rfc822.py: fixed a bug where the address '<>' was converted to None
  instead of an empty string (also fixes the email.Utils module).

- xmlrpclib.py: version 1.0.0; uses precision for doubles.

- test suite: the pickle and cPickle tests were not executing any code
  when run from the standard regresssion test.

Windows

- distutils package: fixed broken Windows installers (bdist_wininst).

- tempfile.py: prevent mysterious warnings when TemporaryFileWrapper
  instances are deleted at process exit time.

- socket.py: prevent mysterious warnings when socket instances are
  deleted at process exit time.

- posixmodule.c: fix a Windows crash with stat() of a filename ending
  in backslash.

Mac

- The Carbon toolbox modules have been upgraded to Universal Headers
  3.4, and experimental CoreGraphics and CarbonEvents modules have
  been added.  All only for framework-enabled MacOSX.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From fredrik@pythonware.com  Fri Dec 21 20:24:11 2001
From: fredrik@pythonware.com (Fredrik Lundh)
Date: Fri, 21 Dec 2001 21:24:11 +0100
Subject: [Python-Dev] RELEASED - Python 2.2 final
References: <200112211954.OAA16972@cj20424-a.reston1.va.home.com>
Message-ID: <001401c18a5d$746ebad0$ced241d5@hagrid>

> On December 21, just in time to be placed under the Christmas tree,
> we're issuing the final release of Python 2.2.

hurray for us!

see y'all after the holidays.

</F>



From fdrake@acm.org  Fri Dec 21 21:12:12 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 21 Dec 2001 16:12:12 -0500 (EST)
Subject: [Python-Dev] Python 2.2.1
Message-ID: <15395.42412.811106.303578@cj42289-a.reston1.va.home.com>

  I've created the branch release22-maint as the Python 2.2
maintenance branch.
  We don't yet have a Patch Czar for 2.2.1, though.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From mwh@python.net  Fri Dec 21 21:27:43 2001
From: mwh@python.net (Michael Hudson)
Date: 21 Dec 2001 21:27:43 +0000
Subject: [Python-Dev] RELEASED - Python 2.2 final
In-Reply-To: Guido van Rossum's message of "Fri, 21 Dec 2001 14:54:36 -0500"
References: <200112211954.OAA16972@cj20424-a.reston1.va.home.com>
Message-ID: <2m8zbwjmb4.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> On December 21, just in time to be placed under the Christmas tree,
> we're issuing the final release of Python 2.2.

Hurrah!

>     http://sf.net/project/shownotes.php?release_id=??????

Oops.

Cheers,
M.

-- 
  ARTHUR:  Why should a rock hum?
    FORD:  Maybe it feels good about being a rock.
                    -- The Hitch-Hikers Guide to the Galaxy, Episode 8


From guido@python.org  Fri Dec 21 21:33:15 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 21 Dec 2001 16:33:15 -0500
Subject: [Python-Dev] RELEASED - Python 2.2 final
In-Reply-To: Your message of "21 Dec 2001 21:27:43 GMT."
 <2m8zbwjmb4.fsf@starship.python.net>
References: <200112211954.OAA16972@cj20424-a.reston1.va.home.com>
 <2m8zbwjmb4.fsf@starship.python.net>
Message-ID: <200112212133.QAA18024@cj20424-a.reston1.va.home.com>

> >     http://sf.net/project/shownotes.php?release_id=??????

Oops, sorry.  The correct URL is:

http://sourceforge.net/project/shownotes.php?release_id=66610

--Guido van Rossum (home page: http://www.python.org/~guido/)


From anthony@interlink.com.au  Sun Dec 23 06:16:28 2001
From: anthony@interlink.com.au (Anthony Baxter)
Date: Sun, 23 Dec 2001 17:16:28 +1100
Subject: [Python-Dev] 2.1.2rc is ready to go.
Message-ID: <200112230616.fBN6GSS07067@mbuna.arbhome.com.au>

the 2.1.2 is ready for release candidate - I've yet to go through the hoops
to cut the rc1, but please please feel free to pull the CVS version and try
it out.

Anthony.




From guido@python.org  Sun Dec 23 14:01:48 2001
From: guido@python.org (Guido van Rossum)
Date: Sun, 23 Dec 2001 09:01:48 -0500
Subject: [Python-Dev] 2.1.2rc is ready to go.
In-Reply-To: Your message of "Sun, 23 Dec 2001 17:16:28 +1100."
 <200112230616.fBN6GSS07067@mbuna.arbhome.com.au>
References: <200112230616.fBN6GSS07067@mbuna.arbhome.com.au>
Message-ID: <200112231401.JAA08934@cj20424-a.reston1.va.home.com>

> the 2.1.2 is ready for release candidate - I've yet to go through the hoops
> to cut the rc1, but please please feel free to pull the CVS version and try
> it out.

Excellent!  When do you plan to release it?  What kind of help do you
want from us?  We'd be happy to follow PEP 101 but it might have to
wait until 2002.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mwh@python.net  Sun Dec 23 18:08:44 2001
From: mwh@python.net (Michael Hudson)
Date: 23 Dec 2001 18:08:44 +0000
Subject: [Python-Dev] Python 2.2.1
In-Reply-To: "Fred L. Drake, Jr."'s message of "Fri, 21 Dec 2001 16:12:12 -0500 (EST)"
References: <15395.42412.811106.303578@cj42289-a.reston1.va.home.com>
Message-ID: <2mzo49yfkj.fsf@starship.python.net>

"Fred L. Drake, Jr." <fdrake@acm.org> writes:

>   I've created the branch release22-maint as the Python 2.2
> maintenance branch.
>   We don't yet have a Patch Czar for 2.2.1, though.

I've just spent half an hour fiddling with gnus to the point that it
should be easy for me to keep a log of checkins that should be put on
the release22-maint branch.

This isn't me volunteering to be Patch Czar, by the way... I don't
know if I'll have time for that, but I hope that doing this will help
whoever it ends up being.

Cheers,
M.

-- 
  About the use of language: it is impossible to sharpen a
  pencil with a blunt axe.  It is equally vain to try to do
  it with ten blunt axes instead.
      -- E.W.Dijkstra, 18th June 1975. Perl did not exist at the time.


From jack@oratrix.nl  Sun Dec 23 22:22:04 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Sun, 23 Dec 2001 23:22:04 +0100
Subject: [Python-Dev] Python 2.2.1
In-Reply-To: Message by Michael Hudson <mwh@python.net> ,
 23 Dec 2001 18:08:44 +0000 , <2mzo49yfkj.fsf@starship.python.net>
Message-ID: <20011223222209.C9CE5E8453@oratrix.oratrix.nl>

Two questions wrt 2.2.1:

- Is there a timeline for it? I couldn't find one in PEP251 (which
stops at 2.2 final). The reason I ask is that there's a serious flaw
in MacPython (has been there since day one) that renders it useless on
multiprocessor Mac OS X machines. I'm waiting for an external party to
fix this (as the problem is in an external library), but I could put a
bit of pressure on if 2.2.1 is due reasonably soon (as I hope).

- The final bits of the Mac tree havne't been checked in yet, should
we move the release22-maint branch when that's done? Can you move a
branch? How? 
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.cwi.nl/~jack        | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 


From Anthony Baxter <anthony@interlink.com.au>  Sun Dec 23 22:25:02 2001
From: Anthony Baxter <anthony@interlink.com.au> (Anthony Baxter)
Date: Mon, 24 Dec 2001 09:25:02 +1100
Subject: [Python-Dev] Python 2.2.1
In-Reply-To: Message from Michael Hudson <mwh@python.net>
 of "23 Dec 2001 18:08:44 -0000." <2mzo49yfkj.fsf@starship.python.net>
Message-ID: <200112232225.fBNMP2910673@mbuna.arbhome.com.au>

>>> Michael Hudson wrote
> I've just spent half an hour fiddling with gnus to the point that it
> should be easy for me to keep a log of checkins that should be put on
> the release22-maint branch.
> 
> This isn't me volunteering to be Patch Czar, by the way... I don't
> know if I'll have time for that, but I hope that doing this will help
> whoever it ends up being.

I'm happy enough to continue doing this - now that it's starting from 
a caught-up position, it's not that hard. The problem with 2.1.2 was
trawling back through 6 months of checkins...

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.



From fdrake@acm.org  Sun Dec 23 22:29:56 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Sun, 23 Dec 2001 17:29:56 -0500 (EST)
Subject: [Python-Dev] Python 2.2.1
In-Reply-To: <20011223222209.C9CE5E8453@oratrix.oratrix.nl>
References: <mwh@python.net>
 <2mzo49yfkj.fsf@starship.python.net>
 <20011223222209.C9CE5E8453@oratrix.oratrix.nl>
Message-ID: <15398.23268.960166.935728@cj42289-a.reston1.va.home.com>

Jack Jansen writes:
 > - The final bits of the Mac tree havne't been checked in yet, should
 > we move the release22-maint branch when that's done? Can you move a
 > branch? How? 

  Yes, it should be moved as necessary to reflect the MacPython
release; this can be done using these commands from the dist/src/
directory:

    cvs tag release22-mac
    cvs tag -r release22-mac -b -F release22-maint 


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From guido@python.org  Sun Dec 23 22:31:52 2001
From: guido@python.org (Guido van Rossum)
Date: Sun, 23 Dec 2001 17:31:52 -0500
Subject: [Python-Dev] Python 2.2.1
In-Reply-To: Your message of "Sun, 23 Dec 2001 23:22:04 +0100."
 <20011223222209.C9CE5E8453@oratrix.oratrix.nl>
References: <20011223222209.C9CE5E8453@oratrix.oratrix.nl>
Message-ID: <200112232231.RAA15943@cj20424-a.reston1.va.home.com>

> Two questions wrt 2.2.1:
> 
> - Is there a timeline for it? I couldn't find one in PEP251 (which
> stops at 2.2 final). The reason I ask is that there's a serious flaw
> in MacPython (has been there since day one) that renders it useless on
> multiprocessor Mac OS X machines. I'm waiting for an external party to
> fix this (as the problem is in an external library), but I could put a
> bit of pressure on if 2.2.1 is due reasonably soon (as I hope).

No timeline -- that sort of depends on how many really bad bugs are
reported, and whether someone volunteers as a release mananger.

> - The final bits of the Mac tree havne't been checked in yet, should
> we move the release22-maint branch when that's done? Can you move a
> branch? How? 

Fred anticipated that; he said he would just move the tags when
necessary.  Don't worry about it.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From Anthony Baxter <anthony@interlink.com.au>  Sun Dec 23 22:34:32 2001
From: Anthony Baxter <anthony@interlink.com.au> (Anthony Baxter)
Date: Mon, 24 Dec 2001 09:34:32 +1100
Subject: [Python-Dev] Python 2.2.1
In-Reply-To: Message from Jack Jansen <jack@oratrix.nl>
 of "Sun, 23 Dec 2001 23:22:04 BST." <20011223222209.C9CE5E8453@oratrix.oratrix.nl>
Message-ID: <200112232234.fBNMYWc10781@mbuna.arbhome.com.au>

>>> Jack Jansen wrote
> - The final bits of the Mac tree havne't been checked in yet, should
> we move the release22-maint branch when that's done? Can you move a
> branch? How? 

D'oh! Hm, I'm not sure how easily that can be done. In theory you should be
able to "cvs tag -d release22-maint" in the Mac-specific subdirectory, then
do the checkins, then "cvs tag -b release22-maint" in the Mac directory again.
So long as there's not been any checkins on the branch in the directories that
you're doing the tag shifting, you should be fine. Probably. Although checking
on a little toy CVS root is probably a great idea :)
In fact, I just did this on a local toy project, and it worked fine.

I'd strongly recommend using cvs tag -d followed by retagging, rather than
using tag -F to try and move it - I've had bad experiences in the past with
-F 

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.



From martin@v.loewis.de  Mon Dec 24 08:32:13 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Mon, 24 Dec 2001 09:32:13 +0100
Subject: [Python-Dev] Python 2.2.1
In-Reply-To: <15398.23268.960166.935728@cj42289-a.reston1.va.home.com>
 (fdrake@acm.org)
References: <mwh@python.net>
 <2mzo49yfkj.fsf@starship.python.net>
 <20011223222209.C9CE5E8453@oratrix.oratrix.nl> <15398.23268.960166.935728@cj42289-a.reston1.va.home.com>
Message-ID: <200112240832.fBO8WDO01324@mira.informatik.hu-berlin.de>

>  > - The final bits of the Mac tree havne't been checked in yet, should
>  > we move the release22-maint branch when that's done? Can you move a
>  > branch? How? 
> 
>   Yes, it should be moved as necessary to reflect the MacPython
> release; this can be done using these commands from the dist/src/
> directory:
> 
>     cvs tag release22-mac
>     cvs tag -r release22-mac -b -F release22-maint 

Maybe I'm misunderstanding. It seems pointless to move a branch: Just
check in to it. In terms of CVS commands

cvs update -rrelease22-maint # get a sandbox of the branch
# make modifications
cvs commit                   # make modifications to the branch
cvs update -A                # return to mainline

What am I missing?

Regards,
Martin


From fdrake@acm.org  Tue Dec 25 16:49:23 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 25 Dec 2001 11:49:23 -0500 (EST)
Subject: [Python-Dev] Python 2.2.1
In-Reply-To: <200112240832.fBO8WDO01324@mira.informatik.hu-berlin.de>
References: <mwh@python.net>
 <2mzo49yfkj.fsf@starship.python.net>
 <20011223222209.C9CE5E8453@oratrix.oratrix.nl>
 <15398.23268.960166.935728@cj42289-a.reston1.va.home.com>
 <200112240832.fBO8WDO01324@mira.informatik.hu-berlin.de>
Message-ID: <15400.44563.436248.81801@cj42289-a.reston1.va.home.com>

Martin v. Loewis writes:
 > Maybe I'm misunderstanding. It seems pointless to move a branch: Just
 > check in to it. In terms of CVS commands

  That would work too; I'm not going to be picky about the process.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From fdrake@acm.org  Wed Dec 26 17:15:40 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 26 Dec 2001 12:15:40 -0500 (EST)
Subject: [Python-Dev] Python 2.2.1
In-Reply-To: <200112232234.fBNMYWc10781@mbuna.arbhome.com.au>
References: <jack@oratrix.nl>
 <20011223222209.C9CE5E8453@oratrix.oratrix.nl>
 <200112232234.fBNMYWc10781@mbuna.arbhome.com.au>
Message-ID: <15402.1468.577251.936398@cj42289-a.reston1.va.home.com>

  Has anything been done to get the release22-maint branch to see the
changes made for MacPython yet?  We should do this before we lose
track of it.
  Jack, any preference for how this is done, or did you already do it?


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From jack@oratrix.nl  Wed Dec 26 22:37:26 2001
From: jack@oratrix.nl (Jack Jansen)
Date: Wed, 26 Dec 2001 23:37:26 +0100
Subject: [Python-Dev] Python 2.2.1
In-Reply-To: Message by "Fred L. Drake, Jr." <fdrake@acm.org> ,
 Wed, 26 Dec 2001 12:15:40 -0500 (EST) , <15402.1468.577251.936398@cj42289-a.reston1.va.home.com>
Message-ID: <20011226223731.262A1E8448@oratrix.oratrix.nl>

Recently, "Fred L. Drake, Jr." <fdrake@acm.org> said:
> 
>   Has anything been done to get the release22-maint branch to see the
> changes made for MacPython yet?  We should do this before we lose
> track of it.
>   Jack, any preference for how this is done, or did you already do it?

They haven\t been checked in yet. There was a snag in the Mac
distriubution that has just been solved, the checkins will follow shortly.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.cwi.nl/~jack        | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 


From fdrake@acm.org  Wed Dec 26 22:39:17 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Wed, 26 Dec 2001 17:39:17 -0500 (EST)
Subject: [Python-Dev] Python 2.2.1
In-Reply-To: <20011226223731.262A1E8448@oratrix.oratrix.nl>
References: <fdrake@acm.org>
 <15402.1468.577251.936398@cj42289-a.reston1.va.home.com>
 <20011226223731.262A1E8448@oratrix.oratrix.nl>
Message-ID: <15402.20885.998208.923051@cj42289-a.reston1.va.home.com>

Jack Jansen writes:
 > They haven\t been checked in yet. There was a snag in the Mac
 > distriubution that has just been solved, the checkins will follow shortly.

  OK, I'll watch for them.  You can make sure the changes hit both the
branch and the trunk yourself, or check into the trunk and we can play
with CVS to move the changes onto the branch for the Mac tree.
  If you want me to do anything to help out with this, I should be
able to lend a hand later this evening or in the morning.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From fdrake@acm.org  Fri Dec 28 04:56:31 2001
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 27 Dec 2001 23:56:31 -0500 (EST)
Subject: [Python-Dev] Documentation for Python 2.2.1
Message-ID: <15403.64383.728258.799543@cj42289-a.reston1.va.home.com>

  I've brought the docs in Python's release22-maint branch up to date
with the corrections that have been made on the trunk.  I'll be
checking in relevant additions to this branch as well as the trunk
when I make the trunk changes.
  When anyone else makes a change on the trunk which should also be
made on the 2.2 maintenance branch, feel free to check it in in both
places, or add a note to the checkin message noting that it should be
made on the branch as well, and I'll try to make sure that gets done.
  The 2.2 maintenance branch has replaced the 2.1 maintenance branch
as the "maintenance version" at http://python.sourceforge.net/; if
anyone thinks we should continue to offer the 2.1 maintenance docs
there, please let me know.
  I don't think we have a Patch Czar for 2.2.1 yet; if anyone would
like to take that duty for the documentation tree, I'd certainly
appreciate the help!
  Thanks!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation



From fdrake@acm.org  Fri Dec 28 04:59:57 2001
From: fdrake@acm.org (Fred L. Drake)
Date: Thu, 27 Dec 2001 23:59:57 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20011228045957.6E9FD286BB@beowolf.digicool.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/maint-docs/

The "maintenance version" of the Python documentation on the Python
development site at SourceForge has been updated to show the docs
being developed for Python 2.2.1.



From fdrake@acm.org  Fri Dec 28 05:03:42 2001
From: fdrake@acm.org (Fred L. Drake)
Date: Fri, 28 Dec 2001 00:03:42 -0500 (EST)
Subject: [Python-Dev] [development doc updates]
Message-ID: <20011228050342.CB129286BB@beowolf.digicool.com>

The development version of the documentation has been updated:

    http://python.sourceforge.net/devel-docs/

A variety of updates made since Python 2.2 was released.



From martin@v.loewis.de  Fri Dec 28 10:13:17 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Fri, 28 Dec 2001 11:13:17 +0100
Subject: [Python-Dev] Re: [XML-SIG] printing Unicode xml to StringIO
In-Reply-To: <3C2C3406.CB6D49DB@lemburg.com> (mal@lemburg.com)
References: <200112271506.fBRF6uF26242@spinaker.ins.cwi.nl> <200112271736.fBRHaom02848@mira.informatik.hu-berlin.de> <3C2C3406.CB6D49DB@lemburg.com>
Message-ID: <200112281013.fBSADHu01819@mira.informatik.hu-berlin.de>

[Over to python-dev. Jaco noticed that writing Unicode objects to
 a StringIO object stopped working in 2.2, see
http://mail.python.org/pipermail/xml-sig/2001-December/006891.html
]

Marc-Andre writes
> Actually, I think that this is a bug in the documentation, not the
> code. StringIO and cStringIO were never meant to work on anything but
> strings and memory buffers. 

IMO, "strings" should include both byte strings and Unicode strings.
Mixing them may not be allowed, but that is a different story. 

In fact, there is an open bug (#216388) that cStringIO rejects Unicode
objects. If that gets fixed, we get the funny scenario that StringIO
rejects Unicode object, whereas cStringIO accepts them.

> The note that Fred added to the docs about StringIO's capability of
> storing Unicode in it's buffer list is simply an artifact of the
> implementation.

There are many developers who take this note literally. Claiming that
this was not intentional is a mistake.

> Please use the .encode() method on Unicode objects before writing
> them to a StringIO object.

If you want to end up with a byte string, this is a good idea. But I
think it is pointless to require encoding them when you want to end up
with a Unicode string; you'd have to invoke unicode() on the result,
for no apparent reason but a bug in the StringIO implementation.

Regards,
Martin


From mal@lemburg.com  Fri Dec 28 10:50:25 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 28 Dec 2001 11:50:25 +0100
Subject: [Python-Dev] Re: [XML-SIG] printing Unicode xml to StringIO
References: <200112271506.fBRF6uF26242@spinaker.ins.cwi.nl> <200112271736.fBRHaom02848@mira.informatik.hu-berlin.de> <3C2C3406.CB6D49DB@lemburg.com> <200112281013.fBSADHu01819@mira.informatik.hu-berlin.de>
Message-ID: <3C2C4E71.2BA58BF3@lemburg.com>

"Martin v. Loewis" wrote:
> 
> [Over to python-dev. Jaco noticed that writing Unicode objects to
>  a StringIO object stopped working in 2.2, see
> http://mail.python.org/pipermail/xml-sig/2001-December/006891.html
> ]
> 
> Marc-Andre writes
> > Actually, I think that this is a bug in the documentation, not the
> > code. StringIO and cStringIO were never meant to work on anything but
> > strings and memory buffers.
> 
> IMO, "strings" should include both byte strings and Unicode strings.
> Mixing them may not be allowed, but that is a different story.
> 
> In fact, there is an open bug (#216388) that cStringIO rejects Unicode
> objects. If that gets fixed, we get the funny scenario that StringIO
> rejects Unicode object, whereas cStringIO accepts them.

StringIO and cStringIO use different methods for storing the
snippets: StringIO makes use of a buffer list which gets
compressed every now and then, while cStringIO uses a raw
memory buffer for this purpose.

Both of these implementation are targetted at providing
a file IO like interface to in-memory "files". Since Python
file object don't magically support Unicode, I wonder where the
idea came from that StringIO/cStringIO should.

That patch I applied to StringIO/cStringIO for 2.2 was
aimed at making these two more compatible to the standard
Python file object. The latter uses the "s#" parser
marker for .write() and thus can also accept memory
buffers. This was previously not possible with either
of the two StringIO implementation (StringIO.py failed
when trying to join different buffer compatible objects,
cStringIO only accepted real string objects).

> > The note that Fred added to the docs about StringIO's capability of
> > storing Unicode in it's buffer list is simply an artifact of the
> > implementation.
> 
> There are many developers who take this note literally. Claiming that
> this was not intentional is a mistake.
>
> > Please use the .encode() method on Unicode objects before writing
> > them to a StringIO object.
> 
> If you want to end up with a byte string, this is a good idea. 

That's the idea behind StringIO objects... they are in-memory file 
object emulators.

> But I
> think it is pointless to require encoding them when you want to end up
> with a Unicode string; you'd have to invoke unicode() on the result,
> for no apparent reason but a bug in the StringIO implementation.

This is a different application. It should be easy enough to
subclass StringIO as UnicodeIO class and then have this class
implement fast Unicode snippet joining. I'm not sure whether
the same can be done with cStringIO's type.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From martin@v.loewis.de  Fri Dec 28 11:04:30 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Fri, 28 Dec 2001 12:04:30 +0100
Subject: [Python-Dev] Re: [XML-SIG] printing Unicode xml to StringIO
In-Reply-To: <3C2C4E71.2BA58BF3@lemburg.com> (mal@lemburg.com)
References: <200112271506.fBRF6uF26242@spinaker.ins.cwi.nl> <200112271736.fBRHaom02848@mira.informatik.hu-berlin.de> <3C2C3406.CB6D49DB@lemburg.com> <200112281013.fBSADHu01819@mira.informatik.hu-berlin.de> <3C2C4E71.2BA58BF3@lemburg.com>
Message-ID: <200112281104.fBSB4UI02359@mira.informatik.hu-berlin.de>


From martin@v.loewis.de  Fri Dec 28 11:25:46 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Fri, 28 Dec 2001 12:25:46 +0100
Subject: [Python-Dev] Re: [XML-SIG] printing Unicode xml to StringIO
In-Reply-To: <3C2C4E71.2BA58BF3@lemburg.com> (mal@lemburg.com)
References: <200112271506.fBRF6uF26242@spinaker.ins.cwi.nl> <200112271736.fBRHaom02848@mira.informatik.hu-berlin.de> <3C2C3406.CB6D49DB@lemburg.com> <200112281013.fBSADHu01819@mira.informatik.hu-berlin.de> <3C2C4E71.2BA58BF3@lemburg.com>
Message-ID: <200112281125.fBSBPkC02403@mira.informatik.hu-berlin.de>

> Both of these implementation are targetted at providing
> a file IO like interface to in-memory "files". Since Python
> file object don't magically support Unicode, I wonder where the
> idea came from that StringIO/cStringIO should.

The exact source of this idea is unknown. However, there are many
early references to it:

- codecs.open returns an "encoded file" which "will only accept
  ... Unicode objects". That is perhaps the earliest precedent of
  a file object supporting Unicode.

- At some point in time, you said that it is a bug that cStringIO
  does not support Unicode strings, see
http://mail.python.org/pipermail/i18n-sig/2000-November/000550.html

- the documentation of StringIO suggests that they should accept
  Unicode.

So I would not blame the users for adopting far-off ideas, when the
Python core itself suggests that these ideas are Pythonic.

> That patch I applied to StringIO/cStringIO for 2.2 was
> aimed at making these two more compatible to the standard
> Python file object. The latter uses the "s#" parser
> marker for .write() and thus can also accept memory
> buffers. This was previously not possible with either
> of the two StringIO implementation (StringIO.py failed
> when trying to join different buffer compatible objects,
> cStringIO only accepted real string objects).

There is nothing wrong with that. The patch should just have
special-cased Unicode objects (and that bug can still be corrected).

Regards,
Martin


From guido@python.org  Fri Dec 28 14:20:18 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 28 Dec 2001 09:20:18 -0500
Subject: [Python-Dev] Re: [XML-SIG] printing Unicode xml to StringIO
In-Reply-To: Your message of "Fri, 28 Dec 2001 12:25:46 +0100."
 <200112281125.fBSBPkC02403@mira.informatik.hu-berlin.de>
References: <200112271506.fBRF6uF26242@spinaker.ins.cwi.nl> <200112271736.fBRHaom02848@mira.informatik.hu-berlin.de> <3C2C3406.CB6D49DB@lemburg.com> <200112281013.fBSADHu01819@mira.informatik.hu-berlin.de> <3C2C4E71.2BA58BF3@lemburg.com>
 <200112281125.fBSBPkC02403@mira.informatik.hu-berlin.de>
Message-ID: <200112281420.JAA22961@cj20424-a.reston1.va.home.com>

Whoa!

- Since we added a note to the docs that StringIO supports Unicode, we
  clearly should continue to support that, and it's a bug if it
  doesn't.

- OTOH, Unicode for cStringIO should be considered at best a feature
  request.  I don't mind if cStringIO doesn't support Unicode -- it
  never has, AFAIK, so it won't break much code.  I don't believe it's
  much faster than StringIO, unless you use the C API (like cPickle
  does).

- Of course, when Unicode is supported, mixing ASCII and Unicode
  should be supported too.  (But not necessarily mixing 8-bit strings
  containing characters in the range \200-\377, since there's no
  default encoding for this range.)

- Since this changed from 2.1 to 2.2, we should restore this
  capability in 2.2.1; I would say that 2.2.1 can't go out until this
  is fixed.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From mal@lemburg.com  Fri Dec 28 15:30:34 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 28 Dec 2001 16:30:34 +0100
Subject: [Python-Dev] Re: [XML-SIG] printing Unicode xml to StringIO
References: <200112271506.fBRF6uF26242@spinaker.ins.cwi.nl> <200112271736.fBRHaom02848@mira.informatik.hu-berlin.de> <3C2C3406.CB6D49DB@lemburg.com> <200112281013.fBSADHu01819@mira.informatik.hu-berlin.de> <3C2C4E71.2BA58BF3@lemburg.com>
 <200112281125.fBSBPkC02403@mira.informatik.hu-berlin.de> <200112281420.JAA22961@cj20424-a.reston1.va.home.com>
Message-ID: <3C2C901A.D61001D9@lemburg.com>

Guido van Rossum wrote:
> 
> Whoa!
> 
> - Since we added a note to the docs that StringIO supports Unicode, we
>   clearly should continue to support that, and it's a bug if it
>   doesn't.

I still believe that the docs are wrong, but nevermind. I'll fix
StringIO.py to continue to support Unicode in addition to strings
and buffer objects. It's basically only about special casing
Unicode in the .write() method.

BTW, I was never aware of the doc changes in this area and the 
test suite didn't bring up the issues either.

> - OTOH, Unicode for cStringIO should be considered at best a feature
>   request.  I don't mind if cStringIO doesn't support Unicode -- it
>   never has, AFAIK, so it won't break much code.  I don't believe it's
>   much faster than StringIO, unless you use the C API (like cPickle
>   does).

Unicode support in cStringIO would require a new implementation
since the machinery uses raw byte buffers.
 
> - Of course, when Unicode is supported, mixing ASCII and Unicode
>   should be supported too.  (But not necessarily mixing 8-bit strings
>   containing characters in the range \200-\377, since there's no
>   default encoding for this range.)

In StringIO.py this is not much of a problem since it uses
a list of snippets. Note that this is also why StringIO.py "supported"
Unicode in the first place (and that's why I think it was more an
artifact of the implementation than true intent).
 
> - Since this changed from 2.1 to 2.2, we should restore this
>   capability in 2.2.1; I would say that 2.2.1 can't go out until this
>   is fixed.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/


From guido@python.org  Fri Dec 28 15:27:02 2001
From: guido@python.org (Guido van Rossum)
Date: Fri, 28 Dec 2001 10:27:02 -0500
Subject: [Python-Dev] Re: [XML-SIG] printing Unicode xml to StringIO
In-Reply-To: Your message of "Fri, 28 Dec 2001 16:30:34 +0100."
 <3C2C901A.D61001D9@lemburg.com>
References: <200112271506.fBRF6uF26242@spinaker.ins.cwi.nl> <200112271736.fBRHaom02848@mira.informatik.hu-berlin.de> <3C2C3406.CB6D49DB@lemburg.com> <200112281013.fBSADHu01819@mira.informatik.hu-berlin.de> <3C2C4E71.2BA58BF3@lemburg.com> <200112281125.fBSBPkC02403@mira.informatik.hu-berlin.de> <200112281420.JAA22961@cj20424-a.reston1.va.home.com>
 <3C2C901A.D61001D9@lemburg.com>
Message-ID: <200112281527.KAA23737@cj20424-a.reston1.va.home.com>

> > - Since we added a note to the docs that StringIO supports Unicode, we
> >   clearly should continue to support that, and it's a bug if it
> >   doesn't.
> 
> I still believe that the docs are wrong, but nevermind. I'll fix
> StringIO.py to continue to support Unicode in addition to strings
> and buffer objects. It's basically only about special casing
> Unicode in the .write() method.

Thanks.

> BTW, I was never aware of the doc changes in this area and the 
> test suite didn't bring up the issues either.

Can you please add something to the test suite that makes sure this
feature works?

> > - OTOH, Unicode for cStringIO should be considered at best a feature
> >   request.  I don't mind if cStringIO doesn't support Unicode -- it
> >   never has, AFAIK, so it won't break much code.  I don't believe it's
> >   much faster than StringIO, unless you use the C API (like cPickle
> >   does).
> 
> Unicode support in cStringIO would require a new implementation
> since the machinery uses raw byte buffers.

That's why I don't care much about it. :-)

> > - Of course, when Unicode is supported, mixing ASCII and Unicode
> >   should be supported too.  (But not necessarily mixing 8-bit strings
> >   containing characters in the range \200-\377, since there's no
> >   default encoding for this range.)
> 
> In StringIO.py this is not much of a problem since it uses
> a list of snippets. Note that this is also why StringIO.py "supported"
> Unicode in the first place (and that's why I think it was more an
> artifact of the implementation than true intent).

But it was useful! :-)

> > - Since this changed from 2.1 to 2.2, we should restore this
> >   capability in 2.2.1; I would say that 2.2.1 can't go out until this
> >   is fixed.

Try to mark the checkin messages as "2.2.1 bugfix", for the 2.2.1
patch czar.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From martin@v.loewis.de  Fri Dec 28 17:02:17 2001
From: martin@v.loewis.de (Martin v. Loewis)
Date: Fri, 28 Dec 2001 18:02:17 +0100
Subject: [Python-Dev] Re: [XML-SIG] printing Unicode xml to StringIO
In-Reply-To: <3C2C901A.D61001D9@lemburg.com> (mal@lemburg.com)
References: <200112271506.fBRF6uF26242@spinaker.ins.cwi.nl> <200112271736.fBRHaom02848@mira.informatik.hu-berlin.de> <3C2C3406.CB6D49DB@lemburg.com> <200112281013.fBSADHu01819@mira.informatik.hu-berlin.de> <3C2C4E71.2BA58BF3@lemburg.com>
 <200112281125.fBSBPkC02403@mira.informatik.hu-berlin.de> <200112281420.JAA22961@cj20424-a.reston1.va.home.com> <3C2C901A.D61001D9@lemburg.com>
Message-ID: <200112281702.fBSH2Ha04205@mira.informatik.hu-berlin.de>

> Unicode support in cStringIO would require a new implementation
> since the machinery uses raw byte buffers.

Not necessarily. You could add a flag saying that whether those bytes
represent Unicode strings or not. If they do, you pick an encoding of
your choice (perhaps unicode-internal), and convert the existing bytes
to that encoding on first sighting of a Unicode string (assuming that
everything so far is in the system encoding). When returning the bytes
to the user, you build a Unicode object if the flag is set.

Of course, you'd still have to touch every method, to analyse the
flag...

Regards,
Martin


From Jack.Jansen@cwi.nl  Mon Dec 31 13:39:09 2001
From: Jack.Jansen@cwi.nl (Jack Jansen)
Date: Mon, 31 Dec 2001 14:39:09 +0100
Subject: [Python-Dev] Offline for a while
Message-ID: <200112311339.fBVDd9O16531@spruit.ins.cwi.nl>

Folks,
I will probably be unreacable by mail for the next few days, the
backup we put in for our move seems to be non-functional.

So don't worry about bounceds to the oratrix.nl address, all should
be well again on wednseday or thusrday (and backspace also doesn't work
on this machine:-)

	Jack