From martin at v.loewis.de  Mon Sep  1 07:04:00 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Mon Sep  1 02:04:01 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCENGFJAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCENGFJAB.tim.one@comcast.net>
Message-ID: <m3d6elkpue.fsf@mira.informatik.hu-berlin.de>

"Tim Peters" <tim.one@comcast.net> writes:

> > This is much worse, then: How can it possibly know what formats the C
> > library expects in the current locale? What if the C library insists that
> > a thousands-separator is used when the locale has one?  etc.
> 
> I'm not sure that's a realistic objection.

Ok. Are you then, overall, in favour of taking the proposed approach?

It is not thread-safe, but only so if somebody calls setlocale in a
different thread, and that is known not to be thread-safe - so I could
live with that limitation.

It is just that the patch does not "feel" right, given that there must
be "native" locale-inaware parsing of floating point constants
somewhere on each platform (atleast on those that support C++98).

Regards,
Martin

From martin at v.loewis.de  Mon Sep  1 07:36:16 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Mon Sep  1 02:36:17 2003
Subject: [Python-Dev] Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <20030814022128.GN3095@async.com.br>
References: <20030814022128.GN3095@async.com.br>
Message-ID: <m3znhpj9s0.fsf@mira.informatik.hu-berlin.de>

Christian Reis <kiko@async.com.br> writes:

> So, in an attempt to garner comments (now that we have 2.3 off the
> chopping block) I'm reposting my PEP proposal (with minor updates).

I can agree with the declared problem of the PEP, and the rationale
for fixing it. Tim also convinced me that the approach taken to solve
it is, technically, acceptable. So I only list issues where I
disagree.

>     This change should also solve the aforementioned thread-safety
>     problems.

It does not, and I think the PEP should point out that it doesn't.

>     This problem was initially reported as a problem in the GTK+
>     libraries [5]; since then it has been correctly diagnosed as an
>     inconsistency in Python's implementation. However, in a fortunate
>     coincidence, the glib library implements a number of
>     LC_NUMERIC-agnostic functions (for an example, see [6]) for reasons
>     similar to those presented in this paper.

One of my early concerns (and I still have this concern) is that the
contributors here appear to take the position "We have this fine code
developed elsewhere, it seems to work, so we copy it. We don't
actually have to understand this code". I would feel more comfortable
if the code was written from scratch for usage in Python, with just
the ideas borrowed from glib. Proper attribution of contributors and
licensing are just one aspect, we really need the submitter of the
code fully understand it, and be capable of reacting to problems
quickly.

That said, I don't actually require that the code is written from
scratch. Instead, a detailed elaboration of how precisely the
implementation is approached, in the PEP, would be good.

The PEP should also point out deficiencies of the approach taken,
e.g. the issue of spelling NaN, inf, etc. If it can be determined not
to be an issue in real life (i.e. for all interesting platforms), this
should be documented as well.

Regards,
Martin

From james at daa.com.au  Mon Sep  1 15:59:32 2003
From: james at daa.com.au (James Henstridge)
Date: Mon Sep  1 02:59:40 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIECMFJAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCIECMFJAB.tim.one@comcast.net>
Message-ID: <3F52EE54.6090800@daa.com.au>

On 31/08/2003 9:25 AM, Tim Peters wrote:

>[James Henstridge]
>  
>
>>>As Christian said, there is code in glib (not to be confused with
>>>glibc: the GNU C library) that could act as a basis for locale
>>>independent float conversion functions in Python.
>>>      
>>>
>
>[Martin v. L?wis]
>  
>
>>I very much doubt that this statement is true. Are you sure this code
>>supports all the platforms where Python runs? E.g. what about the
>>three (!) different floating point formats on a VAX?
>>    
>>
>
>Well, you should look at the patch:  it doesn't know anything about internal
>fp formats -- all conversions are performed in the end by calling the
>platform C's strtod() or snprintf().  What it does do is:
>
>1. For string to double, preprocess the input string to change it to
>   use current-locale spelling before calling the platform C strtod().
>
>2. For double to string, postprocess the result of the platform C
>   snprintf() to replace current-locale spelling with a "standard"
>   spelling.
>
>So this is much more string-munging code than it is floating-point code.
>Indeed, there doesn't appear to be a single floating-point operation in the
>entire patch (apart from calls to platform string<->float functions).
>
>OTOH, despite the claims, it doesn't look threadsafe to me:  there's no
>guarantee, e.g., that the idea of current locale g_ascii_strtod() obtains
>from localeconv() at its start is still in effect by the time
>g_ascii_strtod() gets around to calling strtod().
>
This is true.  However, in practice we found it fixed a number of thread 
safety issues in programs.

Your average localised package usually switches to the user's preferred 
locale on startup, so that it can display strings and messages, and 
occasionally wants to read/write numbers in a locale independent format 
(usually when saving/loading files).  The most common way of doing this 
is the setlocale/strtod/setlocale combo, which has thread safety 
problems and possible reentrancy problems if done wrong.

The method used by g_ascii_strotod() removes the need to switch locale 
when parsing the float, which means that an application using it may 
only need to call setlocale() once on startup and never again.  This 
seems to be the best way to use setlocale w.r.t. thread safety.

The existing locale handling in Python shares this property, but makes 
it difficult for external libraries to format and parse floats in the 
locale's representation.  From what I can see, leaving LC_NUMERIC set to 
the locale value rather than "C" leads to better interoperability.

>  So at best it solves part
>of one relevant problem here (other relevant problems include that platform
>C libraries disagree about how to spell infinities, NaNs and signed zeroes;
>about how many digits to use for an exponent; and about how to round results
>(for example,
>
>  
>
>>>>"%.1f" % 2.25
>>>>        
>>>>
>'2.3'
>  
>
>
>on Windows, but most (not all!) flavors of Unix produce the IEEE-754
>to-nearest/even rounded '2.2' instead)).
>
>It's easy to write portable, perfectly-rounding string<->double conversion
>routines without calling any platform functions.  The rub is that "fast"
>goes out the window then, unless you give up at least one of {portable,
>accurate}.
>  
>
It would be great for Python to have consistent float parsing/formatting 
on every platform in the future.  Making sure that every place where 
Python wants to parse or format a float in a locale independent fashion 
go through a single set of functions should make it easier to drop in a 
new set of routines in the future.  However, getting rid of the 
LC_NUMERIC=C requirement would have real benefits today.

James.

-- 
Email: james@daa.com.au
WWW:   http://www.daa.com.au/~james/




From martin at v.loewis.de  Mon Sep  1 08:15:08 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Mon Sep  1 03:15:10 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <3F52EE54.6090800@daa.com.au>
References: <LNBBLJKPBEHFEDALKOLCIECMFJAB.tim.one@comcast.net>
	<3F52EE54.6090800@daa.com.au>
Message-ID: <m3isodj7zh.fsf@mira.informatik.hu-berlin.de>

James Henstridge <james@daa.com.au> writes:

> The existing locale handling in Python shares this property, but makes
> it difficult for external libraries to format and parse floats in the
> locale's representation.  From what I can see, leaving LC_NUMERIC set
> to the locale value rather than "C" leads to better interoperability.

I think everybody agrees that allowing non-C LC_NUMERIC settings in
the C library is very desirable. My concerns are about the specific
approach taken to implement that change. Or, actually, with an entire
class of approaches: namely those that involve complex algorithms
(i.e. which include a for-statement :) to implement that feature.

Regards,
Martin

From guido at python.org  Mon Sep  1 02:34:36 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Sep  1 04:35:34 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: Your message of "01 Sep 2003 08:03:21 +0200."
	<m3d6elkpue.fsf@mira.informatik.hu-berlin.de> 
References: <LNBBLJKPBEHFEDALKOLCCENGFJAB.tim.one@comcast.net>  
	<m3d6elkpue.fsf@mira.informatik.hu-berlin.de> 
Message-ID: <200309010834.h818Yac22250@12-236-84-31.client.attbi.com>

> It is just that the patch does not "feel" right, given that there must
> be "native" locale-inaware parsing of floating point constants
> somewhere on each platform (atleast on those that support C++98).

FWIW, I have the same feeling, but the idea of having to support
our own version of such code is even more uncomfortable.

Maybe at least we can detect platforms for which we know there is a
native conversion in the library, and not use the hack on those?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Sep  1 02:36:32 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Sep  1 04:37:16 2003
Subject: [Python-Dev] Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: Your message of "01 Sep 2003 08:35:43 +0200."
	<m3znhpj9s0.fsf@mira.informatik.hu-berlin.de> 
References: <20030814022128.GN3095@async.com.br>  
	<m3znhpj9s0.fsf@mira.informatik.hu-berlin.de> 
Message-ID: <200309010836.h818aWF22262@12-236-84-31.client.attbi.com>

> That said, I don't actually require that the code is written from
> scratch. Instead, a detailed elaboration of how precisely the
> implementation is approached, in the PEP, would be good.
> 
> The PEP should also point out deficiencies of the approach taken,
> e.g. the issue of spelling NaN, inf, etc. If it can be determined not
> to be an issue in real life (i.e. for all interesting platforms), this
> should be documented as well.

I think that anything that might still be needed years after the code
is adopted should be in comments in the code, not in a PEP.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From gjc at inescporto.pt  Mon Sep  1 12:07:43 2003
From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro)
Date: Mon Sep  1 07:07:44 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <200309010834.h818Yac22250@12-236-84-31.client.attbi.com>
References: <LNBBLJKPBEHFEDALKOLCCENGFJAB.tim.one@comcast.net>
	<m3d6elkpue.fsf@mira.informatik.hu-berlin.de>
	<200309010834.h818Yac22250@12-236-84-31.client.attbi.com>
Message-ID: <1062414058.21848.4.camel@spectrum.inescn.pt>

A Seg, 2003-09-01 ?s 09:34, Guido van Rossum escreveu:
> > It is just that the patch does not "feel" right, given that there must
> > be "native" locale-inaware parsing of floating point constants
> > somewhere on each platform (atleast on those that support C++98).
> 
> FWIW, I have the same feeling, but the idea of having to support
> our own version of such code is even more uncomfortable.
> 
> Maybe at least we can detect platforms for which we know there is a
> native conversion in the library, and not use the hack on those?

  In case people haven't noticed, the second version of the patch,
submitted following the first patch, already does that.  It adds a
configure.in check for glibc's strtod_l() and uses that instead of glib
code whenever it's available.  Only on non-glibc systems is the glib
code compiled in.

  Regards.

-- 
Gustavo Jo?o Alves Marques Carneiro
<gjc@inescporto.pt> <gustavo@users.sourceforge.net>



From guido at python.org  Mon Sep  1 11:05:35 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Sep  1 13:07:09 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: Your message of "01 Sep 2003 12:00:58 BST."
	<1062414058.21848.4.camel@spectrum.inescn.pt> 
References: <LNBBLJKPBEHFEDALKOLCCENGFJAB.tim.one@comcast.net>
	<m3d6elkpue.fsf@mira.informatik.hu-berlin.de>
	<200309010834.h818Yac22250@12-236-84-31.client.attbi.com> 
	<1062414058.21848.4.camel@spectrum.inescn.pt> 
Message-ID: <200309011705.h81H5Zf22978@12-236-84-31.client.attbi.com>

> A Seg, 2003-09-01 às 09:34, Guido van Rossum escreveu:
> > FWIW, I have the same feeling, but the idea of having to support
> > our own version of such code is even more uncomfortable.
> > 
> > Maybe at least we can detect platforms for which we know there is a
> > native conversion in the library, and not use the hack on those?
> 
>   In case people haven't noticed, the second version of the patch,
> submitted following the first patch, already does that.  It adds a
> configure.in check for glibc's strtod_l() and uses that instead of glib
> code whenever it's available.  Only on non-glibc systems is the glib
> code compiled in.

This doesn't address the first sentence of mine quoted above: I'm
uncomfortable with having this code being part of Python, given that
we have no expertise to maintain it long-term (nor should we need such
expertise, IMO).

Here's yet another idea (which probably has flaws as well): instead of
substituting the locale's decimal separator, rewrite strings like
"3.14" as "314e-2" and rewrite strings like "3.14e5" as "314e3", then
pass to strtod(), which assigns the same meaning to such strings in
all locales.  This removes the question of what decimal separator is
used by the locale completely, and thus removes the last bit of
thread-unsafety from the code.  However, I don't know if underflow can
cause the result to be different, e.g. perhaps 1.23eX could be
computed but 123e(X-2) could not???  (Sounds pretty unlikely on the
face of it since I'd expect any decent conversion algorithm to pretty
much break its input down into a string of digits and an exponent, but
I've never actually studied such algorithms in detail.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.one at comcast.net  Mon Sep  1 15:30:23 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Sep  1 14:30:58 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <m3d6elkpue.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEAAFKAB.tim.one@comcast.net>

[Martin]
> Ok. Are you then, overall, in favour of taking the proposed approach?

It solves part of one problem; I'd rather solve all of it, but can't
volunteer time to do that.

> It is not thread-safe, but only so if somebody calls setlocale in a
> different thread, and that is known not to be thread-safe - so I could
> live with that limitation.

There's no way of using C's locale gimmicks that's threadsafe, short of all
callers agreeing to follow a beyond-standard-C exclusion protocol -- which
is the same as saying "no way" in reality.  So that's part of one problem no
patch of this ilk *can* solve.  It's not that the patch doesn't try hard
enough, it's that this approach is inherently inadequate to solve all of
this particular problem.

> It is just that the patch does not "feel" right, given that there must
> be "native" locale-inaware parsing of floating point constants
> somewhere on each platform (atleast on those that support C++98).

I haven't found one on Windows (doesn't mean it doesn't exist, does mean
it's apparently well hidden if it does exist).

> ...
> One of my early concerns (and I still have this concern) is that the
> contributors here appear to take the position "We have this fine code
> developed elsewhere, it seems to work, so we copy it. We don't
> actually have to understand this code". I would feel more comfortable
> if the code was written from scratch for usage in Python, with just
> the ideas borrowed from glib. Proper attribution of contributors and
> licensing are just one aspect, we really need the submitter of the
> code fully understand it, and be capable of reacting to problems
> quickly.

The patch is certainly more code than is needed to solve the part of the
problem it does solve.  For example, things like

typedef char		gchar;
typedef short		gshort;
typedef long   		glong;
typedef int    		gint;

introduce silly synonyms ("silly" == typing gshort instead of short does
nothing except introduce possibilities for confusion); there are many
definitions like

#define g_ascii_isupper(c) \
  ((g_ascii_table[(guchar) (c)] & G_ASCII_UPPER) != 0)

that are never referenced; the code caters to C99's hexadecimal float
literals but Python doesn't; and so on.  If someone who understood Python
internals read my earlier two-sentence description of how the patch works,
they could write something that works equally well for Python's purposes
with a fraction of the code introduced by the patch.

> ...
> The PEP should also point out deficiencies of the approach taken,
> e.g. the issue of spelling NaN, inf, etc. If it can be determined not
> to be an issue in real life (i.e. for all interesting platforms), this
> should be documented as well.

Well, the patch doesn't even pretend to address other issues with
portability of float literals.  They routinely come up on c.l.py, so of
course users bump into them; when someone is motivated enough to file a bug
report, I shuffle it off to PEP 42, under the "non-accidental 754 support"
heading (which covers many fp issues beyond just literals, of course).

[James Henstridge]
> ...
> Your average localised package usually switches to the user's
> preferred locale on startup, so that it can display strings and
> messages, and occasionally wants to read/write numbers in a locale
> independent format (usually when saving/loading files).  The most
> common way of doing this is the setlocale/strtod/setlocale combo,
> which has thread safety problems and possible reentrancy problems if
> done wrong.

I became acutely aware of the problems here due to the spambayes project,
part of which embeds Python in Outlook 2000/2002.  Outlook routinely runs
more than a dozen threads, and by observation changes locale "frequently".
None of that is documented, Python has no influence over when or why Outlook
decides to switch locale, and neither can Python exclude Outlook's other
threads when the Outlook thread Python is running in becomes active.

Mark Hammond solved our problems there by forcing locale back to "C" every
chance he gets; that's an anti-social and probabilistic approach, but
appears to be the best spambayes can do today.  Having spambayes grow its
own float<->string code doesn't help, because the worst problem spambayes
had is that Python's marshal format uses ASCII strings to store float
literals in .pyc files, so that Python itself can (and does) load insane
float values out of .pyc files if LC_NUMERIC isn't "C" at the time a .pyc
file gets imported.

The only thing that could truly solve spambayes's problems here is for
Python to use a thoroughly thread-safe string->float routine, where
"thoroughly" includes not caring whether other threads switch locale in
mid-stream.

An irony is that Microsoft's *native* locale gimmicks are thread-safe (each
Win32 thread has its own idea of Win32 locale); why Outlook is even mucking
with C's thread-braindead notion of locale is a mystery.

In short, I can't be enthusiastic about the patch because it doesn't solve
the only relevant locale problem I've actually run into.  I understand that
it may well solve many I haven't run into.

OTOH, the specific problem I'm acutely worried about would be better
addressed by changing the way Python marhals float values.

[Guido]
> Maybe at least we can detect platforms for which we know there is a
> native conversion in the library, and not use the hack on those?

I rarely find that piles of conditionalized code are more comprehensible or
reliable; they usually result in mysterious x-platform differences, and
become messier over time as we stumble into more platform library bugs,
quirks, and limitations.

> ...
> Here's yet another idea (which probably has flaws as well): instead of
> substituting the locale's decimal separator, rewrite strings like
> "3.14" as "314e-2" and rewrite strings like "3.14e5" as "314e3", then
> pass to strtod(), which assigns the same meaning to such strings in
> all locales.

This is a harder transformation than s/./locale_decimal_point/.  It does
address the thread-safety issue.  Numerically it's flaky, as only a
perfectly-rounding string->float routine can guarantee to return bit-for-bit
identical results given equivalent (viewed as infinite precision) decimal
representations as inputs, and few platform string->float routines do
perfect rounding.

> This removes the question of what decimal separator is used by the
> locale completely, and thus removes the last bit of thread-unsafety
> from the code.  However, I don't know if underflow can cause the result
> to be different, e.g. perhaps 1.23eX could be computed but 123e(X-2)
> could not???  (Sounds pretty unlikely on the face of it since I'd expect
> any decent conversion algorithm to pretty much break its input down into
> a string of digits and an exponent, but I've never actually studied
> such algorithms in detail.)

Each library is likely fail in its own unique ways.  Here's a cute one:

"""
base = 1.2345678901234567

digits = "12345678901234567"

for exponent in range(-16, -15000, -1):
    string = digits + "0" * (-16 - exponent)
    string += "e%d" % exponent
    derived = float(string)
    assert base == derived, (string, derived)
"""

On Windows, this first fails at exponent -5202, where float(string) delivers
a result a factor of 10 too large.  I was surprised it did that well!  Under
Cygwin Python 2.2.3, it consumed > 14 minutes of CPU time, but never failed.
I believe they're using a derivative of David Gay's excruciatingly complex
IEEE-754 perfect-rounding string<->float routines (which would explain both
why it didn't fail and why it consumed enormous CPU time; the code is
excruciatingly complex because it does perfect rounding quickly for "normal"
inputs, via a large variety of delicate speed tricks; when those tricks
don't apply, it has to simulate unbounded-precision arithmetic to guarantee
perfect rounding).


From kiko at async.com.br  Mon Sep  1 17:44:33 2003
From: kiko at async.com.br (Christian Reis)
Date: Mon Sep  1 15:45:13 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEAAFKAB.tim.one@comcast.net>
References: <m3d6elkpue.fsf@mira.informatik.hu-berlin.de>
	<LNBBLJKPBEHFEDALKOLCAEAAFKAB.tim.one@comcast.net>
Message-ID: <20030901194433.GK1917@async.com.br>

On Mon, Sep 01, 2003 at 02:30:23PM -0400, Tim Peters wrote:
> > One of my early concerns (and I still have this concern) is that the
> > contributors here appear to take the position "We have this fine code
> > developed elsewhere, it seems to work, so we copy it. We don't
> > actually have to understand this code". I would feel more comfortable
> > if the code was written from scratch for usage in Python, with just
> > the ideas borrowed from glib. Proper attribution of contributors and
> > licensing are just one aspect, we really need the submitter of the
> > code fully understand it, and be capable of reacting to problems
> > quickly.
> 
> The patch is certainly more code than is needed to solve the part of the
> problem it does solve.  For example, things like
> 
> typedef char		gchar;
> typedef short		gshort;
> typedef long   		glong;
> typedef int    		gint;
> 
> introduce silly synonyms ("silly" == typing gshort instead of short does
> nothing except introduce possibilities for confusion); there are many
> definitions like
> 
> #define g_ascii_isupper(c) \
>   ((g_ascii_table[(guchar) (c)] & G_ASCII_UPPER) != 0)
> 
> that are never referenced; the code caters to C99's hexadecimal float
> literals but Python doesn't; and so on.  If someone who understood Python
> internals read my earlier two-sentence description of how the patch works,
> they could write something that works equally well for Python's purposes
> with a fraction of the code introduced by the patch.

I would certainly concede this point -- Gustavo's patch is a
proof-of-concept implementation. I do believe that the glib code is a
good starting point for an implementation, and the author has submitted
a written agreement, so the next step would be obtaining approval of the
general approach and then diving in to clean up and minimize the code as
much as possible.

The issue is whether a cleaned up patch is the way python-dev would like
this to go, or if another, perhaps orthogonal, approach -- such as the
conversion Guido has proposed -- would be preferrable.

> > The PEP should also point out deficiencies of the approach taken,
> > e.g. the issue of spelling NaN, inf, etc. If it can be determined not
> > to be an issue in real life (i.e. for all interesting platforms), this
> > should be documented as well.
> 
> Well, the patch doesn't even pretend to address other issues with
> portability of float literals.  They routinely come up on c.l.py, so of

Are these representations (NaN, infinity, etc) LC_NUMERIC-dependent? 
Or, more generally, locale-dependent?

As for the thread-safety issues, it's true, changing locale multiple
times in runtime is bound to confuse the conversion code (and
consequently the interpreter). One way around this would be a complete
reimplementation of the relevant functions, but here's betting that that
alternative would be shunned even more intensely.

Take care,
--
Christian Reis, Senior Engineer, Async Open Source, Brazil.
http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL

From martin at v.loewis.de  Mon Sep  1 22:36:59 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Mon Sep  1 17:37:01 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <200309011705.h81H5Zf22978@12-236-84-31.client.attbi.com>
References: <LNBBLJKPBEHFEDALKOLCCENGFJAB.tim.one@comcast.net>
	<m3d6elkpue.fsf@mira.informatik.hu-berlin.de>
	<200309010834.h818Yac22250@12-236-84-31.client.attbi.com>
	<1062414058.21848.4.camel@spectrum.inescn.pt>
	<200309011705.h81H5Zf22978@12-236-84-31.client.attbi.com>
Message-ID: <m3he3wp4x0.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> Here's yet another idea (which probably has flaws as well): instead of
> substituting the locale's decimal separator, rewrite strings like
> "3.14" as "314e-2" and rewrite strings like "3.14e5" as "314e3", then
> pass to strtod(), which assigns the same meaning to such strings in
> all locales.

Unfortunately, that doesn't work for sprintf, for which you still need
to replace the radixchar.

Regards,
Martin

From gjc at inescporto.pt  Tue Sep  2 00:08:11 2003
From: gjc at inescporto.pt (Gustavo J A M Carneiro)
Date: Mon Sep  1 18:08:11 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <20030901194433.GK1917@async.com.br>
References: <m3d6elkpue.fsf@mira.informatik.hu-berlin.de>
	<LNBBLJKPBEHFEDALKOLCAEAAFKAB.tim.one@comcast.net>
	<20030901194433.GK1917@async.com.br>
Message-ID: <1062454091.3272.11.camel@emperor.homelinux.net>

A Seg, 2003-09-01 ?s 20:44, Christian Reis escreveu:
> On Mon, Sep 01, 2003 at 02:30:23PM -0400, Tim Peters wrote:
> > > One of my early concerns (and I still have this concern) is that the
> > > contributors here appear to take the position "We have this fine code
> > > developed elsewhere, it seems to work, so we copy it. We don't
> > > actually have to understand this code". I would feel more comfortable
> > > if the code was written from scratch for usage in Python, with just
> > > the ideas borrowed from glib. Proper attribution of contributors and
> > > licensing are just one aspect, we really need the submitter of the
> > > code fully understand it, and be capable of reacting to problems
> > > quickly.
> > 
> > The patch is certainly more code than is needed to solve the part of the
> > problem it does solve.  For example, things like
> > 
> > typedef char		gchar;
> > typedef short		gshort;
> > typedef long   		glong;
> > typedef int    		gint;
> > 
> > introduce silly synonyms ("silly" == typing gshort instead of short does
> > nothing except introduce possibilities for confusion); there are many
> > definitions like
> > 
> > #define g_ascii_isupper(c) \
> >   ((g_ascii_table[(guchar) (c)] & G_ASCII_UPPER) != 0)
> > 
> > that are never referenced; the code caters to C99's hexadecimal float
> > literals but Python doesn't; and so on.  If someone who understood Python
> > internals read my earlier two-sentence description of how the patch works,
> > they could write something that works equally well for Python's purposes
> > with a fraction of the code introduced by the patch.
> 
> I would certainly concede this point -- Gustavo's patch is a
> proof-of-concept implementation. I do believe that the glib code is a
> good starting point for an implementation, and the author has submitted
> a written agreement, so the next step would be obtaining approval of the
> general approach and then diving in to clean up and minimize the code as
> much as possible.

  Actually, this code presentation is intentional, for the following two
reasons:
	1- I didn't want to accidentally introduce any bug, so I tried to
copy-paste the code with as little changes as possible;
	2- In the current form, if glib developers find any bug in this code,
we can easily merge the changes back into python.

  Perhaps I was wrong to have done it this way...  Anyway, replacing the
g* types is trivial with any decent text editor.

  Regards.

-- 
Gustavo J. A. M. Carneiro
<gjc@inescporto.pt> <gustavo@users.sourceforge.net>


From martin at v.loewis.de  Mon Sep  1 23:19:09 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Mon Sep  1 18:19:10 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <1062454091.3272.11.camel@emperor.homelinux.net>
References: <m3d6elkpue.fsf@mira.informatik.hu-berlin.de>
	<LNBBLJKPBEHFEDALKOLCAEAAFKAB.tim.one@comcast.net>
	<20030901194433.GK1917@async.com.br>
	<1062454091.3272.11.camel@emperor.homelinux.net>
Message-ID: <m38yp8p2yu.fsf@mira.informatik.hu-berlin.de>

Gustavo J A M Carneiro <gjc@inescporto.pt> writes:

>   Perhaps I was wrong to have done it this way...  Anyway, replacing the
> g* types is trivial with any decent text editor.

It's more than that. We want you to fully understand the patch, and to
spot errors in it even before the glib developers find them. We want
you to provide a minimalistic patch, that just implements the required
functionality, and nothing else. We want the patch to be maintainable
due to it being easy to read and follow, instead of being maintainable
due to the fact that it is identical with some code elsewhere in the
world.

Regards,
Martin


From sholden at holdenweb.com  Mon Sep  1 20:14:54 2003
From: sholden at holdenweb.com (Steve Holden)
Date: Mon Sep  1 19:17:20 2003
Subject: [Python-Dev] readline.clear_history()
In-Reply-To: <2mn0duxc2b.fsf@starship.python.net>
Message-ID: <CGECIJPNNHIFAJKHOLMAKEHFGJAA.sholden@holdenweb.com>

[Michael Hudson]
> Guido van Rossum <guido@python.org> writes:
>
> > Probably.  I'm personally not so stringent for minor stuff like this
> > (hey, I added bool() to 2.2.1 :-), but it seems others are
> > increasingly resisting even minor feature additions in
> micro releases
> > (except in the email package :-).
>
> The problem with adding things like this is that it makes it harder to
> port code from *2.3.1 to 2.3* and at least in principle it would be
> nice if code that works on 2.3.1 works on 2.3 unless it tickles a
> specific bug that has been fixed.
>
> And, come Panther, there are going to be rather a lot of Python 2.3
> installs out there.
>
Just as a matter of interest, do you have any idea how many Mac OSX
licensees there are?

regards
--
Steve Holden                                 http://www.holdenweb.com/
Python Web Programming                http://pydish.holdenweb.com/pwp/
Interview with GvR August 14, 2003       http://www.onlamp.com/python/



From guido at python.org  Mon Sep  1 18:03:29 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Sep  1 20:04:54 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: Your message of "Mon, 01 Sep 2003 14:30:23 EDT."
	<LNBBLJKPBEHFEDALKOLCAEAAFKAB.tim.one@comcast.net> 
References: <LNBBLJKPBEHFEDALKOLCAEAAFKAB.tim.one@comcast.net> 
Message-ID: <200309020003.h8203TD23456@12-236-84-31.client.attbi.com>

[Tim]
> In short, I can't be enthusiastic about the patch because it doesn't
> solve the only relevant locale problem I've actually run into.  I
> understand that it may well solve many I haven't run into.

At this point in your life, Tim, is there any patch you could be truly
enthusiastic about? :-)

I'm asking because I'd like to see the specific problem that started
this thread solved, if necessary using a compromise that means the
solution isn't perfect.  I'm even willing to take a step back in the
status quo, given that the status quo isn't perfect anyway, and that
compromises mean something has to give.

*Maybe* the right solution is that we have to accept a
hard-to-understand overcomplicated piece of code that we don't know
how to maintain (but for which the author asserts that we won't have
to do much maintenance in the foreseeable future).  But *maybe*
there's a simpler solution.

> OTOH, the specific problem I'm acutely worried about would be better
> addressed by changing the way Python marhals float values.

So solve it.  The approach used by binary pickles seems entirely
reasonable.  All we need to do is change the .pyc magic number.
(There's undoubtedly user code in the world that would break because
it requires interoperability between Python versions.  So let the
marshal module grow a way to specify the format.)

> [Guido]
> > Maybe at least we can detect platforms for which we know there is a
> > native conversion in the library, and not use the hack on those?
> 
> I rarely find that piles of conditionalized code are more
> comprehensible or reliable; they usually result in mysterious
> x-platform differences, and become messier over time as we stumble
> into more platform library bugs, quirks, and limitations.

Fair enough.  So *if* we decide to use the donated conversion code, we
should start by using it unconditionally.  I predict that at some
point in the future we'll find a platform whose quirks are not handled
by the donated code, and where it's simpler to use a correct native
equivalent than to try to fix the donated code; but I expect that
point to be pretty far in the future, *or* the platform to be pretty
far from the main stream.

> > ...
> > Here's yet another idea (which probably has flaws as well): instead of
> > substituting the locale's decimal separator, rewrite strings like
> > "3.14" as "314e-2" and rewrite strings like "3.14e5" as "314e3", then
> > pass to strtod(), which assigns the same meaning to such strings in
> > all locales.
> 
> This is a harder transformation than s/./locale_decimal_point/.  It does
> address the thread-safety issue.  Numerically it's flaky, as only a
> perfectly-rounding string->float routine can guarantee to return bit-for-bit
> identical results given equivalent (viewed as infinite precision) decimal
> representations as inputs, and few platform string->float routines do
> perfect rounding.
> 
> > This removes the question of what decimal separator is used by the
> > locale completely, and thus removes the last bit of thread-unsafety
> > from the code.  However, I don't know if underflow can cause the result
> > to be different, e.g. perhaps 1.23eX could be computed but 123e(X-2)
> > could not???  (Sounds pretty unlikely on the face of it since I'd expect
> > any decent conversion algorithm to pretty much break its input down into
> > a string of digits and an exponent, but I've never actually studied
> > such algorithms in detail.)
> 
> Each library is likely fail in its own unique ways.  Here's a cute one:
> 
> """
> base = 1.2345678901234567
> 
> digits = "12345678901234567"
> 
> for exponent in range(-16, -15000, -1):
>     string = digits + "0" * (-16 - exponent)
>     string += "e%d" % exponent
>     derived = float(string)
>     assert base == derived, (string, derived)
> """
> 
> On Windows, this first fails at exponent -5202, where float(string)
> delivers a result a factor of 10 too large.  I was surprised it did
> that well!  Under Cygwin Python 2.2.3, it consumed > 14 minutes of
> CPU time, but never failed.  I believe they're using a derivative of
> David Gay's excruciatingly complex IEEE-754 perfect-rounding
> string<->float routines (which would explain both why it didn't fail
> and why it consumed enormous CPU time; the code is excruciatingly
> complex because it does perfect rounding quickly for "normal"
> inputs, via a large variety of delicate speed tricks; when those
> tricks don't apply, it has to simulate unbounded-precision
> arithmetic to guarantee perfect rounding).

I fail to see the relevance of the example to my proposed hack, except
as a proof that the world isn't perfect -- but we already know that.
Under my proposal, the number of digits converted would never change,
so any sensitivity of the algorithm used to the number of digits
converted would be irrelevant.  I note that the strtod.c code that's
currently in the Python source tree uses a similar (though opposite)
trick: it converts the number to the form 0.<fraction>E<expt> before
handing it off to atof().  So my proposal still stands.  I'm happy to
entertain a proof that it's flawed but not one where the flawed input
has over 5000 digits *and* depends on a flaw in the platform routines.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Sep  1 18:12:20 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Sep  1 20:13:09 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: Your message of "02 Sep 2003 00:18:33 +0200."
	<m38yp8p2yu.fsf@mira.informatik.hu-berlin.de> 
References: <m3d6elkpue.fsf@mira.informatik.hu-berlin.de>
	<LNBBLJKPBEHFEDALKOLCAEAAFKAB.tim.one@comcast.net>
	<20030901194433.GK1917@async.com.br>
	<1062454091.3272.11.camel@emperor.homelinux.net> 
	<m38yp8p2yu.fsf@mira.informatik.hu-berlin.de> 
Message-ID: <200309020012.h820CKD23546@12-236-84-31.client.attbi.com>

> It's more than that. We want you to fully understand the patch, and to
> spot errors in it even before the glib developers find them.

It's unlikely that the glib code would contain errors that Gustavo
could spot, before or after cleaning up the patch (no offense to
Gustavo meant!).  However, a more likely cause of errors would be that
the adoption of the code to a new environment breaks an unspoken
assumption made by the code.  Only truly understanding the code would
reveal such assumptions.

> We want
> you to provide a minimalistic patch, that just implements the required
> functionality, and nothing else. We want the patch to be maintainable
> due to it being easy to read and follow, instead of being maintainable
> due to the fact that it is identical with some code elsewhere in the
> world.

Right.  Just say no to "copy-and-paste code reuse".

--Guido van Rossum (home page: http://www.python.org/~guido/)

From administrator at vt.edu  Mon Sep  1 21:26:35 2003
From: administrator at vt.edu (administrator@vt.edu)
Date: Mon Sep  1 20:26:44 2003
Subject: [Python-Dev] Virus Warning
Message-ID: <200309020026.BPL01149@vivi.cc.vt.edu>


The message you emailed to weiping@vt.edu, dated 09/01/03 20:26:35, contains the W32/Sobig-F virus in the document_9446.pif attachment. The action taken was: deleted the attachment.



From europractice-bounces at ls12sn.cs.uni-dortmund.de  Tue Sep  2 03:35:50 2003
From: europractice-bounces at ls12sn.cs.uni-dortmund.de (europractice-bounces@ls12sn.cs.uni-dortmund.de)
Date: Mon Sep  1 20:36:00 2003
Subject: [Python-Dev] Your message to Europractice awaits moderator approval
Message-ID: <mailman.34.1062462950.294.europractice@ls12.cs.uni-dortmund.de>

Your mail to 'Europractice' with the subject

    Re: Wicked screensaver

Is being held until the list moderator can review it for approval.

The reason it is being held:

    Post by non-member to a members-only list

Either the message will get posted to the list, or you will receive
notification of the moderator's decision.  If you would like to cancel
this posting, please visit the following URL:

    http://ls12-www.cs.uni-dortmund.de/mailman/confirm/europractice/51c62fe262577fff9b37d8ce7c4784dab164ac6a


From tim.one at comcast.net  Mon Sep  1 22:20:55 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Sep  1 21:21:29 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <200309020003.h8203TD23456@12-236-84-31.client.attbi.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEDHFKAB.tim.one@comcast.net>

[Tim]
>> In short, I can't be enthusiastic about the patch because it doesn't
>> solve the only relevant locale problem I've actually run into.  I
>> understand that it may well solve many I haven't run into.

[Guido]
> At this point in your life, Tim, is there any patch you could be truly
> enthusiastic about? :-)

Yes, but I can't be enthusiastic about a hack, and especially not about a
hack that (as I said) doesn't solve the real-life problem spambayes has.

> I'm asking because I'd like to see the specific problem that started
> this thread solved,

At this point, can you state what that specific problem was <wink>?

> if necessary using a compromise that means the solution isn't perfect.
>  I'm even willing to take a step back in the status quo, given that the
> status quo isn't perfect anyway, and that compromises mean something has
> to give.
>
> *Maybe* the right solution is that we have to accept a
> hard-to-understand overcomplicated piece of code that we don't know
> how to maintain (but for which the author asserts that we won't have
> to do much maintenance in the foreseeable future).

I'm finding it hard to believe that anyone other than me and the author has
actually read the patch!  It's easy to understand.  It's over-complicated
for what Python needs, and would be dead easy to understand if the fluff got
chopped.  The *fear* of this code expressed in this thread is baffling to
me, but I suspect it's due to initial shell-shock from the sheer bulk of the
unnecessary code in the patch.

> But *maybe* there's a simpler solution.

>> OTOH, the specific problem I'm acutely worried about would be better
>> addressed by changing the way Python marhals float values.

> So solve it.

Sorry, I don't foresee making time to do that.

> The approach used by binary pickles seems entirely reasonable.

It's the best binary format we've got.  It has problems with 754's special
values (as recorded in PEP 42), and loses precision for VAX D format doubles
(any double format with greater dynamic range or precision than IEEE-754
double).  A decimal string is actually better on all those counts (dynamic
range is no problem then; and *some* platforms can preserve IEEE special
values via to-string-and-back conversion (Windows cannot)).  Decimal strings
lose on correctness only because of locale variations; depending on
platform, they may also lose on speed, but I don't give much weight to speed
here.

> All we need to do is change the .pyc magic number.  (There's undoubtedly
> user code in the world that would break because it requires
> interoperability between Python versions.  So let the marshal module grow
> a way to specify the format.)

> ...
> Fair enough.  So *if* we decide to use the donated conversion code, we
> should start by using it unconditionally.  I predict that at some
> point in the future we'll find a platform whose quirks are not handled
> by the donated code, and where it's simpler to use a correct native
> equivalent than to try to fix the donated code; but I expect that
> point to be pretty far in the future, *or* the platform to be pretty
> far from the main stream.

Do read the patch.  It amounts to

    if decimal_point != '.':
        s/./decimal_point/

in one direction and

    if decimal_point != '.':
        s/decimal_point/./

in the other.  It gets its idea of decimal_point from the platform
localeconv(), so if that doesn't lie it's hard to get wrong.  In the
double->string direction, though, the substitution code appears inadequate
to me, since it doesn't try to strip out thousand-separation characters,
which some locales produce.  For example, on Windows,

>>> locale.setlocale(locale.LC_ALL, "german")
'German_Germany.1252'
>>> locale.format("%g", 123456.0, 1)
'123.456'
>>>

AFAICT, the patch will leave that output as "123.456".  The string->double
direction is much easier to be confident about for this reason.

>>> ...
>>> Here's yet another idea (which probably has flaws as well): instead
>>> of substituting the locale's decimal separator, rewrite strings like
>>> "3.14" as "314e-2" and rewrite strings like "3.14e5" as "314e3",
>>> then pass to strtod(), which assigns the same meaning to such
>>> strings in all locales.

[long example]

> I fail to see the relevance of the example to my proposed hack, except
> as a proof that the world isn't perfect -- but we already know that.

The point is that only perfect-rounding string->float routines can guarantee
to produce identical doubles from mathematically equivalent decimal string
representations.   Finding counterexamples for non-perfect-rounding
libraries is extremely difficult, and/or time-consuming, without studying
the source code of a specific library intensely (almost certainly with more
intensity than its author gave to writing it!), and I don't have time for
that.  It's a potential vulnerability.  Answering whether it's an actual
vulnerability in practice is much more work than I can give to it now.

> Under my proposal, the number of digits converted would never change,
> so any sensitivity of the algorithm used to the number of digits
> converted would be irrelevant.  I note that the strtod.c code that's
> currently in the Python source tree uses a similar (though opposite)
> trick: it converts the number to the form 0.<fraction>E<expt> before
> handing it off to atof().  So my proposal still stands.  I'm happy to
> entertain a proof that it's flawed but not one where the flawed input
> has over 5000 digits *and* depends on a flaw in the platform routines.

As hacks go, it's probably OK.  I don't think it can fail on glibc-based
platforms because I think they do perfect-rounding conversions; the Windows
conversion routines aren't perfect-rounding, but we don't have their source
code so it's impossible for me to give examples offhand where different
results could be delivered, or even to swear that there are (or aren't) such
cases.  I give it a lot of credit for being truly threadsafe.

Note that it doesn't address the other half of the locale conversion problem
(double->string), which, as I noted above, is the harder half (due to
thousands_sep becoming an additional issue).


From tim.one at comcast.net  Mon Sep  1 22:27:17 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Sep  1 21:27:51 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <20030901194433.GK1917@async.com.br>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEDLFKAB.tim.one@comcast.net>

[Christian Reis]
> ...
> Are these representations (NaN, infinity, etc) LC_NUMERIC-dependent?
> Or, more generally, locale-dependent?

Don't know; C89 didn't say anything at all about them, so existing C
practice is all over the map; C99 does say something about them, but whether
a C99 compiler supports them is optional (support for them isn't mandatory;
if a C implementation does choose to support them, then the spellings for
input are standardized, although a locale is allowed to *produce* any
spellings whatsoever).


From martin at v.loewis.de  Tue Sep  2 05:58:53 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Sep  2 00:58:54 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEDHFKAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCGEDHFKAB.tim.one@comcast.net>
Message-ID: <m3ekyzu6q6.fsf@mira.informatik.hu-berlin.de>

"Tim Peters" <tim.one@comcast.net> writes:

> > I'm asking because I'd like to see the specific problem that started
> > this thread solved,
> 
> At this point, can you state what that specific problem was <wink>?

The user was writing a gtk application (using pygtk), where gtk,
internally, would rely on C-library LC_NUMERIC following the local
conventions (I believe gtk would call snprintf to display some message
to the user).  The user then thought that calling locale.setlocale
would be sufficient, but it isn't, and there is no way to fix that
short of rewriting gtk.

> I'm finding it hard to believe that anyone other than me and the author has
> actually read the patch!  It's easy to understand.

On the surface, yes. However, it seems full of hidden assumptions that
are difficult to find out and consider. For example, what if the
platform snprintf choses to output the thousands-separator? I can't
see how that handled in the patch.

Regards,
Martin

From martin at v.loewis.de  Tue Sep  2 06:00:30 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Sep  2 01:00:31 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEDLFKAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCGEDLFKAB.tim.one@comcast.net>
Message-ID: <m3ad9nu6nk.fsf@mira.informatik.hu-berlin.de>

"Tim Peters" <tim.one@comcast.net> writes:

> (support for them isn't mandatory;
> if a C implementation does choose to support them, then the spellings for
> input are standardized, although a locale is allowed to *produce* any
> spellings whatsoever).

Where exactly does it (C99) say that the spellings are locale-specific?

Regards,
Martin

From europractice-bounces at ls12sn.cs.uni-dortmund.de  Tue Sep  2 09:59:03 2003
From: europractice-bounces at ls12sn.cs.uni-dortmund.de (europractice-bounces@ls12sn.cs.uni-dortmund.de)
Date: Tue Sep  2 02:59:14 2003
Subject: [Python-Dev] Your message to Europractice awaits moderator approval
Message-ID: <mailman.4.1062485943.294.europractice@ls12.cs.uni-dortmund.de>

Your mail to 'Europractice' with the subject

    Re: Re: My details

Is being held until the list moderator can review it for approval.

The reason it is being held:

    Post by non-member to a members-only list

Either the message will get posted to the list, or you will receive
notification of the moderator's decision.  If you would like to cancel
this posting, please visit the following URL:

    http://ls12-www.cs.uni-dortmund.de/mailman/confirm/europractice/85d230b79086b9c6d4cbcc59acfbef0ac5d200ae


From cwteoh at tm.net.my  Tue Sep  2 20:49:38 2003
From: cwteoh at tm.net.my (PAUL YOUNG)
Date: Tue Sep  2 07:59:17 2003
Subject: [Python-Dev] you r sending this on purpose or your pc got virus?
	please scan your pc
References: <0HKJ008JPIIR2M@imta1.tm.net.my>
Message-ID: <013c01c37149$7d501f20$0300000a@homerpcpoxxbh6>


----- Original Message ----- 
From: <python-dev@python.org>
To: <cwteoh@tm.net.my>
Sent: Monday, September 01, 2003 10:49 PM
Subject: Re: Details


> See the attached file for details
> 

From sjoerd at acm.org  Tue Sep  2 15:31:17 2003
From: sjoerd at acm.org (Sjoerd Mullender)
Date: Tue Sep  2 08:31:27 2003
Subject: [Python-Dev] configure and setup.py
Message-ID: <20030902123117.9333574130@indus.ins.cwi.nl>

I am trying to build Python on a Unix system (Irix in this case, but
that's irrelevant) and I am getting a bit frustrated.

The problem is this.  I tell configure where to find libraries and
include files using options such as CPPFLAGS='-I...' and LDFLAGS='-L...'
(you can provide those on the configure command line).

Making Python then goes relatively smoothly.  However, building the
dynamically loadable modules with setup.py does *not* work smoothly.
The main problem is that setup.py does not use all relevant information
from the Makefile.  I just submitted a bug report (with patch) for
distutils so that it'll use the LDFLAGS variable from the Makefile, but
that's not enough.

Setup.py checks whether it can find certain libraries (readline comes to
mind, that's the one that triggered my investigation).  There are a
number of extra system-dependent directories hard-coded in setup.py, but
it totally ignores the directories that were specified at configure
time.  I think this is not the right way of doing things.

If any knowledge about where compilers look for libraries should be
built in anywhere, it's in distutils, not setup.py.  Also, and more
importantly, distutils should figure out the extra library directories
that were specified at configure time and add the information to the
list of searchable directories (CCompiler.library_dirs).  If this were
done, setup.py would find my -lreadline and the readline module would
get compiled and I would be able to edit in my interactive Python.

The same goes for include files.

-- Sjoerd Mullender <sjoerd@acm.org>

From tim.one at comcast.net  Tue Sep  2 12:29:55 2003
From: tim.one at comcast.net (Tim Peters)
Date: Tue Sep  2 11:30:33 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <m3ekyzu6q6.fsf@mira.informatik.hu-berlin.de>
Message-ID: <BIEJKCLHCIOIHAGOKOLHOEJFHBAA.tim.one@comcast.net>

[Tim]
>> I'm finding it hard to believe that anyone other than me and the
>> author has actually read the patch!  It's easy to understand.

[Martin]
> On the surface, yes. However, it seems full of hidden assumptions that
> are difficult to find out and consider. For example, what if the
> platform snprintf choses to output the thousands-separator? I can't
> see how that handled in the patch.

I mentioned that one too last night -- it doesn't.

OTOH, *are* there locales that insert thousands_sep?  I don't know.  To get
thousands_sep to appear via Python's locale.format(), in all locales I've
tried so far it requires passing a true value for the optional "grouping"
argument.  Like

>>> import locale
>>> locale.setlocale(locale.LC_ALL, "german")
'German_Germany.1252'
>>> x = 12345.0
>>> locale.format("%g", x)  # no thousands_sep
'12345'
>>> locale.format("%g", x, 1)
'12.345'
>>>

Now going thru locale.py is far from going thru C, but the same thing
happens if I use sprintf() directly from C (no thousands_sep appears,
regardless of how I change locale).  That's on Win2K.  The draft std I have
handy here sez:

    LC_NUMERIC affects the decimal-point character for the
    formatted input/output functions and the string conversion
    functions, as well as the nonmonetary formatting information
    returned by the localeconv function.

There's no support there for the notion that "the formatted (etc)" functions
*can* be affected by thousands_sep, just that fiddling locale can affect
decimal-point and the (passive) values returned by localeconv().


From kiko at async.com.br  Tue Sep  2 13:42:30 2003
From: kiko at async.com.br (Christian Reis)
Date: Tue Sep  2 11:43:15 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHOEJFHBAA.tim.one@comcast.net>
References: <m3ekyzu6q6.fsf@mira.informatik.hu-berlin.de>
	<BIEJKCLHCIOIHAGOKOLHOEJFHBAA.tim.one@comcast.net>
Message-ID: <20030902154230.GB1122@async.com.br>

On Tue, Sep 02, 2003 at 11:29:55AM -0400, Tim Peters wrote:
> OTOH, *are* there locales that insert thousands_sep?  I don't know.  To get
> thousands_sep to appear via Python's locale.format(), in all locales I've
> tried so far it requires passing a true value for the optional "grouping"
> argument.  Like
> 
> >>> import locale
> >>> locale.setlocale(locale.LC_ALL, "german")
> 'German_Germany.1252'
> >>> x = 12345.0
> >>> locale.format("%g", x)  # no thousands_sep
> '12345'
> >>> locale.format("%g", x, 1)
> '12.345'
> >>>

At least with locale.format, if you want grouping, you pass in the third
argument (grouping=1) to format(). An example:

    >>> locale.setlocale(locale.LC_NUMERIC, 'da_DK')
    'da_DK'
    >>> locale.format("%.2f", 71630, 1)
    '71.630,00'

Now, from the glibc docs:

    [...] The SUSv2 specifies one further flag character.

       '      For decimal conversion (i, d, u, f, F, g, G) the output is
              to be  grouped with  thousands'  grouping characters if
              the locale information indicates any.  Note that many
              versions of gcc cannot parse this option  and  will issue
              a warning.  SUSv2 does not include %'F.

So unless I'm mistaken, this wouldn't really be an issue in our case if
explicit grouping isn't requested inside the python conversion functions.

Take care,
--
Christian Reis, Senior Engineer, Async Open Source, Brazil.
http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL

From jepler at unpythonic.net  Tue Sep  2 11:44:03 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Tue Sep  2 11:44:55 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHOEJFHBAA.tim.one@comcast.net>
References: <m3ekyzu6q6.fsf@mira.informatik.hu-berlin.de>
	<BIEJKCLHCIOIHAGOKOLHOEJFHBAA.tim.one@comcast.net>
Message-ID: <20030902154357.GC17554@unpythonic.net>

On Tue, Sep 02, 2003 at 11:29:55AM -0400, Tim Peters wrote:
> OTOH, *are* there locales that insert thousands_sep?  I don't know.
[...]
> There's no support there for the notion that "the formatted (etc)" functions
> *can* be affected by thousands_sep, just that fiddling locale can affect
> decimal-point and the (passive) values returned by localeconv().

The Linux/glibc documentation, which cites SUSv2, seems to imply that no
locale inserts the thousands separator in formatting operations, except
when the ' flag character is included:
       For  some  numeric  conversions  a radix character ('decimal point') or
       thousands' grouping  character  is  used.  The  actual  character  used
       depends on the LC_NUMERIC part of the locale. The POSIX locale uses '.'
       as radix character, and does not have a grouping character.  Thus,
                   printf("%'.2f", 1234567.89);
       results in '1234567.89' in the POSIX locale,  in  '1234567,89'  in  the
       nl_NL locale, and in '1.234.567,89' in the da_DK locale.
[...]
       The  five  flag  characters  above  are defined in the C standard.  The
       SUSv2 specifies one further flag character.

       '      For decimal conversion (i, d, u, f, F, g, G) the output is to be
              grouped with thousands' grouping characters if the locale infor-
              mation indicates any.  Note that many  versions  of  gcc  cannot
              parse  this  option  and  will  issue a warning.  SUSv2 does not
              include %'F.

Jeff

From tim.one at comcast.net  Tue Sep  2 12:49:00 2003
From: tim.one at comcast.net (Tim Peters)
Date: Tue Sep  2 11:49:39 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <m3ad9nu6nk.fsf@mira.informatik.hu-berlin.de>
Message-ID: <BIEJKCLHCIOIHAGOKOLHOEJMHBAA.tim.one@comcast.net>

[Tim]
>> if a C implementation does choose to support them, then the
>> spellings for input are standardized, although a locale is allowed
>> to *produce* any spellings whatsoever).

[martin@v.loewis.de]
> Where exactly does it (C99) say that the spellings are
> locale-specific?

I can't find anything in the std supporting the claim.  For that matter, I
can't find anything in the std supporting the notion that a locale is
allowed to insert thousand-separator characters either (can you?).

There's lots of stuff allowing a locale to *accept* locale-specific
spellings (when parsing strings), in addition to the "C" locale spellings;
the other direction (producing strings) appears much less permissive.


From martin at v.loewis.de  Tue Sep  2 22:16:50 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Sep  2 17:16:52 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHOEJMHBAA.tim.one@comcast.net>
References: <BIEJKCLHCIOIHAGOKOLHOEJMHBAA.tim.one@comcast.net>
Message-ID: <m37k4qki2j.fsf@mira.informatik.hu-berlin.de>

"Tim Peters" <tim.one@comcast.net> writes:

> I can't find anything in the std supporting the claim.  For that matter, I
> can't find anything in the std supporting the notion that a locale is
> allowed to insert thousand-separator characters either (can you?).

No: I'm now convinced that sprintf is *forbidden* to insert the
thousands separator. This is why POSIX added the '-flag (%'f); this
will produce the thousands-separator.

That said: Implementations might choose to ignore the standard in that
respect. This issue just supports my thesis that the patch is
complicated: If I have to read the C99 standard to find out whether it
is correct, it must be complicated. I doubt either the submitters or
the original author of the code did that exercise...

> There's lots of stuff allowing a locale to *accept* locale-specific
> spellings (when parsing strings), in addition to the "C" locale spellings;
> the other direction (producing strings) appears much less permissive.

Indeed. I'm not sure whether this is intentional, though.

Regards,
Martin

From martin at v.loewis.de  Thu Sep  4 19:17:34 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Sep  4 14:17:45 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <3F5611F8.5060606@daa.com.au>
References: <LNBBLJKPBEHFEDALKOLCGEDHFKAB.tim.one@comcast.net>
	<3F5611F8.5060606@daa.com.au>
Message-ID: <m34qztk0pe.fsf@mira.informatik.hu-berlin.de>

James Henstridge <james@daa.com.au> writes:

> In case it is of any help, I put together a modified version of the
> glib code that uses libpython support routines instead of glib support
> routines at:
>     http://www.gnome.org/~james/python-float-convert.c

This is indeed helpful. I like it much more than the code that is on
SF; in particular, the elimination of the table is a good thing.

> It might help people review the code, as it doesn't contain code
> irrelevant to the discussion.  The code could probably be shortened by
> removing support for hexadecimal float representation in
> ascii_strtod().

I would encourage that. There are other format characters that are not
supported either, like the POSIX ' flag. Having limitations in that
approach is fine - they just need to be documented.

Regards,
Martin

From martin at v.loewis.de  Thu Sep  4 20:25:19 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Sep  4 15:25:21 2003
Subject: Solutions for LC_NUMERIC,
	was Re: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <20030903134155.GB649@async.com.br>
References: <m37k4qki2j.fsf@mira.informatik.hu-berlin.de>
	<LNBBLJKPBEHFEDALKOLCCEIEFKAB.tim.one@comcast.net>
	<20030903134155.GB649@async.com.br>
Message-ID: <m38yp5k16y.fsf@mira.informatik.hu-berlin.de>

Christian Reis <kiko@async.com.br> writes:

>     "On glibc-based systems, Python is safe from runtime changes in the
>     LC_NUMERIC locale category. On other platforms, changing LC_NUMERIC
>     at runtime may cause float conversion routines to fail. This may
>     impact the Python interpreter internally."

I could accept such a limitation. However, I maintain that the aspects
must be fully understood, and I agree with Guido that they must be
documented in the documentation proper (i.e. as comments in the source
code, or the user documentation).

So I suggest that you proceed with the PEP and the implementation
as-is. The PEP would need to be enhanced to summarize this discussion
(perhaps listing the alternatives that you just mentioned); the patch
would need to be enhanced to be more simple, readable, and in the
style of the rest of the Python source code, and would need to include
documentation changes and test cases (although some test cases
probably already exist).

Regards,
Martin

From pje at telecommunity.com  Thu Sep  4 16:55:32 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Sep  4 15:55:42 2003
Subject: [Python-Dev] Undocumented functions in 'readline' module
Message-ID: <5.1.1.6.0.20030904155048.00acfd20@mail.rapidsite.net>

While working on the clear_history() patch today, I noticed that there are 
some readline functions in the current CVS that don't appear in the docs:

redisplay()
get_history_item()
get_current_history_length()

Are these intentionally undocumented for some reason, or should I submit a 
doc bug?



From raymond.hettinger at verizon.net  Wed Sep  3 16:55:46 2003
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Thu Sep  4 17:34:49 2003
Subject: [Python-Dev] object.__hash__() vs.  hash()
Message-ID: <000201c37256$5bee3e80$e841fea9@oemcomputer>

object.__hash__() returns id(obj) by default.

__builtin__.hash() uses PyObject_Hash() which is a bit smarter
and avoids id(obj) whenever a comparison function is defined.

Does anyone see a problem with my adding a similiar guard
to object.__hash__()?

This will prevent recurrence of a bug for newstyle objects which
define a comparison function but not a hash function.  For instance,
slice(5).__hash__() is defined, but hash(slice(5)) raises a TypeError.

An alternative solution is to change the logic in the slot 
inheritance to not inherit tp_hash whenever tp_compare or
tp_richcompare is defined.   This approach is a bit more tangled
and may have unforeseen consequences, but has the advantage
of preventing __hash__ from showing up method list for
the subclass:  dir(slice(5)) currently contains '__hash__'.

I prefer the first approach of making object.__hash__() smarter.


Raymond Hettinger




From gminick at hacker.pl  Thu Sep  4 23:04:48 2003
From: gminick at hacker.pl (gminick)
Date: Thu Sep  4 18:21:22 2003
Subject: [Python-Dev] Discordance in documentation...
Message-ID: <20030904200448.GA1164@hannibal>

...or is this just me?

Let's take a look, Reference Lib, 4.2.1 Regular Expression Syntax says:

   "|"
           A|B, where A and B can be arbitrary REs, creates a regular
           expression that will match either A or B.
           [...]
           REs separated by "|" are tried from left to right, and the 
           first one that allows the complete pattern to match is considered 
           the accepted branch. This means that if A matches, B will never 
           be tested, even if it would produce a longer overall match. [...]

And now a little test:

import re
a = "Fuentes Rushdie Marquez"
print re.search("Rushdie|Fuentes", a).group() # returns "Fuentes"

According to the documentation I suspected it will return "Rushdie" 
rather than "Fuentes", but it looks like it returns first part of the
string that matches rather than first part of regular expression.

-- 
[ Wojtek Walczak - gminick (at) underground.org.pl ]
[        <http://gminick.linuxsecurity.pl/>        ]
[ "...rozmaite zwroty, matowe od patyny dawnosci." ]


From postmaster at hermes.oceanet.fr  Thu Sep  4 18:00:40 2003
From: postmaster at hermes.oceanet.fr (postmaster@hermes.oceanet.fr)
Date: Thu Sep  4 18:24:58 2003
Subject: [Python-Dev] VIRUS DANS VOTRE MAIL
Message-ID: <200309041500.h84F0es05747@hermes.oceanet.fr>

                    A L E R T E   V I R A L E

Notre scanner anti-virus à trouvé le virus

	W32/Sobig.f@MM

dans le mail que vous avez envoyé aux adresses suivantes :

-> <cleroy@inter-faces.com>

Il serait préférable de passer votre système à l'antivirus.

Pour référence, voici les entêtes du mail :

------------------------- BEGIN HEADERS -----------------------------
Received: from USER-LPXAO0WZPQ ([219.95.15.228])
	by ns1.certif.net (8.11.6/8.11.6) with ESMTP id h84ExD806964
	for <contact@robostrike.com>; Thu, 4 Sep 2003 16:59:15 +0200
Message-Id: <200309041459.h84ExD806964@ns1.certif.net>
From: <python-dev@python.org>
To: <contact@robostrike.com>
Subject: Re: Your application
Date: Thu, 4 Sep 2003 22:59:52 +0800
X-MailScanner: Found to be clean
Importance: Normal
X-Mailer: Microsoft Outlook Express 6.00.2600.0000
X-MSMail-Priority: Normal
X-Priority: 3 (Normal)
MIME-Version: 1.0
Content-Type: multipart/mixed;
	boundary="_NextPart_000_00CB2F03"
-------------------------- END HEADERS ------------------------------


From halley at play-bow.org  Fri Sep  5 01:15:35 2003
From: halley at play-bow.org (Bob Halley)
Date: Thu Sep  4 20:15:37 2003
Subject: [Python-Dev] Discordance in documentation...
In-Reply-To: <20030904200448.GA1164@hannibal>
References: <20030904200448.GA1164@hannibal>
Message-ID: <m3wucoqeee.fsf@woof.play-bow.org>

gminick <gminick@hacker.pl> writes:

> import re
> a = "Fuentes Rushdie Marquez"
> print re.search("Rushdie|Fuentes", a).group() # returns "Fuentes"
> 
> According to the documentation I suspected it will return "Rushdie" 
> rather than "Fuentes", but it looks like it returns first part of the
> string that matches rather than first part of regular expression.

You are assuming that

        re.search("Rushdie|Fuentes", a)

is evaluated like

        m = re.search("Rushdie", a)
        if m is None:
            m = re.search("Fuentes", a)

But it doesn't work that way.  It works more like:

        m = None
        for i in xrange(0, len(a)):
            s = a[i:]
            m = re.match("Rushdie|Fuentes", s)
            if not m is None:
                break

I.e., try to match the expression at the start of the string; if that
doesn't work, try matching at the next character, etc.

From Martin.vonLoewis at hpi.uni-potsdam.de  Wed Sep  3 19:40:24 2003
From: Martin.vonLoewis at hpi.uni-potsdam.de (Loewis Martin von)
Date: Thu Sep  4 20:27:39 2003
Subject: [Python-Dev] Proper usage of size_t
Message-ID: <5AC0F51A926CD442BC08D55DA7DAB1D4692442@3MXMA1R.hpi.uni-potsdam.de>

On Win64, the Windows SDK compiler gives tons of warnings 
that type conversions may involve truncation. For example,
in 2.3 stringobject.c, a subset of the warnings is

..\Objects\stringobject.c(285) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
..\Objects\stringobject.c(316) : warning C4244: 'function' : conversion from '__int64' to 'int', possible loss of data
..\Objects\stringobject.c(554) : warning C4244: 'function' : conversion from '__int64' to 'int', possible loss of data
..\Objects\stringobject.c(812) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
..\Objects\stringobject.c(2154) : warning C4244: 'function' : conversion from '__int64' to 'int', possible loss of data
..\Objects\stringobject.c(3410) : warning C4267: 'return' : conversion from 'size_t' to 'int', possible loss of data
..\Objects\stringobject.c(3618) : warning C4267: 'return' : conversion from 'size_t' to 'int', possible loss of data
..\Objects\stringobject.c(3737) : warning C4244: '=' : conversion from '__int64' to 'int', possible loss of data
..\Objects\stringobject.c(4072) : warning C4244: '=' : conversion from '__int64' to 'int', possible loss of data
..\Objects\stringobject.c(4076) : warning C4244: '=' : conversion from '__int64' to 'int', possible loss of data

These warnings occur for stuff like

  i = strlen(p);

where i is of type int, but strlen returns size_t, and that might 
exceed what int can store. Likewise, in

  _PyString_Resize(&string, s - PyString_AS_STRING(string));

the pointer difference is of type size_t, but PyString_Resize 
expects int.

I'd be willing to go through and replace "int" by "size_t" 
whereever I find it appropriate, for Python 2.4. I would keep
the current object layout for the moment, but change function
signatures, which would cause ABI breakage on platforms where
sizeof(size_t)!=sizeof(int).

What do you think?

Regards,
Martin

From tjreedy at udel.edu  Thu Sep  4 21:43:31 2003
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu Sep  4 20:43:39 2003
Subject: [Python-Dev] Re: Discordance in documentation...
References: <20030904200448.GA1164@hannibal>
Message-ID: <bj8m7k$nus$1@sea.gmane.org>


"gminick" <gminick@hacker.pl> wrote in message
news:20030904200448.GA1164@hannibal...
> ...or is this just me?
>
> Let's take a look, Reference Lib, 4.2.1 Regular Expression Syntax
says:

It is the Library Reference, not Ref Lib.

>    "|"
>            A|B, where A and B can be arbitrary REs, creates a
regular
>            expression that will match either A or B.
>            [...]
>            REs separated by "|" are tried from left to right, and
the
>            first one that allows the complete pattern to match is
considered
>            the accepted branch. This means that if A matches, B will
never
>            be tested, even if it would produce a longer overall
match. [...]

I think the following version of the last four lines is correct and
clearer.

As the target string is scanned, REs separated by "|" are tried from
left to right.  When one pattern completely matches, that branch is
accepted.  This means that once A matches, B will not be tested
further, even if it would produce a longer overall match.

> And now a little test:
>
> import re
> a = "Fuentes Rushdie Marquez"
> print re.search("Rushdie|Fuentes", a).group() # returns "Fuentes"
>
> According to the documentation I suspected it will return "Rushdie"
> rather than "Fuentes", but it looks like it returns first part of
the
> string that matches rather than first part of regular expression.

As I hope my rewrite makes clearer,  consideration of alternatives is
nested within scanning of the source, and not vice verse as you
inferred from the current doc.

Doc bugs like this can (and should be) reported on SourceForge just
like program bugs.
http://sourceforge.net/tracker/?group_id=5470
That way, the report stays on the open list until someone decides
either to make a fix or that no fix is needed.

Terry J. Reedy




From kiko at async.com.br  Wed Sep  3 11:41:55 2003
From: kiko at async.com.br (Christian Reis)
Date: Thu Sep  4 20:56:26 2003
Subject: Solutions for LC_NUMERIC,
	was Re: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEIEFKAB.tim.one@comcast.net>
References: <m37k4qki2j.fsf@mira.informatik.hu-berlin.de>
	<LNBBLJKPBEHFEDALKOLCCEIEFKAB.tim.one@comcast.net>
Message-ID: <20030903134155.GB649@async.com.br>

On Tue, Sep 02, 2003 at 10:14:10PM -0400, Tim Peters wrote:
> > That said: Implementations might choose to ignore the standard in that
> > respect. This issue just supports my thesis that the patch is
> > complicated: If I have to read the C99 standard to find out whether it
> > is correct, it must be complicated. I doubt either the submitters or
> > the original author of the code did that exercise...
> 
> Well, any time a C library function gets called, you may have to scour the
> standard to answer questions about it -- and if they're libm functions you
> can just trust me that the standard doesn't contain useful answers <wink>.
> That's what comments are for.

Well, I disagree that the patch is complicated (it's not -- the code was
preserved to simplify porting fixes to and from glib, but that's not a
resolution and can be changed); the problem is that the corner cases
here aren't trivial to grasp. We have libc functions that rely on global
state (locale information), with no locking primitives, and no XP
support for locale-independent API.

When I wrote to python-dev initially, I wasn't aware of all the issues
at hand; what I really wanted was to bring up the matter and try and
find ways to solve the problem. The proposal I brought up is a WIP, and
I expected it to change after initial discussion had come up (since all
Martin had said previous to my PEP was "no" ;-)

I have come to agree with Martin (but only after perceiving Tim's
problem with Windows and Outlook) that it would be better if we found
platform-specific locale-independent conversion functions. However, it 
seems that we won't be able to find them on *all* platforms (and, from
Tim's findings -- I don't have access to Windows -- they're missing on a
major platform).

Now, agreeing on the solution depends on finding the correct compromise.
Here's my reasoning:

    - The current implementation is broken -- it's not thread-safe, and
      it makes the locale inconsistent in applications that change
      LC_NUMERIC via the locale module.

    - We should try and use locale-independent primitives where
      available: they offer thread-safety, apart from solving the
      immediate problem.
      
      This is fixed with Gustavo's patch *on glibc platforms*, which
      uses strtod_l and the glibc 2.3 thread-aware uselocale() and
      snprintf() API.

      This is *not* fixed with Gustavo's patch for other platforms.

    - On platforms where locale-independent primitives are unavailable,
      we need to provide our own solution. There are three alternatives
      we have explored here:

        a) massage the data to/from a locale-accepted format and call
           the locale-dependent libc functions. This is *not*
           thread-safe.

        b) provide our own, complete, implementation of strtod and
           snprintf.

        c) link against a library that offers a locale-safe version of
           the functions.

      Alternatives b) and c) would be thread-safe. a) is not.

      The problem with alternative b) is finding somebody with the free
      cycles to implement the function *AND* convincing python-dev that
      the code is correct. Gustavo did some looking through the glibc
      code for the functions in question yesterday and, well, if people
      scoffed at code from glib, I dare them to review that code.

      I assume c) could be provided on Windows with cygwin.

Solution b) above would solve the problem completely. Solution c) here
would solve the problem on platforms where such libraries were
available. If nobody can find, contribute, or accept any of the
solutions, then I suggest using solution a) and release noting the
problem with text such as

    "On glibc-based systems, Python is safe from runtime changes in the
    LC_NUMERIC locale category. On other platforms, changing LC_NUMERIC
    at runtime may cause float conversion routines to fail. This may
    impact the Python interpreter internally."

Note that apart from the glibc part, this is also true today (it's just
that a C library is the only thing able to change LC_NUMERIC right
now). In the worst case (no thread-safe locale) we are not at lot worse
than where we started -- we are not thread-safe, but at least we can get
locale consistency for applications that don't toy with LC_NUMERIC at
runtime (when it's none of their business <wink>).

Take care,
--
Christian Reis, Senior Engineer, Async Open Source, Brazil.
http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL

From europractice-bounces at ls12sn.cs.uni-dortmund.de  Wed Sep  3 05:13:04 2003
From: europractice-bounces at ls12sn.cs.uni-dortmund.de (europractice-bounces@ls12sn.cs.uni-dortmund.de)
Date: Thu Sep  4 20:58:15 2003
Subject: [Python-Dev] Your message to Europractice awaits moderator approval
Message-ID: <mailman.76.1062555184.294.europractice@ls12.cs.uni-dortmund.de>

Your mail to 'Europractice' with the subject

    Re: Thank you!

Is being held until the list moderator can review it for approval.

The reason it is being held:

    Post by non-member to a members-only list

Either the message will get posted to the list, or you will receive
notification of the moderator's decision.  If you would like to cancel
this posting, please visit the following URL:

    http://ls12-www.cs.uni-dortmund.de/mailman/confirm/europractice/a5ca58b7a468a440d33b3d60f58917efc673db7b


From tim.one at comcast.net  Thu Sep  4 22:37:17 2003
From: tim.one at comcast.net (Tim Peters)
Date: Thu Sep  4 21:37:25 2003
Subject: [Python-Dev] Proper usage of size_t
In-Reply-To: <5AC0F51A926CD442BC08D55DA7DAB1D4692442@3MXMA1R.hpi.uni-potsdam.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCGECGFLAB.tim.one@comcast.net>

[Martin von Loewis]
> On Win64, the Windows SDK compiler gives tons of warnings
> that type conversions may involve truncation. For example,
> in 2.3 stringobject.c, a subset of the warnings is
>
> ..\Objects\stringobject.c(285) : warning C4267: '=' : conversion
> from 'size_t' to 'int', possible loss of data
> ..\Objects\stringobject.c(316) : warning C4244: 'function' :
> conversion from '__int64' to 'int', possible loss of data
> ..\Objects\stringobject.c(554) : warning C4244: 'function' :
> conversion from '__int64' to 'int', possible loss of data
> ..\Objects\stringobject.c(812) : warning C4267: 'function' :
> conversion from 'size_t' to 'int', possible loss of data
> ..\Objects\stringobject.c(2154) : warning C4244: 'function' :
> conversion from '__int64' to 'int', possible loss of data
> ..\Objects\stringobject.c(3410) : warning C4267: 'return' :
> conversion from 'size_t' to 'int', possible loss of data
> ..\Objects\stringobject.c(3618) : warning C4267: 'return' :
> conversion from 'size_t' to 'int', possible loss of data
> ..\Objects\stringobject.c(3737) : warning C4244: '=' : conversion
> from '__int64' to 'int', possible loss of data
> ..\Objects\stringobject.c(4072) : warning C4244: '=' : conversion
> from '__int64' to 'int', possible loss of data
> ..\Objects\stringobject.c(4076) : warning C4244: '=' : conversion
> from '__int64' to 'int', possible loss of data
>
> These warnings occur for stuff like
>
>   i = strlen(p);
>
> where i is of type int, but strlen returns size_t, and that might
> exceed what int can store. Likewise, in
>
>   _PyString_Resize(&string, s - PyString_AS_STRING(string));
>
> the pointer difference is of type size_t, but PyString_Resize
> expects int.
>
> I'd be willing to go through and replace "int" by "size_t"
> whereever I find it appropriate, for Python 2.4.

I've done that often in the past, but piecemeal.  Note that another raft of
changes usually follows from this:  it's not only the size of the integral
type that may change, but regardless of platform it's also a change from a
signed type to an unsigned type.  That often triggers a new round of
warnings (e.g., MSVC6 warns when mixing signed and unsigned in comparisons).

> I would keep the current object layout for the moment, but change
> function signatures, which would cause ABI breakage on platforms where
> sizeof(size_t)!=sizeof(int).
>
> What do you think?

It's overdue -- go for it!  +1.


From martin at v.loewis.de  Fri Sep  5 04:56:51 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Sep  4 23:56:52 2003
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
In-Reply-To: <20030903065042.18889.qmail@web40107.mail.yahoo.com>
References: <20030903065042.18889.qmail@web40107.mail.yahoo.com>
Message-ID: <m3d6ehk1k9.fsf@mira.informatik.hu-berlin.de>

Scott Gilbert <xscottg@yahoo.com> writes:

> I'm guessing you're doing housekeeping (instead of waiting diligently for
> the implementation :-), so if you'd like to mark this PEP as abandoned that
> makes sense to me...

I was doing housekeeping indeed (of my mailbox); I wouldn't be pushing
for a resolution on this PEP though. However, if everybody agrees:

PEP editors, can you please mark PEP 296 as Withdrawn?

Regards,
Martin


From mwh at python.net  Wed Sep  3 13:08:13 2003
From: mwh at python.net (Michael Hudson)
Date: Fri Sep  5 00:27:41 2003
Subject: [Python-Dev] readline.clear_history()
In-Reply-To: <CGECIJPNNHIFAJKHOLMAKEHFGJAA.sholden@holdenweb.com> (Steve
	Holden's message of "Mon, 1 Sep 2003 19:14:54 -0400")
References: <CGECIJPNNHIFAJKHOLMAKEHFGJAA.sholden@holdenweb.com>
Message-ID: <2md6eiruxu.fsf@starship.python.net>

"Steve Holden" <sholden@holdenweb.com> writes:

> [Michael Hudson]
>
>> And, come Panther, there are going to be rather a lot of Python 2.3
>> installs out there.
>>
> Just as a matter of interest, do you have any idea how many Mac OSX
> licensees there are?

Nope.  Jaguar apparently shipped 100,000 copies in its first weekend
(including preorders).  It's also on all new macs.  Million or so?

Cheers,
mwh

-- 
  I have a cat, so I know that when she digs her very sharp claws into
  my chest or stomach it's really a sign of affection, but I don't see
  any reason for programming languages to show affection with pain.
                                        -- Erik Naggum, comp.lang.lisp

From guido at python.net  Thu Sep  4 22:27:01 2003
From: guido at python.net (Guido van Rossum)
Date: Fri Sep  5 00:32:07 2003
Subject: [Python-Dev] Undocumented functions in 'readline' module
In-Reply-To: Your message of "Thu, 04 Sep 2003 15:55:32 EDT."
	<5.1.1.6.0.20030904155048.00acfd20@mail.rapidsite.net> 
References: <5.1.1.6.0.20030904155048.00acfd20@mail.rapidsite.net> 
Message-ID: <200309050427.h854R2k28334@12-236-84-31.client.attbi.com>

> While working on the clear_history() patch today, I noticed that there are 
> some readline functions in the current CVS that don't appear in the docs:
> 
> redisplay()
> get_history_item()
> get_current_history_length()
> 
> Are these intentionally undocumented for some reason, or should I submit a 
> doc bug?

By all means submit a fix!

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.net  Thu Sep  4 22:31:21 2003
From: guido at python.net (Guido van Rossum)
Date: Fri Sep  5 00:32:17 2003
Subject: [Python-Dev] Discordance in documentation...
In-Reply-To: Your message of "Thu, 04 Sep 2003 22:04:48 +0200."
	<20030904200448.GA1164@hannibal> 
References: <20030904200448.GA1164@hannibal> 
Message-ID: <200309050431.h854VLm28352@12-236-84-31.client.attbi.com>

> ...or is this just me?

No, there are many others who are just as misguided. :-)

> Let's take a look, Reference Lib, 4.2.1 Regular Expression Syntax says:
> 
>    "|"
>            A|B, where A and B can be arbitrary REs, creates a regular
>            expression that will match either A or B.
>            [...]
>            REs separated by "|" are tried from left to right, and the 
>            first one that allows the complete pattern to match is considered 
>            the accepted branch. This means that if A matches, B will never 
>            be tested, even if it would produce a longer overall match. [...]
> 
> And now a little test:
> 
> import re
> a = "Fuentes Rushdie Marquez"
> print re.search("Rushdie|Fuentes", a).group() # returns "Fuentes"
> 
> According to the documentation I suspected it will return "Rushdie" 
> rather than "Fuentes", but it looks like it returns first part of the
> string that matches rather than first part of regular expression.

There's probably an official answer for this, but maybe this helps you
to explain it: re.search("A|B", s) doesn't first search s for A and
then for B.  Rather, for each successive position of s, it searches
for a match *starting at that point* for "A|B".  So when using
search(), the first match wins.  The rule about which branch of the |
wins only applies when both match at the same point.

(In general, anchored matching is the more fundamental re operation,
and searching is done by trying to match at successive positions until
a match is found.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From gminick at hacker.pl  Fri Sep  5 09:51:28 2003
From: gminick at hacker.pl (gminick)
Date: Fri Sep  5 03:15:16 2003
Subject: [Python-Dev] Re: Discordance in documentation...
In-Reply-To: <bj8m7k$nus$1@sea.gmane.org>
References: <20030904200448.GA1164@hannibal> <bj8m7k$nus$1@sea.gmane.org>
Message-ID: <20030905065128.GA240@hannibal>

On Thu, Sep 04, 2003 at 08:43:31PM -0400, Terry Reedy wrote:
[...]
> I think the following version of the last four lines is correct and
> clearer.
> 
> As the target string is scanned, REs separated by "|" are tried from
> left to right.  When one pattern completely matches, that branch is
> accepted.  This means that once A matches, B will not be tested
> further, even if it would produce a longer overall match.
Well IMHO this one is better, thanks.

[...]
> Doc bugs like this can (and should be) reported on SourceForge just
> like program bugs.
> http://sourceforge.net/tracker/?group_id=5470
I'll do it. I think it can be clearer.

ps. Thanks to Bob Halley for excellent explanation with the code ;-)

-- 
[ Wojtek Walczak - gminick (at) underground.org.pl ]
[        <http://gminick.linuxsecurity.pl/>        ]
[ "...rozmaite zwroty, matowe od patyny dawnosci." ]


From Jack.Jansen at cwi.nl  Fri Sep  5 11:02:15 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Fri Sep  5 04:02:20 2003
Subject: [Python-Dev] readline.clear_history()
In-Reply-To: <CGECIJPNNHIFAJKHOLMAKEHFGJAA.sholden@holdenweb.com>
Message-ID: <43270311-DF77-11D7-9CE2-0030655234CE@cwi.nl>


On Tuesday, September 2, 2003, at 01:14 AM, Steve Holden wrote:
>> And, come Panther, there are going to be rather a lot of Python 2.3
>> installs out there.
>>
> Just as a matter of interest, do you have any idea how many Mac OSX
> licensees there are?

Apple sells about 3 million machines per year (taking their Q3-02 and  
Q3-03
figures and interpolating from that:-) and since last year these boot  
into OSX
by default (and, since the last couple of months they only boot into  
OSX).

Adding the conversion factor from older machines I come to about 5  
million.
And, surprise, that's also what some other people came up with using
a different method:  
<http://www.omnigroup.com/mailman/archive/macosx-talk/2003-August/ 
015570.html>.

But we're really interested in how many people will be running Panther,
and I think that number is going to be lower. Very few people stuck with
10.1, basically only people whose machines wouldn't run 10.2 (pre-G3  
processors),
because 10.2 was really *way* ahead of 10.1. The 10.2->10.3 difference  
is
probably less (to the non-Pythonic general community:-), so how about
1 million 10.3 users by the end of the year, 3-4 million by mid-2004?

Any correspondence of these number with reality is pure chance,  
though:-)
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma  
Goldman


From bac at OCF.Berkeley.EDU  Fri Sep  5 13:46:19 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Sep  5 15:46:32 2003
Subject: [Python-Dev] python-dev Summary for 2003-08-16 through 2003-08-31
	[draft] [re-send]
Message-ID: <3F58E80B.2090801@ocf.berkeley.edu>

[since the original mailing of this didn't seem to get to the list I am 
re-sending]

This is a rather light summary, so finding my usual grammatical errors 
shouldn't be too difficult.  With so many people on vacation I will 
probably wait on sending this summary out for a little while.

On a personal note, this is my 24th summary.  What this means is I have 
now written enough summaries to cover a year's worth.  If you care to 
see some stats on the list read the "Summary Announcements" section. And 
if anyone on the list wants the pickled data I have for this or just 
want to know a specific stat I can look it up for you and tell you.

--------------------------------

python-dev Summary for 2003-08-16 through 2003-08-31
++++++++++++++++++++++++++++++++++++++++++++++++++++
This is a summary of traffic on the `python-dev mailing list`_ from 
August 16, 2003 through August 31, 2003.  It is intended to inform the 
wider Python community of on-going developments on the list.  To comment 
on anything mentioned here, just post to python-list@python.org or 
`comp.lang.python`_ with a subject line mentioning what you are 
discussing. All python-dev members are interested in seeing ideas 
discussed by the community, so don't hesitate to take a stance on 
something.  And if all of this really interests you then get involved 
and join `python-dev`_!

This is the twenty-fourth summary written by Brett Cannon (a year's 
worth of summaries by yours truly now under his belt; does this mean I 
am certifiably insane?).

All summaries are archived at http://www.python.org/dev/summary/ .

Please note that this summary is written using reStructuredText_ which 
can be found at http://docutils.sf.net/rst.html .  Any unfamiliar 
punctuation is probably markup for reST_ (otherwise it is probably 
regular expression syntax or a typo =); you can safely ignore it, 
although I suggest learning reST; its simple and is accepted for `PEP 
markup`_ and gives some perks for the HTML output.  Also, because of the 
wonders of programs that like to reformat text, I cannot guarantee you 
will be able to run the text version of this summary through Docutils_ 
as-is unless it is from the original text file.

.. _PEP Markup: http://www.python.org/peps/pep-0012.html

The in-development version of the documentation for Python can be found 
at http://www.python.org/dev/doc/devel/ and should be used when looking 
up any documentation on something mentioned here.  Python PEPs (Python 
Enhancement Proposals) are located at http://www.python.org/peps/ .  To 
view files in the Python CVS online, go to 
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/ .  Reported bugs 
and suggested patches can be found at the SourceForge_ project page.

.. _python-dev: http://www.python.org/dev/
.. _SourceForge: http://sourceforge.net/tracker/?group_id=5470
.. _python-dev mailing list: 
http://mail.python.org/mailman/listinfo/python-dev
.. _comp.lang.python: http://groups.google.com/groups?q=comp.lang.python
.. _Docutils: http://docutils.sf.net/
.. _reST:
.. _reStructuredText: http://docutils.sf.net/rst.html

.. contents::

.. _last summary: 
http://www.python.org/dev/summary/2003-08-01_2003-08-15.html


=====================
Summary Announcements
=====================
This is the twenty-fourth summary written by me.  Why am I repeating 
this fact since it is mentioned above?  Well, it is significant because 
this means I have written enough summaries to cover a year's worth of 
email traffic on python-dev (had I not taken a summary off back in 
October this milestone would have been hit for the first half of August 
which represented a physical year since I started doing the summaries). 
  I have managed to see a lot happen on python-dev from a new release, 
the first PyCon, and a number of flame wars.  I personally have managed 
to learn a *huge* amount about not just Python the language but how to 
program in Python, C, and how to handle a large programming project.  It 
has truly been worth the experience.

But how much of an experience has it been?  Well, for a long time now I 
have been planning on writing some code to calculate how much email I 
have read, who wrote most of that email, and what threads made up the 
most.  With my move to San Luis Obispo finished I finally had a chance 
to write said code in an imperfect manner (names, for instance, are a 
pain because some people have their name set differently at different 
computers; "Barry Warsaw" compared to "Barry A. Warsaw"; threads are 
worse thanks to the changing of subject titles in the middle of a 
thread) so as to give me some approximate numbers.

I have read 9469 emails that have passed through the python-dev mailing 
list.  The top six emailers (out of approx. 433 unique emailers) have been:

* Brett Cannon (277 emails when you deal with me just using my last 
initial; 2.9% of all emails)
* Barry Warsaw (305 emails when you also count his middle initial; 3.2%)
* Skip Montanaro (481 emails; 5.1%)
* Martin v. L?wis (627 emails, when calculated looking for all names 
that had "Martin" and "wis" in them; 6.6%)
* Tim Peters (694 emails; 7.3%)
* Guido van Rossum (a whopping 1407 emails; 14.8%)

The average person posted 21.9 emails over the emails I covered.  But 
only about 24 people had more than a single percentage (more than 94 
emails) worth of emails accredited to them.  That means that about 5.5% 
of the unique posters on python-dev accounted for 66.8% of all email 
(and I have gotten to know a good amount of those 24.

As for threads (of which there were *very* approx. 1252 unique threads), 
the top five are:

* "type categories" (115 emails; 1.2% of all emails)
* "PEP239 (Rational Numbers) Reference Implementation and new issues" 
(123 emails; 1.3%)
* "PEP-317" (125 emails: 1.3%)
* "python/dist/src/Python import.c,2.210,2.211" (146 emails; 1.5%)
* "Extended Function syntax" (263 emails; 2.8%)

What does this tell you and me?  First, that I contribute to my own pain 
by more than I thought (I can't believe I emailed that many times!). 
Second, I must *really* like reading.  Third, I have an addiction to 
Python.  Fourth, PEPs really do get discussed.  And fifth, Python 
development is alive and well.


OK, enough statistics.  As for this summary, it turned out rather light 
thanks to a couple of things.  One is the shutdown of mail delivery by 
mail.python.org during the SoBig virus' peak.  This not only cut back on 
the number of emails, but also led to me deleting *a lot* of bogus email 
on my other emails accounts.  My blanket deleting may have caught 
legitimate emails so it is possible I accidently deleted some python-dev 
stuff, although if I did it was minimal.  Another contributing factor to 
the light summary is that a lot of regulars on python-dev were on 
vacation.  This looks like it will happen again for the first half of 
September so expect the next summary to be light as well.


=========
Summaries
=========
------------------------------------------------------------------------------
Python using Parrot; new code interpreter or strange evolutionary 
parternship?
------------------------------------------------------------------------------
Pirate_ has now reached version 0.01 alpha.  It lacks classes and 
imports but can run a decent amount of Python code.  At least there is 
now a proof-of-concept that Python running on top of the Parrot_ VM is 
possible.

.. _Pirate: http://pirate.tangentcode.com/
.. _Parrot: http://www.parrotcode.org/

Contributing threads:
   - `pirate 0.01 alpha! 
<http://mail.python.org/pipermail/python-dev/2003-August/037684.html>`__


------------------------
Python 2.3.1 on its way?
------------------------
Raymond Hettinger suggesting pushing for a quick release of Python 2.3.1 
so that the 2.3 branch could be established as a stable version. Several 
bugs and performance enhancements have been committed to the 2.3 
maintenance branch.  Anthony Baxter stepped forward as release czar with 
Raymond Hettinger saying he would help and Barry Warsaw volunteering his 
wisdom as a battle-hardened release czar.

This discussion also brought up the question of whether a .chm help file 
for the Windows distribution would be worth using instead of including 
the HTML distribution of the documentation as it stands now.  It was 
agreed that it was a good thing to have since it allowed for better 
searching.  Tim Peters also discovered the install went faster since it 
would not have to copy a ton of individual HTML files.

Python 2.3.1 has a "verbal" release date of the third week of September; 
there has not been a PEP to set the release schedule officially.

Help would be appreciated in dealing with bug and patch reports on 
SourceForge.  Even if all you do is add a comment saying "this patch 
looks fine" or "I can reproduce this bug" it can be a great help.

Contributing threads:
   - `Py2.3.1 
<http://mail.python.org/pipermail/python-dev/2003-August/037687.html>`__
   - `HTMLHelp for Py2.3.1 
<http://mail.python.org/pipermail/python-dev/2003-August/037866.html>`__
   - `Fixing Patches and Bugs for Py2.3.1 
<http://mail.python.org/pipermail/python-dev/2003-August/037840.html>`__


----------------------------------------
Making looping generators more efficient
----------------------------------------
Would you like to see deeply nested generators be more efficient in 
returning their values?  Clark Evans would and made such a request.  He 
essentially wanted to have nested generator calls propogate their values 
to the first non-generator call directly and thus bypass all of the 
generator maintenance code.  There was no direct reaction to this.

Shane Holloway followed with the idea of having special syntax for when 
you yield each value of an iterator.  The idea, once again, would be to 
speed this common case in the interpreter by skipping some bookkeeping 
overhead.  A few syntax versions were offered, but the idea was all the 
same: special-case ``for item in iterable: yield item`` to something 
like ``yield *iterable``.

Contributing threads:
   - `cooperative generators 
<http://mail.python.org/pipermail/python-dev/2003-August/037708.html>`__
   - `Graph exploration with generators 
<http://mail.python.org/pipermail/python-dev/2003-August/037738.html>`__


--------------------------------------------------
Use of the logging package in the standard library
--------------------------------------------------
Want to help out the development of Python?  Know how to use the logging 
package?  Then python-dev wants you!  There are several modules in the 
stdlib that have home-grown logging code that could (and probably 
should) be using the logging package instead to simplify life.  Read the 
email that started the contributing thread and see if you can't help out 
by converting the module over to using the logging package today!

Contributing threads:
   - `Unification of logging in Python's Standard Library 
<http://mail.python.org/pipermail/python-dev/2003-August/037743.html>`__


-----------------------
Some waxings on PEP 310
-----------------------
PEP 310 proposes the 'with' syntax that came up a while back that 
sparked an immense discussion on python-dev.  The idea was to have a 
more fool-proof way of having an enter and exit method be called before 
executing some specified code.  The common example was acquiring a lock, 
executing some code, and then releasing the lock all without having to 
deal with an explicit try/finally statement.  Samuele Pedroni tried to 
clarify how it should work exactly by requiring __exit__ instead of 
making it optional (read the PEP to understand what this means).

Contributing threads:
   - `PEP 310(with-syntax): close synonym of __exit__ 
<http://mail.python.org/pipermail/python-dev/2003-August/037795.html>`__


------------------------------------------------
Proposed PEP for a 'close' method for generators
------------------------------------------------
Samuele Pedroni has written a pre-PEP on defining a way to have 
generators grow a way to have a 'close' method that is called when their 
execution is finished so as to handle resources correctly.  This is in 
response to not being able to contain yield statements within 
try/finally blocks.

Contributing threads:
   - `pre-PEP: Resource-Release Support for Generators 
<http://mail.python.org/pipermail/python-dev/2003-August/037803.html>`__


-----------------
email-sig created
-----------------
Barry Warsaw has created the `email-sig`_ to steer development of 
version 3 of the email package in hopes of having it done for Python 2.4 .

.. _email-sig: http://www.python.org/sigs/email-sig/

Contributing threads:
   - `New SIG: email-sig <New SIG: email-sig>`__




From tim.one at comcast.net  Sat Sep  6 14:11:14 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sat Sep  6 13:11:17 2003
Subject: [Python-Dev] Why do we flush before truncating?
Message-ID: <LNBBLJKPBEHFEDALKOLCCENJFLAB.tim.one@comcast.net>

   http://www.python.org/sf/801631

gives a failing program on Windows, paraphrased:

f = file('test.dat', 'wb')
f.write('1234567890')   # 10 bytes
f.close()

f = file('test.dat','rb+')
f.read(5)
print f.tell()  # prints 5, as expected

f.truncate()    # leaves the file at 10 bytes
print f.tell()  # prints 10


The problem is that fileobject.c's file_truncate() calls fflush() before
truncating.  The C standard says that the effect of calling fflush() is
undefined if the most recent operation on a stream opened for update was an
input operation.  The stream is indeed opened for update here, and the most
recent operation performed by the *user* was indeed a read.  It so happens
that MS's fflush() changes the file position then.  But the user didn't call
fflush(), Python did, so we can't blame the user for relying on undefined
behavior here.

The problem can be repaired inside file_truncate() by seeking back to the
original file position after the fflush() call -- but the original file
position isn't always available now, so I'd also have to add another call to
_portable_ftell() before the fflush() to find it.

So that gets increasingly complicated.  Much simpler would be to remove this
block of code (which does fix the test program's problem on Windows, by
simply getting rid of the undefined operation):

	/* Flush the file. */
	Py_BEGIN_ALLOW_THREADS
	errno = 0;
	ret = fflush(f->f_fp);
	Py_END_ALLOW_THREADS
	if (ret != 0)
		goto onioerror;

I don't understand why we're flushing the file.  ftruncate() isn't a
standard C function, so the standard sheds no light on why we might be doing
that.  AFAICT, POSIX/SUS doesn't give a reason to flush either:

  http://www.opengroup.org/onlinepubs/007904975/functions/ftruncate.html


From nas-python at python.ca  Sat Sep  6 12:23:40 2003
From: nas-python at python.ca (Neil Schemenauer)
Date: Sat Sep  6 14:16:45 2003
Subject: [Python-Dev] Why do we flush before truncating?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCENJFLAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCENJFLAB.tim.one@comcast.net>
Message-ID: <20030906182340.GA31698@mems-exchange.org>

On Sat, Sep 06, 2003 at 01:11:14PM -0400, Tim Peters wrote:
> I don't understand why we're flushing the file.  ftruncate() isn't a
> standard C function, so the standard sheds no light on why we might be doing
> that.

The fflush call as been there forever. The truncate method was added
in 2.36 by Guido.  I think the code was actually from Jim Roskind:

  http://groups.google.com/groups?selm=199412070213.SAA06932%40infoseek.com

He says:

  Note that since the underlying ftruncate operates on a file
  descriptor (believe it or not), it was necessary to fflush() the
  stream before performing the truncate.  I thought about doing a
  seek() as well, but could not find compelling reason to move the
  stream pointer.

That still gives me no clue as to why the fflush() was deemed
necessary.  I found this posting:

  http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=35E0DB62.1BDD2D30%40taraz.kz&rnum=1

but, AFACK, the reason the program is not working the way the poster
expects is the missing fflush() call before the lseek() call (not
the fflush() before the ftruncate()).

  Neil

From guido at python.org  Sat Sep  6 12:53:36 2003
From: guido at python.org (Guido van Rossum)
Date: Sat Sep  6 14:54:37 2003
Subject: [Python-Dev] Why do we flush before truncating?
In-Reply-To: Your message of "Sat, 06 Sep 2003 13:11:14 EDT."
	<LNBBLJKPBEHFEDALKOLCCENJFLAB.tim.one@comcast.net> 
References: <LNBBLJKPBEHFEDALKOLCCENJFLAB.tim.one@comcast.net> 
Message-ID: <200309061853.h86IraQ31679@12-236-84-31.client.attbi.com>

>    http://www.python.org/sf/801631
> 
> gives a failing program on Windows, paraphrased:
> 
> f = file('test.dat', 'wb')
> f.write('1234567890')   # 10 bytes
> f.close()
> 
> f = file('test.dat','rb+')
> f.read(5)
> print f.tell()  # prints 5, as expected
> 
> f.truncate()    # leaves the file at 10 bytes
> print f.tell()  # prints 10
> 
> 
> The problem is that fileobject.c's file_truncate() calls fflush() before
> truncating.  The C standard says that the effect of calling fflush() is
> undefined if the most recent operation on a stream opened for update was an
> input operation.  The stream is indeed opened for update here, and the most
> recent operation performed by the *user* was indeed a read.  It so happens
> that MS's fflush() changes the file position then.  But the user didn't call
> fflush(), Python did, so we can't blame the user for relying on undefined
> behavior here.
> 
> The problem can be repaired inside file_truncate() by seeking back to the
> original file position after the fflush() call -- but the original file
> position isn't always available now, so I'd also have to add another call to
> _portable_ftell() before the fflush() to find it.
> 
> So that gets increasingly complicated.  Much simpler would be to remove this
> block of code (which does fix the test program's problem on Windows, by
> simply getting rid of the undefined operation):
> 
> 	/* Flush the file. */
> 	Py_BEGIN_ALLOW_THREADS
> 	errno = 0;
> 	ret = fflush(f->f_fp);
> 	Py_END_ALLOW_THREADS
> 	if (ret != 0)
> 		goto onioerror;
> 
> I don't understand why we're flushing the file.  ftruncate() isn't a
> standard C function, so the standard sheds no light on why we might be doing
> that.  AFAICT, POSIX/SUS doesn't give a reason to flush either:
> 
>   http://www.opengroup.org/onlinepubs/007904975/functions/ftruncate.html

ftruncate() is not a standard C function; it's a standard Unix system
call.  It works on a file descriptor (i.e. a small int), not on a
stream (i.e. a FILE *).  The fflush() call is necessary if the last
call was a write, because in that case the stream's buffer may contain
data that the OS file descriptor doesn't have yet.

But ftruncate() is irrelevant, because on Windows, it is never called;
there's a huge #ifdef MS_WINDOWS block containing Windows specific
code, starting with the comment

	/* MS _chsize doesn't work if newsize doesn't fit in 32 bits,
	   so don't even try using it. */

and the ftruncate() call is made in the #else part.

It also looks like the MS_WINDOWS specific code block *does* attempt
to record the current file position and seek back to it -- however it
does this after fflush() has already messed with it.  So perhaps
moving the fflush() call into the #else part and doing something
Windows-specific instead of calling fflush() to ensure the buffer is
flushed inside the MS_WINDOWS part would be the right solution.

I just realize that I have always worked under the assumption that
fflush() after a read is a no-op; I just checked the 89 std and it
says it is undefined.  (I must have picked up that misunderstanding
from some platform-specific man page.)  This can be fixed by doing a
ftell() followed by an fseek() call; this is required to flush the
buffer if there was unwritten output data in the buffer, and is always
allowed.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From bac at OCF.Berkeley.EDU  Sat Sep  6 16:31:52 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sat Sep  6 18:32:42 2003
Subject: [Python-Dev] Changing select.select to accept iterables
Message-ID: <3F5A6058.1080304@ocf.berkeley.edu>

Bug #798046 basically requests that select.select accept iterables 
instead of only lists.  I took a look at the code and beyond 
PyList_Check()s and a function called list2set that expects a Python 
list to be passed that can easily be changed to take in an iterable I 
see no need to keep the restriction.

But this module is old and I suspect it is used a good amount (the thing 
started at ver. 2.1 by Guido back in 1992).  Since it seems to have gone 
this long without being changed to use the iterator protocol I wanted to 
double-check that changing it would be okay.  The only reason I can 
think of why it wouldn't be is performance, as suggested by the OP of 
the bug report.

And in general, when something like this that smacks of, "there is no 
obvious reason to keep it this way", even though it could have been 
changed a while ago and the CVS log seems to not mention anything 
against the change, should the change just be made?  Or should I just 
ask like this to be safe?  I have a sneaking suspicion the answer is, 
"just change it, Brett!  Use your judgment!  Don't be so timid!"  But 
then at least this way I minimize getting chewed out for breaking code.

Then again, being yelled at makes you think like a Dutchman faster than 
anything I know of.  =)

-Brett


From guido at python.org  Sat Sep  6 17:04:40 2003
From: guido at python.org (Guido van Rossum)
Date: Sat Sep  6 19:04:52 2003
Subject: [Python-Dev] Changing select.select to accept iterables
In-Reply-To: Your message of "Sat, 06 Sep 2003 15:31:52 PDT."
	<3F5A6058.1080304@ocf.berkeley.edu> 
References: <3F5A6058.1080304@ocf.berkeley.edu> 
Message-ID: <200309062304.h86N4e601515@12-236-84-31.client.attbi.com>

> Bug #798046 basically requests that select.select accept iterables 
> instead of only lists.  I took a look at the code and beyond 
> PyList_Check()s and a function called list2set that expects a Python 
> list to be passed that can easily be changed to take in an iterable I 
> see no need to keep the restriction.
> 
> But this module is old and I suspect it is used a good amount (the thing 
> started at ver. 2.1 by Guido back in 1992).  Since it seems to have gone 
> this long without being changed to use the iterator protocol I wanted to 
> double-check that changing it would be okay.  The only reason I can 
> think of why it wouldn't be is performance, as suggested by the OP of 
> the bug report.
> 
> And in general, when something like this that smacks of, "there is no 
> obvious reason to keep it this way", even though it could have been 
> changed a while ago and the CVS log seems to not mention anything 
> against the change, should the change just be made?  Or should I just 
> ask like this to be safe?  I have a sneaking suspicion the answer is, 
> "just change it, Brett!  Use your judgment!  Don't be so timid!"  But 
> then at least this way I minimize getting chewed out for breaking code.
> 
> Then again, being yelled at makes you think like a Dutchman faster than 
> anything I know of.  =)

I seem to recall that that code has a long history of being hairy
and full of platform-specific issues, and I'd rather not see it
disturbed by an unspecific desire for more generalization.  Why can't
the OP produce the input in the form of lists?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.one at comcast.net  Sat Sep  6 22:54:34 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sat Sep  6 21:54:39 2003
Subject: [Python-Dev] Changing select.select to accept iterables
In-Reply-To: <200309062304.h86N4e601515@12-236-84-31.client.attbi.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEPEFLAB.tim.one@comcast.net>

[Brett, about <http://www.python.org/sf/798046>]

[Guido]
> I seem to recall that that code has a long history of being hairy
> and full of platform-specific issues, and I'd rather not see it
> disturbed by an unspecific desire for more generalization.  Why can't
> the OP produce the input in the form of lists?

They could, but they specifically already have Sets of socket objects.
That's what C would have used too for select(), if it had sets, so it's a
natural desire.  The SF report notes that when read and write lists change
frequently, it's a lot more efficient to add/remove sockets via O(1) Set
operations.  Under the covers, select() setup takes O(N) time even if the
inputs are native list.

Functions that expect input lists of reasonably small size (thousands, sure;
millions, probably not) can usually be generalized to iterables easily, with
little chance of disruption, by replacing initial "is it a list?" check with
a call to PySequence_Fast(), then
s/PyList_GetItem/PySequence_Fast_GET_ITEM/.


From bac at OCF.Berkeley.EDU  Sat Sep  6 20:29:11 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sat Sep  6 22:30:22 2003
Subject: [Python-Dev] Changing select.select to accept iterables
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEPEFLAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEPEFLAB.tim.one@comcast.net>
Message-ID: <3F5A97F7.7080700@ocf.berkeley.edu>

Tim Peters wrote:

> [Brett, about <http://www.python.org/sf/798046>]
> 
> [Guido]
> 
>>I seem to recall that that code has a long history of being hairy
>>and full of platform-specific issues, and I'd rather not see it
>>disturbed by an unspecific desire for more generalization.  Why can't
>>the OP produce the input in the form of lists?
> 
> 
> They could, but they specifically already have Sets of socket objects.
> That's what C would have used too for select(), if it had sets, so it's a
> natural desire.  The SF report notes that when read and write lists change
> frequently, it's a lot more efficient to add/remove sockets via O(1) Set
> operations.  Under the covers, select() setup takes O(N) time even if the
> inputs are native list.
> 

Tim is right about there not being a restriction of not being able to 
create a list but wanting a better data structure for the job.

And again I suspect Time is right about the socket API designers wanting 
sets if they had it in ANSI C; probably why they call it an fd_set for 
storing what file descriptors to work on.

As for the code being hairy, yes, it does have its share of #ifdefs. 
But the list checks and code directly handling the lists are outside of 
any #ifdefs.  The list codes is literally just the list checks and a 
'for' loop pulling out each item in the list using PyList_GetItem; after 
that everything operates on what was pulled out of the list.  But I 
understand the reluctance of messing with code that has a reputation and 
his a pain to test for something that is just a perk.

> Functions that expect input lists of reasonably small size (thousands, sure;
> millions, probably not) can usually be generalized to iterables easily, with
> little chance of disruption, by replacing initial "is it a list?" check with
> a call to PySequence_Fast(), then
> s/PyList_GetItem/PySequence_Fast_GET_ITEM/.
> 

Is this the preferred way of dealing with places where a small sequence 
and such will be instead of bothering with iterators, or is this a 
hold-over from pre-iterators and thus iterators are the proper way to go 
in new code and in old code where reasonable and relatively painless?

Since Guido has previously expressed reluctance in having me touch this 
code I won't unless he tells me otherwise.

-Brett


From tim.one at comcast.net  Sat Sep  6 23:40:16 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sat Sep  6 22:40:20 2003
Subject: [Python-Dev] Why do we flush before truncating?
In-Reply-To: <20030906182340.GA31698@mems-exchange.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEPJFLAB.tim.one@comcast.net>

[Neil Schemenauer]
> The fflush call as been there forever. The truncate method was added
> in 2.36 by Guido.  I think the code was actually from Jim Roskind:
>
> http://groups.google.com/groups?selm=199412070213.SAA06932%40infoseek.com
>
> He says:
>
>   Note that since the underlying ftruncate operates on a file
>   descriptor (believe it or not), it was necessary to fflush() the
>   stream before performing the truncate.  I thought about doing a
>   seek() as well, but could not find compelling reason to move the
>   stream pointer.
>
> That still gives me no clue as to why the fflush() was deemed
> necessary.

Ack, I glossed over the fileno() call in our file_truncate().  It's usually
a Very Bad Idea to mix stream I/O and lower-level I/O operations without
flushing your pants off, but I'm having a hard time thinking of a specific
reason for doing so in the truncate case.  Better safe than trying to
out-think all possible implementations, though!

> I found this posting:
>
> http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=35E
> 0DB62.1BDD2D30%40taraz.kz&rnum=1
>
> but, AFACK, the reason the program is not working the way the poster
> expects is the missing fflush() call before the lseek() call (not
> the fflush() before the ftruncate()).

I think that's right.  In the Python case, I verified in a debugger that the
file position is 5 immediately before the fflush() call, and 10 immediately
after it.  It's surprising, but apparently OK by the C std.

[Guido
> ftruncate() is not a standard C function;

I suppose that clarifies my immediately preceding

>> ftruncate() isn't a standard C function,

<wink>?

> it's a standard Unix system call.

Yes, and I gave a link to the current POSIX/SUS ftruncate() specification.

> It works on a file descriptor (i.e. a small int), not on a
> stream (i.e. a FILE *).

Right, and I missed that, primarily because Windows doesn't have ftruncate()
so I wasn't looking at that part of the code.

> The fflush() call is necessary if the last call was a write, because in
> that case the stream's buffer may contain data that the OS file
> descriptor doesn't have yet.

I'm not really clear on why that should matter in the specific case of
truncating a file, but will just live with it.

> But ftruncate() is irrelevant, because on Windows, it is never called;
> there's a huge #ifdef MS_WINDOWS block containing Windows specific
> code ...

Right, I wrote that code.  Windows has no way to say "here's a file, change
the size to such-and-such"; the only way is to set the file pointer to the
desired size, and then call the no-argument Win32 SetEndOfFile(); Python
*used* to use the MS C _chsize() function, but that did insane things when
passed a "large" size; the SetEndOfFile() code was introduced as part of
fixing Python's Windows largefile support.

> ...
> It also looks like the MS_WINDOWS specific code block *does* attempt
> to record the current file position and seek back to it

Yes, because the file position must be changed on Windows in order to change
the file size, but *Python's* docs promise that file.truncate() doesn't
change the current position (which is natural behavior under POSIX
ftruncate() but strained on Windows).

> -- however it does this after fflush() has already messed with it.

Note that in the Windows test case, it's not simply that the current
position wasn't preserved across the file.truncate() call, it's also that
the file didn't change size.  It's very easy to fix the former while leaving
the latter broken.

> So perhaps moving the fflush() call into the #else part and doing
> something Windows-specific instead of calling fflush() to ensure the
> buffer is flushed inside the MS_WINDOWS part would be the right solution.
>
> I just realize that I have always worked under the assumption that
> fflush() after a read is a no-op; I just checked the 89 std and it
> says it is undefined.  (I must have picked up that misunderstanding
> from some platform-specific man page.)  This can be fixed by doing a
> ftell() followed by an fseek() call; this is required to flush the
> buffer if there was unwritten output data in the buffer, and is always
> allowed.

That's what I was hoping to avoid, but I don't care anymore:  after staring
it some more, I'm convinced that the current file_truncate() endures a
ridiculous amount of complexity trying to gain a tiny bit of speed in what
has to be a rare operation.


From tim.one at comcast.net  Sat Sep  6 23:47:22 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sat Sep  6 22:47:27 2003
Subject: [Python-Dev] Changing select.select to accept iterables
In-Reply-To: <3F5A97F7.7080700@ocf.berkeley.edu>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEPKFLAB.tim.one@comcast.net>

[Tim]
>> Functions that expect input lists of reasonably small size
>> (thousands, sure; millions, probably not) can usually be generalized
>> to iterables easily, with little chance of disruption, by replacing
>> initial "is it a list?" check with a call to PySequence_Fast(), then
>> s/PyList_GetItem/PySequence_Fast_GET_ITEM/.

[Brett C.]
> Is this the preferred way of dealing with places where a small
> sequence and such will be instead of bothering with iterators, or is
> this a hold-over from pre-iterators and thus iterators are the proper
> way to go in new code and in old code where reasonable and relatively
> painless?

It's rare that you *know* an input will be at worst of reasonably small
size, and that's why PySequence_Fast() isn't used more often (plus, I
suppose, that it's relatively little known).  It's perfect for select()
inputs, which are not only certain to be reasonably small in working
programs (no working program can use millions of sockets simultaneously),
but also can't get any benefit from the "early out" possibilities afforded
by using the iterator protocol instead.

> Since Guido has previously expressed reluctance in having me touch
> this code I won't unless he tells me otherwise.

The change you'd like to make is very low-risk if you use the
PySequence_Fast() approach, mainly simple editor substitutions, and Guido
will like the idea of passing Sets to select() once he tries it <wink>.


From guido at python.org  Sat Sep  6 21:49:16 2003
From: guido at python.org (Guido van Rossum)
Date: Sat Sep  6 23:49:22 2003
Subject: [Python-Dev] Changing select.select to accept iterables
In-Reply-To: Your message of "Sat, 06 Sep 2003 19:29:11 PDT."
	<3F5A97F7.7080700@ocf.berkeley.edu> 
References: <LNBBLJKPBEHFEDALKOLCKEPEFLAB.tim.one@comcast.net>  
	<3F5A97F7.7080700@ocf.berkeley.edu> 
Message-ID: <200309070349.h873nGr01747@12-236-84-31.client.attbi.com>

> > [Brett, about <http://www.python.org/sf/798046>]
> > 
> > [Guido]
> > 
> >>I seem to recall that that code has a long history of being hairy
> >>and full of platform-specific issues, and I'd rather not see it
> >>disturbed by an unspecific desire for more generalization.  Why can't
> >>the OP produce the input in the form of lists?
> > 
> > 
> > They could, but they specifically already have Sets of socket objects.
> > That's what C would have used too for select(), if it had sets, so it's a
> > natural desire.  The SF report notes that when read and write lists change
> > frequently, it's a lot more efficient to add/remove sockets via O(1) Set
> > operations.  Under the covers, select() setup takes O(N) time even if the
> > inputs are native list.

[Brett]
> Tim is right about there not being a restriction of not being able to 
> create a list but wanting a better data structure for the job.
> 
> And again I suspect Time is right about the socket API designers wanting 
> sets if they had it in ANSI C; probably why they call it an fd_set for 
> storing what file descriptors to work on.

If Python had sets 10 years agon, select would've used sets too. :-)

> As for the code being hairy, yes, it does have its share of #ifdefs. 
> But the list checks and code directly handling the lists are outside of 
> any #ifdefs.  The list codes is literally just the list checks and a 
> 'for' loop pulling out each item in the list using PyList_GetItem; after 
> that everything operates on what was pulled out of the list.  But I 
> understand the reluctance of messing with code that has a reputation and 
> his a pain to test for something that is just a perk.
> 
> > Functions that expect input lists of reasonably small size
> > (thousands, sure; millions, probably not) can usually be
> > generalized to iterables easily, with little chance of disruption,
> > by replacing initial "is it a list?" check with a call to
> > PySequence_Fast(), then
> > s/PyList_GetItem/PySequence_Fast_GET_ITEM/.
> > 
> 
> Is this the preferred way of dealing with places where a small sequence 
> and such will be instead of bothering with iterators, or is this a 
> hold-over from pre-iterators and thus iterators are the proper way to go 
> in new code and in old code where reasonable and relatively painless?
> 
> Since Guido has previously expressed reluctance in having me touch this 
> code I won't unless he tells me otherwise.

Since Tim thinks it's okay to change (and since I now know what the OP
wanted) you have my blessing to give this a try.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Sep  6 21:55:46 2003
From: guido at python.org (Guido van Rossum)
Date: Sat Sep  6 23:55:51 2003
Subject: [Python-Dev] Why do we flush before truncating?
In-Reply-To: Your message of "Sat, 06 Sep 2003 22:40:16 EDT."
	<LNBBLJKPBEHFEDALKOLCOEPJFLAB.tim.one@comcast.net> 
References: <LNBBLJKPBEHFEDALKOLCOEPJFLAB.tim.one@comcast.net> 
Message-ID: <200309070355.h873tkU01779@12-236-84-31.client.attbi.com>

> Ack, I glossed over the fileno() call in our file_truncate().  It's usually
> a Very Bad Idea to mix stream I/O and lower-level I/O operations without
> flushing your pants off, but I'm having a hard time thinking of a specific
> reason for doing so in the truncate case.  Better safe than trying to
> out-think all possible implementations, though!

I think the fflush() is there for the following case: a file is opened
for update, some data is written that overwrites some bytes in the
middle but not yet flushed, and then the file is truncated to that
position.  Since ftruncate works on the file descriptor, you'd want
the data flushed before truncating, otherwise things just get too
complicated.

> I suppose that clarifies my immediately preceding
> 
> >> ftruncate() isn't a standard C function,
> 
> <wink>?

I somehow misread what you wrote as "is a standard C function".

> > I just realize that I have always worked under the assumption that
> > fflush() after a read is a no-op; I just checked the 89 std and it
> > says it is undefined.  (I must have picked up that misunderstanding
> > from some platform-specific man page.)  This can be fixed by doing a
> > ftell() followed by an fseek() call; this is required to flush the
> > buffer if there was unwritten output data in the buffer, and is always
> > allowed.
> 
> That's what I was hoping to avoid, but I don't care anymore:  after staring
> it some more, I'm convinced that the current file_truncate() endures a
> ridiculous amount of complexity trying to gain a tiny bit of speed in what
> has to be a rare operation.

Right.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From bac at OCF.Berkeley.EDU  Sat Sep  6 22:16:29 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Sep  7 00:17:25 2003
Subject: [Python-Dev] Changing select.select to accept iterables
In-Reply-To: <200309070349.h873nGr01747@12-236-84-31.client.attbi.com>
References: <LNBBLJKPBEHFEDALKOLCKEPEFLAB.tim.one@comcast.net>
	<3F5A97F7.7080700@ocf.berkeley.edu>
	<200309070349.h873nGr01747@12-236-84-31.client.attbi.com>
Message-ID: <3F5AB11D.4070409@ocf.berkeley.edu>

Guido van Rossum wrote:
> 
> Since Tim thinks it's okay to change (and since I now know what the OP
> wanted) you have my blessing to give this a try.
> 

OK.  I will go with Tim's suggestion and just do the brain-dead 
PySequence_Fast/PySequence_Fast_GET_ITEM substitution solution and not 
bother with the PyObject_GetIter/PyIter_Next solution.

-Brett


From bac at OCF.Berkeley.EDU  Sat Sep  6 23:32:25 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Sep  7 01:33:22 2003
Subject: [Python-Dev] Changing select.select to accept iterables
In-Reply-To: <200309070349.h873nGr01747@12-236-84-31.client.attbi.com>
References: <LNBBLJKPBEHFEDALKOLCKEPEFLAB.tim.one@comcast.net>
	<3F5A97F7.7080700@ocf.berkeley.edu>
	<200309070349.h873nGr01747@12-236-84-31.client.attbi.com>
Message-ID: <3F5AC2E9.2010207@ocf.berkeley.edu>

Guido van Rossum wrote:

>>>[Brett, about <http://www.python.org/sf/798046>]
> 
> Since Tim thinks it's okay to change (and since I now know what the OP
> wanted) you have my blessing to give this a try.
> 

OK, done.  I tore out the PyList_Check in select_select() and just put 
the PySequence_Fast() call in list2set(), which I renamed seq2set(), so 
as to centralize the code and make managing refcounts simpler.  I pasted 
in an interpreter session at the end of this email showing some checking 
I did to make sure I didn't botch anything.  If someone could 
double-check them and see if they can think of something I may have 
missed I would appreciate it.

I made the docstring (as seen below) say that the arguments must be 
sequences since that is what PySequence_Fast says it takes in, although 
it obviously works with containers.  Any objections?

Assuming no one can think of anything that I missed I will then commit 
(with appropriate Misc/NEWS and doc changes), change the bug report to 
an feature request since that is what it really is, and close it.

-Brett



[passes test_select...]
 >>> import test.test_select
timeout = 0
'testing...\n'
timeout = 1
'testing...\n'
timeout = 2
'testing...\n'
timeout = 4
'testing...\n'
timeout = 8
'testing...\n'
timeout = 16
'testing...\n'
timeout = None
'testing...\n'
timeout = None
'testing...\n'
timeout = None
'testing...\n'
timeout = None
'testing...\n'
timeout = None
''
EOF
[24363 refs]

[works with sets as requested...]
 >>> from sets import Set
[26557 refs]
 >>> from select import select
[26559 refs]
 >>> read_FILE = file("README", 'r')
[26567 refs]
 >>> write_FILE = file("@test", 'w')
[26574 refs]
 >>> print select(Set([write_FILE]), Set([read_FILE]), [], 0)
([<open file '@test', mode 'w' at 0x5051e0>], [<open file 'README', mode 
'r' at 0x486550>], [])
[26607 refs]


[modified docstring to say it works with sequences...]
 >>> help(select)
Help on built-in function select:

select(...)
     select(rlist, wlist, xlist[, timeout]) -> (rlist, wlist, xlist)

     Wait until one or more file descriptors are ready for some kind of I/O.
     The first three arguments are sequences of file descriptors to be 
waited for:
...


[new error message on improper arguments...]
 >>> print select(1, 2, 3, 0)
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
TypeError: arguments 1-3 must be sequences
[39435 refs]

[managed to not screw up refcounts =) ...]
 >>> print select(Set([write_FILE]), Set([read_FILE]), [], 0)
([<open file '@test', mode 'w' at 0x5051e0>], [<open file 'README', mode 
'r' at 0x486550>], [])
[39435 refs]
 >>> print select(Set([write_FILE]), Set([read_FILE]), [], 0)
([<open file '@test', mode 'w' at 0x5051e0>], [<open file 'README', mode 
'r' at 0x486550>], [])
[39435 refs]


[... and of course still works the way it originally did.]
 >>> print select([write_FILE], [read_FILE], [], 0)
([<open file '@test', mode 'w' at 0x5051e0>], [<open file 'README', mode 
'r' at 0x486550>], [])
[39435 refs]


From skip at mojam.com  Sun Sep  7 08:00:34 2003
From: skip at mojam.com (Skip Montanaro)
Date: Sun Sep  7 08:00:43 2003
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200309071200.h87C0Yi17146@manatee.mojam.com>


Bug/Patch Summary
-----------------

467 open / 4093 total bugs (-1)
185 open / 2358 total patches (-2)

New Bugs
--------

absolute import patch breaks external users of test.regrtest (2003-08-31)
	http://python.org/sf/798274
Clarify trailing comma in func arg list (2003-09-01)
	http://python.org/sf/798652
windows sys.path contains nonexistant directory (2003-09-01)
	http://python.org/sf/798876
distutils ignored LDFLAGS in Makefile (2003-09-02)
	http://python.org/sf/799088
CPPFLAGS should not be aded to ldshard command (2003-09-02)
	http://python.org/sf/799104
documentation for sys.platform is unclear (2003-09-02)
	http://python.org/sf/799369
tk_focusNext() fails (2003-09-02)
	http://python.org/sf/799428
color highlighting not working on EDIT windows (2003-09-04)
	http://python.org/sf/800432
build fails (2003-09-04)
	http://python.org/sf/800710
test_dumbdbm failing (2003-09-04)
	http://python.org/sf/800824
Doc bug in PEP-0278 (2003-09-04)
	http://python.org/sf/800828
Making "|" directive from REs a bit clearer (2003-09-04)
	http://python.org/sf/800899
Bad RE in scanf example (2003-09-05)
	http://python.org/sf/801306
New slice behaviour.  Bug or feature? (2003-09-05)
	http://python.org/sf/801349

New Patches
-----------

More urllib2 examples (2003-08-31)
	http://python.org/sf/798244
imaplib : Add support for the THREAD command (2003-08-31)
	http://python.org/sf/798297
Improve "veryhigh.tex" API docs (2003-09-01)
	http://python.org/sf/798638
add HTTPResponse.getheaders() (2003-09-03)
	http://python.org/sf/800236
Add 'clear_history()' to 'readline' module (2003-09-04)
	http://python.org/sf/800697
zipimport 64-bit fix (2003-09-06)
	http://python.org/sf/801821
Adding rsplit() to string and unicode objects. (2003-09-06)
	http://python.org/sf/801847

Closed Bugs
-----------

Fatal Python Error during program shutdown (2001-02-05)
	http://python.org/sf/231207
filecmp.dircmp case sensitivity bug (2001-08-20)
	http://python.org/sf/453515
Double underscore needs clarification (2002-02-19)
	http://python.org/sf/520325
setting file buffer size is unreliable (2002-09-02)
	http://python.org/sf/603724
__file__ attribute missing from dynamicly loaded module (2003-03-05)
	http://python.org/sf/698282
WINDOW in py_curses.h needs ncurses-devel (2003-03-11)
	http://python.org/sf/701751
Documentation formatting bugs (2003-04-25)
	http://python.org/sf/727692
recent-files defaulted to strange place and bad permissions (2003-07-31)
	http://python.org/sf/780887
pynche does not start (2003-07-31)
	http://python.org/sf/780996
urllib output: Worker thread.. (2003-08-08)
	http://python.org/sf/785584
build fails on Tru64 Unix (osf1V5) (2003-08-13)
	http://python.org/sf/788183
Unsupported 'locale' setting causes Idle to crash on startup (2003-08-13)
	http://python.org/sf/788378
sgmllib parser problem (2003-08-23)
	http://python.org/sf/793753
pyconfig.h defines _POSIX_C_SOURCE, conflicting with feature (2003-08-23)
	http://python.org/sf/793764
cygwin builds do not embed (2003-08-24)
	http://python.org/sf/794140
email module param parsing bug (2003-08-24)
	http://python.org/sf/794466
locale.setlocale(locale.LC_ALL, "de") raises exception (2003-08-29)
	http://python.org/sf/797447

Closed Patches
--------------

Editing of __str__ and __repr__ docs (2003-04-25)
	http://python.org/sf/727789
curses has_key emulation fix (2003-06-23)
	http://python.org/sf/759208
Add a stream type and a merge type to itertoolsmodule.c (2003-07-19)
	http://python.org/sf/774414
os.path.exists should use os.access when possible (2003-08-10)
	http://python.org/sf/786237
explicitly provide a buffer in PyFile_SetBufSize() (2003-08-13)
	http://python.org/sf/788249
add SafeConfigParser to __all__ (2003-08-18)
	http://python.org/sf/790443
inconsistency with implementation(logging) (2003-08-19)
	http://python.org/sf/791153
timetuple() returns a struct_time (2003-08-20)
	http://python.org/sf/792338
__file__ attribute missing from dynamicly loaded module (2003-08-25)
	http://python.org/sf/794826

From hernan at orgmf.com.ar  Sun Sep  7 16:52:36 2003
From: hernan at orgmf.com.ar (Boriz Izaguirre)
Date: Sun Sep  7 09:54:50 2003
Subject: HTMLHelp for Py2.3.1 (Was: Re: [Python-Dev] Py2.3.1)
Message-ID: <HPEBIMANNCHHDMDBGCHHMEPDDKAA.hernan@orgmf.com.ar>

[Thomas]
>[Fred L. Drake]
>> IDLE currently looks for the index.html file in a few places (which
>> depend on platform); if it can't find it, it uses the documentation on
>> python.org.
>>
>> It should be too hard to change it to load the HTML Help viewer if it
>> finds the .chm file on Windows, and to still fall back to the HTML or
>> the online documentation if the .chm can't be found.
>
>Changing it is trivial, EditorWindow.help_url must point to Python23.chm
>(if it exists).  I can do this.

Beware that for .html you want webbrowser.open(url) and for .chm you
want os.startfile(url)
There used to be a patch in Idle-dev for this.
http://tinyurl.com/mj1s

>Even nicer would be context-sensitive keyword help, but it seems IDLE
>doesn't support it, right?

Standard .chm format already includes useful index data to do
context-sensitive search. A couple of years ago I made an Idle extension
to manage this. It wasn't difficult then, and I think it's even easier
now. The problem I found is that you pass the selected text (the one you
are looking for) to HTMLHelp system by calling a Win32 API.
You'll need win32all installed. That's the way PythonWin works, by the way.
I guess this makes it a no-no for standard Python, right?

Regards,
-Hernan



From goodger at python.org  Sun Sep  7 10:56:03 2003
From: goodger at python.org (David Goodger)
Date: Sun Sep  7 09:57:11 2003
Subject: [Python-Dev] buffer('abc') == 'abc' is False ?!
In-Reply-To: <m3d6ehk1k9.fsf@mira.informatik.hu-berlin.de>
References: <20030903065042.18889.qmail@web40107.mail.yahoo.com>
	<m3d6ehk1k9.fsf@mira.informatik.hu-berlin.de>
Message-ID: <3F5B38F3.8050302@python.org>

Martin v. L?wis wrote:
> PEP editors, can you please mark PEP 296 as Withdrawn?

Done.

-- David Goodger


From gward at python.net  Sun Sep  7 13:19:00 2003
From: gward at python.net (Greg Ward)
Date: Sun Sep  7 12:19:04 2003
Subject: [Python-Dev] Documenting branch policy
Message-ID: <20030907161900.GA3195@cthulhu.gerg.ca>

[cc'ing pydotorg since this involves a change to web content, but
followup to python-dev please]

Has anyone taken the time to sit down and document the development
process surrounding release and maintenance branches?  Speaking as an
infrequent contributor to Python, it's never been entirely clear to me
how to deal with branches.  Seems to me like adding a paragraph or two
to this page:

  http://www.python.org/dev/process.html

would be a good start.  Here's a first crack, based largely on
guesswork, extrapolation from my day job (which also uses maintenance
branches), and common sense:

"""
<h4>CVS Branch Policy</h4>

<p>The Python development process uses CVS branches for a couple
of purposes:
<ul>
  <li>maintenance branches, for minor Python releases (2.2.3, 2.3.1,
      etc.)
  <li>release branches, for the exclusive use of the release manager(s)
  <li>feature branches, for large, disruptive additions/changes that take
      a long time to stabilize
</ul>

<h5>Maintenance branches</h5>

<p>Every major Python version (2.1, 2.2, 2.3, etc.) is accompanied by a
<em>maintenance branch</em>.  Minor Python releases are always made from
a maintenance branch -- e.g. Python 2.2.3 was made on the
<code>release22-maint</code> branch, and Python 2.3.1 will be made on
<code>release23-maint</code>.  Maintenance branches are long-lived; they
remain open as long as minor releases based on the corresponding
major release are a possibility.  Maintenance branches are for bug fixes
<em>only</em>; maintaining backwards compatibility at every level is
imperative.  (In particular, all extension modules and bytecode for
Python 2.<em>x</em>.<em>y</em> must continue to work with Python
2.<em>x</em>.<em>y+1</em> -- binary compatibility in the C API and
bytecode must be maintained.)</p>

<p>Any Python developer may checkin bug fixes on a maintenance branch;
it is the release manager's responsibility to create the maintenance
branch after releasing a new major Python version.</p>

<h5>Release branches</h5>

<p>Every release up to and including the final release for a new major
Python version is accompanied by a <em>release branches</em>.  For
example, there were release branches for 2.3a1, 2.3a2, 2.3b1, 2.3b2,
2.3c1, and 2.3 itself; there will <em>not</em> be release branches for
2.3.1, 2.3.2, and so forth.  Release branches allow the release manager
to make small, last-minute changes without being affected by mainline
development.  It is the release manager's responsibility to merge
changes from each release branch back onto the trunk after the release
is finished.

<p>Unless you are one of the release managers for a new major Python
version, you should have nothing to do with release branches.</p>

<h5>Feature branches</h5>

<p>Occasionally, a new feature will be deemed large and disruptive
enough that it will be added on a branch.  For example, the "new-style
classes" feature of Python 2.2 was added on <code>descr-branch</code>
(the non-obvious name stems from <em>descriptors</em>, the new C-level
data structure underlying new-style classes), which took several months
to stabilize before it was merged onto the trunk.  Managing a feature
branch is tricky business, and the longer it takes to stabilize the new
feature, the trickier it gets.

<p>Feature branches are generally created and used by a single
developer.  Do not create a feature branch without discussing it on
python-dev.</p>
"""

Please let me know how close this is to describing reality!

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
Cheops' Law: Nothing *ever* gets built on schedule or within budget.

From martin at v.loewis.de  Sun Sep  7 20:16:07 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sun Sep  7 15:16:10 2003
Subject: [Python-Dev] Re: [Pydotorg] Documenting branch policy
In-Reply-To: <20030907161900.GA3195@cthulhu.gerg.ca>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
Message-ID: <m3y8x01kbc.fsf@mira.informatik.hu-berlin.de>

Greg Ward <gward@python.net> writes:

> Has anyone taken the time to sit down and document the development
> process surrounding release and maintenance branches? 

You mean PEP 6 and PEP 102?

> Maintenance branches are for bug fixes
> <em>only</em>;

This question is heavily debated. Alex Martelli, for example, favours
a policy where new (in a strict sense) features are acceptable if they
don't break anything, and are "minor", in some sense.

> maintaining backwards compatibility at every level is imperative.
> (In particular, all extension modules and bytecode for Python
> 2.<em>x</em>.<em>y</em> must continue to work with Python
> 2.<em>x</em>.<em>y+1</em> -- binary compatibility in the C API and
> bytecode must be maintained.)</p>

Even that requirement got dropped at one time, on grounds of the
specific API being irrelevant for all practical applications.

> <p>Any Python developer may checkin bug fixes on a maintenance branch;
> it is the release manager's responsibility to create the maintenance
> branch after releasing a new major Python version.</p>

The typical guidelines apply: If in doubt, post to SF.

Regards,
Martin

From gward at python.net  Sun Sep  7 16:30:05 2003
From: gward at python.net (Greg Ward)
Date: Sun Sep  7 15:30:09 2003
Subject: [Python-Dev] Re: [Pydotorg] Documenting branch policy
In-Reply-To: <m3y8x01kbc.fsf@mira.informatik.hu-berlin.de>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
	<m3y8x01kbc.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20030907193005.GA1106@cthulhu.gerg.ca>

On 07 September 2003, Martin v. L?wis said:
> Greg Ward <gward@python.net> writes:
> 
> > Has anyone taken the time to sit down and document the development
> > process surrounding release and maintenance branches? 
> 
> You mean PEP 6 and PEP 102?

Ahh, PEP 6 is what I was looking for.  Never mind my proposed change
then.  Thanks --

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
"Question authority!"  "Oh yeah?  Says who?"

From Jack.Jansen at cwi.nl  Sun Sep  7 23:25:08 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Sun Sep  7 16:25:14 2003
Subject: [Python-Dev] New functionality in micro releases (was: Documenting
	branch policy)
In-Reply-To: <m3y8x01kbc.fsf@mira.informatik.hu-berlin.de>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
	<m3y8x01kbc.fsf@mira.informatik.hu-berlin.de>
Message-ID: <5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>


On 7-sep-03, at 21:15, Martin v. L?wis wrote:
>> Maintenance branches are for bug fixes
>> <em>only</em>;
>
> This question is heavily debated. Alex Martelli, for example, favours
> a policy where new (in a strict sense) features are acceptable if they
> don't break anything, and are "minor", in some sense.

With Python 2.3 included in MacOSX 10.3 I would be heavily opposed to
this. I know, I've done it myself all the time in the past (with 
MacPython for
OS9), but with Python 2.3 we have the situation that the majority of
Python installations (I think it's safe to take the guess that MacOSX 
10.3
installations will soon outnumber all other Pythons together) will stay
at 2.3 until the next release of MacOSX.

This means that by adding even the slightest bit of functionality you
will make life difficult for developers: they themselves will probably
track Python releases, but their customers most likely won't. Note
that the MacPython infrastructure (bundlebuilder and such) really 
invites
people to distribute their applications to third parties. And judging
by the discussions on pythonmac-sig a lot of people are already doing 
this.

Even bug fixes already have an impact (because the developer won't see
a bug that will show up on the deployment machine), but adding 
functionality
would exacerbate the issue.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From just at letterror.com  Sun Sep  7 23:36:43 2003
From: just at letterror.com (Just van Rossum)
Date: Sun Sep  7 16:36:30 2003
Subject: [Python-Dev] New functionality in micro releases (was:
	Documenting branch policy)
In-Reply-To: <5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
Message-ID: <r01050400-1026-FFA1FD76E17211D79506003065D5E7E4@[10.0.0.23]>

Jack Jansen wrote:

> On 7-sep-03, at 21:15, Martin v. L?wis wrote:
> > This question is heavily debated. Alex Martelli, for example,
> > favours a policy where new (in a strict sense) features are
> > acceptable if they don't break anything, and are "minor", in some
> > sense.
> 
> With Python 2.3 included in MacOSX 10.3 I would be heavily opposed to
> this. I know, I've done it myself all the time in the past (with
> MacPython for OS9), but with Python 2.3 we have the situation that
> the majority of Python installations (I think it's safe to take the
> guess that MacOSX 10.3 installations will soon outnumber all other
> Pythons together) will stay at 2.3 until the next release of MacOSX.

It's been a huge pain with 2.2 vs. 2.2.1 even: all stuff written for
2.2.1 that uses booleans does _not_ work on plain 2.2, which is what's
installed on OSX 10.2. Which sortof kills the advantage of having a
Python installed with the OS to begin with. In retrospect, I would have
been strongly against adding booleans to 2.2.1 for this reason. Apart
from real bugs, I think every program that works on 2.x.y should work on
2.x.z, regardless of whether y > z or y < z.

Just

From guido at python.org  Sun Sep  7 16:23:38 2003
From: guido at python.org (Guido van Rossum)
Date: Sun Sep  7 18:24:09 2003
Subject: [Python-Dev] Re: [Pydotorg] Documenting branch policy
In-Reply-To: Your message of "07 Sep 2003 21:15:51 +0200."
	<m3y8x01kbc.fsf@mira.informatik.hu-berlin.de> 
References: <20030907161900.GA3195@cthulhu.gerg.ca>  
	<m3y8x01kbc.fsf@mira.informatik.hu-berlin.de> 
Message-ID: <200309072223.h87MNd311094@12-236-84-31.client.attbi.com>

> > Maintenance branches are for bug fixes
> > <em>only</em>;
> 
> This question is heavily debated. Alex Martelli, for example, favours
> a policy where new (in a strict sense) features are acceptable if they
> don't break anything, and are "minor", in some sense.

Which pretty much matches the policy I used informally before all this
was written down in PEPs.  (Admittedly, 1.5.2 went way beyond this,
and I don't wish to go back to those days.)

> > maintaining backwards compatibility at every level is imperative.
> > (In particular, all extension modules and bytecode for Python
> > 2.<em>x</em>.<em>y</em> must continue to work with Python
> > 2.<em>x</em>.<em>y+1</em> -- binary compatibility in the C API and
> > bytecode must be maintained.)</p>
> 
> Even that requirement got dropped at one time, on grounds of the
> specific API being irrelevant for all practical applications.

So the rule becomes all extension modules and bytecode "in actual use"
must continue to work.  BTW I don't think we've ever changed the
bytecode in a micro release after 1.5.2, and I want to continue to be
strict about that. 

> > <p>Any Python developer may checkin bug fixes on a maintenance branch;
> > it is the release manager's responsibility to create the maintenance
> > branch after releasing a new major Python version.</p>
> 
> The typical guidelines apply: If in doubt, post to SF.

Or discuss it here.  (I personally don't see new bugs on SF, but I do
read all of python-dev -- though this should come as no surprise, as
according to Brett's stats, I also write most of it. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From hpk at trillke.net  Mon Sep  8 01:50:00 2003
From: hpk at trillke.net (holger krekel)
Date: Sun Sep  7 18:50:11 2003
Subject: [Python-Dev] PyPy "Berlin" sprint (29th Sep - 4th Oct 2003)
Message-ID: <20030908005000.D22698@prim.han.de>

hello python developers, 

the fourth PyPy coding sprint will take place in Berlin (Germany) from

    (morning of) 29th of September to (evening of) 4th of October 2003. 

All Python developers are welcome to join the sprint. 
See below for participation details. 


What is PyPy?
-------------

PyPy is a remimplementation of Python in the Python language itself. 
Simplicity and flexibilty are the foremost goals.  With PyPy we want it
to be easy to extend the language or generate a more minimal language 
implementation.

One key technique to making this feasible is to specialize our abstract/general 
PyPy-Python implementation into native code (e.g. C-code).  Eventually the
concepts of PSYCO and Stackless are to be integrated into PyPy. 

There is, of course, much more to it so feel free to consult some
documents especially our Oscon-2003 paper on this page:

    http://codespeak.net/pypy/index.cgi?doc

We know that many areas and issues are still somewhat vague
although PyPy can already run many python programs [*]. Fortunately, Python 
is pretty tolerant about expressing ideas in vague ways.  After all, 
hacking at PyPy aims (and already proved!) to preserve the fun of 
hacking in Python. 


History, status and Berlin goals
--------------------------------

The project started in Jan/Feb 2003 with an initiative from Christian 
Tismer, Holger Krekel and Armin Rigo. During the course many more
developers joined our sprints and there are now between 5 and 15 people
involved in developing PyPy.  

The three sprints resulted in a fully working interpreter, some development 
infrastructure and the "standard object space" which is an abstraction for 
operations on standard (CPython) objects. 

There is no public release yet, you more or less have to install
a subversion-client and checkout the trunk.  Here is our 'howtosvn' 
which contains up-to-date clients for multiple platforms along
with some instructions how to get started:

    http://codespeak.net/pypy/index.cgi?doc/devel/howtosvn.html

Main goals for the Berlin sprint are 

- integration/enhancement of a Python parser and the compiler package 

- generating native code from our Python interpreter/types implementations

- enhancing/correcting/completing what we have (various tasks)


How to participate in Berlin (29th of Sept. to 4th of October)
--------------------------------------------------------------

If you are interested to participate please subscribe at our 
sprint organization list 

    http://codespeak.net/mailman/pypy-sprint

and please list yourself at 

    http://codespeak.net/moin/pypy/moin.cgi/SprintAttendants

so that we can organize accomodation and arrange details. Costs
should be pretty low (except from travelling costs, of course).
Currently nine people are scheduled to come.  

Btw, the sprint starts early on the 29th of september so it's best to
arrive on sunday, 28th. Also we probably want to have one day
off during the sprint week to do some sight (or cafe+pub) seeing. 
Actually it's much better to participate the whole week to better
get into it. 

cheers,

    holger 
    

[*] and currently only 20-40 thousand times slower than CPython! which clearly 
    shows that we are successfully follow the "premature optimization is the root 
    of all evil" maxime. 

From bac at OCF.Berkeley.EDU  Wed Sep  3 02:11:03 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Sep  7 21:44:07 2003
Subject: [Python-Dev] python-dev Summary for 2003-08-16 through 2003-08-31
	[draft]
Message-ID: <3F55A217.50403@ocf.berkeley.edu>

This is a rather light summary, so finding my usual grammatical errors 
shouldn't be too difficult.  With so many people on vacation I will 
probably wait on sending this summary out for a little while.

On a personal note, this is my 24th summary.  What this means is I have 
now written enough summaries to cover a year's worth.  If you care to 
see some stats on the list read the "Summary Announcements" section. 
And if anyone on the list wants the pickled data I have for this or just 
want to know a specific stat I can look it up for you and tell you.

--------------------------------

python-dev Summary for 2003-08-16 through 2003-08-31
++++++++++++++++++++++++++++++++++++++++++++++++++++
This is a summary of traffic on the `python-dev mailing list`_ from 
August 16, 2003 through August 31, 2003.  It is intended to inform the 
wider Python community of on-going developments on the list.  To comment 
on anything mentioned here, just post to python-list@python.org or 
`comp.lang.python`_ with a subject line mentioning what you are 
discussing. All python-dev members are interested in seeing ideas 
discussed by the community, so don't hesitate to take a stance on 
something.  And if all of this really interests you then get involved 
and join `python-dev`_!

This is the twenty-fourth summary written by Brett Cannon (a year's 
worth of summaries by yours truly now under his belt; does this mean I 
am certifiably insane?).

All summaries are archived at http://www.python.org/dev/summary/ .

Please note that this summary is written using reStructuredText_ which 
can be found at http://docutils.sf.net/rst.html .  Any unfamiliar 
punctuation is probably markup for reST_ (otherwise it is probably 
regular expression syntax or a typo =); you can safely ignore it, 
although I suggest learning reST; its simple and is accepted for `PEP 
markup`_ and gives some perks for the HTML output.  Also, because of the 
wonders of programs that like to reformat text, I cannot guarantee you 
will be able to run the text version of this summary through Docutils_ 
as-is unless it is from the original text file.

.. _PEP Markup: http://www.python.org/peps/pep-0012.html

The in-development version of the documentation for Python can be found 
at http://www.python.org/dev/doc/devel/ and should be used when looking 
up any documentation on something mentioned here.  Python PEPs (Python 
Enhancement Proposals) are located at http://www.python.org/peps/ .  To 
view files in the Python CVS online, go to 
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/ .  Reported bugs 
and suggested patches can be found at the SourceForge_ project page.

.. _python-dev: http://www.python.org/dev/
.. _SourceForge: http://sourceforge.net/tracker/?group_id=5470
.. _python-dev mailing list: 
http://mail.python.org/mailman/listinfo/python-dev
.. _comp.lang.python: http://groups.google.com/groups?q=comp.lang.python
.. _Docutils: http://docutils.sf.net/
.. _reST:
.. _reStructuredText: http://docutils.sf.net/rst.html

.. contents::

.. _last summary: 
http://www.python.org/dev/summary/2003-08-01_2003-08-15.html


=====================
Summary Announcements
=====================
This is the twenty-fourth summary written by me.  Why am I repeating 
this fact since it is mentioned above?  Well, it is significant because 
this means I have written enough summaries to cover a year's worth of 
email traffic on python-dev (had I not taken a summary off back in 
October this milestone would have been hit for the first half of August 
which represented a physical year since I started doing the summaries). 
  I have managed to see a lot happen on python-dev from a new release, 
the first PyCon, and a number of flame wars.  I personally have managed 
to learn a *huge* amount about not just Python the language but how to 
program in Python, C, and how to handle a large programming project.  It 
has truly been worth the experience.

But how much of an experience has it been?  Well, for a long time now I 
have been planning on writing some code to calculate how much email I 
have read, who wrote most of that email, and what threads made up the 
most.  With my move to San Luis Obispo finished I finally had a chance 
to write said code in an imperfect manner (names, for instance, are a 
pain because some people have their name set differently at different 
computers; "Barry Warsaw" compared to "Barry A. Warsaw"; threads are 
worse thanks to the changing of subject titles in the middle of a 
thread) so as to give me some approximate numbers.

I have read 9469 emails that have passed through the python-dev mailing 
list.  The top six emailers (out of approx. 433 unique emailers) have been:

* Brett Cannon (277 emails when you deal with me just using my last 
initial; 2.9% of all emails)
* Barry Warsaw (305 emails when you also count his middle initial; 3.2%)
* Skip Montanaro (481 emails; 5.1%)
* Martin v. L?wis (627 emails, when calculated looking for all names 
that had "Martin" and "wis" in them; 6.6%)
* Tim Peters (694 emails; 7.3%)
* Guido van Rossum (a whopping 1407 emails; 14.8%)

The average person posted 21.9 emails over the emails I covered.  But 
only about 24 people had more than a single percentage (more than 94 
emails) worth of emails accredited to them.  That means that about 5.5% 
of the unique posters on python-dev accounted for 66.8% of all email 
(and I have gotten to know a good amount of those 24.

As for threads (of which there were *very* approx. 1252 unique threads), 
the top five are:

* "type categories" (115 emails; 1.2% of all emails)
* "PEP239 (Rational Numbers) Reference Implementation and new issues" 
(123 emails; 1.3%)
* "PEP-317" (125 emails: 1.3%)
* "python/dist/src/Python import.c,2.210,2.211" (146 emails; 1.5%)
* "Extended Function syntax" (263 emails; 2.8%)

What does this tell you and me?  First, that I contribute to my own pain 
by more than I thought (I can't believe I emailed that many times!). 
Second, I must *really* like reading.  Third, I have an addiction to 
Python.  Fourth, PEPs really do get discussed.  And fifth, Python 
development is alive and well.


OK, enough statistics.  As for this summary, it turned out rather light 
thanks to a couple of things.  One is the shutdown of mail delivery by 
mail.python.org during the SoBig virus' peak.  This not only cut back on 
the number of emails, but also led to me deleting *a lot* of bogus email 
on my other emails accounts.  My blanket deleting may have caught 
legitimate emails so it is possible I accidently deleted some python-dev 
stuff, although if I did it was minimal.  Another contributing factor to 
the light summary is that a lot of regulars on python-dev were on 
vacation.  This looks like it will happen again for the first half of 
September so expect the next summary to be light as well.


=========
Summaries
=========
------------------------------------------------------------------------------
Python using Parrot; new code interpreter or strange evolutionary 
parternship?
------------------------------------------------------------------------------
Pirate_ has now reached version 0.01 alpha.  It lacks classes and 
imports but can run a decent amount of Python code.  At least there is 
now a proof-of-concept that Python running on top of the Parrot_ VM is 
possible.

.. _Pirate: http://pirate.tangentcode.com/
.. _Parrot: http://www.parrotcode.org/

Contributing threads:
   - `pirate 0.01 alpha! 
<http://mail.python.org/pipermail/python-dev/2003-August/037684.html>`__


------------------------
Python 2.3.1 on its way?
------------------------
Raymond Hettinger suggesting pushing for a quick release of Python 2.3.1 
so that the 2.3 branch could be established as a stable version. 
Several bugs and performance enhancements have been committed to the 2.3 
maintenance branch.  Anthony Baxter stepped forward as release czar with 
Raymond Hettinger saying he would help and Barry Warsaw volunteering his 
wisdom as a battle-hardened release czar.

This discussion also brought up the question of whether a .chm help file 
for the Windows distribution would be worth using instead of including 
the HTML distribution of the documentation as it stands now.  It was 
agreed that it was a good thing to have since it allowed for better 
searching.  Tim Peters also discovered the install went faster since it 
would not have to copy a ton of individual HTML files.

Python 2.3.1 has a "verbal" release date of the third week of September; 
there has not been a PEP to set the release schedule officially.

Help would be appreciated in dealing with bug and patch reports on 
SourceForge.  Even if all you do is add a comment saying "this patch 
looks fine" or "I can reproduce this bug" it can be a great help.

Contributing threads:
   - `Py2.3.1 
<http://mail.python.org/pipermail/python-dev/2003-August/037687.html>`__
   - `HTMLHelp for Py2.3.1 
<http://mail.python.org/pipermail/python-dev/2003-August/037866.html>`__
   - `Fixing Patches and Bugs for Py2.3.1 
<http://mail.python.org/pipermail/python-dev/2003-August/037840.html>`__


----------------------------------------
Making looping generators more efficient
----------------------------------------
Would you like to see deeply nested generators be more efficient in 
returning their values?  Clark Evans would and made such a request.  He 
essentially wanted to have nested generator calls propogate their values 
to the first non-generator call directly and thus bypass all of the 
generator maintenance code.  There was no direct reaction to this.

Shane Holloway followed with the idea of having special syntax for when 
you yield each value of an iterator.  The idea, once again, would be to 
speed this common case in the interpreter by skipping some bookkeeping 
overhead.  A few syntax versions were offered, but the idea was all the 
same: special-case ``for item in iterable: yield item`` to something 
like ``yield *iterable``.

Contributing threads:
   - `cooperative generators 
<http://mail.python.org/pipermail/python-dev/2003-August/037708.html>`__
   - `Graph exploration with generators 
<http://mail.python.org/pipermail/python-dev/2003-August/037738.html>`__


--------------------------------------------------
Use of the logging package in the standard library
--------------------------------------------------
Want to help out the development of Python?  Know how to use the logging 
package?  Then python-dev wants you!  There are several modules in the 
stdlib that have home-grown logging code that could (and probably 
should) be using the logging package instead to simplify life.  Read the 
email that started the contributing thread and see if you can't help out 
by converting the module over to using the logging package today!

Contributing threads:
   - `Unification of logging in Python's Standard Library 
<http://mail.python.org/pipermail/python-dev/2003-August/037743.html>`__


-----------------------
Some waxings on PEP 310
-----------------------
PEP 310 proposes the 'with' syntax that came up a while back that 
sparked an immense discussion on python-dev.  The idea was to have a 
more fool-proof way of having an enter and exit method be called before 
executing some specified code.  The common example was acquiring a lock, 
executing some code, and then releasing the lock all without having to 
deal with an explicit try/finally statement.  Samuele Pedroni tried to 
clarify how it should work exactly by requiring __exit__ instead of 
making it optional (read the PEP to understand what this means).

Contributing threads:
   - `PEP 310(with-syntax): close synonym of __exit__ 
<http://mail.python.org/pipermail/python-dev/2003-August/037795.html>`__


------------------------------------------------
Proposed PEP for a 'close' method for generators
------------------------------------------------
Samuele Pedroni has written a pre-PEP on defining a way to have 
generators grow a way to have a 'close' method that is called when their 
execution is finished so as to handle resources correctly.  This is in 
response to not being able to contain yield statements within 
try/finally blocks.

Contributing threads:
   - `pre-PEP: Resource-Release Support for Generators 
<http://mail.python.org/pipermail/python-dev/2003-August/037803.html>`__


-----------------
email-sig created
-----------------
Barry Warsaw has created the `email-sig`_ to steer development of 
version 3 of the email package in hopes of having it done for Python 2.4 .

.. _email-sig: http://www.python.org/sigs/email-sig/

Contributing threads:
   - `New SIG: email-sig <New SIG: email-sig>`__


From mfb at lotusland.dyndns.org  Sun Sep  7 23:13:02 2003
From: mfb at lotusland.dyndns.org (Matthew F. Barnes)
Date: Sun Sep  7 23:25:47 2003
Subject: [Python-Dev] Consistent logging in the standard library
Message-ID: <4101.192.168.1.1.1062990782.squirrel@server.lotusland.dyndns.org>

Having just read the "Use of the logging package in the standard library"
section of Brett's latest python-dev summary [1], I thought I might try to
generate some discussion on this topic since I brought it up in the first
place.

There was actually a bit of a longer discussion of this topic on
comp.lang.python [2], where A.M. Kuchling pointed out a number of good
issues that will need to be considered and suggested that I write a PEP on
the topic.  In fact I've been working on one since, but it's not yet ready
to be submitted (it's my first, so it's taking awhile).

The basic idea in the PEP is this: maintain a level of indirection between
the logger objects and the modules or class instances invoking them, so
that custom application-specific loggers can be substituted for the
"standard" ones.

The method for doing so is modelled after the socket module's new timeout
functionality [3], where each module that performs logging defines a pair
of module-level functions named getdefaultlogger() and setdefaultlogger()
that return and accept (respectively) a logging.Logger instance.  The
initial default logger for each of these modules is named after the module
itself (i.e. logging.getLogger(__name__)) and its behavior mimics that of
the Python 2.3 version of the module in terms of format and destination of
the logging messages.

So an application that wants to override the logging behavior of a
standard library module can do so simply by passing a customized logger
object to the module's setdefaultlogger() function.  This works out even
better for modules where all the logging is encapsulated in a class.

The PEP calls for each class that performs logging to define an attribute
called self.logger, and upon instantiation initialize self.logger to the
module's *current* default logger (again, modelled after how the timeout
value for a socket object is initialized).  This allows each instance of a
class to have its own unique logging behavior.

The first item for discussion is whether this seems like a reasonable
approach.  The second is identifying which standard library modules are
candidates for modification using this approach.  I've identified the
following by simply searching for the word "logging" in the source code,
but I'm not sure whether this list is complete:

        asyncore
        BaseHTTPServer
        cgi
        doctest
        imaplib
        unittest
        the distutils package

I have patches ready for asyncore and BaseHTTPServer and doctest, which
fit the model I described above quite nicely.  I determined that no
changes are necessary for the cgi module, since it doesn't even use the
logging functions it defines (initlog, dolog, etc.).  I'm currently
working on a way to replicate imaplib's command history using LogRecords,
which may call for a new type of logging.BufferingHandler subclass that
maintains a circular queue of LogRecords.  And the distutils package seems
prepped for using the logging package already.

The oddball is unittest.  Unittest is what motivated me to bring up the
logging issue in the first place, but I can't (or more precisely - am not
in a position to) decide what direction that module should take w.r.t. if
and how it should use the logging package.  Should the TextTestRunner
class be modified to use a logging.Logger instead of a _WritelnDecorator,
or should a separate TestRunner subclass (a LoggingTestRunner perhaps) be
defined instead?  I'm hoping that Steve Purcell or others may have some
thoughts on this.

I would also appreciate hearing any alternatives to the approach I've
suggested, as well as comments on the strengths and weaknesses of each.

Matthew Barnes


    [1]
<http://mail.python.org/pipermail/python-dev/2003-September/037938.html>
    [2]
<http://groups.google.com/groups?hl=en&lr=lang_en&ie=UTF-8&oe=UTF-8&threadm=3a8e83d2.0308182217.7ccaf883%40posting.google.com&rnum=1&prev=/groups%3Fq%3DUnification%2BPython%2BBarnes%26hl%3Den%26lr%3Dlang_en%26ie%3DUTF-8%26oe%3DUTF-8%26selm%3D3a8e83d2.0308182217.7ccaf883%2540posting.google.com%26rnum%3D1>
    [3] <http://www.python.org/doc/2.3/lib/module-socket.html>

From skip at pobox.com  Sun Sep  7 19:04:33 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Sep  8 09:07:05 2003
Subject: [Python-Dev] New functionality in micro releases (was: Documenting
	branch policy)
In-Reply-To: <5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
	<m3y8x01kbc.fsf@mira.informatik.hu-berlin.de>
	<5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
Message-ID: <16219.47489.553300.299720@montanaro.dyndns.org>

>>>>> "Jack" == Jack Jansen <Jack.Jansen@cwi.nl> writes:

    Jack> On 7-sep-03, at 21:15, Martin v. L?wis wrote:
    >>> Maintenance branches are for bug fixes
    >>> <em>only</em>;
    >> 
    >> This question is heavily debated. Alex Martelli, for example, favours
    >> a policy where new (in a strict sense) features are acceptable if they
    >> don't break anything, and are "minor", in some sense.

    Jack> ... but with Python 2.3 we have the situation that the majority of
    Jack> Python installations (I think it's safe to take the guess that
    Jack> MacOSX 10.3 installations will soon outnumber all other Pythons
    Jack> together) will stay at 2.3 until the next release of MacOSX.

I wouldn't be so sure of that.  Aside from the many Linux distributions
which include some version of Python, it's clear that HP/Compaq is including
some version of Python with their new Windows computers (based on all the
newbie questions about this fielded at webmaster@python.org).  Still, the
number of vendors delivering Python with their computers and the long string
of problems caused by RedHat shipping Python 1.5.2 w/ 7.x long after 2.0 was
history suggests that micro releases be reserved entirely for bug fixes.

Skip

From barry at python.org  Mon Sep  8 14:18:30 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Sep  8 09:18:32 2003
Subject: [Python-Dev] Documenting branch policy
In-Reply-To: <20030907161900.GA3195@cthulhu.gerg.ca>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
Message-ID: <1063027104.16344.9.camel@anthem>

 On Sun, 2003-09-07 at 12:19, Greg Ward wrote:

> <h5>Release branches</h5>
> 
> <p>Every release up to and including the final release for a new major
> Python version is accompanied by a <em>release branches</em>.

There's even some debate about these (I've caught up with the thread, so
I know you've withdrawn this change).  I've been backing off the use of
release branches because they create more complexity in a world where
none of us have much time to deal with it.  For 2.3, python-dev was
really good about heeding calls for checkin freezes, and the time
between wanting to cut the release and actually doing it was pretty
short, so I don't see much need for release branches.  I updated PEP 101
to reflect this.

-Barry



From harri.pasanen at trema.com  Mon Sep  8 16:39:56 2003
From: harri.pasanen at trema.com (Harri Pasanen)
Date: Mon Sep  8 09:40:07 2003
Subject: [Python-Dev] New functionality in micro releases (was:
	=?iso-8859-1?q?Documenting	branch?= policy)
In-Reply-To: <16219.47489.553300.299720@montanaro.dyndns.org>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
	<5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
	<16219.47489.553300.299720@montanaro.dyndns.org>
Message-ID: <200309081539.56700.harri.pasanen@trema.com>

On Monday 08 September 2003 01:04, Skip Montanaro wrote:
> >>>>> "Jack" == Jack Jansen <Jack.Jansen@cwi.nl> writes:
>
>     Jack> ... but with Python 2.3 we have the situation that the
> majority of Jack> Python installations (I think it's safe to take
> the guess that Jack> MacOSX 10.3 installations will soon outnumber
> all other Pythons Jack> together) will stay at 2.3 until the next
> release of MacOSX.
>
> I wouldn't be so sure of that.  Aside from the many Linux
> distributions which include some version of Python, it's clear that
> HP/Compaq is including some version of Python with their new
> Windows computers (based on all the newbie questions about this
> fielded at webmaster@python.org).  Still, the number of vendors
> delivering Python with their computers and the long string of
> problems caused by RedHat shipping Python 1.5.2 w/ 7.x long after
> 2.0 was history suggests that micro releases be reserved entirely
> for bug fixes.
>

Seconded, from where I'm sitting MaxOSX is not even on the radar,  but 
Linux is all over the place, on various platforms, and all those we 
use have Python out of the box.  RedHat on Itanium still seems to 
have 1.5.2 for instance.   Mandrake 9.1 is on 2.2.2.
Mandrake 9.2 will have Python 2.3.  

Personally I would have guessed there to be more Mandrake Linux 
installations than MacOSX installations - but in reality I have no 
idea.

Just a datapoint...

Harri

From guido at python.org  Mon Sep  8 07:59:02 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Sep  8 09:59:21 2003
Subject: [Python-Dev] New functionality in micro releases (was:
	Documenting branch policy)
In-Reply-To: Your message of "Sun, 07 Sep 2003 18:04:33 CDT."
	<16219.47489.553300.299720@montanaro.dyndns.org> 
References: <20030907161900.GA3195@cthulhu.gerg.ca>
	<m3y8x01kbc.fsf@mira.informatik.hu-berlin.de>
	<5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl> 
	<16219.47489.553300.299720@montanaro.dyndns.org> 
Message-ID: <200309081359.h88Dx2G11829@12-236-84-31.client.attbi.com>

>     Jack> ... but with Python 2.3 we have the situation that the majority of
>     Jack> Python installations (I think it's safe to take the guess that
>     Jack> MacOSX 10.3 installations will soon outnumber all other Pythons
>     Jack> together) will stay at 2.3 until the next release of MacOSX.

[Skip]
> I wouldn't be so sure of that.  Aside from the many Linux distributions
> which include some version of Python, it's clear that HP/Compaq is including
> some version of Python with their new Windows computers (based on all the
> newbie questions about this fielded at webmaster@python.org).  Still, the
> number of vendors delivering Python with their computers and the long string
> of problems caused by RedHat shipping Python 1.5.2 w/ 7.x long after 2.0 was
> history suggests that micro releases be reserved entirely for bug fixes.

I agree with Jack & Just that MacOSX is a strong argument for keeping
new features, no matter how harmless, out of micro releases.  But I'm
not sure how Red Hat shipping 1.5.2 proves this point; there were no
micro releases of 1.5 after that.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Mon Sep  8 10:09:55 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Sep  8 10:10:20 2003
Subject: [Python-Dev] New functionality in micro releases (was:
	Documenting branch policy)
In-Reply-To: <200309081359.h88Dx2G11829@12-236-84-31.client.attbi.com>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
	<m3y8x01kbc.fsf@mira.informatik.hu-berlin.de>
	<5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
	<16219.47489.553300.299720@montanaro.dyndns.org>
	<200309081359.h88Dx2G11829@12-236-84-31.client.attbi.com>
Message-ID: <16220.36275.971661.113208@montanaro.dyndns.org>


    Guido> But I'm not sure how Red Hat shipping 1.5.2 proves this point;
    Guido> there were no micro releases of 1.5 after that.

True, but it demonstrates how much agony changes to the language -
regardless of the versioning - can cause.

Skip


From guido at python.org  Mon Sep  8 08:19:48 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Sep  8 10:20:16 2003
Subject: [Python-Dev] New functionality in micro releases (was:
	Documenting branch policy)
In-Reply-To: Your message of "Mon, 08 Sep 2003 09:09:55 CDT."
	<16220.36275.971661.113208@montanaro.dyndns.org> 
References: <20030907161900.GA3195@cthulhu.gerg.ca>
	<m3y8x01kbc.fsf@mira.informatik.hu-berlin.de>
	<5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
	<16219.47489.553300.299720@montanaro.dyndns.org>
	<200309081359.h88Dx2G11829@12-236-84-31.client.attbi.com> 
	<16220.36275.971661.113208@montanaro.dyndns.org> 
Message-ID: <200309081419.h88EJmA11895@12-236-84-31.client.attbi.com>

>     Guido> But I'm not sure how Red Hat shipping 1.5.2 proves this point;
>     Guido> there were no micro releases of 1.5 after that.
> 
> True, but it demonstrates how much agony changes to the language -
> regardless of the versioning - can cause.
> 
> Skip

Then that's an argument against changes to the language in general.
We've had that argument, and the resulting policy is specified in PEPs
4 and 5 (amongst others).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Mon Sep  8 11:08:30 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Sep  8 11:08:35 2003
Subject: [Python-Dev] New functionality in micro releases (was:
	Documenting branch policy)
In-Reply-To: <200309081419.h88EJmA11895@12-236-84-31.client.attbi.com>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
	<m3y8x01kbc.fsf@mira.informatik.hu-berlin.de>
	<5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
	<16219.47489.553300.299720@montanaro.dyndns.org>
	<200309081359.h88Dx2G11829@12-236-84-31.client.attbi.com>
	<16220.36275.971661.113208@montanaro.dyndns.org>
	<200309081419.h88EJmA11895@12-236-84-31.client.attbi.com>
Message-ID: <16220.39790.575212.507229@montanaro.dyndns.org>


    >> True, but it demonstrates how much agony changes to the language -
    >> regardless of the versioning - can cause.

    Guido> Then that's an argument against changes to the language in
    Guido> general.  We've had that argument, and the resulting policy is
    Guido> specified in PEPs 4 and 5 (amongst others).

I wasn't trying to suggest the language shouldn't evolve, just that ...

Oh, never mind.  I'm sure you actually know what I meant.

S

From fdrake at acm.org  Mon Sep  8 14:27:52 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon Sep  8 13:28:02 2003
Subject: [Python-Dev] New functionality in micro releases (was:
	Documenting branch policy)
In-Reply-To: <r01050400-1026-FFA1FD76E17211D79506003065D5E7E4@[10.0.0.23]>
References: <5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
	<r01050400-1026-FFA1FD76E17211D79506003065D5E7E4@[10.0.0.23]>
Message-ID: <16220.48152.685701.574897@grendel.zope.com>


Just van Rossum writes:
 > It's been a huge pain with 2.2 vs. 2.2.1 even: all stuff written for
 > 2.2.1 that uses booleans does _not_ work on plain 2.2, which is what's

No code written for 2.2.1 should be using bool(), True, or False;
those were not documented in 2.2.1 *at all* as far as I can tell.  The
description of bool() was added to the docs in 2.2.3, and I'm not sure
why, though it is noted that it was added in 2.2.1.  I'd be happy to
rip it out of the documentation, or add a stronger compatibility
warning, for 2.2.4.

 > installed on OSX 10.2. Which sortof kills the advantage of having a
 > Python installed with the OS to begin with. In retrospect, I would have
 > been strongly against adding booleans to 2.2.1 for this reason. Apart
 > from real bugs, I think every program that works on 2.x.y should work on
 > 2.x.z, regardless of whether y > z or y < z.

Depending on just what you mean "every program that works", this could
completely prevent these bugfix releases.  We might not be able to
remove a core dump since it would allow code to run that was not run
before, thereby changing the behavior of the code.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From python at rcn.com  Mon Sep  8 19:29:10 2003
From: python at rcn.com (Raymond Hettinger)
Date: Mon Sep  8 18:33:50 2003
Subject: [Python-Dev] New functionality in micro releases (was:Documenting
	branch policy)
References: <5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl><r01050400-1026-FFA1FD76E17211D79506003065D5E7E4@[10.0.0.23]>
	<16220.48152.685701.574897@grendel.zope.com>
Message-ID: <007201c37658$a077a0e0$a511c797@oemcomputer>

[Just van Rossum]
>  > Apart
>  > from real bugs, I think every program that works on 2.x.y should work on
>  > 2.x.z, regardless of whether y > z or y < z.

[Fred Drake]
> Depending on just what you mean "every program that works", this could
> completely prevent these bugfix releases.  We might not be able to
> remove a core dump since it would allow code to run that was not run
> before, thereby changing the behavior of the code.

JvR did qualify it with:  "apart from real bugs".  The only question is how
"real" does it have to be.  I think all of the following should be fair game:

  * not matching documented behavior (for example: '%F" % x)
  * usability fixes
  * performance issues (strptime caching for example)
  * something that can only be crashed by an obscure case
     (Armin Rigo's itertools.izip hack)
  * leaks (the massive leak in the array module)
  * security fixes
  * enhancing the docs (chm files, examples, new docs, etc.)
  * adding capabilities to standalone Tools/scripts or IDLE
  * updates to separately maintained packages (like "email")
     so that between major releases, Python won't get terribly
     far behind the stable releases of the package.
  
I think of micro releases as being the same as service patches which are
supposed to be painless upgrades that any admin or user can load without
fear of breaking existing code. That should likely be the standard rather
than requiring that all programs work irrespective of the micro version.

If an application is affected by a bug in an earlier micro version, it
isn't terribly unreasonable to ask the users to download the service patch
which should be relatively effortless as compared with installing a new
major release.


Raymond Hettinger

From guido at python.org  Mon Sep  8 17:13:33 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Sep  8 19:13:52 2003
Subject: [Python-Dev] New functionality in micro releases (was:Documenting
	branch policy)
In-Reply-To: Your message of "Mon, 08 Sep 2003 18:29:10 EDT."
	<007201c37658$a077a0e0$a511c797@oemcomputer> 
References: <5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl><r01050400-1026-FFA1FD76E17211D79506003065D5E7E4@[10.0.0.23]>
	<16220.48152.685701.574897@grendel.zope.com> 
	<007201c37658$a077a0e0$a511c797@oemcomputer> 
Message-ID: <200309082313.h88NDX712738@12-236-84-31.client.attbi.com>

>   * not matching documented behavior (for example: '%F" % x)

Depends on how you fix these.  Fixing the code may break more code
than fixing the docs (assuming the mismatch existed for a long time)!

>   * usability fixes

What are these?  Seems like a loophole the size of a large truck.

>   * performance issues (strptime caching for example)
>   * something that can only be crashed by an obscure case
>      (Armin Rigo's itertools.izip hack)
>   * leaks (the massive leak in the array module)
>   * security fixes

But every attempt should be made to fix these without changing APIs.

>   * enhancing the docs (chm files, examples, new docs, etc.)
>   * adding capabilities to standalone Tools/scripts or IDLE

Questionable, if these are installed.  Esp. now that IDLE is part of
the standard library.

>   * updates to separately maintained packages (like "email")
>      so that between major releases, Python won't get terribly
>      far behind the stable releases of the package.

Again, this should be done with the utmost backwards compatibility in
mind, and even then, new features are new features, and thus may cause
the problem Jack and Just warn about.

> I think of micro releases as being the same as service patches which are
> supposed to be painless upgrades that any admin or user can load without
> fear of breaking existing code. That should likely be the standard rather
> than requiring that all programs work irrespective of the micro version.
> 
> If an application is affected by a bug in an earlier micro version, it
> isn't terribly unreasonable to ask the users to download the service patch
> which should be relatively effortless as compared with installing a new
> major release.

Right, but keep in mind that few people except hardcore Python users
are going to upgrade their Python, no matter how painless, just to run
some small piece of software they downloaded.  They'll just toss the
download as one of so many broken things, proof that free software
sucks.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Mon Sep  8 21:02:13 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Sep  8 20:02:24 2003
Subject: [Python-Dev] New functionality in micro releases
	(was:Documenting branch policy)
In-Reply-To: <200309082313.h88NDX712738@12-236-84-31.client.attbi.com>
References: <Your message of "Mon, 08 Sep 2003 18:29:10 EDT."
	<007201c37658$a077a0e0$a511c797@oemcomputer>
	<5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
	<r01050400-1026-FFA1FD76E17211D79506003065D5E7E4@[10.0.0.23]>
	<16220.48152.685701.574897@grendel.zope.com>
	<007201c37658$a077a0e0$a511c797@oemcomputer>
Message-ID: <5.1.1.6.0.20030908195222.01f1a860@telecommunity.com>

At 04:13 PM 9/8/03 -0700, Guido van Rossum wrote:
>Right, but keep in mind that few people except hardcore Python users
>are going to upgrade their Python, no matter how painless, just to run
>some small piece of software they downloaded.  They'll just toss the
>download as one of so many broken things, proof that free software
>sucks.

I've even seen *developers* do this...  Somebody who works in my department 
here griped about PEAK not working on their machine and when I went to 
troubleshoot it, I found they had a lesser version (2.2.1) of Python than 
the docs explicitly required (2.2.2, which fixed a couple of relevant 
new-style class bugs).  (Makes me think I should put some code in to check 
the Python version and exit with a helpful error message.)

It does suggest that being able to specify the version of Python required 
by a script or module, would be a helpful idiom.  sys.requireversion(), 
perhaps?  Guess I should make a quick check to be sure you haven't already 
used the time machine and put this in...  :)

Of course, the sad bit is that even if there were a sys.requireversion(), 
it wouldn't be in the versions where it was actually needed: the older ones!


From sholden at holdenweb.com  Mon Sep  8 22:06:39 2003
From: sholden at holdenweb.com (Steve Holden)
Date: Tue Sep  9 00:52:25 2003
Subject: [Python-Dev] New functionality in micro releases
	(was:Documentingbranch policy)
In-Reply-To: <200309082313.h88NDX712738@12-236-84-31.client.attbi.com>
Message-ID: <CGECIJPNNHIFAJKHOLMAGEIFGOAA.sholden@holdenweb.com>

[Raymond Hettinger]
> >   * not matching documented behavior (for example: '%F" % x)
>
[Guido]
> Depends on how you fix these.  Fixing the code may break more code
> than fixing the docs (assuming the mismatch existed for a long time)!
>
Such as the socket mismatch that was fixed in (IIRC) 1.6, to loud
complaints from half the network programming world? This still bites
occasionally when old cord has to be brought forward to more recent
versions (though, of course, it isn't hard to make the necessary
changes).

[...]
> >   * updates to separately maintained packages (like "email")
> >      so that between major releases, Python won't get terribly
> >      far behind the stable releases of the package.
>
> Again, this should be done with the utmost backwards compatibility in
> mind, and even then, new features are new features, and thus may cause
> the problem Jack and Just warn about.
>
Yup. Any potential for code breakage is going to cause somebody
somewhere a certain amount of pain. It's these edge cases that cause
most of the problems, since overall the version-to-version compatibility
picture is actually remarkably good. But people will cheerfully ignore
the 99.99% of their code that *doesn't* break, and bitch about the
little bits that do :-)

> > I think of micro releases as being the same as service
> patches which are
> > supposed to be painless upgrades that any admin or user can
> load without
> > fear of breaking existing code. That should likely be the
> standard rather
> > than requiring that all programs work irrespective of the
> micro version.
> >
> > If an application is affected by a bug in an earlier micro
> version, it
> > isn't terribly unreasonable to ask the users to download
> the service patch
> > which should be relatively effortless as compared with
> installing a new
> > major release.
>
> Right, but keep in mind that few people except hardcore Python users
> are going to upgrade their Python, no matter how painless, just to run
> some small piece of software they downloaded.  They'll just toss the
> download as one of so many broken things, proof that free software
> sucks.
>
If you want something to be true sufficiently hard, anything convenient
will be seen as supporting evidence.

Raymond did, however, pinpoint the reason why I haven't yet started
using True and False.

what-*me*-conservative-ly y'rs  - steve
--
Steve Holden                                 http://www.holdenweb.com/
Python Web Programming                http://pydish.holdenweb.com/pwp/
Interview with GvR August 14, 2003       http://www.onlamp.com/python/



From eppstein at ics.uci.edu  Mon Sep  8 18:48:50 2003
From: eppstein at ics.uci.edu (David Eppstein)
Date: Tue Sep  9 01:07:48 2003
Subject: [Python-Dev] Re: New functionality in micro releases
	(was:Documenting branch policy)
References: <007201c37658$a077a0e0$a511c797@oemcomputer>
	<5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
	<r01050400-1026-FFA1FD76E17211D79506003065D5E7E4@[10.0.0.23]>
	<16220.48152.685701.574897@grendel.zope.com>
	<007201c37658$a077a0e0$a511c797@oemcomputer>
	<200309082313.h88NDX712738@12-236-84-31.client.attbi.com>
	<5.1.1.6.0.20030908195222.01f1a860@telecommunity.com>
Message-ID: <eppstein-AE4E8F.17485008092003@sea.gmane.org>

In article <5.1.1.6.0.20030908195222.01f1a860@telecommunity.com>,
 "Phillip J. Eby" <pje@telecommunity.com> wrote:

> It does suggest that being able to specify the version of Python required 
> by a script or module, would be a helpful idiom.  sys.requireversion(), 
> perhaps?  Guess I should make a quick check to be sure you haven't already 
> used the time machine and put this in...  :)
> 
> Of course, the sad bit is that even if there were a sys.requireversion(), 
> it wouldn't be in the versions where it was actually needed: the older ones!

Ok, so what's needed isn't a new sys.feature, it's some short 
boilerplate code you could include in all your python apps, to make sure 
anyone who tries to run it on an old system is informed of the problem.

Something like:

import sys
class PythonTooOld(Exception): pass
if sys.version < '2.2':
    raise PythonTooOld, 'This code needs Python 2.2 or greater.'

-- 
David Eppstein                      http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science


From aleaxit at yahoo.com  Tue Sep  9 08:20:37 2003
From: aleaxit at yahoo.com (Alex Martelli)
Date: Tue Sep  9 01:20:43 2003
Subject: [Python-Dev] New functionality in micro releases (was:Documenting
	branch policy)
In-Reply-To: <5.1.1.6.0.20030908195222.01f1a860@telecommunity.com>
References: <Your message of "Mon, 08 Sep 2003 18:29:10 EDT."
	<007201c37658$a077a0e0$a511c797@oemcomputer>
	<007201c37658$a077a0e0$a511c797@oemcomputer>
	<5.1.1.6.0.20030908195222.01f1a860@telecommunity.com>
Message-ID: <200309090720.37252.aleaxit@yahoo.com>

On Tuesday 09 September 2003 02:02 am, Phillip J. Eby wrote:
   ...
> Of course, the sad bit is that even if there were a sys.requireversion(),
> it wouldn't be in the versions where it was actually needed: the older
> ones!

Actually this would "fail-RIGHT": on older versions the program would
die right there with an AttributeException and a traceback about a
line such as sys.requireversion('2.3.2'), which is already a clear
enough indication of exactly what is wrong -- on newer ones with an
even clearer exception, of course.  In any case, one could dress up
the sys.requireversion call in a try/except to make its absence no problem.


Alex


From fdrake at acm.org  Mon Sep  8 23:37:59 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue Sep  9 01:49:37 2003
Subject: [Python-Dev] New functionality in micro releases
	(was:Documenting branch policy)
In-Reply-To: <5.1.1.6.0.20030908195222.01f1a860@telecommunity.com>
References: <Your message of "Mon, 08 Sep 2003 18:29:10 EDT."
	<007201c37658$a077a0e0$a511c797@oemcomputer>
	<5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
	<r01050400-1026-FFA1FD76E17211D79506003065D5E7E4@[10.0.0.23]>
	<16220.48152.685701.574897@grendel.zope.com>
	<007201c37658$a077a0e0$a511c797@oemcomputer>
	<5.1.1.6.0.20030908195222.01f1a860@telecommunity.com>
Message-ID: <16221.15623.111594.33933@grendel.zope.com>


Phillip J. Eby writes:
 > It does suggest that being able to specify the version of Python required 
 > by a script or module, would be a helpful idiom.  sys.requireversion(), 
 > perhaps?  Guess I should make a quick check to be sure you haven't already 
 > used the time machine and put this in...  :)

I'm not aware of any such thing.

 > Of course, the sad bit is that even if there were a sys.requireversion(), 
 > it wouldn't be in the versions where it was actually needed: the older ones!

Comparison with sys.version_info would be trivial, and seems
sufficient:

if sys.version_info < (2,2,2):
  print "Get a newer version of Python!"
else:
  main()


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From mfb at lotusland.dyndns.org  Tue Sep  9 02:25:11 2003
From: mfb at lotusland.dyndns.org (Matthew F. Barnes)
Date: Tue Sep  9 02:38:11 2003
Subject: [Python-Dev] New functionality in micro 
	releases(was:Documenting branch policy)
In-Reply-To: <16221.15623.111594.33933@grendel.zope.com>
References: <Your message of "Mon, 08 Sep 2003 18:29:10 
	EDT."<007201c37658$a077a0e0$a511c797@oemcomputer><5FE933B2-E171-11D7-A
	B7D-000A27B19B96@cwi.nl><r01050400-1026-FFA1FD76E17211D79506003065D5E7
	E4@[10.0.0.23]><16220.48152.685701.574897@grendel.zope.com><007201c376
	58$a077a0e0$a511c797@oemcomputer><5.1.1.6.0.20030908195222.01f1a860@te
	lecommunity.com> <16221.15623.111594.33933@grendel.zope.com>
Message-ID: <4269.192.168.1.1.1063088711.squirrel@server.lotusland.dyndns.org>

Fred L. Drake, Jr. said:
> Comparison with sys.version_info would be trivial, and seems
> sufficient:
>
> if sys.version_info < (2,2,2):
>   print "Get a newer version of Python!"
> else:
>   main()

FWIW - I often use something close to this near the top of my own modules
(realizing that it gets skipped when the -O option is provided):

assert sys.version_info >= (2,2,2), 'requires Python 2.2.2 or better'

Matthew Barnes

From Jack.Jansen at cwi.nl  Tue Sep  9 12:26:54 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Tue Sep  9 05:27:01 2003
Subject: [Python-Dev] Documenting branch policy
In-Reply-To: <1063027104.16344.9.camel@anthem>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
	<1063027104.16344.9.camel@anthem>
Message-ID: <C03EDAB0-E2A7-11D7-8D13-000A27B19B96@cwi.nl>


On 8-sep-03, at 15:18, Barry Warsaw wrote:

>> <p>Every release up to and including the final release for a new major
>> Python version is accompanied by a <em>release branches</em>.
>
> There's even some debate about these (I've caught up with the thread, 
> so
> I know you've withdrawn this change).  I've been backing off the use of
> release branches because they create more complexity in a world where
> none of us have much time to deal with it.

They served me well for the MacPython-OS9 releases. But as 2.3 is going 
to
be the last of those anyway I could live without them, I guess.

Although: they'd still be useful in case of unforeseen problems with a
distribution that are not code-related. Think of things like the Windows
installer turning out to be broken on certain machine, so you want to 
build
the installer again (but not Python itself).
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From barry at python.org  Tue Sep  9 13:09:36 2003
From: barry at python.org (Barry Warsaw)
Date: Tue Sep  9 08:09:37 2003
Subject: [Python-Dev] New functionality in micro releases
	(was:Documenting branch policy)
In-Reply-To: <007201c37658$a077a0e0$a511c797@oemcomputer>
References: <5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
	<r01050400-1026-FFA1FD76E17211D79506003065D5E7E4@[10.0.0.23]>
	<16220.48152.685701.574897@grendel.zope.com>
	<007201c37658$a077a0e0$a511c797@oemcomputer>
Message-ID: <1063109371.16781.21.camel@anthem>

On Mon, 2003-09-08 at 18:29, Raymond Hettinger wrote:

>   * updates to separately maintained packages (like "email")
>      so that between major releases, Python won't get terribly
>      far behind the stable releases of the package.

You have to take things on a case-by-case basis.  I actually agonized
quite a bit about the email package in 2.2, but after discussing things
with Guido and the mimelib community, there was general consensus that
not fixing/upgrading the package was worse than doing so.  But I doubt
we'll do the same thing for Python 2.3, because the package is more
stable now, has been integrated longer, and the planned changes are
probably more radical.

This does point out a problem with the batteries-included philosophy
though: it's hard to upgrade packages that come with Python.  Say we
release email 3.0 and you want to use it in Python 2.3, what do you do? 
You can't just do a distutils install, because site-packages is searched
after the standard library.  I think we need more flexibility in
installing distutils packages, so we can install in various locations
that override standard packages (e.g. user-centric, system-centric,
etc.).

In general though, it's a good thing to default to a policy where micro
releases are bug fixes only.  Given the reality of the resources
available, time between major releases, and the fact that Python
development is a wholly volunteer effort, some amount of ugliness is
bound to invade the process.  That's where the BDFL uses his judgement
to make a decision and we live with it.

Take the True/False thing.  OOH, it was a new feature added to a micro
release.  OTOH it was backward compatible, helped future compatibility,
and was easily coded around to avoid micro-version skew.  Taken as a
whole, and given when the decision was made, it was probably a good one
although it caused a little bit of pain.

-Barry



From barry at python.org  Tue Sep  9 13:14:11 2003
From: barry at python.org (Barry Warsaw)
Date: Tue Sep  9 08:14:12 2003
Subject: [Python-Dev] Documenting branch policy
In-Reply-To: <C03EDAB0-E2A7-11D7-8D13-000A27B19B96@cwi.nl>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
	<1063027104.16344.9.camel@anthem>
	<C03EDAB0-E2A7-11D7-8D13-000A27B19B96@cwi.nl>
Message-ID: <1063109647.16781.24.camel@anthem>

On Tue, 2003-09-09 at 05:26, Jack Jansen wrote:
> On 8-sep-03, at 15:18, Barry Warsaw wrote:
> 
> >> <p>Every release up to and including the final release for a new major
> >> Python version is accompanied by a <em>release branches</em>.
> >
> > There's even some debate about these (I've caught up with the thread, 
> > so
> > I know you've withdrawn this change).  I've been backing off the use of
> > release branches because they create more complexity in a world where
> > none of us have much time to deal with it.
> 
> They served me well for the MacPython-OS9 releases. But as 2.3 is going 
> to
> be the last of those anyway I could live without them, I guess.
> 
> Although: they'd still be useful in case of unforeseen problems with a
> distribution that are not code-related. Think of things like the Windows
> installer turning out to be broken on certain machine, so you want to 
> build
> the installer again (but not Python itself).

That's a good point.  Release branches are probably overkill for
alpha/beta/rc releases, but probably make sense for final releases.  We
actually did create one for 2.3, but I screwed up when I named it and
that caused a tiny bit of pain in moving to the maintenance branch. 
OTOH, for a final release, maybe release branch == maintenance branch.

-Barry



From fdrake at acm.org  Tue Sep  9 09:16:47 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue Sep  9 08:16:53 2003
Subject: [Pydotorg] Re: [Python-Dev] Documenting branch policy
In-Reply-To: <1063109647.16781.24.camel@anthem>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
	<1063027104.16344.9.camel@anthem>
	<C03EDAB0-E2A7-11D7-8D13-000A27B19B96@cwi.nl>
	<1063109647.16781.24.camel@anthem>
Message-ID: <16221.50351.725375.52522@grendel.zope.com>


Barry Warsaw writes:
 > That's a good point.  Release branches are probably overkill for
 > alpha/beta/rc releases, but probably make sense for final releases.  We
 > actually did create one for 2.3, but I screwed up when I named it and
 > that caused a tiny bit of pain in moving to the maintenance branch. 
 > OTOH, for a final release, maybe release branch == maintenance branch.

I think that's sufficient.  There's no need for them to be separate.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From gward at python.net  Tue Sep  9 09:24:47 2003
From: gward at python.net (Greg Ward)
Date: Tue Sep  9 08:24:53 2003
Subject: [Python-Dev] Documenting branch policy
In-Reply-To: <1063109647.16781.24.camel@anthem>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
	<1063027104.16344.9.camel@anthem>
	<C03EDAB0-E2A7-11D7-8D13-000A27B19B96@cwi.nl>
	<1063109647.16781.24.camel@anthem>
Message-ID: <20030909122447.GB15042@cthulhu.gerg.ca>

On 09 September 2003, Barry Warsaw said:
> That's a good point.  Release branches are probably overkill for
> alpha/beta/rc releases, but probably make sense for final releases.  We
> actually did create one for 2.3, but I screwed up when I named it and
> that caused a tiny bit of pain in moving to the maintenance branch. 
> OTOH, for a final release, maybe release branch == maintenance branch.

Well, if nothing else this thread has demonstrated that I'm *not* the only
one who's a little confused by the release/maintenance branching policy.
*phew*!

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
Just because you're paranoid doesn't mean they *aren't* out to get you.

From anthony at interlink.com.au  Wed Sep 10 02:18:30 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Sep  9 11:18:52 2003
Subject: [Python-Dev] New functionality in micro releases (was:Documenting
	branch policy) 
In-Reply-To: <1063109371.16781.21.camel@anthem> 
Message-ID: <200309091518.h89FIURh024097@localhost.localdomain>

>>> Barry Warsaw wrote
> This does point out a problem with the batteries-included philosophy
> though: it's hard to upgrade packages that come with Python.  Say we
> release email 3.0 and you want to use it in Python 2.3, what do you do? 
> You can't just do a distutils install, because site-packages is searched
> after the standard library.  I think we need more flexibility in
> installing distutils packages, so we can install in various locations
> that override standard packages (e.g. user-centric, system-centric,
> etc.).

The approach taken by the XML add-on package doesn't seem to be the 
right approach, either. At the moment your choices are to manually install
the new code over the old, or else have a separate install area that's
manually added to the start of the PYTHONPATH (this is what mailman
does, right?) Neither of these approaches seem ideal. The latter is what
I do for various systems that need newer versions of packages that are
bundled with python, it'd be nice if there was a better way - but I can't
see one.

Anthony

From tim.one at comcast.net  Tue Sep  9 12:25:50 2003
From: tim.one at comcast.net (Tim Peters)
Date: Tue Sep  9 11:25:48 2003
Subject: [Python-Dev] New functionality in micro releases
	(was:Documentingbranch policy) 
In-Reply-To: <200309091518.h89FIURh024097@localhost.localdomain>
Message-ID: <BIEJKCLHCIOIHAGOKOLHOEPGHDAA.tim.one@comcast.net>

[Barry Warsaw]
> ...
> This does point out a problem with the batteries-included philosophy

I was still basically asleep when I saw this msg, and my bleary eyes first
read that as "barriers-included".

> though: it's hard to upgrade packages that come with Python.

Indeed.  Batteries, barriers, same thing <wink>.


From barry at python.org  Tue Sep  9 16:50:15 2003
From: barry at python.org (Barry Warsaw)
Date: Tue Sep  9 11:50:16 2003
Subject: [Python-Dev] New functionality in micro releases
	(was:Documenting branch policy)
In-Reply-To: <200309091518.h89FIURh024097@localhost.localdomain>
References: <200309091518.h89FIURh024097@localhost.localdomain>
Message-ID: <1063122609.16781.48.camel@anthem>

On Tue, 2003-09-09 at 11:18, Anthony Baxter wrote:

> The approach taken by the XML add-on package doesn't seem to be the 
> right approach, either. At the moment your choices are to manually install
> the new code over the old, or else have a separate install area that's
> manually added to the start of the PYTHONPATH (this is what mailman
> does, right?)

Right, and it sucks because there are multiple top-level entry points
into the system (e.g. command line and cron scripts) and each has to
mangle the path very early on.  Then there's the pain of not being able
to run for the source directory because the configure/install procedure
sets up the path hacking module with the proper paths.  Blech.

-Barry



From guido at python.org  Tue Sep  9 10:17:15 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Sep  9 12:17:37 2003
Subject: [Python-Dev] New functionality in micro releases
	(was:Documentingbranch policy)
In-Reply-To: Your message of "Mon, 08 Sep 2003 21:06:39 EDT."
	<CGECIJPNNHIFAJKHOLMAGEIFGOAA.sholden@holdenweb.com> 
References: <CGECIJPNNHIFAJKHOLMAGEIFGOAA.sholden@holdenweb.com> 
Message-ID: <200309091617.h89GHFb13948@12-236-84-31.client.attbi.com>

> [Guido]
> > Depends on how you fix these.  Fixing the code may break more code
> > than fixing the docs (assuming the mismatch existed for a long time)!

[Steve]
> Such as the socket mismatch that was fixed in (IIRC) 1.6, to loud
> complaints from half the network programming world? This still bites
> occasionally when old cord has to be brought forward to more recent
> versions (though, of course, it isn't hard to make the necessary
> changes).

That was not a micro release.  I'm glad we fixed that one then; now it
would be a much bigger disaster.  (And it was not a matter of
adjusting to the docs, but adjusting to the fact that not all socets
deal with host/port pairs.)

> [...]

> Yup. Any potential for code breakage is going to cause somebody
> somewhere a certain amount of pain. It's these edge cases that cause
> most of the problems, since overall the version-to-version compatibility
> picture is actually remarkably good. But people will cheerfully ignore
> the 99.99% of their code that *doesn't* break, and bitch about the
> little bits that do :-)

For people with a large code base and real customers, the only working
approach is long, careful testing with each new Python release.  Hence
the call for Python-in-a-tie.  (Though I haven't heard much about that
recently -- I've even heard a call to upgrade the tie to 2.3.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pinki_mallick at mentorg.com  Tue Sep  9 12:25:18 2003
From: pinki_mallick at mentorg.com (Mallick, Pinki)
Date: Tue Sep  9 14:25:24 2003
Subject: [Python-Dev] memory related issues in Python - bug??
Message-ID: <F7DECA8E801AD51195F100508BB89DA607CC8FFC@svr-orw-exc-04.wv.mentorg.com>

Hi,

I have a very big XML file that I am trying to parse using xml.dom and xml.dom.minidom. This file has around 100 nodes. Each of these nodes(say 'child_X') have a number of child nodes and each of these child nodes in turn has a text node as its child. So when I try to retrieve all the data from each 'child_X' node, I use this code, 

grandChildren = child_X.childNodes()
for grandChildNode in grandChildren:
	equation = equation + str(grandChildNode.firstChild.data)


This works properly for all files. But when I am trying to use it on a very large XML file, one of the node's data is incomplete. eg. if grandChildNode.toxml() returns "<mi>y18</mi>" for this node, then "str(grandChildNode.firstChild.data)" returns only "y" instead of "y18".

This looks like some memory problem. Is there any workaround for this problem.

Thanks for any suggestion or help,
Pinki Mallick

From fdrake at acm.org  Tue Sep  9 15:31:41 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue Sep  9 14:31:50 2003
Subject: [Python-Dev] memory related issues in Python - bug??
In-Reply-To: <F7DECA8E801AD51195F100508BB89DA607CC8FFC@svr-orw-exc-04.wv.mentorg.com>
References: <F7DECA8E801AD51195F100508BB89DA607CC8FFC@svr-orw-exc-04.wv.mentorg.com>
Message-ID: <16222.7309.203774.926825@grendel.zope.com>


Mallick, Pinki writes:
 > I have a very big XML file that I am trying to parse using xml.dom
 > and xml.dom.minidom. This file has around 100 nodes. Each of these
...

This message was also posted to the XML-SIG (where it belongs) and the
webmaster address for python.org (where it doesn't).  Please don't
spam several lists with a question like this.

I've responded to the question in the appropriate forum.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From aahz at pythoncraft.com  Tue Sep  9 16:16:31 2003
From: aahz at pythoncraft.com (Aahz)
Date: Tue Sep  9 15:16:35 2003
Subject: [Python-Dev] memory related issues in Python - bug??
In-Reply-To: <F7DECA8E801AD51195F100508BB89DA607CC8FFC@svr-orw-exc-04.wv.mentorg.com>
References: <F7DECA8E801AD51195F100508BB89DA607CC8FFC@svr-orw-exc-04.wv.mentorg.com>
Message-ID: <20030909191631.GA1022@panix.com>

On Tue, Sep 09, 2003, Mallick, Pinki wrote:
> 
> I have a very big XML file that I am trying to parse using xml.dom
> and xml.dom.minidom. 

You've sent this question to two inappropriate places:
webmaster@python.org and python-dev@python.org

Please use the Usenet group comp.lang.python (or subscribe to
python-list) and post your question there.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

From martin at v.loewis.de  Tue Sep  9 21:30:51 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Sep  9 16:30:52 2003
Subject: [Python-Dev] New functionality in micro releases (was:Documenting
	branch policy)
In-Reply-To: <200309091518.h89FIURh024097@localhost.localdomain>
References: <200309091518.h89FIURh024097@localhost.localdomain>
Message-ID: <m3k78h7lid.fsf@mira.informatik.hu-berlin.de>

Anthony Baxter <anthony@interlink.com.au> writes:

> The approach taken by the XML add-on package doesn't seem to be the 
> right approach, either. At the moment your choices are to manually install
> the new code over the old, or else have a separate install area that's
> manually added to the start of the PYTHONPATH (this is what mailman
> does, right?)

Are you referring to PyXML here? It allows to extend/replace the
existing library with neither copying the new code over the old nor
with adding anything to PYTHONPATH. Instead, the core library expects
to be replaced, and is actively looking for a replacement.

Regards,
Martin

From guido at python.org  Tue Sep  9 22:32:51 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Sep 10 00:33:13 2003
Subject: [Python-Dev] Quick: want to give a Python talk at Linuxworld NY?
Message-ID: <200309100432.h8A4Wps15627@12-236-84-31.client.attbi.com>

I was just approached by Chris DiBona (chris at damagestudios dot
com) about talking at the next Linuxworld in New York, in January.
There are plenty of PHP and Perl talks but no Python talk yet.  Time
is of the essence, they are finalizing the program *now*.  Reward is a
free pass to everything.  Spread the word!

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Sep 10 00:15:37 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Sep 10 02:16:00 2003
Subject: [Python-Dev] Consistent logging in the standard library
In-Reply-To: Your message of "Sun, 07 Sep 2003 22:13:02 CDT."
	<4101.192.168.1.1.1062990782.squirrel@server.lotusland.dyndns.org> 
References: <4101.192.168.1.1.1062990782.squirrel@server.lotusland.dyndns.org>
Message-ID: <200309100615.h8A6FbM15765@12-236-84-31.client.attbi.com>

> The method for doing so is modelled after the socket module's new timeout
> functionality [3], where each module that performs logging defines a pair
> of module-level functions named getdefaultlogger() and setdefaultlogger()
> that return and accept (respectively) a logging.Logger instance.  The
> initial default logger for each of these modules is named after the module
> itself (i.e. logging.getLogger(__name__)) and its behavior mimics that of
> the Python 2.3 version of the module in terms of format and destination of
> the logging messages.

Hm...  Isn't the logging module's ability to set separate *handlers*
sufficient to get the desired flexibility?  Rather than the
application specifying a different *logger* object, the application
can specify a different *handler* object.  I think this was the
originally intended design.  It places less of a burden on the modules
that use logging (no need to define getdefaultlogger() and
setdefaultlogger()) and makes it possible for the different handlers
to be specified by the logging module's configuration file, without
the application having to support this.

Then what remains is a convention for the name of the logger used by
standard library modules; perhaps the module name should suffice?

> So an application that wants to override the logging behavior of a
> standard library module can do so simply by passing a customized logger
> object to the module's setdefaultlogger() function.  This works out even
> better for modules where all the logging is encapsulated in a class.
> 
> The PEP calls for each class that performs logging to define an attribute
> called self.logger, and upon instantiation initialize self.logger to the
> module's *current* default logger (again, modelled after how the timeout
> value for a socket object is initialized).  This allows each instance of a
> class to have its own unique logging behavior.

What behavior exactly are you thinking of?  A logger object has almost
no semantics apart from passing log events on to handlers; by design,
all the work is done in the handlers.

> The first item for discussion is whether this seems like a reasonable
> approach.  The second is identifying which standard library modules are
> candidates for modification using this approach.  I've identified the
> following by simply searching for the word "logging" in the source code,
> but I'm not sure whether this list is complete:
> 
>         asyncore
>         BaseHTTPServer
>         cgi
>         doctest
>         imaplib
>         unittest
>         the distutils package
> 
> I have patches ready for asyncore and BaseHTTPServer and doctest, which
> fit the model I described above quite nicely.

I think it would be useful to let these use the logging module even
without the proposed setdefaultlogger() etc. architecture.

> I determined that no changes are necessary for the cgi module, since
> it doesn't even use the logging functions it defines (initlog,
> dolog, etc.).

They're intended for use by CGI applications.  They should probably be
rewritten to direct the calls to the logging module.

> I'm currently working on a way to replicate imaplib's command
> history using LogRecords, which may call for a new type of
> logging.BufferingHandler subclass that maintains a circular queue of
> LogRecords.

That might be a useful feature on its own for the logging package.
Note though, that at a grander scale, the RotatingFileLogger class
already supports a way to keep only the last N messages (for much
larger N, though :-).

> And the distutils package seems prepped for using the logging
> package already.
> 
> The oddball is unittest.  Unittest is what motivated me to bring up the
> logging issue in the first place, but I can't (or more precisely - am not
> in a position to) decide what direction that module should take w.r.t. if
> and how it should use the logging package.  Should the TextTestRunner
> class be modified to use a logging.Logger instead of a _WritelnDecorator,
> or should a separate TestRunner subclass (a LoggingTestRunner perhaps) be
> defined instead?  I'm hoping that Steve Purcell or others may have some
> thoughts on this.

To reach Steve, you should probably post to the PyUnit list (sorry,
you'll have to Google for details, I only vaguely recall that there is
such a list).

Personally, I think that unittest is such a special case that it might
be best to leave it alone...

> I would also appreciate hearing any alternatives to the approach I've
> suggested, as well as comments on the strengths and weaknesses of each.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From vilmanis at internode.on.net  Wed Sep 10 23:09:29 2003
From: vilmanis at internode.on.net (Yuri Vilmanis)
Date: Wed Sep 10 08:40:37 2003
Subject: [Python-Dev] Base-n integer formatting
Message-ID: <200309102209.29603.vilmanis@internode.on.net>

I have three feature requests for you: no. 1 is quite fundamental, no. 2 would 
be a 'nice' useful feature and no. 3 is a bit more esoteric, but very useful 
in certain applications

1) print formatting for integers is currently only available for octal, hex, 
and decimal. This seems rather odd, as string to integer conversion is 
available for bases 2 through 36. End result: when I want to do things in 
base 6 (rare, but it has happened twice) or much more commonly in binary, I 
can read the numbers in, but cannot print the results - ie quite useless. 

2) Input and output formatting for unary (which is commonly referred to as 
base 1, yes, I know it's not 'really' base 1 by the general deinition of 
base, but "base 1" is the convention) would also be good, as would roman 
numeral support, especially in word-processing applications or IO from any 
formatted document with roman page/section numbers

3) negative bases: I have needed this more times than I think some people 
would care to imagine. These representations have several very useful 
properties such as implementation-independent signdness (hence arbirtrary 
word length), and some more interesting properties I won't expand on here.

Context: I find python very handy not only as a programming/scripting language 
but also as a scratchpad for various number theory investigations. 
Unfortunately the output formatting leaves all but the most basic 
investigations out in the cold, and lack of negative base input/output makes 
it harder to use python to look for certain patterns in result sets, not to 
mention getting the data in...

Thanks,
	Yuri


From mchermside at ingdirect.com  Wed Sep 10 10:30:08 2003
From: mchermside at ingdirect.com (Chermside, Michael)
Date: Wed Sep 10 09:30:19 2003
Subject: [Python-Dev] Base-n integer formatting
Message-ID: <7F171EB5E155544CAC4035F0182093F042126F@INGDEXCHSANC1.ingdirect.com>

Yuri:

I think you have some interesting ideas, but this is really not the
best place to present them. The newsgroup comp.lang.python, also a
mailing list at python-list@python.org, is really the best place to
discuss these ideas. Python-dev is intended for discussion about
the development of Python. New ideas or language features are best
introduced first on c.l.py, then based on the feedback there, written
up as a PEP, *THEN* folks will start discussing it on python-dev as
they figure out HOW (and perhaps whether) to implement it. There's
an exception -- regular Python developers sometimes introduce ideas
on this list, but that's a very short list of people, and it's a 
bit different, because if they bring it up it's because they're
already volunteering to implement it (and probably have a prototype
up and running already).

If you'll contact me _separately_, I'll be glad to give you my
thoughts on your suggestions.

-- Michael Chermside


-----Original Message-----
From: Yuri Vilmanis [mailto:vilmanis@internode.on.net]
Sent: Wednesday, September 10, 2003 8:39 AM
To: python-dev@python.org
Subject: [Python-Dev] Base-n integer formatting


I have three feature requests for you: no. 1 is quite fundamental, no. 2 would 
be a 'nice' useful feature and no. 3 is a bit more esoteric, but very useful 
in certain applications

1) print formatting for integers is currently only available for octal, hex, 
and decimal. This seems rather odd, as string to integer conversion is 
available for bases 2 through 36. End result: when I want to do things in base 6 (rare, but it has happened twice) or much more commonly in binary, I 
can read the numbers in, but cannot print the results - ie quite useless. 
2) Input and output formatting for unary (which is commonly referred to as base 1, yes, I know it's not 'really' base 1 by the general deinition of base, but "base 1" is the convention) would also be good, as would roman numeral support, especially in word-processing applications or IO from any formatted document with roman page/section numbers

3) negative bases: I have needed this more times than I think some people would care to imagine. These representations have several very useful 
properties such as implementation-independent signdness (hence arbirtrary word length), and some more interesting properties I won't expand on here.

Context: I find python very handy not only as a programming/scripting language 
but also as a scratchpad for various number theory investigations. 
Unfortunately the output formatting leaves all but the most basic 
investigations out in the cold, and lack of negative base input/output makes 
it harder to use python to look for certain patterns in result sets, not to mention getting the data in...

Thanks,
	Yuri


_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev


This email may contain confidential or privileged information. If you believe you have received the message in error, please notify the sender and delete the message without copying or disclosing it.


From mfb at lotusland.dyndns.org  Wed Sep 10 09:29:46 2003
From: mfb at lotusland.dyndns.org (Matthew F. Barnes)
Date: Wed Sep 10 09:43:11 2003
Subject: [Python-Dev] Consistent logging in the standard library
In-Reply-To: <200309100615.h8A6FbM15765@12-236-84-31.client.attbi.com>
References: <4101.192.168.1.1.1062990782.squirrel@server.lotusland.dyndns.org> 
	<200309100615.h8A6FbM15765@12-236-84-31.client.attbi.com>
Message-ID: <3412.192.168.1.1.1063200586.squirrel@server.lotusland.dyndns.org>

Guido van Rossum said:
> Hm...  Isn't the logging module's ability to set separate *handlers*
> sufficient to get the desired flexibility?  Rather than the
> application specifying a different *logger* object, the application
> can specify a different *handler* object.  I think this was the
> originally intended design.  It places less of a burden on the modules
> that use logging (no need to define getdefaultlogger() and
> setdefaultlogger()) and makes it possible for the different handlers
> to be specified by the logging module's configuration file, without
> the application having to support this.

I was thinking of the logging hierarchy when I came up with this idea.  If
a module references a specific logger object explicitly when a logging
message needs to be issued (e.g. logging.getLogger(__name__).info(msg)),
then the module's logging output is forever handled at a fixed place in
the hierarchy.

An application may want to organize things differently.  For example, it
may want to redirect asyncore's log messages to a logger named
"myapp.network.asyncore" so that its messages are propagated to the
"myapp.network" and "myapp" loggers defined by the application.  Those
loggers could apply additional handlers or filters to asyncore's log
messages.  In such a case the application could set this up as follows:

    asyncore.setdefaultlogger(logging.getLogger('myapp.network.asyncore'))


> What behavior exactly are you thinking of?  A logger object has almost
> no semantics apart from passing log events on to handlers; by design,
> all the work is done in the handlers.

I used the word "behavior" to refer to the series of handlers and filters
that are applied to a log message.  Different instances of the same class
could log to different logger objects, and thereby have different "logging
behaviors".  Unfortunately none of the modules I've identified in the
stdlib seem to provide a motivating example of why you'd ever want to do
this.  I can try to elaborate further if the benefit is not clear.


>> I determined that no changes are necessary for the cgi module, since
>> it doesn't even use the logging functions it defines (initlog,
>> dolog, etc.).
>
> They're intended for use by CGI applications.  They should probably be
> rewritten to direct the calls to the logging module.

I don't have much experience with this module so correct me if I'm wrong,
but to my knowledge the cgi module does not call its own logging
functions.  This implies to me that it's essentially up to the CGI
application to perform its own logging.  If that's true then I wonder if
CGI applications would be better off just be using the logging package
directly, rather than routing log messages through the cgi module.

The existing log functions in cgi should probably remain for backwards
compatibility, but I just have to wonder if it's really worth updating
them.


>> I'm currently working on a way to replicate imaplib's command
>> history using LogRecords, which may call for a new type of
>> logging.BufferingHandler subclass that maintains a circular queue of
>> LogRecords.
>
> That might be a useful feature on its own for the logging package.
> Note though, that at a grander scale, the RotatingFileLogger class
> already supports a way to keep only the last N messages (for much
> larger N, though :-).

Thanks, I'll look into that.


> To reach Steve, you should probably post to the PyUnit list (sorry,
> you'll have to Google for details, I only vaguely recall that there is
> such a list).

If Pyunit-interest is what you mean, the last time I checked it didn't
seem to be very active anymore and was filled with mostly spam.  I'll try
to reach Steve directly.


> Personally, I think that unittest is such a special case that it might
> be best to leave it alone...

I think I agree with you.  Unless Steve has any ideas, I'm going to drop
it from the forthcoming PEP.


From guido at python.org  Wed Sep 10 08:18:00 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Sep 10 10:18:26 2003
Subject: [Python-Dev] Base-n integer formatting
In-Reply-To: Your message of "Wed, 10 Sep 2003 22:09:29 +0930."
	<200309102209.29603.vilmanis@internode.on.net> 
References: <200309102209.29603.vilmanis@internode.on.net> 
Message-ID: <200309101418.h8AEI0f16363@12-236-84-31.client.attbi.com>

Yuri,

Those conversions are so easily coded in Python itself that I see no
reason to build them into the language or standard library.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From Paul.Moore at atosorigin.com  Wed Sep 10 17:49:21 2003
From: Paul.Moore at atosorigin.com (Moore, Paul)
Date: Wed Sep 10 11:50:09 2003
Subject: [Python-Dev] RE: [python-win32] pb with string conversion to float
	and GetObject call
Message-ID: <16E1010E4581B049ABC51D4975CEDB8803060C76@UKDCX001.uk.int.atosorigin.com>

From: popov [mailto:outlook@evpopov.com]

> Here's a strange one:
> 
> print float('8.4')
> 
> Works as expected.
> But:
> 
> import win32com.client
> obj = win32com.client.GetObject ("winmgmts:")
> print float('8.4')
> 
> Throws a 'ValueError: invalid literal for float(): 8.4' exception on the
> print line !
> If you use '8,4' instead of '8.4', then it works (!). So it seems that some
> locals have been modified here by the call to GetObject (without this call -
> leaving only the import statement - the problem does not show up).
> Note that using 'LDAP:' instead of 'winmgmts:', everything is ok. So maybe
> the culprit is the winmgmts provider...
> If someone could test this and let me know the results. Thx.

There's a discussion going on on python-dev at the moment (the messages
mention LC_NUMERIC in the subject) on basically this issue. Python's
float conversion routines rely on the underlying C locale remaining
unchanged. It's known that 3rd party extensions can violate this, and
also that Microsoft MAPI mucks up the locale behind the scenes. From
what you are reporting, it looks like the WMI stuff also does this.

I don't know of any fix for sure - you might be able to use the functions
from the locale module to set the locale value back to "C". Assuming that
an acceptable fix is found, something should be in Python 2.4 (I don't know
if the fix will be deemed appropriate for a maintenance release of 2.3).

Hope this helps,
Paul.

PS I'm copying this to python-dev, as evidence of another area where this
   issue comes up - I hope that's acceptable.

From mcherm at mcherm.com  Wed Sep 10 09:58:25 2003
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed Sep 10 11:58:26 2003
Subject: [Python-Dev] Consistent logging in the standard library
Message-ID: <1063209505.3f5f4a21b564e@mcherm.com>

Quoth Matthew F. Barnes:
> An application may want to organize things differently.  For example, it
> may want to redirect asyncore's log messages to a logger named
> "myapp.network.asyncore"

Seems like an uncommon request, but one which shouldn't be made
impossible for those who want it. But couldn't it be achieved by 
writing a handler which simply re-logged the message at the new
spot in the hierarchy?




From fdrake at acm.org  Wed Sep 10 13:04:07 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed Sep 10 12:04:18 2003
Subject: [Python-Dev] Consistent logging in the standard library
In-Reply-To: <1063209505.3f5f4a21b564e@mcherm.com>
References: <1063209505.3f5f4a21b564e@mcherm.com>
Message-ID: <16223.19319.350549.596704@grendel.zope.com>


Michael Chermside writes:
 > Seems like an uncommon request, but one which shouldn't be made
 > impossible for those who want it. But couldn't it be achieved by 
 > writing a handler which simply re-logged the message at the new
 > spot in the hierarchy?

That would solve that use case handily; if that's the right approach,
such a handler should be provided as part of the standard set of
handlers.

The use case for a .setlogger() method on the various objects that
perform logging is different, so that is still needed.

Generally, I'm not convinced that a getdefaultlogger() is needed or
helpful (it seems an implementation default for the module), though
I'm not at all opposed to the setdefaultlogger() for the affected
modules.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From kiko at async.com.br  Wed Sep 10 14:08:59 2003
From: kiko at async.com.br (Christian Reis)
Date: Wed Sep 10 12:10:25 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEAAFKAB.tim.one@comcast.net>
References: <m3d6elkpue.fsf@mira.informatik.hu-berlin.de>
	<LNBBLJKPBEHFEDALKOLCAEAAFKAB.tim.one@comcast.net>
Message-ID: <20030910160858.GE2405@async.com.br>

On Mon, Sep 01, 2003 at 02:30:23PM -0400, Tim Peters wrote:
> There's no way of using C's locale gimmicks that's threadsafe, short of all
> callers agreeing to follow a beyond-standard-C exclusion protocol -- which
> is the same as saying "no way" in reality.  So that's part of one problem no
> patch of this ilk *can* solve.  It's not that the patch doesn't try hard
> enough, it's that this approach is inherently inadequate to solve all of
> this particular problem.
> 
> > It is just that the patch does not "feel" right, given that there must
> > be "native" locale-inaware parsing of floating point constants
> > somewhere on each platform (atleast on those that support C++98).
> 
> I haven't found one on Windows (doesn't mean it doesn't exist, does mean
> it's apparently well hidden if it does exist).

Just to follow up, today I found a thread on opengroup.org that
discusses locale-safe APIs in the C library. They don't suggest anything
very positive in the way of standardization :-/

http://www.opengroup.org/austin/mailarchives/austin-group-l/msg00763.html

Take care,
--
Christian Reis, Senior Engineer, Async Open Source, Brazil.
http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL

From Mariselaschwebach at pcvo.com  Wed Sep 10 22:32:24 2003
From: Mariselaschwebach at pcvo.com (Women Helpline)
Date: Wed Sep 10 15:37:26 2003
Subject: [Python-Dev] Luscious, Seductive L|ps can be yours
Message-ID: <551x9i9qj5mae9$7u-6@rh1.c.xp.d6>

An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20030910/b1cc80f2/attachment-0001.htm
From mfb at lotusland.dyndns.org  Wed Sep 10 17:21:47 2003
From: mfb at lotusland.dyndns.org (Matthew F. Barnes)
Date: Wed Sep 10 17:35:06 2003
Subject: [Python-Dev] Consistent logging in the standard library
In-Reply-To: <16223.19319.350549.596704@grendel.zope.com>
References: <1063209505.3f5f4a21b564e@mcherm.com> 
	<16223.19319.350549.596704@grendel.zope.com>
Message-ID: <30174.130.76.32.145.1063228907.squirrel@server.lotusland.dyndns.org>

Fred L. Drake, Jr. said:
>
> Michael Chermside writes:
>  > Seems like an uncommon request, but one which shouldn't be made
>  > impossible for those who want it. But couldn't it be achieved by
>  > writing a handler which simply re-logged the message at the new
>  > spot in the hierarchy?
>
> That would solve that use case handily; if that's the right approach,
> such a handler should be provided as part of the standard set of
> handlers.

I agree, this seems much cleaner than what I've proposed.

In fact, the "ForwardingHandler" idea could be submitted to SourceForge as
a separate patch for logging, and then the issue drops out of this
"consistent logging" PEP altogether.

So that changes my proposal as follows:
- The getdefaultlogger() and setdefaultlogger() functions go away.
- The self.logger attribute of class instances can be initialized to
logging.getLogger(__name__).
- Some kind of "ForwardingHandler" can be added to the module's logger to
forward messages elsewhere in the logging hierarchy.  But this is not
likely to be a common use case.

Matthew Barnes

From fuerte at sci.fi  Thu Sep 11 20:23:05 2003
From: fuerte at sci.fi (Harri Pesonen)
Date: Thu Sep 11 12:23:02 2003
Subject: [Python-Dev] Making python C-api thread safe
Message-ID: <3F60A169.3070409@sci.fi>

Background: I'm new to Python. I just embedded Python into a 
multi-threaded application, and extended Python by callbacks as well. 
During this process I realised that Python is not internally as 
beautiful as externally. It is not thread safe. This surprised me, 
because Python is a quite young programming language. I started thinking 
how to fix this, how to make Python thread safe. Currently Python does 
not support multiprocessor machines or hyperthreading processors. It is 
only a matter of time when this has to be fixed, better soon than later.

How to make C-api thread safe: It's quite simple, in fact. Thread should 
first call



From aahz at pythoncraft.com  Thu Sep 11 13:44:22 2003
From: aahz at pythoncraft.com (Aahz)
Date: Thu Sep 11 12:44:27 2003
Subject: [Python-Dev] Making python C-api thread safe
In-Reply-To: <3F60A169.3070409@sci.fi>
References: <3F60A169.3070409@sci.fi>
Message-ID: <20030911164421.GA897@panix.com>

On Thu, Sep 11, 2003, Harri Pesonen wrote:
>
> Background: I'm new to Python. 

This issue gets discussed regularly on comp.lang.python; python-dev is
for discussion of specific changes to the Python language and libraries.
Until you are familiar with the way threading works in Python and have
some concrete proposal to make, please keep the discussion on c.l.py.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

From fuerte at sci.fi  Thu Sep 11 20:53:10 2003
From: fuerte at sci.fi (Harri Pesonen)
Date: Thu Sep 11 12:53:11 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
Message-ID: <3F60A876.7020204@sci.fi>

Sorry about the previous post! :-)

Background: I'm new to Python. I just embedded Python into a 
multi-threaded application, and extended Python by callbacks as well. 
During this process I realised that Python is not internally as 
beautiful as externally. It is not thread safe. This surprised me, 
because Python is a quite young programming language. I started thinking 
how to fix this, how to make Python thread safe. Currently Python does 
not support multiprocessor machines or hyperthreading processors. It is 
only a matter of time when this has to be fixed, better soon than later.

How to make C-API thread safe: It's quite simple, in fact. Thread should 
first call

PyThreadState* Py_NewInterpreter()

and use this thread state pointer in every following API call. This 
thread state pointer should point to a memory structure, that holds 
everything that Python interpreter needs. There should be no global 
variables, not even None, except

const char* Py_GetVersion()

and such. So every Python function should have this thread state as the 
first argument:

int PyRun_SimpleString(PyThreadState*tstate, char *command)

and so on. Also callbacks should have it, about every API function.

Advantages: The following functions would not be needed any more, it 
would make the API much simpler:

void PyEval_InitThreads( )
void PyEval_AcquireLock( )
void PyEval_ReleaseLock( )
void PyEval_AcquireThread( PyThreadState *tstate)
void PyEval_ReleaseThread( PyThreadState *tstate)
PyThreadState* PyEval_SaveThread( )
void PyEval_RestoreThread( PyThreadState *tstate)
Py_BEGIN_ALLOW_THREADS
Py_END_ALLOW_THREADS
Py_BLOCK_THREADS
Py_UNBLOCK_THREADS
PyInterpreterState* PyInterpreterState_New( )
void PyInterpreterState_Clear( PyInterpreterState *interp)
void PyInterpreterState_Delete( PyInterpreterState *interp)
PyThreadState* PyThreadState_New( PyInterpreterState *interp)
void PyThreadState_Clear( PyThreadState *tstate)
void PyThreadState_Delete( PyThreadState *tstate)
PyThreadState* PyThreadState_Get( )
PyThreadState* PyThreadState_Swap( PyThreadState *tstate)

The current documentation says: "In order to support multi-threaded 
Python programs, the interpreter regularly releases and reacquires the 
lock -- by default, every 100 bytecode instructions". This would not be 
needed anymore. It would make the interpreter faster.

Making this change would take a couple of weeks, if the job would be 
divided between modules to different persons. The change is trivial, but 
it would have to be made everywhere.

Disadvantages: Everything old would break. Thread safe Python should 
have a version number significantly higher, like Python 3. Old 
applications would work with old Python versions, with old Python DLLs, 
but they would need to be rewritten for the new API. It would be a 
trivial task, but still.

Disclaimer: I already got mail saying: "

This issue gets discussed regularly on comp.lang.python; python-dev is
for discussion of specific changes to the Python language and libraries.
Until you are familiar with the way threading works in Python and have
some concrete proposal to make, please keep the discussion on c.l.py."

The current situation is very unfortunate, having a single threaded implementation in the modern computer age. I have a concrete proposal, which would break everything :-) but I think that is the only solution.

Thanks for reading,

Harri




From pje at telecommunity.com  Thu Sep 11 14:10:06 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Sep 11 13:10:24 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <3F60A876.7020204@sci.fi>
Message-ID: <5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>

At 07:53 PM 9/11/03 +0300, Harri Pesonen wrote:
>Sorry about the previous post! :-)
>
>Background: I'm new to Python. I just embedded Python into a 
>multi-threaded application, and extended Python by callbacks as well. 
>During this process I realised that Python is not internally as beautiful 
>as externally. It is not thread safe. This surprised me, because Python is 
>a quite young programming language. I started thinking how to fix this, 
>how to make Python thread safe. Currently Python does not support 
>multiprocessor machines or hyperthreading processors. It is only a matter 
>of time when this has to be fixed, better soon than later.
>
>How to make C-API thread safe: It's quite simple, in fact. Thread should 
>first call
>
>PyThreadState* Py_NewInterpreter()
>
>and use this thread state pointer in every following API call. This thread 
>state pointer should point to a memory structure, that holds everything 
>that Python interpreter needs. There should be no global variables, not 
>even None, except

This only makes individual Python interpreters thread-safe.  That might 
certainly be useful for some embedded applications, but it does nothing for 
thread-safety *within* an interpreter.

In other words, Python would *still* need a global interpreter lock, in 
order for threaded programs written in Python to behave properly.

So, your solution would help only creators of multithreaded embedded 
applications, wherein each thread desired an isolated Python 
interpreter.  Meanwhile, it would impose both a one-time and ongoing cost 
penalty on *all other* Python users, without giving them any benefit in return.


From fuerte at sci.fi  Thu Sep 11 21:47:49 2003
From: fuerte at sci.fi (Harri Pesonen)
Date: Thu Sep 11 13:47:46 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
References: <5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
Message-ID: <3F60B545.6030100@sci.fi>

Phillip J. Eby wrote:

> At 07:53 PM 9/11/03 +0300, Harri Pesonen wrote:
>
>> Sorry about the previous post! :-)
>>
>> Background: I'm new to Python. I just embedded Python into a 
>> multi-threaded application, and extended Python by callbacks as well. 
>> During this process I realised that Python is not internally as 
>> beautiful as externally. It is not thread safe. This surprised me, 
>> because Python is a quite young programming language. I started 
>> thinking how to fix this, how to make Python thread safe. Currently 
>> Python does not support multiprocessor machines or hyperthreading 
>> processors. It is only a matter of time when this has to be fixed, 
>> better soon than later.
>>
>> How to make C-API thread safe: It's quite simple, in fact. Thread 
>> should first call
>>
>> PyThreadState* Py_NewInterpreter()
>>
>> and use this thread state pointer in every following API call. This 
>> thread state pointer should point to a memory structure, that holds 
>> everything that Python interpreter needs. There should be no global 
>> variables, not even None, except
>
>
> This only makes individual Python interpreters thread-safe.  That 
> might certainly be useful for some embedded applications, but it does 
> nothing for thread-safety *within* an interpreter.
>
> In other words, Python would *still* need a global interpreter lock, 
> in order for threaded programs written in Python to behave properly.
>
> So, your solution would help only creators of multithreaded embedded 
> applications, wherein each thread desired an isolated Python 
> interpreter.  Meanwhile, it would impose both a one-time and ongoing 
> cost penalty on *all other* Python users, without giving them any 
> benefit in return.

You are right, I thought only about making interpreters thread safe, 
thanks for pointing this out. But this is still needed, it is 
unavoidable in the long run.

What do you mean about the penalty? If you mean having the state pointer 
in stack in every call, I don't think so.

If we think about making Python threads thread-safe, within the 
interpreter, then I think that it should be left for the application 
deverloper, in principle. He should create "critical sections" and other 
locks when needed (I don't know how it is done currently). I can imagine 
that None and other values will cause problems, when two threads 
increment or decrement the reference count at the same time. Of course 
there is a simple solution for that, InterlockedIncrement and 
InterlockedDecrement in Win32 at least.

I can imagine that there are other problems with garbage collector, and 
so on. I don't want to go into details about this, I got my message 
through, I was almost losing sleep when I realised the current situation.

But my basic message is this: Python needs to be made thread safe. 
Making the individual interpreters thread safe is trivial, and benefits 
many people, and is a necessary first step; making threads within 
interpreter thread safe is possible as well, at least if you leave 
something for the developer, as you should, as you do in every other 
programming language as well.

Harri



From brian at sweetapp.com  Thu Sep 11 12:06:43 2003
From: brian at sweetapp.com (Brian Quinlan)
Date: Thu Sep 11 14:05:57 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <3F60B545.6030100@sci.fi>
Message-ID: <001001c3788f$75798320$21795418@dell1700>

> Making the individual interpreters thread safe is trivial, and
benefits
> many people, 

I'm not sure that I believe this statement. Isn't this of benefit only
to people who want to have multiple interpreter instances running
concurrently in an embedding situation? I doubt that there is an army of
people who want to do that.

> and is a necessary first step; making threads within
> interpreter thread safe is possible as well, at least if you leave
> something for the developer, as you should, as you do in every other
> programming language as well.

Of course it is possible (and would likely be of benefit to more than a
handful of Python users). The problem is that it requires effort, would
likely require massive code breakage and the locking required would
likely significantly decrease Python's single-thread performance.

Cheers,
Brian


From aahz at pythoncraft.com  Thu Sep 11 15:21:36 2003
From: aahz at pythoncraft.com (Aahz)
Date: Thu Sep 11 14:21:41 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <001001c3788f$75798320$21795418@dell1700>
References: <3F60B545.6030100@sci.fi> <001001c3788f$75798320$21795418@dell1700>
Message-ID: <20030911182136.GA12857@panix.com>

Please, people, let's take this off python-dev and move it to
comp.lang.python.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

From dave at boost-consulting.com  Thu Sep 11 16:11:54 2003
From: dave at boost-consulting.com (David Abrahams)
Date: Thu Sep 11 15:13:05 2003
Subject: [Python-Dev] PyUnicode_FromEncodedObject
Message-ID: <uhe3j16o5.fsf@boost-consulting.com>


I'd post a SF bug but it keeps insisting I log in, even after I have
already.

There's a bug, either in the code or docs for
PyUnicode_FromEncodedObject.  The docs read:

  "Unicode objects are passed back as-is with incremented
  refcount. Note: These cannot be decoded; passing a non-NULL value
  for encoding will result in a TypeError."


'tain't so; the following shows that the error is unconditional.



    if (obj == NULL) {
	PyErr_BadInternalCall();
	return NULL;
    }

#if 0
    /* For b/w compatibility we also accept Unicode objects provided
       that no encodings is given and then redirect to
       PyObject_Unicode() which then applies the additional logic for
       Unicode subclasses.

       NOTE: This API should really only be used for object which
             represent *encoded* Unicode !

    */
	if (PyUnicode_Check(obj)) {
	    if (encoding) {
		PyErr_SetString(PyExc_TypeError,
				"decoding Unicode is not supported");
	    return NULL;
	    }
	return PyObject_Unicode(obj);
	    }
#else
    if (PyUnicode_Check(obj)) {
	PyErr_SetString(PyExc_TypeError,
			"decoding Unicode is not supported");
	return NULL;
	}
#endif


-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com


From fuerte at sci.fi  Thu Sep 11 23:16:03 2003
From: fuerte at sci.fi (Harri Pesonen)
Date: Thu Sep 11 15:16:06 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <5.1.1.6.0.20030911142317.02b88640@telecommunity.com>
References: <5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
	<5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
	<5.1.1.6.0.20030911142317.02b88640@telecommunity.com>
Message-ID: <3F60C9F3.4060107@sci.fi>

Phillip J. Eby wrote:

> At 08:47 PM 9/11/03 +0300, Harri Pesonen wrote:
>
>> But my basic message is this: Python needs to be made thread safe. 
>> Making the individual interpreters thread safe is trivial, and 
>> benefits many people, and is a necessary first step;
>
>
> It's far from trivial - you're talking about invalidating every piece 
> of C code written for Python over a multi-year people by dozens upon 
> dozens of extension authors.

The change is trivial in Python C API. I already said that it would 
break everything outside the Python distribution, but the change in 
other applications is also trivial.

> It doesn't benefit many people: only those using isolated interpreters 
> embedded in a multithreaded C program. 

I don't know how many people are writing threads in Python, either. I 
guess that not so many. In my case I only need a thread safe 
interpreter, I don't create threads in Python code. So just having what 
I described would be enough for me: no need for global interpreter lock, 
and Python would be really multithreading. It would benefit many people, 
I'm sure.

>
>> making threads within interpreter thread safe is possible as well, at 
>> least if you leave something for the developer, as you should, as you 
>> do in every other programming language as well.
>
>
> You misunderstand.  Those "critical sections" are for the most part in 
> Python's C code, not in the Python script. 

Yes, I'm aware of the None problem at least (only one instance of it). 
Please enlighten me about the other critical sections? Object 
allocation/freeing?

> I'm guessing you haven't done much writing of C extensions for Python 
> (or Python core C), or else you'd realize why trying to make 
> INCREF/DECREF threadsafe would absolutely decimate performance.  
> Reference count updates happen *way* too often in normal code flow.
>
I also knew that already. But how else can you do it?

Of course, changing Python to not have a single None would help a lot. 
Or, perhaps it could have a single None, but in case of None, the 
reference count would have no meaning, it would never be deallocated, 
because it would be checked in code. Maybe it does it already, I don't know.

I'm also wondering why this problem has not been addressed before? If I 
had the power to change Python, this would be the first thing I did.

Harri



From jeremy at alum.mit.edu  Thu Sep 11 16:32:45 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu Sep 11 15:32:55 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <3F60C9F3.4060107@sci.fi>
References: <5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
	<5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
	<5.1.1.6.0.20030911142317.02b88640@telecommunity.com>
	<3F60C9F3.4060107@sci.fi>
Message-ID: <1063308764.2069.235.camel@localhost.localdomain>

On Thu, 2003-09-11 at 15:16, Harri Pesonen wrote:
> I'm also wondering why this problem has not been addressed before? If I 
> had the power to change Python, this would be the first thing I did.

Try coming up with a patch.  I expect it would be considered provided
that it was maximally backwards compatible with the existing C API and
did not reduce performance on benchmarks like pystone.

That is to say, sure it would be nice, but at what cost?

Jeremy



From fuerte at sci.fi  Fri Sep 12 00:31:25 2003
From: fuerte at sci.fi (Harri Pesonen)
Date: Thu Sep 11 16:31:22 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <1063308764.2069.235.camel@localhost.localdomain>
References: <5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>	
	<5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>	
	<5.1.1.6.0.20030911142317.02b88640@telecommunity.com>	
	<3F60C9F3.4060107@sci.fi>
	<1063308764.2069.235.camel@localhost.localdomain>
Message-ID: <3F60DB9D.8060205@sci.fi>

Jeremy Hylton wrote:

>On Thu, 2003-09-11 at 15:16, Harri Pesonen wrote:
>  
>
>>I'm also wondering why this problem has not been addressed before? If I 
>>had the power to change Python, this would be the first thing I did.
>>    
>>
>
>Try coming up with a patch.  I expect it would be considered provided
>that it was maximally backwards compatible with the existing C API and
>did not reduce performance on benchmarks like pystone.
>
It would be totally incompatible with the existing C API. The 
performance would be better.

>That is to say, sure it would be nice, but at what cost?
>  
>
At cost of breaking up with the past, but eventually it has to be done.

If I had time, I would create this multithreading Python myself, call it 
a different language perhaps. I don't know if the Python licence allows 
this.

Harri

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20030911/a31103f3/attachment.htm
From martin at v.loewis.de  Thu Sep 11 21:46:35 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Sep 11 16:46:35 2003
Subject: [Python-Dev] PyUnicode_FromEncodedObject
In-Reply-To: <uhe3j16o5.fsf@boost-consulting.com>
References: <uhe3j16o5.fsf@boost-consulting.com>
Message-ID: <m3k78ft5nu.fsf@mira.informatik.hu-berlin.de>

David Abrahams <dave@boost-consulting.com> writes:

> I'd post a SF bug but it keeps insisting I log in, even after I have
> already.

It does not insist, it just explains that you have logged
on. Unfortunately, there is no way to make this message disappear once
you have logged on, so all of us see it all the time.

> 
> There's a bug, either in the code or docs for
> PyUnicode_FromEncodedObject.  The docs read:
> 
>   "Unicode objects are passed back as-is with incremented
>   refcount. Note: These cannot be decoded; passing a non-NULL value
>   for encoding will result in a TypeError."

Please do submit a bug report for that.

Regards,
Martin

From martin at v.loewis.de  Thu Sep 11 21:54:11 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Sep 11 16:54:13 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <3F60C9F3.4060107@sci.fi>
References: <5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
	<5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
	<5.1.1.6.0.20030911142317.02b88640@telecommunity.com>
	<3F60C9F3.4060107@sci.fi>
Message-ID: <m3fzj3t5b4.fsf@mira.informatik.hu-berlin.de>

Harri Pesonen <fuerte@sci.fi> writes:

> The change is trivial in Python C API. I already said that it would
> break everything outside the Python distribution, but the change in
> other applications is also trivial.

If it is trivial, would you mind posting a patch somewhere?

> > You misunderstand.  Those "critical sections" are for the most part
> > in Python's C code, not in the Python script.
> 
> Yes, I'm aware of the None problem at least (only one instance of
> it). Please enlighten me about the other critical sections? Object
> allocation/freeing?

Yes, that, plus:
- allocation of/access to small numbers
- access to global variables in extension modules
  (e.g. cursesmodule.c:PyCursesError)
- type objects
etc.

> Of course, changing Python to not have a single None would help a
> lot. Or, perhaps it could have a single None, but in case of None, the
> reference count would have no meaning, it would never be deallocated,
> because it would be checked in code. Maybe it does it already, I don't
> know.

So how can you know a patch would be trivial?

> I'm also wondering why this problem has not been addressed before? If
> I had the power to change Python, this would be the first thing I did.

Please go ahead and post a patch. You might find that the patch is
difficult to write, and, once written, will have many errors. Once
those are fixed, you might find that Python becomes painfully slow,
and lose a lot of portability.

Regards,
Martin

From martin at v.loewis.de  Fri Sep 12 00:07:56 2003
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Sep 11 17:08:40 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <3F60DB9D.8060205@sci.fi>
References: <5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>		<5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>		<5.1.1.6.0.20030911142317.02b88640@telecommunity.com>		<3F60C9F3.4060107@sci.fi>	<1063308764.2069.235.camel@localhost.localdomain>
	<3F60DB9D.8060205@sci.fi>
Message-ID: <3F60E42C.1040501@v.loewis.de>

Harri Pesonen wrote:

> If I had time, I would create this multithreading Python myself, call it 
> a different language perhaps. I don't know if the Python licence allows 
> this.

The license is only on the implementation (source code), not on the 
language itself.
You are free to provide alternative implementations of the language, 
without having to ask anybody for permission.

But please don't call such a thing trivial when you know it is 
inherently complex.

Regards,
Martin


From greg at cosc.canterbury.ac.nz  Fri Sep 12 16:11:28 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Sep 11 23:11:46 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <3F60C9F3.4060107@sci.fi>
Message-ID: <200309120311.h8C3BSe01781@oma.cosc.canterbury.ac.nz>

> Yes, I'm aware of the None problem at least (only one instance of it). 
> Please enlighten me about the other critical sections? Object 
> allocation/freeing?

None is only the molecule at the tip of the iceberg.  If you're
talking about threads within a Python interpreter, doing just about
anything at all to any Python object is a critical section. The amount
of locking required to deal with that at a fine-grained level would
totally kill performance, not to mention being hugely tedious and
error-prone to implement.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From dave at boost-consulting.com  Fri Sep 12 00:35:40 2003
From: dave at boost-consulting.com (David Abrahams)
Date: Thu Sep 11 23:36:48 2003
Subject: [Python-Dev] PyUnicode_FromEncodedObject
In-Reply-To: <m3k78ft5nu.fsf@mira.informatik.hu-berlin.de> (Martin v.
	=?iso-8859-1?q?L=F6wis's?= message of "11 Sep 2003 22:46:13 +0200")
References: <uhe3j16o5.fsf@boost-consulting.com>
	<m3k78ft5nu.fsf@mira.informatik.hu-berlin.de>
Message-ID: <ullsuwuer.fsf@boost-consulting.com>

martin@v.loewis.de (Martin v. L?wis) writes:

> David Abrahams <dave@boost-consulting.com> writes:
>
>> I'd post a SF bug but it keeps insisting I log in, even after I have
>> already.
>
> It does not insist, it just explains that you have logged
> on. Unfortunately, there is no way to make this message disappear once
> you have logged on, so all of us see it all the time.

No, seriously.  There is no "submit new" button/link for me.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com


From fuerte at sci.fi  Fri Sep 12 08:56:55 2003
From: fuerte at sci.fi (Harri Pesonen)
Date: Fri Sep 12 00:56:56 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <5.1.1.6.0.20030911162016.02027750@telecommunity.com>
References: <5.1.1.6.0.20030911142317.02b88640@telecommunity.com>
	<5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
	<5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
	<5.1.1.6.0.20030911142317.02b88640@telecommunity.com>
	<5.1.1.6.0.20030911162016.02027750@telecommunity.com>
Message-ID: <3F615217.8060106@sci.fi>

Phillip J. Eby wrote:

> Please do not CC: my mail to Python-Dev again; I intentionally did not 
> include python-dev on my CC: because it was asked that we move this 
> thread elsewhere.
>
> At 10:16 PM 9/11/03 +0300, Harri Pesonen wrote:
>
>> Phillip J. Eby wrote:
>>
>>> At 08:47 PM 9/11/03 +0300, Harri Pesonen wrote:
>>>
>>>> But my basic message is this: Python needs to be made thread safe. 
>>>> Making the individual interpreters thread safe is trivial, and 
>>>> benefits many people, and is a necessary first step;
>>>
>>>
>>>
>>> It's far from trivial - you're talking about invalidating every 
>>> piece of C code written for Python over a multi-year people by 
>>> dozens upon dozens of extension authors.
>>
>>
>> The change is trivial in Python C API. I already said that it would 
>> break everything outside the Python distribution, but the change in 
>> other applications is also trivial.
>
>
> How do you propose that C code called *from* Python *receive* the 
> threadstate pointer? 

Exactly like that. Is there a problem? I'm suggesting that every 
function call gets that pointer, unless the function can get it from 
some other argument, that contains a pointer to it.

>
>>> It doesn't benefit many people: only those using isolated 
>>> interpreters embedded in a multithreaded C program.
>>
>>
>> I don't know how many people are writing threads in Python, either. I 
>> guess that not so many. In my case I only need a thread safe 
>> interpreter, I don't create threads in Python code. So just having 
>> what I described would be enough for me: no need for global 
>> interpreter lock, and Python would be really multithreading. It would 
>> benefit many people, I'm sure.
>
>
> Obviously, it's enough for you, or you wouldn't be proposing it.  What 
> does it do for me?  Nothing whatsoever, except add needless overhead 
> and make me rewrite every C extension I've ever written for Python.  
> So, by and large, you're not going to get much support for your change 
> from Python developers, especially those who write C extensions, or 
> depend on extensions written by others. 

Probably. That's why I'm thinking now that the language should be called 
something else, like MPython for "multi-threading Python". It would be 
99% compatible with the existing Python syntax, but have different 
internals.

>
>> Yes, I'm aware of the None problem at least (only one instance of 
>> it). Please enlighten me about the other critical sections? Object 
>> allocation/freeing?
>
>
> Data structure manipulations, e.g. all use of dictionaries.  Python 
> spends most of its time doing dictionary lookups or modifications, all 
> of which need to be protected. 

After sleeping over night, I think that I got it. :-) The simple 
solution is, that each thread created in Python gets its own independent 
interpreter state as well. And there could be a separate thread-global 
interpreter state for shared memory access. Access to this global state 
would always be synchronized. There could even be multiple named global 
states, so that the thread interlocking could be minimized. The python 
syntax for creating objects in this global state should be invented:

synchronize a = "abcd"

Also when creating the new thread, it's arguments would be copied from 
the creating state to the new state.

What does it sound? Of course it would be incompatible with the current 
threading system in Python, but it would be totally multithreading, no 
global interpreter lock needed. It would be faster than current Python, 
there would be no need to free or acquire the lock when calling OS 
functions, and no need to check how many byte codes have been processed, 
etc.

>
>>> I'm guessing you haven't done much writing of C extensions for 
>>> Python (or Python core C), or else you'd realize why trying to make 
>>> INCREF/DECREF threadsafe would absolutely decimate performance.
>>> Reference count updates happen *way* too often in normal code flow.
>>
>> I also knew that already. But how else can you do it?
>
>
> The way it's done now!  :) 

I understand why the current Python works like it does. But I think that 
it's time for the next generation. If you don't do it, and I have no 
time now to do it, I'm still sure that this is done at some point, 
rather sooner than later.

>
>> Of course, changing Python to not have a single None would help a 
>> lot. Or, perhaps it could have a single None, but in case of None, 
>> the reference count would have no meaning, it would never be 
>> deallocated, because it would be checked in code. Maybe it does it 
>> already, I don't know.
>
>
> I really don't mean to be rude (another reason I'm writing this to you 
> privately), but this paragraph shows you are *really* new to Python 
> both at the level of coding in Python and coding with Python's C API.  
> I wish I could explain in detail why, but there's really far too much 
> that you don't understand and it would take me too long.  I will 
> attempt to summarize a very few points, however: first, identity 
> (pointer comparison) is a part of the Python language, so you can't 
> have multiple None instances any more than you can have more than one 
> value be NULL in C.  Second, at the C level, all Python objects 
> (including None) have an absolutely uniform API, so having refcount 
> behavior be different for different kinds of objects is not at all 
> practical.  Third, if you had more than one Py_None at the C level, 
> you'd either have to make Py_None a macro, or rewrite all the C.  If 
> you don't think that's a problem, you have absolutely no idea how much 
> C code out there is written to the Python API. 

Yes, Py_None would be a macro. All access to interpreter state would go 
through the interpreter state pointer that is always in stack, the first 
argument each C API function gets. That pointer should be named so that 
the macros will always work ("tState", for example, so that Py_None 
macro would expand to tState->mPy_None, for example).

>
>> I'm also wondering why this problem has not been addressed before?
>
>
> It has; the cure is worse than the disease.  A few years ago, somebody 
> wrote a "free-threading" version of Python, which locked individual 
> data objects rather than use the global interpreter lock.  The 
> performance for single-threaded programs was abominable, and the 
> performance gain even on multiprocessor machines was not thought worth 
> the cost.  So the project was scrapped. 

There would be no locking in my proposal, except when accessing the 
shared memory global thread state.

I don't know, I got mail about writing a PEP. It is clear that it would 
not be accepted, because it would break the existing API. The change is 
so big that I think that it has to be called a different language.

This is the last message I will make about this matter (before actually 
starting to code it), so I'm posting this to python-list as well, 
because this is too important to be ignored. Python *needs* to be 
free-threading...

Harri



From martin at v.loewis.de  Fri Sep 12 06:56:01 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Fri Sep 12 01:56:03 2003
Subject: [Python-Dev] PyUnicode_FromEncodedObject
In-Reply-To: <ullsuwuer.fsf@boost-consulting.com>
References: <uhe3j16o5.fsf@boost-consulting.com>
	<m3k78ft5nu.fsf@mira.informatik.hu-berlin.de>
	<ullsuwuer.fsf@boost-consulting.com>
Message-ID: <m33cf27dp8.fsf@mira.informatik.hu-berlin.de>

David Abrahams <dave@boost-consulting.com> writes:

> > It does not insist, it just explains that you have logged
> > on. Unfortunately, there is no way to make this message disappear once
> > you have logged on, so all of us see it all the time.
> 
> No, seriously.  There is no "submit new" button/link for me.

What happens if you navigate to

http://sourceforge.net/tracker/?group_id=5470&atid=105470

? Do you have cookies enabled in your browser?

Regards,
Martin

From dave at boost-consulting.com  Fri Sep 12 08:38:49 2003
From: dave at boost-consulting.com (David Abrahams)
Date: Fri Sep 12 07:39:57 2003
Subject: [Python-Dev] PyUnicode_FromEncodedObject
In-Reply-To: <m33cf27dp8.fsf@mira.informatik.hu-berlin.de> (Martin v.
	=?iso-8859-1?q?L=F6wis's?= message of "12 Sep 2003 07:55:47 +0200")
References: <uhe3j16o5.fsf@boost-consulting.com>
	<m3k78ft5nu.fsf@mira.informatik.hu-berlin.de>
	<ullsuwuer.fsf@boost-consulting.com>
	<m33cf27dp8.fsf@mira.informatik.hu-berlin.de>
Message-ID: <ubrtqw81i.fsf@boost-consulting.com>

martin@v.loewis.de (Martin v. L?wis) writes:

> David Abrahams <dave@boost-consulting.com> writes:
>
>> > It does not insist, it just explains that you have logged
>> > on. Unfortunately, there is no way to make this message disappear once
>> > you have logged on, so all of us see it all the time.
>> 
>> No, seriously.  There is no "submit new" button/link for me.
>
> What happens if you navigate to
>
> http://sourceforge.net/tracker/?group_id=5470&atid=105470
>
> ? Do you have cookies enabled in your browser?

Yep.  Submitted now.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com


From python at rcn.com  Fri Sep 12 14:17:18 2003
From: python at rcn.com (Raymond Hettinger)
Date: Fri Sep 12 14:48:13 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
Message-ID: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>

[Raymond]
> > I would like to backport this patch to Py2.3.1.  
> > The effort to provide a full mapping interface to all
> > mapping like objects was attempted in Py2.3 and several
> > modules for the bsddb package were updated, but this
> > one was missed and the package was left half converted.
> > 
> > IIRC, dbhash and bsddb don't affect the Apple MacIntosh
> > users.  Also, since this effort was started for bsddb and
> > only half completed, I view it to be a bit of a bugfix as
> > well as being featurelike.  It certainly affects the usability
> > of the module (the looping example and related text in 
> > the docs were both wrong -- that would not have happened
> > if the normal looping expectations were supported).

[GvR]
> Can you discuss this on python-dev?

Guys, are you okay with backporting this?


Raymond Hettinger





Index: __init__.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/bsddb/__init__.py,v
retrieving revision 1.5
retrieving revision 1.6
diff -C2 -d -r1.5 -r1.6
*** __init__.py 24 Apr 2003 16:02:44 -0000 1.5
--- __init__.py 12 Sep 2003 06:33:37 -0000 1.6
***************
*** 53,58 ****
  #----------------------------------------------------------------------
  
  
! class _DBWithCursor:
      """
      A simple wrapper around DB that makes it look like the bsddbobject in
--- 53,59 ----
  #----------------------------------------------------------------------
  
+ import UserDict
  
! class _DBWithCursor(UserDict.DictMixin):
      """
      A simple wrapper around DB that makes it look like the bsddbobject in
***************
*** 145,148 ****
--- 146,157 ----
          return self.db.sync()
  
+     def __iter__(self):
+         try:
+             yield self.first()[0]
+             next = self.next
+             while 1:
+                 yield next()[0]
+         except _bsddb.DBNotFoundError:
+             return
  
  #----------------------------------------------------------------------


From skip at pobox.com  Fri Sep 12 15:02:13 2003
From: skip at pobox.com (Skip Montanaro)
Date: Fri Sep 12 15:02:27 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
Message-ID: <16226.6197.573407.585746@montanaro.dyndns.org>


    >> > IIRC, dbhash and bsddb don't affect the Apple MacIntosh users.

I don't know what you mean here.  Do you mean Mac OS 9 or Mac OS X?  I
use bsddb all the time on Mac OS X.

    Raymond> [GvR]
    >> Can you discuss this on python-dev?

    Raymond> Guys, are you okay with backporting this?

What are we discussing?  Just adding an iterator to objects opened with the
old bsddb API?

Skip

From python at rcn.com  Fri Sep 12 15:19:01 2003
From: python at rcn.com (Raymond Hettinger)
Date: Fri Sep 12 15:23:55 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
	<16226.6197.573407.585746@montanaro.dyndns.org>
Message-ID: <005a01c37962$b94a0e20$30bc958d@oemcomputer>

[Raymond]
>     >> > IIRC, dbhash and bsddb don't affect the Apple MacIntosh users.
[Skip]
> I don't know what you mean here.  Do you mean Mac OS 9 or Mac OS X?  I
> use bsddb all the time on Mac OS X.

Hmm, I'll have to fix the docs too.  They currently list bsddb as being
only for Unix and Windows.


[Skip]
> What are we discussing?  Just adding an iterator to objects opened with the
> old bsddb API?

Yes. Iterators and the rest of the mapping API (__iter__, pop, 
popitem, iteritems, setdefault, etc).

Like dumbdbm and shelve, this is done by including UserDict.DictMixin
as a base class.


Raymond Hettinger




#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

From skip at pobox.com  Fri Sep 12 15:55:44 2003
From: skip at pobox.com (Skip Montanaro)
Date: Fri Sep 12 16:10:09 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <005a01c37962$b94a0e20$30bc958d@oemcomputer>
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
	<16226.6197.573407.585746@montanaro.dyndns.org>
	<005a01c37962$b94a0e20$30bc958d@oemcomputer>
Message-ID: <16226.9408.548163.145368@montanaro.dyndns.org>


    >> I don't know what you mean here.  Do you mean Mac OS 9 or Mac OS X?
    >> I use bsddb all the time on Mac OS X.

    Raymond> Hmm, I'll have to fix the docs too.  They currently list bsddb
    Raymond> as being only for Unix and Windows.

For non-GUI stuff Mac OS X is more Unix than Macintosh. ;-)

    Raymond> Yes. Iterators and the rest of the mapping API (__iter__, pop,
    Raymond> popitem, iteritems, setdefault, etc).

    Raymond> Like dumbdbm and shelve, this is done by including
    Raymond> UserDict.DictMixin as a base class.

I say go for it unless there's a serious backward compatibility problem.  I
suspect the vast majority of the people who use bsddb do so through the
anydbm module, so if usage that way doesn't break you should be okay.

Skip





From jeremy at alum.mit.edu  Fri Sep 12 16:44:15 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Fri Sep 12 16:44:29 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
Message-ID: <1063399454.2069.299.camel@localhost.localdomain>

On Fri, 2003-09-12 at 14:17, Raymond Hettinger wrote:
> [Raymond]
> > > I would like to backport this patch to Py2.3.1.  
> > > The effort to provide a full mapping interface to all
> > > mapping like objects was attempted in Py2.3 and several
> > > modules for the bsddb package were updated, but this
> > > one was missed and the package was left half converted.
> > > 
> > > IIRC, dbhash and bsddb don't affect the Apple MacIntosh
> > > users.  Also, since this effort was started for bsddb and
> > > only half completed, I view it to be a bit of a bugfix as
> > > well as being featurelike.  It certainly affects the usability
> > > of the module (the looping example and related text in 
> > > the docs were both wrong -- that would not have happened
> > > if the normal looping expectations were supported).
> 
> [GvR]
> > Can you discuss this on python-dev?
> 
> Guys, are you okay with backporting this?

Isn't this just the sort of little feature that was the subject of
recent discussion on the dangers of backporting.  It would mean that
someone writing an app against Python 2.3 could never be sure whether
the feature existed.  In practice, that would mean developers using
2.3.x would only find out about problems after deployment when they bump
into a user with the original 2.3.

I'm not sure how convincing I find this argument, but it's got some
merit.

Jeremy



From python at rcn.com  Fri Sep 12 19:50:40 2003
From: python at rcn.com (Raymond Hettinger)
Date: Fri Sep 12 20:11:51 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
	<1063399454.2069.299.camel@localhost.localdomain>
Message-ID: <008a01c3798a$f7198b40$b439c797@oemcomputer>

[Raymond]
> > > > I would like to backport this patch to Py2.3.1.  
> > > > The effort to provide a full mapping interface to all
> > > > mapping like objects was attempted in Py2.3 and several
> > > > modules for the bsddb package were updated, but this
> > > > one was missed and the package was left half converted.
> > > > 
> > > > IIRC, dbhash and bsddb don't affect the Apple MacIntosh
> > > > users.  Also, since this effort was started for bsddb and
> > > > only half completed, I view it to be a bit of a bugfix as
> > > > well as being featurelike.  It certainly affects the usability
> > > > of the module (the looping example and related text in 
> > > > the docs were both wrong -- that would not have happened
> > > > if the normal looping expectations were supported).
[GvR]
> > > Can you discuss this on python-dev?
[Raymond] 
> > Guys, are you okay with backporting this?
[Jeremy]
> Isn't this just the sort of little feature that was the subject of
> recent discussion on the dangers of backporting.  It would mean that
> someone writing an app against Python 2.3 could never be sure whether
> the feature existed.  In practice, that would mean developers using
> 2.3.x would only find out about problems after deployment when they bump
> into a user with the original 2.3.
> 
> I'm not sure how convincing I find this argument, but it's got some
> merit.

It has some.  That's why GvR had me bring it to python-dev
so you guys could help decide on the proper balance.
But for that one thought, the decision to apply is clear cut.

Thoughts in favor of applying:
* the existing interface is a pain and is misdocumented in Py2.3.0
* the patch was already half complete for 2.3.0 (applied
   to dbobj and dbshelve) __init__.py was just missed).
* the patch was already applied to dumbdbm and shelve for Py2.3.0
* if needed, it's not hard to write code that automatically
   adjusts for Py2.3.0:
       if not f.__class__.__bases__:
            f.__class__.__bases = (UserDict.DictMixin,)
* waiting another 18 months to put this in just results in that
   many more installed Pythons out there without the patch


Raymond

From martin at v.loewis.de  Sat Sep 13 00:52:37 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sat Sep 13 00:52:50 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <008a01c3798a$f7198b40$b439c797@oemcomputer>
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
	<1063399454.2069.299.camel@localhost.localdomain>
	<008a01c3798a$f7198b40$b439c797@oemcomputer>
Message-ID: <m3isnxxpbe.fsf@mira.informatik.hu-berlin.de>

"Raymond Hettinger" <python@rcn.com> writes:

> It has some.  That's why GvR had me bring it to python-dev
> so you guys could help decide on the proper balance.
> But for that one thought, the decision to apply is clear cut.

There is no proper balance. If you believe in the policy "no new
features", this policy is absolute - it would be pointless to accept
some new features, but reject others on grounds of the policy.

This policy is mutually exclusive with "new features are fine as long
as they don't break anything". You can get opinions, but you cannot
get consensus.

Regards,
Martin


From python at rcn.com  Sat Sep 13 02:14:12 2003
From: python at rcn.com (Raymond Hettinger)
Date: Sat Sep 13 02:19:06 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer><1063399454.2069.299.camel@localhost.localdomain><008a01c3798a$f7198b40$b439c797@oemcomputer>
	<m3isnxxpbe.fsf@mira.informatik.hu-berlin.de>
Message-ID: <007c01c379be$40efa460$b439c797@oemcomputer>

[Raymond]
> > It has some.  That's why GvR had me bring it to python-dev
> > so you guys could help decide on the proper balance.
> > But for that one thought, the decision to apply is clear cut.

[Martin]
> There is no proper balance. If you believe in the policy "no new
> features", this policy is absolute - it would be pointless to accept
> some new features, but reject others on grounds of the policy.
> 
> This policy is mutually exclusive with "new features are fine as long
> as they don't break anything". You can get opinions, but you cannot
> get consensus.

Well put!

Even bugfixes are features in that something will work under Py2.3.1
that won't work under Py2.3.0.   So, even an absolutist policy on our
end shouldn't create a false sense of security or relieve a developer
from testing his or her application on the lowest numbered Python
for which it is claimed to run (including micro-releases).

That has always been true.  If you developed something under Py2.2.3
and expected it to run under Py2.2.0, the only way to be sure would be
to test it with Py2.2.0.

In this particular case, we are (IMHO) much better off adding an
iterable interface than leaving current situation with "db.first()"
and "while 1: db.next()" wrapped in a try/except for an 
undocumented exception class -- yuck.  Worse still, the docs
for Py2.3.0 cannot be changed and they are wrong on several
counts (the methods return items instead of keys; EOF is signaled
by an exception instead of None; and the example doesn't run).


Raymond


 


#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

From arigo at tunes.org  Sat Sep 13 09:46:53 2003
From: arigo at tunes.org (Armin Rigo)
Date: Sat Sep 13 09:48:38 2003
Subject: [Python-Dev] pre-PEP: Resource-Release Support for Generators
In-Reply-To: <200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
References: <5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch>
	<00a401c36c0f$76952160$c304a044@oemcomputer>
	<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
Message-ID: <20030913134653.GA12752@vicky.ecs.soton.ac.uk>

Hello,

(My reply didn't seem to have reached python-dev... second try.)

On Tue, Aug 26, 2003 at 01:50:52PM -0700, Guido van Rossum wrote:
> Another comment on Samuele's PEP: It is sort of sad that the *user* of
> a generator has to know that the generator's close() must be called.
> Normally, the beauty of using a try/finally for cleanup is that your
> callers don't need to know about it.  But I see no way around this.

What about letting the 'for' handle this? It's the most common way generators
are used. When a 'for' loop on a generator-iterator finishes it would call the
close() method of the iterator, which on generators would (say) simulate a 
return from the latest 'yield'.

Well, close() might not be such a good name because it would probably break
exsiting code (e.g. closing files unexpectedly), but __exit__() might do. In
other words we could import some of the proposed functionality of the 'with'
keyword (PEP 310) into 'for'. I think it makes sense because 'for' is already
defined in term of implicit calls to next(), the only method of iterators; so
if a second method is added, 'for' can be taught about it too.


Armin


From pedronis at bluewin.ch  Sat Sep 13 10:18:51 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Sat Sep 13 10:17:11 2003
Subject: [Python-Dev] pre-PEP: Resource-Release Support for
  Generators
In-Reply-To: <20030913134653.GA12752@vicky.ecs.soton.ac.uk>
References: <200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
	<5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch>
	<00a401c36c0f$76952160$c304a044@oemcomputer>
	<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
Message-ID: <5.2.1.1.0.20030913160918.02773bc0@pop.bluewin.ch>

At 14:46 13.09.2003 +0100, Armin Rigo wrote:
>Hello,
>
>(My reply didn't seem to have reached python-dev... second try.)
>
>On Tue, Aug 26, 2003 at 01:50:52PM -0700, Guido van Rossum wrote:
> > Another comment on Samuele's PEP: It is sort of sad that the *user* of
> > a generator has to know that the generator's close() must be called.
> > Normally, the beauty of using a try/finally for cleanup is that your
> > callers don't need to know about it.  But I see no way around this.
>
>What about letting the 'for' handle this? It's the most common way generators
>are used. When a 'for' loop on a generator-iterator finishes it would call the
>close() method of the iterator, which on generators would (say) simulate a
>return from the latest 'yield'.
>
>Well, close() might not be such a good name because it would probably break
>exsiting code (e.g. closing files unexpectedly), but __exit__() might do. In
>other words we could import some of the proposed functionality of the 'with'
>keyword (PEP 310) into 'for'. I think it makes sense because 'for' is already
>defined in term of implicit calls to next(), the only method of iterators; so
>if a second method is added, 'for' can be taught about it too.

I expect generators to grow also a __exit__ if they grow a close and PEP 
310 is accepted,
this idiom will then be common

with g = gen()
  for v in g:
   ...

but it will be common for files and file-like too. It can be reasonable to 
discuss
whether we want a special syntax then that conflates 'with' and 'for' behavior.
But we can experiment a while before seeing whether writing the idiom is 
unbearably
tedious ;).

OTOH conflating 'with' and 'for' just for generators seems a rather ad-hoc 
breaking of
orthoganility of the two, you could not write anymore code like this:

g = gen()
for v in g:
   ... do something up to a point ...
...
for v in g:
   ...

now this is rare but still breaking orthoganility of primitives is 
something I would think twice about.

regards. 


From pedronis at bluewin.ch  Sat Sep 13 10:23:20 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Sat Sep 13 10:21:40 2003
Subject: [Python-Dev] pre-PEP: Resource-Release Support for
  Generators
In-Reply-To: <5.2.1.1.0.20030913160918.02773bc0@pop.bluewin.ch>
References: <20030913134653.GA12752@vicky.ecs.soton.ac.uk>
	<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
	<5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch>
	<00a401c36c0f$76952160$c304a044@oemcomputer>
	<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
Message-ID: <5.2.1.1.0.20030913162229.026d3248@pop.bluewin.ch>


>
>
>OTOH conflating 'with' and 'for' just for generators seems a rather ad-hoc 
>breaking of
>orthoganility of the two, you could not write anymore code like this:

sorry, hmm, orthogonality that I know what is is.


From worktogether at runbox.com  Sat Sep 13 18:42:54 2003
From: worktogether at runbox.com (Dr. Newell)
Date: Sat Sep 13 18:42:52 2003
Subject: [Python-Dev] To  You  as  a  Nicer  Person
Message-ID: <417-220039613224254410@runbox.com>

Dear python-dev:

You may ask yourself
      >    How to make people like you and respect you
      >    How to win friends
      >    How to let your conduct help your health, work, job, career, success, relationships, spirit,  mind, well-being, ...
      >    How to make your life smoother and happier 
      >    How to do whatever you like without being unpleasant to other people 
      >    How to develop good conduct in your children or students 
      >    How to make the world peaceful and better

You can find all the answers to these questions, and much more, in this great handbook:
 
" Complete Conduct Principles for the 21st Century " by Dr. John Newton 

It is the best educational GIFT idea for children, friends, relatives, classmates, students, parents, teachers, other educators, ..., particularly at this special time. 

BENEFITS to each individual reader: Many! -- such as for health, work, job, career, success, self-improvement, education, relationships, spirit, mind, well-being, and much more -- almost all the areas that are important to you in the 21st century. People around you will benefit, too. (Please see the preface of the book for details.) 

EVERYONE may find this handbook useful and helpful, regardless of age (from children to oldsters), occupation, rank, status, gender, religious belief, race, nationality, country, or region. 

If you are a parent or a teacher, you can learn how to develop good conduct in your children or students from this handbook. Please advise your children or students to read the book. It will result in great benefits for both you and them. 

This book is a must for EVERYONE to be better prepared for personal conduct for the rest of the 21st century.

The book's content is obvious from its title. The complete useful conduct principles cover not only what we should do, but also what we should not do -- especially those faults people make often and easily. 

This timely, unique, and very important handbook is designed to suit most people, and is self-contained and user-friendly. 

The book was also praised as "a compendium of concisely expressed, practical, informative, pertinent, workable advice" by Michael J. Carson, a professional book reviewer.

This book is significantly different and better than competitive works.

"Unlike most books of this subject, it is NOT a religious book, nor is a collection of old conduct rules." 

Some of its innovative contents may help solve problems that the Western culture cannot. 

The book's merit and importance have been recognized and praised by many experts, elected public officials, and world leaders.

As a result of popular demand, the book has been a top "Featured Item" in some top on-line bookstores in some subjects, such as "Personal Practical Guides" in "Nonfiction" and "Reference", "Conduct" in "Health, Mind & Body", "Life skills guides", "Ethics & Moral Philosophy", ...   

"The book will also be effective for violence prevention for the whole society." said some experts.

How to make the world peaceful and better ---
You can find the solution in the book.
Let's work together to make the world peaceful and better! 

The author, John Newton, holds a Ph.D. from MIT, and does researches at Harvard. His long-term research on "The personal conduct in the human society of the 21st century" resulted in this book. 

Before the human beings went into the 21st century, the compassionate, merciful, courageous and farsighted Dr. Newton had issued a number of warning predictions, some of which have already been proved in the new century.

The book is published by NCWO, headquartered beside Harvard University and MIT, two leading institutes of new knowledge and literature. The publisher has a deep commitment to publishing only the best.

NCWO is an educational, non-profit, non-partisan, and honorary organization; it endeavors to make the 21st century nicer than ever before. To accomplish its mission, NCWO is proud to introduce this book.

The Web site of NCWO has been chosen as one of "Top Non Profit Sites Chosen by Type Non Profit Editors".

The book is available in two types of binding: Hardcover (ISBN 0967370574; case bound, Smyth sewn; with dust jacket) and Paperback (ISBN 0967370582; perfect bound). Both editions are unabridged, and are printed on 60 lb, natural, acid-free, excellent and healthful paper. You can get the book from many fine on-line bookstores and traditional bookstores. 

For your convenience, (if you wish for an Internet link; otherwise you may skip this section)  I herewith provide you with a link directly to the book page of each edition in Half.com by eBay, a popular on-line discount mall:

for  paperback:
http://half.ebay.com/cat/buy/prod.cgi?cpid=2425993

for hardcover:
http://half.ebay.com/cat/buy/prod.cgi?cpid=2425992


Please forward this e-mail to people you know -- children, friends, relatives, classmates, students, parents, teachers, other educators, ..., because they can benefit from it, too. This can be a wonderful kindness you provide to them!


Sincerely yours,
(python-dev, best wishes to you!)
Christopher Newell, Ph.D.
Cambridge, Massachusetts, USA

P.S.
Some educational units, ranging from the level of nation or state to individual school or university, have ordered the book as textbook, reference book, gift to students, or as an active action to prevent school violence, to improve education and to benefit students, teachers & parents. 

To have more people benefit from the book, please consider suggesting to the schools -- your children attend, or you yourself attend, have attended before, teach at, or serve -- that the book be used for fundraising for the schools. The book is an ideal fundraising tool. For example, it may used as a premium or a re-sale product for the fundraising. The successful fundraising will significantly help school education. Better yet, each supporter and his/her family will benefit from the book. Suggesting to the parent-teacher associations (organizations) (PTA/PTO) of the schools is also a good idea.



From raymond.hettinger at verizon.net  Sat Sep 13 21:05:18 2003
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sat Sep 13 21:27:13 2003
Subject: [Python-Dev] SF #805304:  bug or feature?
Message-ID: <000201c37a5e$aa6b9e80$ed24c797@oemcomputer>

Fred reported that super objects don't respond well to language constructs
that bypass attribute lookup.   For example, supobj.__setitem__(name,value) 
works if __setitem__() is defined in the target, but supobj[name]=value
will raise a TypeError.      www.python.org/sf/805304

Once the fix is approved, there is a question of whether it should
be backported.  The case against it is that some programs written 
under Py2.3.1 or Py2.2.4 won't run on Py2.3.0 or Py2.2.3 and the 
patch can be viewed as an API expansion.  The other point of view
is that super objects should have always behaved this way and that
the patch just fixes buggy behavior.


Raymond 





From skip at mojam.com  Sun Sep 14 08:00:34 2003
From: skip at mojam.com (Skip Montanaro)
Date: Sun Sep 14 08:00:43 2003
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200309141200.h8EC0Yi25498@manatee.mojam.com>


Bug/Patch Summary
-----------------

475 open / 4113 total bugs (+10)
193 open / 2371 total patches (+8)

New Bugs
--------

cgi.CGIHTTPRequestHandler remembers QUERY_STRING (2003-09-07)
	http://python.org/sf/801992
Mode argument of dumbdbm does not work (2003-09-07)
	http://python.org/sf/802128
tkFont may reuse font names (2003-09-08)
	http://python.org/sf/802310
Bogus use of Tkinter.PhotoImage can give sig11 (2003-09-08)
	http://python.org/sf/802417
uu.decode prints to stderr (2003-09-09)
	http://python.org/sf/803413
sgmllib doesn't support hex or Unicode character references (2003-09-09)
	http://python.org/sf/803422
plat-mac/applesingle.py needs cosmetic changes (2003-09-09)
	http://python.org/sf/803498
urllib/urllib2(?) timeouts (2003-09-10)
	http://python.org/sf/803634
Crypto terminology for crypto hash function (2003-09-10)
	http://python.org/sf/804113
invalid use of setlocale (2003-09-11)
	http://python.org/sf/804543
PyUnicode_FromEncodedObject (2003-09-12)
	http://python.org/sf/805015
Inappropriate error received using socket timeout (2003-09-12)
	http://python.org/sf/805194
_tkinter compilation fails (2003-09-12)
	http://python.org/sf/805200
super instances don't support item assignment (2003-09-12)
	http://python.org/sf/805304
IDLE can't save UTF-8 files on Windows (2003-09-13)
	http://python.org/sf/805728
Minor Python Intro update (2003-09-13)
	http://python.org/sf/805788

New Patches
-----------

Suggested change in how ntpath.expanduser works (2003-09-07)
	http://python.org/sf/802159
better parser error message for non-EOL following line cont. (2003-09-07)
	http://python.org/sf/802188
[_ssl.c] SSL_write() called with -1 as size (2003-09-10)
	http://python.org/sf/803998
A ForwardingHandler for logging (2003-09-10)
	http://python.org/sf/804180
unittest's main don't handle TestSuite in command line (2003-09-10)
	http://python.org/sf/804212
Lib/email/Encoders.py iso-2022 is 7bit (2003-09-11)
	http://python.org/sf/804885
fix --without-threads (2003-09-13)
	http://python.org/sf/805604
PTH related fixes (2003-09-13)
	http://python.org/sf/805613
make test_fcntl 64bit clean (2003-09-13)
	http://python.org/sf/805626
Fix linking with shared libpython (2003-09-13)
	http://python.org/sf/805678
Highlight builtins (2003-09-13)
	http://python.org/sf/805830

Closed Bugs
-----------

Add docs for 'basestring' (2002-05-24)
	http://python.org/sf/560286
htmllib.HTMLParser.anchorlist problem (2003-03-28)
	http://python.org/sf/711632
2 problems with IDLE (2003-07-21)
	http://python.org/sf/775061
RESET_ERROR is not defined(logging module) (2003-08-21)
	http://python.org/sf/792649
select module doesn't allow any iterable. (2003-08-31)
	http://python.org/sf/798046
test_dumbdbm failing (2003-09-04)
	http://python.org/sf/800824
Making "|" directive from REs a bit clearer (2003-09-04)
	http://python.org/sf/800899
Bad RE in scanf example (2003-09-05)
	http://python.org/sf/801306

Closed Patches
--------------

Allow os.access to handle Unicode file name (2003-08-17)
	http://python.org/sf/790000
implement htmllib.HTMLParser.reset (2003-08-22)
	http://python.org/sf/793021
zipimport 64-bit fix (2003-09-06)
	http://python.org/sf/801821

From martin at v.loewis.de  Sun Sep 14 18:08:57 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sun Sep 14 18:09:10 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <007c01c379be$40efa460$b439c797@oemcomputer>
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
	<1063399454.2069.299.camel@localhost.localdomain>
	<008a01c3798a$f7198b40$b439c797@oemcomputer>
	<m3isnxxpbe.fsf@mira.informatik.hu-berlin.de>
	<007c01c379be$40efa460$b439c797@oemcomputer>
Message-ID: <m3wucbc9ae.fsf@mira.informatik.hu-berlin.de>

"Raymond Hettinger" <python@rcn.com> writes:

> In this particular case, we are (IMHO) much better off adding an
> iterable interface than leaving current situation with "db.first()"
> and "while 1: db.next()" wrapped in a try/except for an 
> undocumented exception class -- yuck.  

As nobody has protested against this specific change (yet?), I guess
you can go ahead and apply it. In doing so, you have to accept the
burden of taking blame in case this turns out to be a bad idea.

> Worse still, the docs for Py2.3.0 cannot be changed and they are
> wrong on several counts (the methods return items instead of keys;
> EOF is signaled by an exception instead of None; and the example
> doesn't run).

That would be a documentation bug. However, documentation bugs are
generally best fixed by correcting the documentation, not by adjusting
the code to match the documentation (atleast, for maintenance
releases).

Regards,
Martin


From martin at v.loewis.de  Sun Sep 14 18:12:42 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sun Sep 14 18:12:42 2003
Subject: [Python-Dev] SF #805304:  bug or feature?
In-Reply-To: <000201c37a5e$aa6b9e80$ed24c797@oemcomputer>
References: <000201c37a5e$aa6b9e80$ed24c797@oemcomputer>
Message-ID: <m3smmzc945.fsf@mira.informatik.hu-berlin.de>

"Raymond Hettinger" <raymond.hettinger@verizon.net> writes:

> Once the fix is approved, there is a question of whether it should
> be backported.  The case against it is that some programs written 
> under Py2.3.1 or Py2.2.4 won't run on Py2.3.0 or Py2.2.3 and the 
> patch can be viewed as an API expansion.  The other point of view
> is that super objects should have always behaved this way and that
> the patch just fixes buggy behavior.

Whether it is a bug or not can be best clarified by looking at the
documentation: If it is documented that attributes of super objects
always follow MRO, then there is a bug. That might be a doc bug, but

Special cases aren't special enough to break the rules.

makes clear it wouldn't be a doc bug, but a bug in the code.

OTOH, if this is not documented at all, you clearly have a doc bug.

Regards,
Martin


From jeremy at alum.mit.edu  Sun Sep 14 18:21:15 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Sun Sep 14 18:21:33 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <m3wucbc9ae.fsf@mira.informatik.hu-berlin.de>
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
	<1063399454.2069.299.camel@localhost.localdomain>
	<008a01c3798a$f7198b40$b439c797@oemcomputer>
	<m3isnxxpbe.fsf@mira.informatik.hu-berlin.de>
	<007c01c379be$40efa460$b439c797@oemcomputer>
	<m3wucbc9ae.fsf@mira.informatik.hu-berlin.de>
Message-ID: <1063578075.2093.67.camel@localhost.localdomain>

On Sun, 2003-09-14 at 18:08, Martin v. L?wis wrote:
> "Raymond Hettinger" <python@rcn.com> writes:
> 
> > In this particular case, we are (IMHO) much better off adding an
> > iterable interface than leaving current situation with "db.first()"
> > and "while 1: db.next()" wrapped in a try/except for an 
> > undocumented exception class -- yuck.  
> 
> As nobody has protested against this specific change (yet?), I guess
> you can go ahead and apply it. In doing so, you have to accept the
> burden of taking blame in case this turns out to be a bad idea.

Perhaps we could wait to hear from Jack or Just or someone who was
concerned about feature creep in maintenance releases.  I think we
should decide now whether to follow the policy they propose or we should
drop it.  Right now, it feels like it's in limbo.

Jeremy



From tim.one at comcast.net  Sun Sep 14 18:36:09 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Sep 14 18:37:57 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <1063578075.2093.67.camel@localhost.localdomain>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEDIFOAB.tim.one@comcast.net>

[Jeremy]
> Perhaps we could wait to hear from Jack or Just or someone who was
> concerned about feature creep in maintenance releases.  I think we
> should decide now whether to follow the policy they propose or we
> should drop it.  Right now, it feels like it's in limbo.

Guido needs to set the direction here.  Slamming new features into micro
releases became irresistible after the 1.5.2 experience, where a very long
time passed without a new Python release (not even a bugfix release).  That
naturally caused people to want to get every improvement in ASAP.

We were able to counteract that successfully in the early days of
PythonLabs, because then we pumped out two minor releases per year, and it
was much easier to sell "leave the micro release a pure bugfix release --
the next minor release is only a few months away".

Then 2.3 stretched out over 18 months, and there's no plan for 2.4 in
evidence yet.  Given this most recent history, and the unlikelihood that
PythonLabs's (or a workalike equivalent's) early days will repeat itself
soon, if I had a minor feature I really wanted to get in, I'd try to get it
into a 2.3 micro release.

OTOH, if Guido decides to go back to a schedule-driven release for 2.4, and
it's not terribly far in the future, then it will again be much easier to
sell 2.3 micro releases as bugfix-only (and stick to that).

We're all pretty creative about what we'll call "a bug", when it suits a
feature we believe in <0.7 wink>.


From niemeyer at conectiva.com  Sun Sep 14 20:06:37 2003
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Sun Sep 14 20:07:14 2003
Subject: [Python-Dev] datetime issues
Message-ID: <20030915000637.GC27698@ibook>

Greetings!

Is there any reason for not having a single keyword parameter for
every function accepting a tzinfo instance? I've noticed that some
require 'tz', and others 'tzinfo'. Is it too late to fix that?

Also, is there any further work going on to improve the datetime module?
I'm not suggesting we should mirror mx.DateTime functionalities, but two
features I miss from mx.DateTime is the DateFrom(), which does its best
to parse a given string, and the RelativeDateTime() functionality.

This has probably been raised in the past, so if that's the case, I'm
sorry. Marc-Andre, what's your opinion about reusing code from
mx.DateTime into datetime?

Anyway, reusing code or not, I'll probably put sometime on it in the
future, if this looks interesting to everyone (after I finish my current
python pendencies, like SRE's recursivity removal).

-- 
Gustavo Niemeyer
http://niemeyer.net

From tim.one at comcast.net  Sun Sep 14 20:35:35 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Sep 14 20:37:26 2003
Subject: [Python-Dev] datetime issues
In-Reply-To: <20030915000637.GC27698@ibook>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEEEFOAB.tim.one@comcast.net>

[Gustavo Niemeyer]
> Is there any reason for not having a single keyword parameter for
> every function accepting a tzinfo instance?

Just the lack of anyone pointing it out in the module's 18-month review
process <wink>.

> I've noticed that some require 'tz', and others 'tzinfo'. Is it too
> late to fix that?

Of course, although it's not too late to add synonyms.  It appears that the
functions "with a lot of arguments" use tzinfo, and those with only one or
two arguments tz now.  I doubt anyone would bother to use the tz keyword-arg
name for a few-argument function (now(), fromtimestamp(), astimezone()), so
I'd rather that those few grow a tzinfo synonym.  tz can be deprecated too,
if you're bold.

> Also, is there any further work going on to improve the datetime
> module?

Not that I'm aware of.  I'm not working on it anymore (other than keeping my
out for bug reports, of which there have been blessedly few).

> I'm not suggesting we should mirror mx.DateTime functionalities, but
> two features I miss from mx.DateTime is the DateFrom(), which does its
> best to parse a given string, and the RelativeDateTime() functionality.
>
> This has probably been raised in the past, so if that's the case, I'm
> sorry. Marc-Andre, what's your opinion about reusing code from
> mx.DateTime into datetime?
>
> Anyway, reusing code or not, I'll probably put sometime on it in the
> future, if this looks interesting to everyone (after I finish my
> current python pendencies, like SRE's recursivity removal).

If Guido wanted to stay out of the relatively clear <snort> time zone
business, I can imagine his interest in trying to guess how to parse date
and time strings.  Apart from Brett's strptime(), I believe the email module
has some useful string->datetime code.  It may be good to fold such stuff in
this area as Python already supports into datetime somehow.


From arigo at tunes.org  Mon Sep 15 06:48:36 2003
From: arigo at tunes.org (Armin Rigo)
Date: Mon Sep 15 06:50:28 2003
Subject: [Python-Dev] pre-PEP: Resource-Release Support for Generators
In-Reply-To: <5.2.1.1.0.20030913160918.02773bc0@pop.bluewin.ch>
References: <200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
	<5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch>
	<00a401c36c0f$76952160$c304a044@oemcomputer>
	<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
	<5.2.1.1.0.20030913160918.02773bc0@pop.bluewin.ch>
Message-ID: <20030915104836.GA1143@vicky.ecs.soton.ac.uk>

Hello Samuele,

On Sat, Sep 13, 2003 at 04:18:51PM +0200, Samuele Pedroni wrote:
> OTOH conflating 'with' and 'for' just for generators seems a rather ad-hoc 
> breaking of
> orthoganility of the two, you could not write anymore code like this:
> 
> g = gen()
> for v in g:
>   ... do something up to a point ...
> ...
> for v in g:
>   ...

I had thought about this. This occurs when you 'break' out of the first loop. 
I would say that NOT calling the __exit__() method in this specific case seems
quite intuitive, the 'break' meaning 'just exit from the loop now without any
further processing, skipping the 'else' part if present'.


Armin


From sholden at holdenweb.com  Mon Sep 15 07:59:36 2003
From: sholden at holdenweb.com (Steve Holden)
Date: Mon Sep 15 08:03:00 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEDIFOAB.tim.one@comcast.net>
Message-ID: <CGECIJPNNHIFAJKHOLMACEKLHAAA.sholden@holdenweb.com>

> -----Original Message-----
> From: python-dev-bounces@python.org
> [mailto:python-dev-bounces@python.org]On Behalf Of Tim Peters
> Sent: Sunday, September 14, 2003 6:36 PM
> To: jeremy@alum.mit.edu; Martin v. L?wis
> Cc: Raymond Hettinger; python-dev@python.org
> Subject: RE: [Python-Dev] python/dist/src/Lib/bsddb
> __init__.py,1.5,1.6
>
>
> [Jeremy]
> > Perhaps we could wait to hear from Jack or Just or someone who was
> > concerned about feature creep in maintenance releases.  I think we
> > should decide now whether to follow the policy they propose or we
> > should drop it.  Right now, it feels like it's in limbo.
>
> Guido needs to set the direction here.  Slamming new features
> into micro
> releases became irresistible after the 1.5.2 experience,
> where a very long
> time passed without a new Python release (not even a bugfix
> release).  That
> naturally caused people to want to get every improvement in ASAP.
>
> We were able to counteract that successfully in the early days of
> PythonLabs, because then we pumped out two minor releases per
> year, and it
> was much easier to sell "leave the micro release a pure
> bugfix release --
> the next minor release is only a few months away".
>
> Then 2.3 stretched out over 18 months, and there's no plan for 2.4 in
> evidence yet.  Given this most recent history, and the
> unlikelihood that
> PythonLabs's (or a workalike equivalent's) early days will
> repeat itself
> soon, if I had a minor feature I really wanted to get in, I'd
> try to get it
> into a 2.3 micro release.
>
> OTOH, if Guido decides to go back to a schedule-driven
> release for 2.4, and
> it's not terribly far in the future, then it will again be
> much easier to
> sell 2.3 micro releases as bugfix-only (and stick to that).
>
> We're all pretty creative about what we'll call "a bug", when
> it suits a
> feature we believe in <0.7 wink>.
>
So, what we need is a plan for a 2.4 release, then. How soon would it
need to be, though? And who can commit time to making it happen. Seems
to me that some formerly active core developers are currently busy in
other realms. Is it feasible to have a 2.4 plan, or are we ready to
contemplate (gasp) 3.0? What help can such as I, not a practiced C coder
and fairly heavily committed and thinly spread, be?

regards
--
Steve Holden                                 http://www.holdenweb.com/
Python Web Programming                http://pydish.holdenweb.com/pwp/
Interview with GvR August 14, 2003       http://www.onlamp.com/python/




From mcherm at mcherm.com  Mon Sep 15 08:04:51 2003
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon Sep 15 08:04:56 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
Message-ID: <1063627491.3f65aae37a3b7@mcherm.com>

[Jeremy]
> Isn't this just the sort of little feature that was the subject of
> recent discussion on the dangers of backporting.  It would mean that
> someone writing an app against Python 2.3 could never be sure whether
> the feature existed.  In practice, that would mean developers using
> 2.3.x would only find out about problems after deployment when they bump
> into a user with the original 2.3.
> 
> I'm not sure how convincing I find this argument, but it's got some
> merit.

[Martin]
> There is no proper balance. If you believe in the policy "no new
> features", this policy is absolute - it would be pointless to accept
> some new features, but reject others on grounds of the policy.

I don't really understand this absolutist position. Suppose (hypothetical
example made extreme for retorical purposes) we discover that there was
a HUGE bug in release 2.3. Using a variable named "Q" as the argument of
a for loop causes the interpreter to seg-fault. This is CLEARLY a bad 
thing... capital one-letter variable names are uncommon, but not THAT 
uncommon! So I suppose that 2.3.1 would need to include a fix for this 
bug.

But THEN anyone writing code for 2.3.x could never be SURE whether the
bug existed. If they used a capital Q in a for loop, their code would
eventually be run by somebody with 2.3.0, causing a crash. The only way
to be safe would be to test their code on 2.3.0. How is this "bug fix"
any different from a "new feature"?

I understand the argument "avoid new features in maintanence releases
because there we value stability FAR more than we value features". I
understand the argument "avoid new features in maintanence releases
because features should be added on the main trunk to encourage the
(eventual) use of new versions of Python". But I don't understand
the argument "PROHIBIT new features in maintanence releases because
then programs written under the maintanence release might not run
under the original release", because to me it seems to be logically
equivalent to "prohibit maintanence releases".

[Martin]
> This policy is mutually exclusive with "new features are fine as long
> as they don't break anything". You can get opinions, but you cannot
> get consensus.

If "consensus" is taken to mean "universal agreement", then I'd agree,
but if we mean "general agreement or accord" (ie, everyone gets heard
then NEARLY everyone agrees and the naysayers decide they can live
with it), then we might have a shot. I'd certainly say (and I think
most here would agree) that even ONE seriously intended firmly held 
argument against a particular new feature (in a maint. release) would 
make me think VERY carefully before allowing that feature.

-- Michael Chermside


From amk at amk.ca  Mon Sep 15 08:42:58 2003
From: amk at amk.ca (A.M. Kuchling)
Date: Mon Sep 15 08:43:06 2003
Subject: [Python-Dev] Re: datetime issues
References: <20030915000637.GC27698@ibook>
	<LNBBLJKPBEHFEDALKOLCMEEEFOAB.tim.one@comcast.net>
Message-ID: <oprvjj9wplqmnqtt@news.gmane.org>

On Sun, 14 Sep 2003 20:35:35 -0400, Tim Peters <tim.one@comcast.net> wrote:
> If Guido wanted to stay out of the relatively clear <snort> time zone
> business, I can imagine his interest in trying to guess how to parse date
> and time strings.  Apart from Brett's strptime(), I believe the email 
> module
> has some useful string->datetime code.

The PyXML package includes a parser (xml.utils.iso8601) for the date/time 
format most commonly used in XML applications.  That and a parser for 
RFC822 dates would meet the needs of many technical applications (though 
not ones where users can enter dates -- but that problem may be too 
complicated for the stdlib).  I'll happily convert the iso8601 module for 
addition to the standard library if date parsing is deemed a topic of 
interest.

--amk


From amk at amk.ca  Mon Sep 15 08:51:02 2003
From: amk at amk.ca (A.M. Kuchling)
Date: Mon Sep 15 08:51:19 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
References: <LNBBLJKPBEHFEDALKOLCCEDIFOAB.tim.one@comcast.net>
	<CGECIJPNNHIFAJKHOLMACEKLHAAA.sholden@holdenweb.com>
Message-ID: <oprvjknc03qmnqtt@news.gmane.org>

On Mon, 15 Sep 2003 07:59:36 -0400, Steve Holden <sholden@holdenweb.com> 
wrote:
> other realms. Is it feasible to have a 2.4 plan, or are we ready to
> contemplate (gasp) 3.0?

I'd be in favor of setting an arbitrary date for a 2.4 release, and when 
that date rolls around stabilizing and releasing the CVS tree, no matter 
what's been implemented or not implemented.  If we've made major language 
revisions by that time, fine, but it's worthwhile even if 2.4 is just 2.3 + 
(various bugfixes deemed too risky for 2.3.1) + (a few new modules).

PEP 320 estimates February 2005, a 19-month gap matching that between 2.2 
and 2.3.  That seems a bit too long, especially if there are no large 
changes that require testing; aiming for autumn 2004, beginning to freeze 
in August, seems reasonable.

--amk


From barry at python.org  Mon Sep 15 09:10:22 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Sep 15 09:10:33 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <oprvjknc03qmnqtt@news.gmane.org>
References: <LNBBLJKPBEHFEDALKOLCCEDIFOAB.tim.one@comcast.net>
	<CGECIJPNNHIFAJKHOLMACEKLHAAA.sholden@holdenweb.com>
	<oprvjknc03qmnqtt@news.gmane.org>
Message-ID: <1063631421.10716.2.camel@anthem>

On Mon, 2003-09-15 at 08:51, A.M. Kuchling wrote:

> I'd be in favor of setting an arbitrary date for a 2.4 release, and when 
> that date rolls around stabilizing and releasing the CVS tree, no matter 
> what's been implemented or not implemented.  If we've made major language 
> revisions by that time, fine, but it's worthwhile even if 2.4 is just 2.3 + 
> (various bugfixes deemed too risky for 2.3.1) + (a few new modules).
> 
> PEP 320 estimates February 2005, a 19-month gap matching that between 2.2 
> and 2.3.  That seems a bit too long, especially if there are no large 
> changes that require testing; aiming for autumn 2004, beginning to freeze 
> in August, seems reasonable.

+1

-Barry



From barry at python.org  Mon Sep 15 09:14:17 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Sep 15 09:14:22 2003
Subject: [Python-Dev] Re: datetime issues
In-Reply-To: <oprvjj9wplqmnqtt@news.gmane.org>
References: <20030915000637.GC27698@ibook>
	<LNBBLJKPBEHFEDALKOLCMEEEFOAB.tim.one@comcast.net>
	<oprvjj9wplqmnqtt@news.gmane.org>
Message-ID: <1063631656.10716.7.camel@anthem>

On Mon, 2003-09-15 at 08:42, A.M. Kuchling wrote:

> I believe the email 
> > module
> > has some useful string->datetime code.
> 
> The PyXML package includes a parser (xml.utils.iso8601) for the date/time 
> format most commonly used in XML applications.  That and a parser for 
> RFC822 dates would meet the needs of many technical applications (though 
> not ones where users can enter dates -- but that problem may be too 
> complicated for the stdlib).  I'll happily convert the iso8601 module for 
> addition to the standard library if date parsing is deemed a topic of 
> interest.

Yep, I think email._parseaddr.parsedate{,_tz}() is an RFC 2822 compliant
date parser.  Note that there's also Unix-from dates which are slightly
different.

I'd also like to see factored out various date formating functions, so
that the module is more or less symmetric. 

-Barry



From nas-python at python.ca  Mon Sep 15 10:47:12 2003
From: nas-python at python.ca (Neil Schemenauer)
Date: Mon Sep 15 10:39:57 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <oprvjknc03qmnqtt@news.gmane.org>
References: <LNBBLJKPBEHFEDALKOLCCEDIFOAB.tim.one@comcast.net>
	<CGECIJPNNHIFAJKHOLMACEKLHAAA.sholden@holdenweb.com>
	<oprvjknc03qmnqtt@news.gmane.org>
Message-ID: <20030915144712.GA7448@mems-exchange.org>

On Mon, Sep 15, 2003 at 08:51:02AM -0400, A.M. Kuchling wrote:
> I'd be in favor of setting an arbitrary date for a 2.4 release, and when 
> that date rolls around stabilizing and releasing the CVS tree, no matter 
> what's been implemented or not implemented.

+1

From tim.one at comcast.net  Mon Sep 15 10:44:07 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Sep 15 10:44:18 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <oprvjknc03qmnqtt@news.gmane.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEGNFOAB.tim.one@comcast.net>

[Andrew Kuchling]
> I'd be in favor of setting an arbitrary date for a 2.4 release, and
> when that date rolls around stabilizing and releasing the CVS tree,
> no matter what's been implemented or not implemented.  If we've made
> major language revisions by that time, fine, but it's worthwhile even
> if 2.4 is just 2.3 + (various bugfixes deemed too risky for 2.3.1) +
> (a few new modules).
>
> PEP 320 estimates February 2005, a 19-month gap matching that between
> 2.2 and 2.3.  That seems a bit too long, especially if there are no
> large changes that require testing; aiming for autumn 2004, beginning
> to freeze in August, seems reasonable.

I'd like to see 2.4 released no later than 29-Jul-2004.  That's a year after
2.3 got released.  I don't have nearly as many years left to me as most of
you do <wink>.


From barry at python.org  Mon Sep 15 10:55:31 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Sep 15 10:55:37 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEGNFOAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCKEGNFOAB.tim.one@comcast.net>
Message-ID: <1063637731.10716.25.camel@anthem>

On Mon, 2003-09-15 at 10:44, Tim Peters wrote:

> I'd like to see 2.4 released no later than 29-Jul-2004.

I like the tradition this sets.  I can't think of a better birthday
present to one of the best Canadian bassists in the world.

-Barry



From jeremy at alum.mit.edu  Mon Sep 15 10:57:02 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon Sep 15 10:56:56 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <oprvjknc03qmnqtt@news.gmane.org>
References: <LNBBLJKPBEHFEDALKOLCCEDIFOAB.tim.one@comcast.net>
	<CGECIJPNNHIFAJKHOLMACEKLHAAA.sholden@holdenweb.com>
	<oprvjknc03qmnqtt@news.gmane.org>
Message-ID: <1063637821.2093.132.camel@localhost.localdomain>

On Mon, 2003-09-15 at 08:51, A.M. Kuchling wrote:
> I'd be in favor of setting an arbitrary date for a 2.4 release, and when 
> that date rolls around stabilizing and releasing the CVS tree, no matter 
> what's been implemented or not implemented.  If we've made major language 
> revisions by that time, fine, but it's worthwhile even if 2.4 is just 2.3 + 
> (various bugfixes deemed too risky for 2.3.1) + (a few new modules).
> 
> PEP 320 estimates February 2005, a 19-month gap matching that between 2.2 
> and 2.3.  That seems a bit too long, especially if there are no large 
> changes that require testing; aiming for autumn 2004, beginning to freeze 
> in August, seems reasonable.

In the absence of any plans for major new features, I think this is a
good plan.  A rough schedule might be:

2.4a1  April
2.4a2  late May/early June
2.4b1  July 9
2.4b2  Aug. 6
2.4 f  Sep. 3

We should also make a plan for a 2.3 maintenance release.  There have
been a lot of checkins, but I haven't gotten a sense for how significant
the bugs were.

Jeremy



From neal at metaslash.com  Mon Sep 15 11:03:13 2003
From: neal at metaslash.com (Neal Norwitz)
Date: Mon Sep 15 11:03:21 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <1063637821.2093.132.camel@localhost.localdomain>
References: <LNBBLJKPBEHFEDALKOLCCEDIFOAB.tim.one@comcast.net>
	<CGECIJPNNHIFAJKHOLMACEKLHAAA.sholden@holdenweb.com>
	<oprvjknc03qmnqtt@news.gmane.org>
	<1063637821.2093.132.camel@localhost.localdomain>
Message-ID: <20030915150313.GI1240@epoch.metaslash.com>

On Mon, Sep 15, 2003 at 10:57:02AM -0400, Jeremy Hylton wrote:
> 
> We should also make a plan for a 2.3 maintenance release.  There have
> been a lot of checkins, but I haven't gotten a sense for how significant
> the bugs were.

I think 2.3 has been great!  There were a few memory leaks plugged.  
I don't even recall any crashes specific to 2.3.  I'm very happy with
the apparent robustness of 2.3.  Maybe nobody is using it? :-)

As far as the release schedule for 2.4, I haven't heard any squawking
recently about Python changing too fast.  But then again, I have no
time to read c.l.p anymore. Maybe there's a correlation? :-)

Neal


From barry at python.org  Mon Sep 15 11:08:37 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Sep 15 11:08:43 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <20030915150313.GI1240@epoch.metaslash.com>
References: <LNBBLJKPBEHFEDALKOLCCEDIFOAB.tim.one@comcast.net>
	<CGECIJPNNHIFAJKHOLMACEKLHAAA.sholden@holdenweb.com>
	<oprvjknc03qmnqtt@news.gmane.org>
	<1063637821.2093.132.camel@localhost.localdomain>
	<20030915150313.GI1240@epoch.metaslash.com>
Message-ID: <1063638517.10716.32.camel@anthem>

On Mon, 2003-09-15 at 11:03, Neal Norwitz wrote:

> I think 2.3 has been great!  There were a few memory leaks plugged.  
> I don't even recall any crashes specific to 2.3.  I'm very happy with
> the apparent robustness of 2.3.  Maybe nobody is using it? :-)

I'm using it for everything, and I agree!  It's a very solid release.  I
still think we should plan for a 2.3.1 soonish, but the hold up may just
be finding a release manager to own it.  I'd rather spread the love this
time. :) 

> As far as the release schedule for 2.4, I haven't heard any squawking
> recently about Python changing too fast.

We can't let that stand, now can we? :)  Maybe Guido can post about his
big plans for 2.4.  There's still time to get PEPpy too.

-Barry



From anthony at interlink.com.au  Mon Sep 15 11:11:35 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Sep 15 11:13:01 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6 
In-Reply-To: <1063638517.10716.32.camel@anthem> 
Message-ID: <200309151511.h8FFBZ0j029922@localhost.localdomain>


>>> Barry Warsaw wrote 
> I'm using it for everything, and I agree!  It's a very solid release.  I
> still think we should plan for a 2.3.1 soonish, but the hold up may just
> be finding a release manager to own it.  I'd rather spread the love this
> time. :) 

I've already put my hand up for it, and Raymond's offered to help. I'm thinking
about maybe starting this process late next week? 

(I've already done at least one minor release before, so I've a fair idea
of what's involved. Hey, hang on, didn't I write PEP102 for this? ;)

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From pje at telecommunity.com  Mon Sep 15 11:26:27 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Sep 15 11:26:41 2003
Subject: [Python-Dev] pre-PEP: Resource-Release Support for
  Generators
In-Reply-To: <20030915104836.GA1143@vicky.ecs.soton.ac.uk>
References: <5.2.1.1.0.20030913160918.02773bc0@pop.bluewin.ch>
	<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
	<5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch>
	<00a401c36c0f$76952160$c304a044@oemcomputer>
	<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
	<5.2.1.1.0.20030913160918.02773bc0@pop.bluewin.ch>
Message-ID: <5.1.1.6.0.20030915112135.03013ec0@telecommunity.com>

At 11:48 AM 9/15/03 +0100, Armin Rigo wrote:
>Hello Samuele,
>
>On Sat, Sep 13, 2003 at 04:18:51PM +0200, Samuele Pedroni wrote:
> > OTOH conflating 'with' and 'for' just for generators seems a rather ad-hoc
> > breaking of
> > orthoganility of the two, you could not write anymore code like this:
> >
> > g = gen()
> > for v in g:
> >   ... do something up to a point ...
> > ...
> > for v in g:
> >   ...
>
>I had thought about this. This occurs when you 'break' out of the first loop.
>I would say that NOT calling the __exit__() method in this specific case seems
>quite intuitive, the 'break' meaning 'just exit from the loop now without any
>further processing, skipping the 'else' part if present'.

Hmmm...  You realize this is also going to break the similarity between:

i = iter(something)
while 1:
     try:
         j=i.next()
     except StopIteration:
         break

and

for j in iter(something):
     pass


The former is already complex enough as a mental model.  I think it gets 
altogether too complex if one also has to consider the enter/leave 
issue.  This strikes me as an area like try/except vs. try/finally: it 
really should be a separate block, just for the sake of explicitness.  As 
much as it would be cool to have the automatic release, I think I'd rather use:

with i = iter(something):
     for j in i:
         ...

And make the resource management very visible.


From skip at pobox.com  Mon Sep 15 11:32:37 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Sep 15 11:32:49 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <1063638517.10716.32.camel@anthem>
References: <LNBBLJKPBEHFEDALKOLCCEDIFOAB.tim.one@comcast.net>
	<CGECIJPNNHIFAJKHOLMACEKLHAAA.sholden@holdenweb.com>
	<oprvjknc03qmnqtt@news.gmane.org>
	<1063637821.2093.132.camel@localhost.localdomain>
	<20030915150313.GI1240@epoch.metaslash.com>
	<1063638517.10716.32.camel@anthem>
Message-ID: <16229.56213.235834.239788@montanaro.dyndns.org>


    Barry> On Mon, 2003-09-15 at 11:03, Neal Norwitz wrote:
    >> I think 2.3 has been great!  There were a few memory leaks plugged.
    >> I don't even recall any crashes specific to 2.3.  I'm very happy with
    >> the apparent robustness of 2.3.  Maybe nobody is using it? :-)

    Barry> I'm using it for everything, and I agree!  

Ditto on my development machine, except I just cvs up periodically, so I'm
generally running from CVS as of a few days ago, not even 2.3.  I do intend
to update to 2.3 on the Mojam webserver in a short while.  That should
provide a nice speed boost over the 2.2.3 I'm currently running there.

    >> As far as the release schedule for 2.4, I haven't heard any squawking
    >> recently about Python changing too fast.

    Barry> We can't let that stand, now can we? :) 

I suspect that's simply because the focus was on library and performance
improvements instead of new language features.

    Barry> There's still time to get PEPpy too.

I would like to get back to PEP 304 (Control of pyc file location).  I got
stuck when someone raised some good objections about how it would play on
Windows with it's multiple rooted file system, but if those problems can be
ironed out I think it would be a good addition to the system.

Skip

From niemeyer at conectiva.com  Mon Sep 15 12:26:51 2003
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Mon Sep 15 12:27:32 2003
Subject: [Python-Dev] datetime issues
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEEEFOAB.tim.one@comcast.net>
References: <20030915000637.GC27698@ibook>
	<LNBBLJKPBEHFEDALKOLCMEEEFOAB.tim.one@comcast.net>
Message-ID: <20030915162651.GA30600@ibook.distro.conectiva>

> [Gustavo Niemeyer]
> > Is there any reason for not having a single keyword parameter for
> > every function accepting a tzinfo instance?
> 
> Just the lack of anyone pointing it out in the module's 18-month review
> process <wink>.

Would you belive that I was.. erm.. sleeping!? :-)

> > I've noticed that some require 'tz', and others 'tzinfo'. Is it too
> > late to fix that?
> 
> Of course, although it's not too late to add synonyms.  It appears
> that the functions "with a lot of arguments" use tzinfo, and those
> with only one or two arguments tz now.  I doubt anyone would bother to
> use the tz keyword-arg name for a few-argument function (now(),
> fromtimestamp(), astimezone()), so I'd rather that those few grow a
> tzinfo synonym.  tz can be deprecated too, if you're bold.

It'd be nice to have these synonyms. I'll put it in my ever growing
todo list.

> > Also, is there any further work going on to improve the datetime
> > module?
> 
> Not that I'm aware of.  I'm not working on it anymore (other than
> keeping my out for bug reports, of which there have been blessedly
> few).
> 
> > I'm not suggesting we should mirror mx.DateTime functionalities, but
> > two features I miss from mx.DateTime is the DateFrom(), which does
> > its best to parse a given string, and the RelativeDateTime()
> > functionality.
> >
> > This has probably been raised in the past, so if that's the case, I'm
> > sorry. Marc-Andre, what's your opinion about reusing code from
> > mx.DateTime into datetime?
> >
> > Anyway, reusing code or not, I'll probably put sometime on it in the
> > future, if this looks interesting to everyone (after I finish my
> > current python pendencies, like SRE's recursivity removal).
> 
> If Guido wanted to stay out of the relatively clear <snort> time zone
> business, I can imagine his interest in trying to guess how to parse
> date and time strings.  Apart from Brett's strptime(), I believe the
> email module has some useful string->datetime code.  It may be good to
> fold such stuff in this area as Python already supports into datetime
> somehow.

Cool. I'll give it a try as well. I have a working relativedelta
implementation now. I'll polish it a little further and publish
it somewhere for tests.

Thanks!

-- 
Gustavo Niemeyer
http://niemeyer.net

From amk at amk.ca  Mon Sep 15 12:29:11 2003
From: amk at amk.ca (A.M. Kuchling)
Date: Mon Sep 15 12:29:18 2003
Subject: [Python-Dev] Re: datetime issues
References: <20030915000637.GC27698@ibook>
	<LNBBLJKPBEHFEDALKOLCMEEEFOAB.tim.one@comcast.net>
	<oprvjj9wplqmnqtt@news.gmane.org> <1063631656.10716.7.camel@anthem>
Message-ID: <oprvjuqxnrqmnqtt@news.gmane.org>

On 15 Sep 2003 09:14:17 -0400, Barry Warsaw <barry@python.org> wrote:
> I'd also like to see factored out various date formating functions, so
> that the module is more or less symmetric.

Probably this is PEP territory.  Unless someone else wants to do it, I'll 
volunteer to produce a draft PEP for adding parsing and formatting 
functions.

--amk
 


From amk at amk.ca  Mon Sep 15 12:44:29 2003
From: amk at amk.ca (A.M. Kuchling)
Date: Mon Sep 15 12:44:37 2003
Subject: [Python-Dev] Re: Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
References: <LNBBLJKPBEHFEDALKOLCCEDIFOAB.tim.one@comcast.net>
	<CGECIJPNNHIFAJKHOLMACEKLHAAA.sholden@holdenweb.com>
	<oprvjknc03qmnqtt@news.gmane.org>
	<1063637821.2093.132.camel@localhost.localdomain>
Message-ID: <oprvjvgfe2qmnqtt@news.gmane.org>

On Mon, 15 Sep 2003 10:57:02 -0400, Jeremy Hylton <jeremy@alum.mit.edu> 
wrote:
> We should also make a plan for a 2.3 maintenance release.  There have
> been a lot of checkins, but I haven't gotten a sense for how significant
> the bugs were.

I watch the checkins for the sake of the 2.4 document, and haven't seen any 
bugfixes that seemed really critical.  No one in c.l.py seems to be running 
into any of the memory leaks that were fixed, and none of the fixed bugs 
seem likely to silently lose or corrupt data.

Is it worth holding 2.3.1 until after Panther ships, on the assumption that 
its release will turn up more users and more bugs?  If not, 2.3.1's timing 
can probably be left up to the release manager, who can make the release 
whenever he feels like it.

--amk


From niemeyer at conectiva.com  Mon Sep 15 12:50:26 2003
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Mon Sep 15 12:51:04 2003
Subject: [Python-Dev] Re: datetime issues
In-Reply-To: <oprvjj9wplqmnqtt@news.gmane.org>
References: <20030915000637.GC27698@ibook>
	<LNBBLJKPBEHFEDALKOLCMEEEFOAB.tim.one@comcast.net>
	<oprvjj9wplqmnqtt@news.gmane.org>
Message-ID: <20030915165026.GB30600@ibook.distro.conectiva>

> >If Guido wanted to stay out of the relatively clear <snort> time zone
> >business, I can imagine his interest in trying to guess how to parse date
> >and time strings.  Apart from Brett's strptime(), I believe the email 
> >module
> >has some useful string->datetime code.
> 
> The PyXML package includes a parser (xml.utils.iso8601) for the date/time 
> format most commonly used in XML applications.  That and a parser for 

Cool! Thanks for mentioning it!

> RFC822 dates would meet the needs of many technical applications (though 
> not ones where users can enter dates -- but that problem may be too 
> complicated for the stdlib). 

I do not agree. I've done a small research and there seems to be many
nice generic implementations of these parsers, which would benefit
technical applications *and* hand made dates (some of them below[1]),
and it doesn't look that complex. I'll be working to provide trial
implementation collecting the best features described in these papers in
the near future.

Btw, the relativedelta is mostly ready. That's the first step to provide
relative dates in the parser.

> I'll happily convert the iso8601 module for addition to the standard
> library if date parsing is deemed a topic of interest.

I'm not yet sure if date parsing and other improvements in the date/time
area is a topic of general interest, but at least I am interested. :-)

[1]
http://www.egenix.com/files/python/mxDateTime.html
http://ringmaster.arc.nasa.gov/tools/time_formats.html
http://www.thinkage.ca/english/gcos/expl/b/lib/0tosec.html
http://search.cpan.org/author/MUIR/Time-modules-2003.0211/lib/Time/ParseDate.pm

-- 
Gustavo Niemeyer
http://niemeyer.net

From niemeyer at conectiva.com  Mon Sep 15 12:55:14 2003
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Mon Sep 15 12:55:51 2003
Subject: [Python-Dev] Re: datetime issues
In-Reply-To: <oprvjuqxnrqmnqtt@news.gmane.org>
References: <20030915000637.GC27698@ibook>
	<LNBBLJKPBEHFEDALKOLCMEEEFOAB.tim.one@comcast.net>
	<oprvjj9wplqmnqtt@news.gmane.org> <1063631656.10716.7.camel@anthem>
	<oprvjuqxnrqmnqtt@news.gmane.org>
Message-ID: <20030915165514.GC30600@ibook.distro.conectiva>

> >I'd also like to see factored out various date formating functions, so
> >that the module is more or less symmetric.
> 
> Probably this is PEP territory.  Unless someone else wants to do it,
> I'll volunteer to produce a draft PEP for adding parsing and
> formatting functions.

Please, don't discard the generic parser in the PEP.

-- 
Gustavo Niemeyer
http://niemeyer.net

From mwh at python.net  Mon Sep 15 14:01:37 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Sep 15 14:03:04 2003
Subject: [Python-Dev] refleak status
Message-ID: <A67DCC06-E7A6-11D7-97A6-0003931DF95C@python.net>

After a week or so of frantically searching for these things, I sort of 
forgot about the reference leak hunt I started in Python's tests.  A 
week ago I ran my hacked regrtest.py again, made some notes, tried & 
failed to send them to python-dev, went on holiday, came back, digged a 
little more and am trying to send again...

real leaks i think i've fixed (am syncing up to CVS now, will test 
overnight).

     test_class leaked [66, 66, 66, 66] references
     test_hotshot leaked [111, 111, 111, 111] references

real leaks i cant fix:

     test_compile leaked [45, 45, 45, 45] references
         lambda a,a:0 worst offender.  confused.

red herrings:

     test_codeccallbacks leaked [2, 2, 2, 2] references
         callback registry
     test_mutants leaked [-1489, 1520, -755, -183] references
         randomized test

suspected bugs in _Py_RefTotal tracking:

     test_descr leaked [2, 2, 2, 2] references
         something to do with resurrection
     test_gc leaked [18, 18, 18, 18] references
         test_trashcan ... not a real leak, it seems

suspected red herrings:

     test_cpickle leaked [6, 3, 2, 1] references
     test_logging leaked [82, 82, 82, 82] references
         (bet this is registry stuff of one sort or another)
     test_threadedtempfile leaked [-74, 3, 3, 161] references
     test_unicode leaked [7, 7, 7, 7] references
         registries again

apparent refleaks that still need digging:

     test_minidom leaked [1, 1, 1, 1] references
     test_new leaked [2, 2, 2, 2] references
     test_pkg leaked [8, 8, 9, 9] references
     test_pkgimport leaked [5, 6, 6, 6] references
     test_sax leaked [1762, 1762, 1762, 1762] references

Wouldn't like to claim anything about the following tests:

6 tests failed:
     test_dircache test_doctest test_getopt test_multifile
     test_warnings test_zipimport
27 tests skipped:
     test_al test_bsddb test_bsddb3 test_cd test_cl test_curses test_dl
     test_email_codecs test_gdbm test_gl test_imgfile test_largefile
     test_linuxaudiodev test_locale test_mpz test_nis
     test_normalization test_ossaudiodev test_pep277 test_poll
     test_socket_ssl test_socketserver test_sunaudiodev test_timeout
     test_urllibnet test_winreg test_winsound
Those skips are all expected on darwin.

(test_threaded_import deadlocks, so add that to the list)

Should investigate why the failing tests fail when run repeatedly and
check that they're not hiding ref leaks.

Cheers,
mwh


From bac at OCF.Berkeley.EDU  Mon Sep 15 14:20:21 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Sep 15 14:20:36 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <1063637821.2093.132.camel@localhost.localdomain>
References: <LNBBLJKPBEHFEDALKOLCCEDIFOAB.tim.one@comcast.net>	<CGECIJPNNHIFAJKHOLMACEKLHAAA.sholden@holdenweb.com>	<oprvjknc03qmnqtt@news.gmane.org>
	<1063637821.2093.132.camel@localhost.localdomain>
Message-ID: <3F6602E5.1040300@ocf.berkeley.edu>

Jeremy Hylton wrote:

> On Mon, 2003-09-15 at 08:51, A.M. Kuchling wrote:
> 
>>I'd be in favor of setting an arbitrary date for a 2.4 release, and when 
>>that date rolls around stabilizing and releasing the CVS tree, no matter 
>>what's been implemented or not implemented.  If we've made major language 
>>revisions by that time, fine, but it's worthwhile even if 2.4 is just 2.3 + 
>>(various bugfixes deemed too risky for 2.3.1) + (a few new modules).
>>
>>PEP 320 estimates February 2005, a 19-month gap matching that between 2.2 
>>and 2.3.  That seems a bit too long, especially if there are no large 
>>changes that require testing; aiming for autumn 2004, beginning to freeze 
>>in August, seems reasonable.
> 
> 
> In the absence of any plans for major new features, I think this is a
> good plan.  A rough schedule might be:
> 
> 2.4a1  April
> 2.4a2  late May/early June
> 2.4b1  July 9
> 2.4b2  Aug. 6
> 2.4 f  Sep. 3
> 

+1

We could also try to schedule micro releases two months after the 
initial release.  That seems to have been a decent amount of time to get 
in bug reports and fix them.  Then again, standardizing micro releases 
might cause people to hold off on using the newest release and thus we 
might get less testing from users.

-Brett


From recht at netbsd.org  Mon Sep 15 17:07:59 2003
From: recht at netbsd.org (Marc Recht)
Date: Mon Sep 15 17:08:15 2003
Subject: [Python-Dev] Python and POSIX
Message-ID: <35230000.1063660079@leeloo.intern.geht.de>

Hi!

Currently Python configure process sets quite a few POSIX/XOPEN macros (eg. 
_POSIX_C_SOURCE/_XOPEN_SOURCE) to archieve portability. The is a good thing 
- for the interpreter. The problem with this is that these macros are 
exported to Python extensions via pyconfig.h. That way every extension 
which uses pyconfig.h is limited to POSIX/XOPEN functionality and not all 
library features of the OS (eg. RPC) are available for the extension. For 
the systems that supported it the Python configure process also sets macros 
which activate all library functions (_NETBSD_SOURCE, __EXTENSIONS__) and 
thus avoiding POSIX as a work-around. On all systems I'm aware of this is 
the same feature-set you get if you don't set any macro at all, but 
unfortunately not all systems support such a macro.
So, why not split pyconfig.h up into an external and internal header? That 
way the the interpreter could be POSIX confirming (which is indeed a good 
thing) without enforcing POSIX to the extensions.

Regards,
Marc

From martin at v.loewis.de  Mon Sep 15 17:58:14 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Mon Sep 15 18:02:57 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <1063627491.3f65aae37a3b7@mcherm.com>
References: <1063627491.3f65aae37a3b7@mcherm.com>
Message-ID: <m3pti1ww7d.fsf@mira.informatik.hu-berlin.de>

Michael Chermside <mcherm@mcherm.com> writes:

> I don't really understand this absolutist position. 

I think the rationale is that developers don't actively *use* bug
fixes, whereas they do use new features.

So when you tell them "we have booleans now", they will drop
everything else and start putting booleans all over the place. In the
middle of that, they think "wait, what about 2.2", and they will test
2.2.1, and find that it works just fine. They run into problems only
after they release the software.

With bug fixes, it is different: People won't actively change their
working code to deliberately formulate it in a way that breaks on
older Python versions. Instead, upon reading that the bug has been
fixed, they typically think "does not concern me", and proceed with
whatever they were doing, anyway. In some cases, they may happen to
write new code that relies on a recent bug fix. When they discover
that older versions don't work with their code, they just bump the
prerequisites, or add a work-around.

In addition, there is always the concern that new code may break
existing code, so the amount of new code should be kept to the
smallest possible minimum. Adding new features is beyond that minimum,
so it should not be done (regardless of whether that new code is
likely to break existing code, upon visual inspection).

Regards,
Martin

From martin at v.loewis.de  Mon Sep 15 18:05:12 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Mon Sep 15 18:05:29 2003
Subject: [Python-Dev] Python and POSIX
In-Reply-To: <35230000.1063660079@leeloo.intern.geht.de>
References: <35230000.1063660079@leeloo.intern.geht.de>
Message-ID: <m3llspwvvr.fsf@mira.informatik.hu-berlin.de>

Marc Recht <recht@netbsd.org> writes:

> On all systems I'm aware of this is the same feature-set you get if
> you don't set any macro at all, but unfortunately not all systems
> support such a macro.

You should take a look at Solaris, then. Depending on whether you
define XPG macros (I forgot which ones specifically), the socket
functions mutate from BSD (e.g. connect(3BSD)) to XPG functions
(connect(3XPG)). This is implemented by exposing

#pragma redefine_extname connect __xpg_connect

or some such in socket.h, if the XPG macro is defined.

Likewise, on HP-UX, you select one of the three (!) curses libraries
on the system through these macros.

> So, why not split pyconfig.h up into an external and internal header?

That would be possible if one would
a) provide a patch, and
b) could demonstrate that, on all relevant systems, lack of such macros
   cannot possible have an effect on proper operation of extension modules.

In particular, don't try to make LFS support "internal": Extensions
modules are known to break if they use a different "struct stat" from
the one that the core uses.

Regards,
Martin

From python at rcn.com  Tue Sep 16 00:46:04 2003
From: python at rcn.com (Raymond Hettinger)
Date: Tue Sep 16 00:50:54 2003
Subject: [Python-Dev] Re: Re: python/dist/src/Lib/bsddb __init__.py, 1.5,
	1.6
References: <LNBBLJKPBEHFEDALKOLCCEDIFOAB.tim.one@comcast.net><CGECIJPNNHIFAJKHOLMACEKLHAAA.sholden@holdenweb.com><oprvjknc03qmnqtt@news.gmane.org><1063637821.2093.132.camel@localhost.localdomain>
	<oprvjvgfe2qmnqtt@news.gmane.org>
Message-ID: <004101c37c0d$701be040$0c06a044@oemcomputer>

> Is it worth holding 2.3.1 until after Panther ships, on the assumption that 
> its release will turn up more users and more bugs?  If not, 2.3.1's timing 
> can probably be left up to the release manager, who can make the release 
> whenever he feels like it.

Last month, we worked out a tentantive plan to start the  2.3.1 release 
process in the third week of September.  I think we should stick with
that.  The initial round of bugfixes is an important step for 2.3 being
viewed as stable and to show everyone that development/support
has not vanished.

If critical bugs surface later, that can be a consideration for 2.3.2.
There is no reason to be stingy with the initial bugfix release.



Raymond 



#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

From bac at OCF.Berkeley.EDU  Tue Sep 16 01:09:07 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Sep 16 01:09:20 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
Message-ID: <3F669AF3.8050605@ocf.berkeley.edu>

Some of you may remember that I was working on closing a bug for Red Hat 
6.1 where time.tzset was being detected by 'configure' when it shouldn't 
have (it's busted on that platform) for 2.3.0 .  Unfortunately a working 
patch never surfaced in time to be thoroughly tested.

Well, Kurt Kaiser wrote up a patch ( http://www.python.org/sf/762934 ) 
that seems sound.  I have gotten zero responses from anyone to test it 
on Red Hat 6.1 and a remember that some people here were helping to test 
last time (I also remember this led to the creation of a wiki page 
listing platforms various people could test things on but I can't find 
the page).

I would like to get this in for 2.3.1 (assuming this is considered 
reasonable; not sure after the discussion that has gone on over what 
should go into a micro release) if possible.  So if anyone is running 
Red Hat 6.1 or has easy access to one I would appreciate it if you took 
a look at the patch and gave it a go (don't forget the various steps 
needed to make the patch work; read the follow-ups to see what needs to 
be done).

-Brett


From mal at lemburg.com  Tue Sep 16 03:40:59 2003
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue Sep 16 03:40:54 2003
Subject: [Python-Dev] datetime issues
In-Reply-To: <20030915000637.GC27698@ibook>
References: <20030915000637.GC27698@ibook>
Message-ID: <3F66BE8B.7060307@lemburg.com>

Gustavo Niemeyer wrote:
> Also, is there any further work going on to improve the datetime module?
> I'm not suggesting we should mirror mx.DateTime functionalities, but two
> features I miss from mx.DateTime is the DateFrom(), which does its best
> to parse a given string, and the RelativeDateTime() functionality.
> 
> This has probably been raised in the past, so if that's the case, I'm
> sorry. Marc-Andre, what's your opinion about reusing code from
> mx.DateTime into datetime?

The mxDateTime code comes under the eGenix Public License, so
reuse is permittted under the terms of the license. That said,
I doubt that you want to maintain a date/time parser in the
Python core -- I certainly don't :-)

> Anyway, reusing code or not, I'll probably put sometime on it in the
> future, if this looks interesting to everyone (after I finish my current
> python pendencies, like SRE's recursivity removal).


-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Sep 16 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::


From mwh at python.net  Tue Sep 16 04:59:28 2003
From: mwh at python.net (Michael Hudson)
Date: Tue Sep 16 04:58:58 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
In-Reply-To: <3F669AF3.8050605@ocf.berkeley.edu> (Brett C.'s message of
	"Mon, 15 Sep 2003 22:09:07 -0700")
References: <3F669AF3.8050605@ocf.berkeley.edu>
Message-ID: <2mznh5az2n.fsf@starship.python.net>

"Brett C." <bac@OCF.Berkeley.EDU> writes:

> I would like to get this in for 2.3.1 (assuming this is considered
> reasonable; not sure after the discussion that has gone on over what
> should go into a micro release) if possible.  So if anyone is running
> Red Hat 6.1 or has easy access to one I would appreciate it if you
> took a look at the patch and gave it a go

I no longer have access to RH 6.1 (the starship is now Debian testing).

Cheers,
mwh

-- 
  Of course, it obviously is beta hardware so such things are to be
  expected, but that doesn't mean that you can't point your fingers
  and generate a nelson style HAHA at a multi billion dollar
  corporation's expense.                   -- CmdrTaco on slashdot.org

From arigo at tunes.org  Tue Sep 16 06:50:04 2003
From: arigo at tunes.org (Armin Rigo)
Date: Tue Sep 16 06:52:03 2003
Subject: [Python-Dev] pre-PEP: Resource-Release Support for Generators
In-Reply-To: <5.1.1.6.0.20030915112135.03013ec0@telecommunity.com>
References: <5.2.1.1.0.20030913160918.02773bc0@pop.bluewin.ch>
	<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
	<5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch>
	<00a401c36c0f$76952160$c304a044@oemcomputer>
	<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
	<5.2.1.1.0.20030913160918.02773bc0@pop.bluewin.ch>
	<5.1.1.6.0.20030915112135.03013ec0@telecommunity.com>
Message-ID: <20030916105004.GA11247@vicky.ecs.soton.ac.uk>

Hello Phillip,

On Mon, Sep 15, 2003 at 11:26:27AM -0400, Phillip J. Eby wrote:
> i = iter(something)
> while 1:
>     try:
>         j=i.next()
>     except StopIteration:
>         break
> 
> (...) is already complex enough as a mental model.

Makes sense. Then let's make sure, if both 'with' and 'yield within
try:finally' are accepted, that they are compatible, e.g. by having and
__exit__() method on generators (and not just a close()).


Armin


From recht at netbsd.org  Tue Sep 16 06:59:01 2003
From: recht at netbsd.org (Marc Recht)
Date: Tue Sep 16 06:59:06 2003
Subject: [Python-Dev] Python and POSIX
In-Reply-To: <m3llspwvvr.fsf@mira.informatik.hu-berlin.de>
References: <35230000.1063660079@leeloo.intern.geht.de>
	<m3llspwvvr.fsf@mira.informatik.hu-berlin.de>
Message-ID: <46570000.1063709941@leeloo.intern.geht.de>

> You should take a look at Solaris, then. Depending on whether you
> define XPG macros (I forgot which ones specifically), the socket
> functions mutate from BSD (e.g. connect(3BSD)) to XPG functions
> (connect(3XPG)). This is implemented by exposing
>
># pragma redefine_extname connect __xpg_connect
>
> or some such in socket.h, if the XPG macro is defined.
I had a quick glance at the Solaris 8 headers and it seems that's 
influenced by __EXTENSIONS__, too. So it _should_ be the same as without 
the macros.

> Likewise, on HP-UX, you select one of the three (!) curses libraries
> on the system through these macros.
Oh. Didn't know about HP-UX. That's weird.

>> So, why not split pyconfig.h up into an external and internal header?
>
> That would be possible if one would
> a) provide a patch, and
OK. I justed wanted to verify if there's enough interest (and no general 
rejection) in such a patch and if anyone else is already working on it.

> b) could demonstrate that, on all relevant systems, lack of such macros
>    cannot possible have an effect on proper operation of extension
> modules.
That's a bit harder since I don't have access to all "relevant" systems. I 
could test on the open-source x86 platforms, though. And IIRC CompileFarm 
has Solaris, too.
And I _believe_ that - since these changes came up with Python 2.3 (with 
the exception of _POSIX_1_SOURCE and _POSIX_SOURCE) - that it will break 
not much or anything at all if at least _XOPEN_SOURCE are moved 
_XOPEN_SOURCE_EXTENDED to an internal header (or an internal part of 
pyconfig.h). (As a first step.)

But, you're right there shouldn't be a rash decision and this needs much 
testing.
I think I'll just go on and provide a patch as basis for further discussion.

Regards,
Marc


From neal at metaslash.com  Tue Sep 16 08:40:17 2003
From: neal at metaslash.com (Neal Norwitz)
Date: Tue Sep 16 08:40:21 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
In-Reply-To: <3F669AF3.8050605@ocf.berkeley.edu>
References: <3F669AF3.8050605@ocf.berkeley.edu>
Message-ID: <20030916124017.GB13879@epoch.metaslash.com>

On Mon, Sep 15, 2003 at 10:09:07PM -0700, Brett C. wrote:
> 
> Well, Kurt Kaiser wrote up a patch ( http://www.python.org/sf/762934 ) 
> that seems sound.  I have gotten zero responses from anyone to test it 
> on Red Hat 6.1 and a remember that some people here were helping to test 
> last time (I also remember this led to the creation of a wiki page 
> listing platforms various people could test things on but I can't find 
> the page).

        http://www.python.org/cgi-bin/moinmoin/PythonTesters

From niemeyer at conectiva.com  Tue Sep 16 09:54:36 2003
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Tue Sep 16 09:55:27 2003
Subject: [Python-Dev] datetime issues
In-Reply-To: <3F66BE8B.7060307@lemburg.com>
References: <20030915000637.GC27698@ibook> <3F66BE8B.7060307@lemburg.com>
Message-ID: <20030916135436.GA2509@ibook.distro.conectiva>

> The mxDateTime code comes under the eGenix Public License, so
> reuse is permittted under the terms of the license. That said,
> I doubt that you want to maintain a date/time parser in the
> Python core -- I certainly don't :-)

Thanks. I've got the relativedelta type ready. It's based on your
specification (thanks for it), but uses no code from your module (you'll
notice that clearly when looking at the code). Indeed, it has even a
different algorithm, which solves some of the current "problems" with
your implementation (like getting 03/03 when adding one month to 31/01).
You're free to adopt it, if you feel like so. I'll be releasing that
code under the PSF license.

I'll probably work on a generic parser, once I get some more time.
Depending on the results, I'll submit it for review and possible
inclusion as well.

Here is the documentation of relativedelta, FWIW:

"""
The relativedelta type is based on the specification of the excelent
work done by M.-A. Lemburg in his mx.DateTime extension. However,
notice that this type does *NOT* implement the same algorithm as
his work. Do *NOT* expect it to behave like mx.DateTime's counterpart.

There's two different ways to build a relativedelta instance. The
first one is passing it two date/datetime classes:

    relativedelta(datetime1, datetime2)

And the other way is to use the following keyword arguments:

    year, month, day, hour, minute, seconds, microseconds:
        Absolute information.

    years, months, weeks, days, hours, minutes, seconds, microseconds:
        Relative information, may be negative.

    weekday:
        Tuple with (wday, nth), specifying the nth relative weekday.

Here is the behavior of operations with relativedelta:

1) Calculate the absolute year, using the 'year' argument, or the
   original datetime year, if the argument is not present.

2) Add the relative 'years' argument to the absolute year.

3) Do steps 1 and 2 for month/months.

4) Calculate the absolute day, using the 'day' argument, or the
   original datetime day, if the argument is not present. Then,
   subtract from the day until it fits in the year and month
   found after their operations.

5) Add the relative 'days' argument to the absolute day. Notice
   that the 'weeks' argument is multiplied by 7 and added to
   'days'.

6) Do steps 1 and 2 for hour/hours, minute/minutes, second/seconds,
   microsecond/microseconds.

7) If the 'weekday' argument is present, calculate the weekday,
   with the given (wday, nth) tuple. wday is the index of the
   weekday (0-6, 0=Mon), and nth is the number of weeks to add
   forward or backward, depending on its signal. Notice that if
   the calculated date is already Monday, for example, using
   (0, 1) or (0, -1) won't change the day.
"""

And here is a little demo:

 >>> from dtutil import *; from datetime import *

 >>> relativedelta(datetime.now(),datetime(1983,9,10,10,00))
 relativedelta(years=+20, days=+6, minutes=+25, seconds=+4,
 microseconds=+231402)

 >>> datetime(1983,9,10,10,00)+relativedelta(years=+20, days=+6,
 minutes=+25, seconds=+4, microseconds=+231402)
 datetime.datetime(2003, 9, 16, 10, 25, 4, 231402)

 >>> relativedelta(datetime(2000,2,29),datetime(2001,3,29))
 relativedelta(years=-1, months=-1)
 >>> datetime(2001,3,29)+relativedelta(years=-1, months=-1)
 datetime.datetime(2000, 2, 29, 0, 0)
 >>> relativedelta(datetime(2000,2,29),datetime(2001,3,30))
 relativedelta(years=-1, months=-1)
 >>> datetime(2001,3,30)+relativedelta(years=-1, months=-1)
 datetime.datetime(2000, 2, 29, 0, 0)

 >>> relativedelta(datetime(2000,2,29),datetime(2001,2,28))
 relativedelta(months=-11, days=-28)
 >>> relativedelta(datetime(2000,2,28),datetime(2001,2,28))
 relativedelta(years=-1)
 >>> relativedelta(datetime(2000,2,27),datetime(2001,2,28))
 relativedelta(years=-1, days=-1)

 >>> relativedelta(datetime(2001,2,28),datetime(2000,2,29))
 relativedelta(years=+1)
 >>> relativedelta(datetime(2001,2,28),datetime(2000,2,28))
 relativedelta(years=+1)
 >>> relativedelta(datetime(2001,2,28),datetime(2000,2,27))
 relativedelta(years=+1, days=+1)

 >>> import calendar
 >>> date.today()+relativedelta(weekday=(calendar.MONDAY, +2))
 datetime.date(2003, 9, 29)
 >>> date.today()+relativedelta(weekday=(calendar.MONDAY, -1))
 datetime.date(2003, 9, 15)

-- 
Gustavo Niemeyer
http://niemeyer.net

From skip at pobox.com  Tue Sep 16 10:17:54 2003
From: skip at pobox.com (Skip Montanaro)
Date: Tue Sep 16 10:18:06 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
In-Reply-To: <3F669AF3.8050605@ocf.berkeley.edu>
References: <3F669AF3.8050605@ocf.berkeley.edu>
Message-ID: <16231.7058.987290.575924@montanaro.dyndns.org>


    Brett> Well, Kurt Kaiser wrote up a patch (
    Brett> http://www.python.org/sf/762934 ) that seems sound.  I have
    Brett> gotten zero responses from anyone to test it on Red Hat 6.1 and a
    Brett> remember that some people here were helping to test last time (I
    Brett> also remember this led to the creation of a wiki page listing
    Brett> platforms various people could test things on but I can't find
    Brett> the page).

That would be:

    http://www.python.org/cgi-bin/moinmoin/PythonTesters

I see Kurt Kaiser as the only person listed who can test for RH < 6.2.
Perhaps you should broadcast a call for testers to c.l.py, referring to the
above page and asking in particular for RH 6.1 folks to chime in.

Skip

From tim at zope.com  Tue Sep 16 10:34:23 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 10:34:49 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <1063684862.2093.193.camel@localhost.localdomain>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEOIFOAB.tim@zope.com>

[Tim]
>> $9 = {_ob_next = 0x0, _ob_prev = 0x0, ob_refcnt = 0,
>>       ob_type = 0x8130160, length = 1, str = 0x41903130,
>>       hash = -1, defenc = 0x0}
>>
>> that certainly means the object has been released already.

[Jeremy]
> This is code that is trying to revive an object from the freelist, so
> it isn't a big surprise that the object looks like it has been freed.

Doh -- yup, of course.

> Now here's a very strange thing.  If I apply the following patch, the
> problem goes away.  The patch doesn't make a lot of sense to me, so
> I'm guessing that it's just a lucky patch -- avoiding the problem
> without fixing the cause.

We should move this to python-dev.  OK, I just did <wink>.  Another clue
about the cause coming up.

> The thing that is curious is that we change the test against
> unicode->str[0], which is probably an unsigned int,

Why?  Are you doing a UCS4 build?  The expansion of Py_UNICODE (unicode->str
is an array of Py_UNICODE thingies) can't be guessed from where I sit (it
expands to "unsigned short" on Windows, but that's because that's hardcoded
in PC\pyconfig.h; I'm not sure where you get its expansion from).

> so that we also test that it is > 0.  In the specific case that
> crashes, the debugger says the value is -875836469.

That's probably a great clus!  In hex, that's 0xcbcbcbcb, which is the
special byte pattern the debug pymalloc sprays into freshly-allocated
memory.  IOW, unicode->str contains (in a release build) uninitialized heap
trash at this point, and so there's nothing it can be tested against that
makes any sense.  This suggests a deeper bug in the Unicode support.

> Does that make any sense?  Do we actually have a signed value here?

You have to figure out what Py_UNICODE expands to in your build to answer
that one.

> Or does it get converted to a signed value when the comparison occurs?

If gdb showed you -875836469 in response to "unicode->str[0]" in isolation,
I have to guess Py_UNICODE is expanding to a signed 4-byte type in your
build.

> Jeremy
>
> Index: unicodeobject.c
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/Objects/unicodeobject.c,v
> retrieving revision 2.196
> diff -c -r2.196 unicodeobject.c
> *** unicodeobject.c	16 Sep 2003 03:41:45 -0000	2.196
> --- unicodeobject.c	16 Sep 2003 03:55:53 -0000
> ***************
> *** 130,138 ****
>       /* Resizing shared object (unicode_empty or single character
>          objects) in-place is not allowed. Use PyUnicode_Resize()
>          instead ! */
>       if (unicode == unicode_empty ||
>   	(unicode->length == 1 &&
> ! 	 unicode->str[0] < 256 &&
>   	 unicode_latin1[unicode->str[0]] == unicode)) {
>           PyErr_SetString(PyExc_SystemError,
>                           "can't resize shared unicode objects");
> --- 130,142 ----
>       /* Resizing shared object (unicode_empty or single character
>          objects) in-place is not allowed. Use PyUnicode_Resize()
>          instead ! */
>       if (unicode == unicode_empty ||
>   	(unicode->length == 1 &&
> ! 	 unicode->str[0] < 256 && unicode->str[0] > 0 &&
>   	 unicode_latin1[unicode->str[0]] == unicode)) {
>           PyErr_SetString(PyExc_SystemError,
>                           "can't resize shared unicode objects");

Belaboring the obvious, if unicode->str[0] is really heap trash at this
point, there's no safe test.


From jeremy at zope.com  Tue Sep 16 11:05:19 2003
From: jeremy at zope.com (Jeremy Hylton)
Date: Tue Sep 16 11:05:50 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEOIFOAB.tim@zope.com>
References: <LNBBLJKPBEHFEDALKOLCMEOIFOAB.tim@zope.com>
Message-ID: <1063724718.4868.2.camel@localhost.localdomain>

I disabled the KEEPALIVE_SIZE_LIMIT, as you suggested, and that also
fixed the problem.  The comment describe the code is really unnerving:

/* Limit for the Unicode object free list stay alive optimization.

   ...

   Note: This is an experimental feature ! If you get core dumps when
   using Unicode objects, turn this feature off.

*/

Why was this experimental feature released with Python if it causes core
dumps?

Jeremy



From tim at zope.com  Tue Sep 16 12:12:26 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 12:12:48 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <1063724718.4868.2.camel@localhost.localdomain>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEPCFOAB.tim@zope.com>

[Jeremy Hylton]
> I disabled the KEEPALIVE_SIZE_LIMIT, as you suggested, and that also
> fixed the problem.

We don't know why, though, nor do we have a small test case, right?

>  The comment describe the code is really unnerving:
>
> /* Limit for the Unicode object free list stay alive optimization.
>
>    ...
>
>    Note: This is an experimental feature ! If you get core dumps when
>    using Unicode objects, turn this feature off.
>
> */
>
> Why was this experimental feature released with Python if it causes
> core dumps?

Good question <wink>.

Going back to an earlier msg:

> Index: unicodeobject.c
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/Objects/unicodeobject.c,v
> retrieving revision 2.196
> diff -c -r2.196 unicodeobject.c
> *** unicodeobject.c	16 Sep 2003 03:41:45 -0000	2.196
> --- unicodeobject.c	16 Sep 2003 03:55:53 -0000
> ***************
> *** 130,138 ****
>       /* Resizing shared object (unicode_empty or single character
>          objects) in-place is not allowed. Use PyUnicode_Resize()
>          instead ! */
>       if (unicode == unicode_empty ||
>   	(unicode->length == 1 &&
> ! 	 unicode->str[0] < 256 &&
>   	 unicode_latin1[unicode->str[0]] == unicode)) {
>           PyErr_SetString(PyExc_SystemError,
>                           "can't resize shared unicode objects");
> --- 130,142 ----
>       /* Resizing shared object (unicode_empty or single character
>          objects) in-place is not allowed. Use PyUnicode_Resize()
>          instead ! */
>       if (unicode == unicode_empty ||
>   	(unicode->length == 1 &&
> ! 	 unicode->str[0] < 256 && unicode->str[0] > 0 &&
>   	 unicode_latin1[unicode->str[0]] == unicode)) {
>           PyErr_SetString(PyExc_SystemError,
>                           "can't resize shared unicode objects");

> Belaboring the obvious, if unicode->str[0] is really heap trash at
> this point, there's no safe test.

On second thought, the test could be correct even if it does read up heap
trash:  the

    unicode_latin1[unicode->str[0]] == unicode

part is a very strong condition.  If str[0] does contain heap trash, it
can't pass, but it could cause Purify (etc) to whine about reading
uninitialized memory.  If we don't care about that,

     unicode->length == 1 &&
      (unsigned int)unicode->str[0] < 256U &&
      unicode_latin1[unicode->str[0]] == unicode

would be a safe and correct guard.

Still, it doesn't look to me like the code *expected* that str could contain
uninitialized heap trash at this point, so I'd like someone who thinks they
understand this code to think about how that could happen (it apparently is
happening, so "beats me" isn't good enough <wink>).


From patmiller at llnl.gov  Tue Sep 16 12:24:45 2003
From: patmiller at llnl.gov (Pat Miller)
Date: Tue Sep 16 12:24:50 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <200309120311.h8C3BSe01781@oma.cosc.canterbury.ac.nz>
References: <200309120311.h8C3BSe01781@oma.cosc.canterbury.ac.nz>
Message-ID: <3F67394D.305@llnl.gov>

I used to work on the SISAL compiler which was a fine-grained parallel
functional language.  It reference counted its allocated data.

Updating the reference count is a critical section.
We were spending 40% of our time blocked for reference counting on only 4 threads.

We eventually taught the compiler to do reference count optimizations
(e.g. put all readers of an object before the writers), but these
optimizations are not suitable to the Python interpreter.

With this same reasoning, you can infer why C++ smart pointers are
also a bad idea for threaded code.

Pat


-- 
Patrick Miller | (925) 423-0309 | http://www.llnl.gov/CASC/people/pmiller

All you need in this life is ignorance and confidence, and then success is
sure. -- Mark Twain


From jeremy at zope.com  Tue Sep 16 12:31:20 2003
From: jeremy at zope.com (Jeremy Hylton)
Date: Tue Sep 16 12:31:45 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEPCFOAB.tim@zope.com>
References: <LNBBLJKPBEHFEDALKOLCOEPCFOAB.tim@zope.com>
Message-ID: <1063729880.4868.8.camel@localhost.localdomain>

On Tue, 2003-09-16 at 12:12, Tim Peters wrote:
> [Jeremy Hylton]
> > I disabled the KEEPALIVE_SIZE_LIMIT, as you suggested, and that also
> > fixed the problem.
> 
> We don't know why, though, nor do we have a small test case, right?

Not a small test case, no, but I have boiled it down a bit.  If you run
the tests from a Zope checkout on the Zope-2_7-branch, you can provoke
the crash with "test.py Template."

> Going back to an earlier msg:
> 
> > Index: unicodeobject.c
> > ===================================================================
> > RCS file: /cvsroot/python/python/dist/src/Objects/unicodeobject.c,v
> > retrieving revision 2.196
> > diff -c -r2.196 unicodeobject.c
> > *** unicodeobject.c	16 Sep 2003 03:41:45 -0000	2.196
> > --- unicodeobject.c	16 Sep 2003 03:55:53 -0000
> > ***************
> > *** 130,138 ****
> >       /* Resizing shared object (unicode_empty or single character
> >          objects) in-place is not allowed. Use PyUnicode_Resize()
> >          instead ! */
> >       if (unicode == unicode_empty ||
> >   	(unicode->length == 1 &&
> > ! 	 unicode->str[0] < 256 &&
> >   	 unicode_latin1[unicode->str[0]] == unicode)) {
> >           PyErr_SetString(PyExc_SystemError,
> >                           "can't resize shared unicode objects");
> > --- 130,142 ----
> >       /* Resizing shared object (unicode_empty or single character
> >          objects) in-place is not allowed. Use PyUnicode_Resize()
> >          instead ! */
> >       if (unicode == unicode_empty ||
> >   	(unicode->length == 1 &&
> > ! 	 unicode->str[0] < 256 && unicode->str[0] > 0 &&
> >   	 unicode_latin1[unicode->str[0]] == unicode)) {
> >           PyErr_SetString(PyExc_SystemError,
> >                           "can't resize shared unicode objects");
> 
> > Belaboring the obvious, if unicode->str[0] is really heap trash at
> > this point, there's no safe test.
> 
> On second thought, the test could be correct even if it does read up heap
> trash:  the
> 
>     unicode_latin1[unicode->str[0]] == unicode
> 
> part is a very strong condition.  If str[0] does contain heap trash, it
> can't pass, but it could cause Purify (etc) to whine about reading
> uninitialized memory.  If we don't care about that,

It could also cause a segmentation violation depending on the value of
the stack trash it reads.  At least, that's what appears to be going
wrong.

> 
>      unicode->length == 1 &&
>       (unsigned int)unicode->str[0] < 256U &&
>       unicode_latin1[unicode->str[0]] == unicode
> 
> would be a safe and correct guard.

That sounds better.

> Still, it doesn't look to me like the code *expected* that str could contain
> uninitialized heap trash at this point, so I'd like someone who thinks they
> understand this code to think about how that could happen (it apparently is
> happening, so "beats me" isn't good enough <wink>).

I certainly don't understand the code yet.

Jeremy



From bac at OCF.Berkeley.EDU  Tue Sep 16 12:53:41 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Sep 16 12:54:02 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
In-Reply-To: <16231.7058.987290.575924@montanaro.dyndns.org>
References: <3F669AF3.8050605@ocf.berkeley.edu>
	<16231.7058.987290.575924@montanaro.dyndns.org>
Message-ID: <3F674015.8090801@ocf.berkeley.edu>

Skip Montanaro wrote:

>     Brett> Well, Kurt Kaiser wrote up a patch (
>     Brett> http://www.python.org/sf/762934 ) that seems sound.  I have
>     Brett> gotten zero responses from anyone to test it on Red Hat 6.1 and a
>     Brett> remember that some people here were helping to test last time (I
>     Brett> also remember this led to the creation of a wiki page listing
>     Brett> platforms various people could test things on but I can't find
>     Brett> the page).
> 
> That would be:
> 
>     http://www.python.org/cgi-bin/moinmoin/PythonTesters
> 
> I see Kurt Kaiser as the only person listed who can test for RH < 6.2.
> Perhaps you should broadcast a call for testers to c.l.py, referring to the
> above page and asking in particular for RH 6.1 folks to chime in.
> 

Jordan Krushen pointed out that the patch is actually opened against RH 
6.2, not 6.1 like I originally said.  But I don't think it is that big 
of an issue since there is a bug report for this against SuSE.  I think 
the key thing here is an older version of glibc since that would be what 
defines the tzset function.

But I will consider sending a note out to c.l.py about this.  Thanks for 
the suggestion, Skip.

-Brett


From tim at zope.com  Tue Sep 16 12:58:21 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 12:58:45 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <1063729880.4868.8.camel@localhost.localdomain>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEPIFOAB.tim@zope.com>

[Jeremy Hylton]
>>> I disabled the KEEPALIVE_SIZE_LIMIT, as you suggested, and that also
>>> fixed the problem.

[Tim
>> We don't know why, though, nor do we have a small test case, right?

[Jeremy]
> Not a small test case, no, but I have boiled it down a bit.  If you
> run the tests from a Zope checkout on the Zope-2_7-branch, you can
> provoke the crash with "test.py Template."

Sorry, I cannot.  I tried a fresh Zope-2_7-branch, on Windows, with

    released Python 2.3, release build
    current CVS Python, release build
    current CVS Python, debug build

"test.py Template" gave the same result in all cases:  77 tests run, 1
failure (AssertionError: HTML Output Changed) in checkStringExpressions.

But note that Py_UNICODE is unsigned short on this box, and reading a trash
one of those as an index isn't nearly as likely to reach into unmapped
memory as on a box where you get 4 bytes of trash.

>> ...
>> On second thought, the test could be correct even if it does read up
>> heap trash:  the
>>
>>     unicode_latin1[unicode->str[0]] == unicode
>>
>> part is a very strong condition.  If str[0] does contain heap trash,
>> it can't pass, but it could cause Purify (etc) to whine about reading
>> uninitialized memory.  If we don't care about that,

> It could also cause a segmentation violation depending on the value of
> the stack trash it reads.  At least, that's what appears to be going
> wrong.

Right, I meant the condition is very strong *if* the unicode->str[0] part is
known to lie in range(256).  unicode_latin1 is statically declared to
contain 256 elements, so if the trash index is in range(256) then the index
resolves to legitimate memory.

>>      unicode->length == 1 &&
>>       (unsigned int)unicode->str[0] < 256U &&
>>       unicode_latin1[unicode->str[0]] == unicode

>> would be a safe and correct guard.

> That sounds better.

To worm around something that shouldn't happen at all, sure <wink>.

>> Still, it doesn't look to me like the code *expected* that str could
>> contain uninitialized heap trash at this point, so I'd like someone
>> who thinks they understand this code to think about how that could
>> happen (it apparently is happening, so "beats me" isn't good enough
>> <wink>).

> I certainly don't understand the code yet.

Me neither, but offhand didn't see anything glaringly wrong.  If some
internal Unicode operation decided to allocate a short Unicode string, but
freed it before filling in any of the string bytes, I suppose the keepalive
optimization would retain a Unicode object with uninitialized str space on
the unicode free list.  A subsequent _PyUnicode_New could grab that and try
to boost its ->str size.  That could explain it.


From jeremy at alum.mit.edu  Tue Sep 16 12:58:43 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue Sep 16 12:58:47 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <1063729880.4868.8.camel@localhost.localdomain>
References: <LNBBLJKPBEHFEDALKOLCOEPCFOAB.tim@zope.com>
	<1063729880.4868.8.camel@localhost.localdomain>
Message-ID: <1063731523.4868.11.camel@localhost.localdomain>

On Tue, 2003-09-16 at 12:31, Jeremy Hylton wrote:
> On Tue, 2003-09-16 at 12:12, Tim Peters wrote:
> > [Jeremy Hylton]
> > > I disabled the KEEPALIVE_SIZE_LIMIT, as you suggested, and that also
> > > fixed the problem.
> > 
> > We don't know why, though, nor do we have a small test case, right?
> 
> Not a small test case, no, but I have boiled it down a bit.  If you run
> the tests from a Zope checkout on the Zope-2_7-branch, you can provoke
> the crash with "test.py Template."

Okay.  I have a smaller test case, though I don't know what I couldn't
find it earlier.  If you run "test.py DocumentTemplate UnicodeStr" by
itself, you get the crash.

The DocumentTemplate implementation has C and Python implementations of
some utilities.  It uses the C by default and falls back to Python if
the C extension can't be imported.  If I change the code so that it
always uses the Python implementation, the crash goes away.

Jeremy



From tim at zope.com  Tue Sep 16 13:11:13 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 13:12:00 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEPIFOAB.tim@zope.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEPKFOAB.tim@zope.com>

[Jeremy]
>> Not a small test case, no, but I have boiled it down a bit.  If you
>> run the tests from a Zope checkout on the Zope-2_7-branch, you can
>> provoke the crash with "test.py Template."

[Tim]
> Sorry, I cannot.  I tried a fresh Zope-2_7-branch, on Windows, with
>
>     released Python 2.3, release build
>     current CVS Python, release build
>     current CVS Python, debug build
>
> "test.py Template" gave the same result in all cases:  77 tests run, 1
> failure (AssertionError: HTML Output Changed) in
> checkStringExpressions.
>
> But note that Py_UNICODE is unsigned short on this box, and reading a
> trash one of those as an index isn't nearly as likely to reach into
> unmapped memory as on a box where you get 4 bytes of trash.

What I *could* do, though, is add this to unicode_resize():

    if (unicode != unicode_empty && unicode->length == 1) {
        assert(unicode->str[0] != 0xcbcbU);
    }

That checks for the debug pymalloc's "uninitialized memory" byte pattern
appropriate for this box, and that assert does indeed trigger when running

    test.py Template

It fires while running checkUnicodeInserts, but checkUnicodeInserts does not
fail in isolation.  So checkUnicodeInserts is the victim of something an
earlier test left on the unicode freelist.


From tim at zope.com  Tue Sep 16 13:15:53 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 13:16:58 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <1063731523.4868.11.camel@localhost.localdomain>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEPLFOAB.tim@zope.com>

[Jeremy Hylton]
> Okay.  I have a smaller test case, though I don't know what I couldn't
> find it earlier.  If you run "test.py DocumentTemplate UnicodeStr" by
> itself, you get the crash.

As before, nothing crashes here, but running that test does trigger the
debug-build-specific assert I added here (see last msg).  Good sleuthing!

> The DocumentTemplate implementation has C and Python implementations
> of some utilities.  It uses the C by default and falls back to Python
> if the C extension can't be imported.  If I change the code so that it
> always uses the Python implementation, the crash goes away.

I'm guessing

   lib/python/DocumentTemplate/cDocumentTemplate.c


From theller at python.net  Tue Sep 16 13:45:40 2003
From: theller at python.net (Thomas Heller)
Date: Tue Sep 16 13:45:22 2003
Subject: [Python-Dev] Re: HTMLHelp for Py2.3.1
In-Reply-To: <HPEBIMANNCHHDMDBGCHHMEPDDKAA.hernan@orgmf.com.ar> (Boriz
	Izaguirre's message of "Sun, 7 Sep 2003 15:52:36 +0200")
References: <HPEBIMANNCHHDMDBGCHHMEPDDKAA.hernan@orgmf.com.ar>
Message-ID: <8yook4or.fsf@python.net>

"Boriz Izaguirre" <hernan@orgmf.com.ar> writes:

> [Thomas]
>>[Fred L. Drake]
>>> IDLE currently looks for the index.html file in a few places (which
>>> depend on platform); if it can't find it, it uses the documentation on
>>> python.org.
>>>
>>> It should be too hard to change it to load the HTML Help viewer if it
>>> finds the .chm file on Windows, and to still fall back to the HTML or
>>> the online documentation if the .chm can't be found.
>>
>>Changing it is trivial, EditorWindow.help_url must point to Python23.chm
>>(if it exists).  I can do this.
>
> Beware that for .html you want webbrowser.open(url) and for .chm you
> want os.startfile(url)
> There used to be a patch in Idle-dev for this.
> http://tinyurl.com/mj1s
>

It seems this patch is already applied in 2.3.

>>Even nicer would be context-sensitive keyword help, but it seems IDLE
>>doesn't support it, right?
>
> Standard .chm format already includes useful index data to do
> context-sensitive search. A couple of years ago I made an Idle extension
> to manage this. It wasn't difficult then, and I think it's even easier
> now. The problem I found is that you pass the selected text (the one you
> are looking for) to HTMLHelp system by calling a Win32 API.

I know this, and how to do the windows side. What I meant was that I
don't know enough about IDLE (or Tkinter) to implement the functionality
to find the word under the cursor.

> You'll need win32all installed. That's the way PythonWin works, by the way.
> I guess this makes it a no-no for standard Python, right?
>

I don't think win32all wraps the HTMLHelp api, does it?
OTOH, I'm waiting for a reason to include ctypes with core python ;-)


> Regards,
> -Hernan
>

Thomas


From theller at python.net  Tue Sep 16 13:47:06 2003
From: theller at python.net (Thomas Heller)
Date: Tue Sep 16 13:46:46 2003
Subject: [Python-Dev] Windows installer and .pyc files mtime
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEDJFJAB.tim.one@comcast.net> (Tim
	Peters's message of "Sat, 30 Aug 2003 22:29:35 -0400")
References: <LNBBLJKPBEHFEDALKOLCGEDJFJAB.tim.one@comcast.net>
Message-ID: <4qzck4md.fsf@python.net>

"Tim Peters" <tim.one@comcast.net> writes:

> [Thomas Heller]
>> Temporary download url:
>>
>> http://starship.python.net/crew/theller/python-2.3.1-beta1.exe
>>
>> This is built from CVS 23maint branch, some hours ago.  The uncommited
>> changes I made are here, together with additional comments:
>>
>> http://www.python.org/sf/796919
>>
>> Only slightly tested so far.
>
> Slightly more now, on Win98SE.  Cool!  I thought something was wrong because
> the install on Win98SE went faster than I was used to.  Turns out that's
> just because we're installing 1,200+ fewer files than we used to (thanks to
> shipping all the HTML docs in a single .chm file).
>
> Everything I tried worked fine.  I think the *default* should be not to
> compile .pyc files.  Regardless of the default, popping up a DOS box during
> compilation doesn't bother me at all.

Great, I'll change it. But when there's an option to compile .pyc,
shouldn't there also be an option for .pyo (and should it use -O or -OO)?

Thomas


From tim at zope.com  Tue Sep 16 13:58:30 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 13:58:44 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEPKFOAB.tim@zope.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEPOFOAB.tim@zope.com>

About Jeremy's segfault, we've debugged this in an IM session, and it's a
bug in Python's (2.1, 2.2, 2.3, doesn't matter) unicodeobject.c.  We'll fix
it.  You're most likely to see a segfault under a UCS4 build.  Just another
reason to upgrade to Windows <wink>.


From tim at zope.com  Tue Sep 16 14:13:30 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 14:13:44 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKEPIFOAB.tim@zope.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEAAFPAB.tim@zope.com>

[Tim]
> ...
> If some internal Unicode operation decided to allocate a short Unicode
> string, but freed it before filling in any of the string bytes, I
> suppose the keepalive optimization would retain a Unicode object with
> uninitialized str space on the unicode free list.  A subsequent
> _PyUnicode_New could grab that and try to boost its ->str size.  That
> could explain it.

And that turned out to be the case.  One example (there are more) is in
PyUnicode_DecodeASCII():  the local

    PyUnicodeObject *v;

gets initialized:

    v = _PyUnicode_New(size);

Suppose size is 1.  Suppose the string coming in is "\xc8".  The first
iteration of the loop sets a "ordinal not in range(128)" error, and jumps to
onError:

 onError:
    Py_XDECREF(v);

unicode_dealloc() then stuffs v on unicode_freelist, and because the length
"is small" (size == 1), v->str[] is retained, still holding uninitialized
heap trash.

A later _PyUnicode_New() grabs this off unicode_freelist, decides to boost
the str space via unicode_resize(), and the latter blows up in the

    	 unicode_latin1[unicode->str[0]] == unicode)) {

check because unicode->str[0] happens to be gigantically negative on
Jeremy's box, and the preceding

	 unicode->str[0] < 256 &&

check is too weak.  Or is there an implicit assumption that Py_UNICODE is
always an unsigned type (in which case, why isn't the literal 256U?; and in
which case, it doesn't seem to be true on Jeremy's box).


From tim at zope.com  Tue Sep 16 14:47:00 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 14:48:12 2003
Subject: [Python-Dev] Windows installer and .pyc files mtime
In-Reply-To: <4qzck4md.fsf@python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEAEFPAB.tim@zope.com>

[Tim]
>> Everything I tried worked fine.  I think the *default* should be not
>> to compile .pyc files.  Regardless of the default, popping up a DOS
>> box during compilation doesn't bother me at all.

[Thomas Heller]
> Great, I'll change it. But when there's an option to compile .pyc,
> shouldn't there also be an option for .pyo (and should it use -O or
> -OO)?

Sounds like YAGNI.  In the years I did the Windows installer, nobody asked
for compilation.  I haven't seen anyone here ask for it either, just a few
people saying they think other people might want it.  Since I wasn't one of
them, you're going to have to ask someone who thinks they can channel these
oddly unassertive users <wink>.


From martin at v.loewis.de  Tue Sep 16 15:52:26 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Sep 16 15:52:46 2003
Subject: [Python-Dev] Python and POSIX
In-Reply-To: <46570000.1063709941@leeloo.intern.geht.de>
References: <35230000.1063660079@leeloo.intern.geht.de>
	<m3llspwvvr.fsf@mira.informatik.hu-berlin.de>
	<46570000.1063709941@leeloo.intern.geht.de>
Message-ID: <m3ekyg5x51.fsf@mira.informatik.hu-berlin.de>

Marc Recht <recht@netbsd.org> writes:

> I had a quick glance at the Solaris 8 headers and it seems that's
> influenced by __EXTENSIONS__, too. So it _should_ be the same as
> without the macros.

While that is true, I would hope that you come to the conclusion that
these macros can potentially do much more than simply hiding library
functions. Instead, their *primary* purpose is to request Single Unix
behaviour in case a platform implements both the standard behaviour,
and the traditional one (regardless of which tradition the system
follows).

So I would be *very* careful to enable these macros only for a few
translation units; I'm pretty sure that the "law" would be on the
vendor's side if inconsistent usage of these defines would be found to
cause problems, in Irix 27, or ValueBSD 3.

> That's a bit harder since I don't have access to all "relevant"
> systems. I could test on the open-source x86 platforms, though. And
> IIRC CompileFarm has Solaris, too.

If the change is only for 2.4, that would be fine - that code
hopefully will get testing on several other platforms before 2.4 is
released.

> And I _believe_ that - since these changes came up with Python 2.3
> (with the exception of _POSIX_1_SOURCE and _POSIX_SOURCE) - that it
> will break not much or anything at all if at least _XOPEN_SOURCE are
> moved _XOPEN_SOURCE_EXTENDED to an internal header (or an internal
> part of pyconfig.h).

Believes should not be trusted too much when it comes to
portability. If you can think of an obscure interpretation of some
standard, it is almost certain that there is some vendor out there who
has chosen that interpretation.

Regards,
Martin


From martin at v.loewis.de  Tue Sep 16 15:55:46 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Sep 16 15:56:21 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEPCFOAB.tim@zope.com>
References: <LNBBLJKPBEHFEDALKOLCOEPCFOAB.tim@zope.com>
Message-ID: <m3ad945wzh.fsf@mira.informatik.hu-berlin.de>

"Tim Peters" <tim@zope.com> writes:

> Still, it doesn't look to me like the code *expected* that str could contain
> uninitialized heap trash at this point, so I'd like someone who thinks they
> understand this code to think about how that could happen (it apparently is
> happening, so "beats me" isn't good enough <wink>).

I would like to see a stack trace.

Regards,
Martin

From martin at v.loewis.de  Tue Sep 16 16:00:27 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Sep 16 16:00:45 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <1063729880.4868.8.camel@localhost.localdomain>
References: <LNBBLJKPBEHFEDALKOLCOEPCFOAB.tim@zope.com>
	<1063729880.4868.8.camel@localhost.localdomain>
Message-ID: <m365js5wro.fsf@mira.informatik.hu-berlin.de>

Jeremy Hylton <jeremy@zope.com> writes:

> > Still, it doesn't look to me like the code *expected* that str could contain
> > uninitialized heap trash at this point, so I'd like someone who thinks they
> > understand this code to think about how that could happen (it apparently is
> > happening, so "beats me" isn't good enough <wink>).
> 
> I certainly don't understand the code yet.

The code you quote is about the "singleton" Unicode character
(i.e. those with code points < 256). If you create a Unicode object S
with len(S)==1, and ord(S) < 256, you don't necessarily get a new
object, but you may get an existing (cached) one. If you then apply
PyUnicde_Resize to that object, you get an error because resizing that
object would require changing all shared copies, which is bad because
Unicode object are immutable.

I agree with Tim that, in no case, resize should be called for a
garbage string - only valid strings (i.e. with truly allocated memory)
should ever be resized.

Regards,
Martin


From martin at v.loewis.de  Tue Sep 16 16:05:18 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Sep 16 16:05:50 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEAAFPAB.tim@zope.com>
References: <LNBBLJKPBEHFEDALKOLCCEAAFPAB.tim@zope.com>
Message-ID: <m3y8wo4hz5.fsf@mira.informatik.hu-berlin.de>

"Tim Peters" <tim@zope.com> writes:

> Or is there an implicit assumption that Py_UNICODE is always an
> unsigned type (in which case, why isn't the literal 256U?

Likely, the author of the code was subconsciously assuming
so. However, that assumption is invalid, and sharing should only
happen for ordinals in range(256).

Regards,
Martin

From tim.one at comcast.net  Tue Sep 16 16:46:31 2003
From: tim.one at comcast.net (Tim Peters)
Date: Tue Sep 16 16:46:42 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <m365js5wro.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEBGFPAB.tim.one@comcast.net>

[Martin v. L?wis]
> ...
> I agree with Tim that, in no case, resize should be called for a
> garbage string - only valid strings (i.e. with truly allocated memory)
> should ever be resized.

unicode->str pointed to properly allocated memory in the failing cases, but
to uninitialized allocated memory.  So str[0] was just some random
collection of bits from the heap, and checking

    random bits < 256

wasn't enough to weed out the cases where the random bits happened to look
like a negative integer.  For Jeremy, str[0] was 0xcbcbcbcb (for me it was
0xcbcb), and using that as an index blew up.

...

[Tim]
>> Or is there an implicit assumption that Py_UNICODE is always an
>> unsigned type (in which case, why isn't the literal 256U?

[Martin]
> Likely, the author of the code was subconsciously assuming
> so. However, that assumption is invalid, and sharing should only
> happen for ordinals in range(256).

I've since patched Jeremy's patch to try to live with that Py_UNICODE may be
a signed type too.


From mal at lemburg.com  Tue Sep 16 17:22:26 2003
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue Sep 16 17:22:28 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects
 unicodeobject.c, 2.197, 2.198
In-Reply-To: <E19zMTW-0004cb-00@sc8-pr-cvs1.sourceforge.net>
References: <E19zMTW-0004cb-00@sc8-pr-cvs1.sourceforge.net>
Message-ID: <3F677F12.80409@lemburg.com>

tim_one@users.sourceforge.net wrote:
> Update of /cvsroot/python/python/dist/src/Objects
> In directory sc8-pr-cvs1:/tmp/cvs-serv17421/Objects
> 
> Modified Files:
> 	unicodeobject.c 
> Log Message:
> On c.l.py, Martin v. L?wis said that Py_UNICODE could be of a signed type,
> so fiddle Jeremy's fix to live with that.  Also added more comments.

Note that the implementation will bomb in several places if
Py_UNICODE is a signed type.

Py_UNICODE was never intended to be a signed type, so the proper
fix would be to add logic so that Py_UNICODE gets forced to be an
unsigned type.

The only case where Py_UNICODE could become signed is via
a compiler that defines wchar_t to be signed -- rather unlikely.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Sep 16 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::


From tim.one at comcast.net  Tue Sep 16 17:39:58 2003
From: tim.one at comcast.net (Tim Peters)
Date: Tue Sep 16 17:40:04 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects
	unicodeobject.c, 2.197, 2.198
In-Reply-To: <3F677F12.80409@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEBMFPAB.tim.one@comcast.net>

[M.-A. Lemburg]
> Note that the implementation will bomb in several places if
> Py_UNICODE is a signed type.
>
> Py_UNICODE was never intended to be a signed type, so the proper
> fix would be to add logic so that Py_UNICODE gets forced to be an
> unsigned type.

Jeremy believed Py_UNICODE was already an unsigned type on his box, and that
was the box with the segfaults.  I don't know.  Comparison with a signed int
confused the issues to the point where we gave up and just fixed it <wink>.

> The only case where Py_UNICODE could become signed is via
> a compiler that defines wchar_t to be signed -- rather unlikely.

The C standard requires wchar_t to be an integer type, but doesn't constrain
it further to an unsigned integer type.  I don't like relying on
non-standard assumptions in cases where there's little-to-no cost in not
relying on them.  For example, the cast I put in with this patch is probably
a nop on most boxes, just forcing an unsigned comparison (which must have
been the original intent, if Py_UNICODE was assumed to be an unsigned type).


From gjc at inescporto.pt  Tue Sep 16 18:04:42 2003
From: gjc at inescporto.pt (Gustavo J A M Carneiro)
Date: Tue Sep 16 18:06:34 2003
Subject: Solutions for LC_NUMERIC, was Re: [Python-Dev] Re: Be Honest
	about LC_NUMERIC [REPOST]
In-Reply-To: <m38yp5k16y.fsf@mira.informatik.hu-berlin.de>
References: <m37k4qki2j.fsf@mira.informatik.hu-berlin.de>
	<LNBBLJKPBEHFEDALKOLCCEIEFKAB.tim.one@comcast.net>
	<20030903134155.GB649@async.com.br>
	<m38yp5k16y.fsf@mira.informatik.hu-berlin.de>
Message-ID: <1063749875.20959.15.camel@emperor.homelinux.net>

A Qua, 2003-09-03 ?s 22:32, Martin v. L?wis escreveu:
> Christian Reis <kiko@async.com.br> writes:
> 
> >     "On glibc-based systems, Python is safe from runtime changes in the
> >     LC_NUMERIC locale category. On other platforms, changing LC_NUMERIC
> >     at runtime may cause float conversion routines to fail. This may
> >     impact the Python interpreter internally."
> 
> I could accept such a limitation. However, I maintain that the aspects
> must be fully understood, and I agree with Guido that they must be
> documented in the documentation proper (i.e. as comments in the source
> code, or the user documentation).
> 
> So I suggest that you proceed with the PEP and the implementation
> as-is. The PEP would need to be enhanced to summarize this discussion
> (perhaps listing the alternatives that you just mentioned); the patch
> would need to be enhanced to be more simple, readable, and in the
> style of the rest of the Python source code, and would need to include
> documentation changes and test cases (although some test cases
> probably already exist).

  I have updated the patch[1], with James Henstridge's cleaned up code
(and some more cleanup).  I am aware it still needs more comments and
test cases, but interested parties might want to start looking at it.

  Please note that, after some discussion, we have concluded it is
better not to use glibc locale extensions to implement thread-safe float
conversion.  The main reasons are: (1) simplicity (no ifdefs); (2)
consistency between glibc and non-glibc platforms.   We should accept
the fact python programs should not be allowed change locale after
creating threads, period.  Perhaps python itself should raise an
exception if this is ever attempted.  This is subject to discussion, of
course! ;)

  Regards.

[1]
http://sourceforge.net/tracker/download.php?group_id=5470&atid=305470&file_id=61688&aid=774665

-- 
Gustavo J. A. M. Carneiro
<gjc@inescporto.pt> <gustavo@users.sourceforge.net>


From lists at webcrunchers.com  Tue Sep 16 16:50:09 2003
From: lists at webcrunchers.com (John D.)
Date: Tue Sep 16 18:44:29 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <005a01c37962$b94a0e20$30bc958d@oemcomputer>
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
	<16226.6197.573407.585746@montanaro.dyndns.org>
Message-ID: <v03110778bb8d2740509d@[192.168.0.2]>

Raymond writes:


>[Raymond]
>>     >> > IIRC, dbhash and bsddb don't affect the Apple MacIntosh users.
>[Skip]
>> I don't know what you mean here.  Do you mean Mac OS 9 or Mac OS X?  I
>> use bsddb all the time on Mac OS X.
>
>Hmm, I'll have to fix the docs too.  They currently list bsddb as being
>only for Unix and Windows.

While you are at it,  please add a few more examples of how to use "bsddb3"
in particular the "dbtables" section.   Like how do you select all from a table.

JD



From tim at zope.com  Tue Sep 16 21:34:50 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 21:35:40 2003
Subject: [Python-Dev] RE: [Zope-CVS] CVS: Products/Ape/lib/apelib/zodb3 -
	serializers.py:1.3storage.py:1.8
In-Reply-To: <200309162100.h8GL08H01454@cvs.baymountain.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEECOFPAB.tim@zope.com>

[Shane Hathaway]
> Modified Files:
> 	serializers.py storage.py
> Log Message:
> Fixed Python 2.3 compatibility.
>
> - If you append to a list while it is being pickled, Python 2.3 will
>   pick up the new items.  Previous versions of Python did not.

How do you grow a list while it's being pickled?  Must be another thread
doing that, and if so that's mighty dubious.  If you're pickling to a
genuine file object, then cPickle releases the GIL around its fwrite()
calls, so that's one way.  But in the context of the patch:

+        p.persistent_id = lambda ob: None  # Stop recording references
         p.dump(unmanaged)
         s = outfile.getvalue()
         event.addUnmanagedPersistentObjects(unmanaged)

the use of .getvalue() suggests the pickle target isn't a real file.  Then
cPickle never releases the GIL (cStringIO doesn't either), and no other
thread should dare muck with a Python object (like the Python list getting
pickled) without holding the GIL.

However this ends up happening, if you instead shrink a list while it's
being pickled, Pythons earlier than 2.3 can end up trying to pickle random
recycled memory.  Their cPickles captured the list length once at the start
of pickling the list, and that coding optimization is appropriate only if
the GIL is never released.  2.3 uses the iteration protocol instead, so
"sees" dynamic changes in the list size.

Copying Python-Dev because this may reveal a new mountain of ways to crash
the interpreter:  mutate objects in devious ways while they're getting
cPickled.


From bac at OCF.Berkeley.EDU  Tue Sep 16 23:09:44 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Sep 16 23:09:49 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
In-Reply-To: <3F674015.8090801@ocf.berkeley.edu>
References: <3F669AF3.8050605@ocf.berkeley.edu>
	<16231.7058.987290.575924@montanaro.dyndns.org>
	<3F674015.8090801@ocf.berkeley.edu>
Message-ID: <3F67D078.1000106@ocf.berkeley.edu>

Brett C. wrote:

> Skip Montanaro wrote:
> 
>>     Brett> Well, Kurt Kaiser wrote up a patch (
>>     Brett> http://www.python.org/sf/762934 ) that seems sound.  I have
>>     Brett> gotten zero responses from anyone to test it on Red Hat 6.1 
>> and a
>>     Brett> remember that some people here were helping to test last 
>> time (I
>>     Brett> also remember this led to the creation of a wiki page listing
>>     Brett> platforms various people could test things on but I can't find
>>     Brett> the page).
>>
>> That would be:
>>
>>     http://www.python.org/cgi-bin/moinmoin/PythonTesters
>>
>> I see Kurt Kaiser as the only person listed who can test for RH < 6.2.
>> Perhaps you should broadcast a call for testers to c.l.py, referring 
>> to the
>> above page and asking in particular for RH 6.1 folks to chime in.
>>
> 
> Jordan Krushen pointed out that the patch is actually opened against RH 
> 6.2, not 6.1 like I originally said.  But I don't think it is that big 
> of an issue since there is a bug report for this against SuSE.  I think 
> the key thing here is an older version of glibc since that would be what 
> defines the tzset function.
> 

Well, Jordan emailed me personally with the result of running RH 6.1 and 
RH 6.2 under VMWare; both passed (thanks, Jordan)!  So, does anyone have 
any objections to me checking this in, or do people feel this is not 
enough testing to show the patch does what it needs to?

And a question on patching configure.pre.in: is that all I have to patch 
or do I need to run reconfigure and then also patch configure.in?

-Brett


From martin at v.loewis.de  Wed Sep 17 00:26:57 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Wed Sep 17 00:27:20 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
In-Reply-To: <3F67D078.1000106@ocf.berkeley.edu>
References: <3F669AF3.8050605@ocf.berkeley.edu>
	<16231.7058.987290.575924@montanaro.dyndns.org>
	<3F674015.8090801@ocf.berkeley.edu>
	<3F67D078.1000106@ocf.berkeley.edu>
Message-ID: <m3u17c2g6m.fsf@mira.informatik.hu-berlin.de>

"Brett C." <bac@OCF.Berkeley.EDU> writes:

> And a question on patching configure.pre.in: is that all I have to
> patch or do I need to run reconfigure and then also patch configure.in?

There is no configure.pre.in in Python.

Martin

From hernan at orgmf.com.ar  Wed Sep 17 02:53:18 2003
From: hernan at orgmf.com.ar (=?us-ascii?Q?Hernan_Martinez_Foffani?=)
Date: Wed Sep 17 02:55:24 2003
Subject: [Python-Dev] RE: HTMLHelp for Py2.3.1
In-Reply-To: <8yook4or.fsf@python.net>
Message-ID: <HPEBIMANNCHHDMDBGCHHAECFDLAA.hernan@orgmf.com.ar>

[Thomas]
>[Hernan]
>> Standard .chm format already includes useful index data to do
>> context-sensitive search. A couple of years ago I made an Idle
>> extension to manage this. It wasn't difficult then, and I think it's
>> even easier now. The problem I found is that you pass the selected
>> text (the one you are looking for) to HTMLHelp system by calling a
>> Win32 API. 
> I know this, and how to do the windows side. What I meant was that I
> don't know enough about IDLE (or Tkinter) to implement the
> functionality to find the word under the cursor.

I'm pretty sure that some IDLE extension used to have such functionality.
It wasn't that difficult. It was a patch I did a couple of years now
and wasn't finish. If I can find it I'll tell you my approach.

>> You'll need win32all installed. That's the way PythonWin works, by
>> the way. I guess this makes it a no-no for standard Python, right?
>> 
> I don't think win32all wraps the HTMLHelp api, does it?

Yes, it does. I think Mark(?) did it for his PythonWin IDE.

> OTOH, I'm waiting for a reason to include ctypes with core python ;-)

So there is one. ;-)

-Hernan


From mal at lemburg.com  Wed Sep 17 04:08:13 2003
From: mal at lemburg.com (M.-A. Lemburg)
Date: Wed Sep 17 04:08:23 2003
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEBMFPAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCIEBMFPAB.tim.one@comcast.net>
Message-ID: <3F68166D.5030206@lemburg.com>

Tim Peters wrote:
> [M.-A. Lemburg]
> 
>>Note that the implementation will bomb in several places if
>>Py_UNICODE is a signed type.
>>
>>Py_UNICODE was never intended to be a signed type, so the proper
>>fix would be to add logic so that Py_UNICODE gets forced to be an
>>unsigned type.
> 
> Jeremy believed Py_UNICODE was already an unsigned type on his box, and that
> was the box with the segfaults.  I don't know.  Comparison with a signed int
> confused the issues to the point where we gave up and just fixed it <wink>.

That sounds more like compiler bug to me. What's bothering me
is that such compares are done in other places too, so a more
general solution would be better.

>>The only case where Py_UNICODE could become signed is via
>>a compiler that defines wchar_t to be signed -- rather unlikely.
> 
> The C standard requires wchar_t to be an integer type, but doesn't constrain
> it further to an unsigned integer type.  I don't like relying on
> non-standard assumptions in cases where there's little-to-no cost in not
> relying on them.  For example, the cast I put in with this patch is probably
> a nop on most boxes, just forcing an unsigned comparison (which must have
> been the original intent, if Py_UNICODE was assumed to be an unsigned type).

No question there, but wouldn't it be easier to test such a platform
and then fallback to "unigned int" in case wchar_t is found to
be a signed value ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Sep 17 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::


From shane at zope.com  Wed Sep 17 08:34:50 2003
From: shane at zope.com (Shane Hathaway)
Date: Wed Sep 17 08:35:31 2003
Subject: [Python-Dev] RE: [Zope-CVS] CVS: Products/Ape/lib/apelib/zodb3 -
 serializers.py:1.3storage.py:1.8
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEECOFPAB.tim@zope.com>
References: <LNBBLJKPBEHFEDALKOLCEECOFPAB.tim@zope.com>
Message-ID: <Pine.LNX.4.50.0309170756120.18346-100000@shane.zope.com>

On Tue, 16 Sep 2003, Tim Peters wrote:

> [Shane Hathaway]
> > Modified Files:
> > 	serializers.py storage.py
> > Log Message:
> > Fixed Python 2.3 compatibility.
> >
> > - If you append to a list while it is being pickled, Python 2.3 will
> >   pick up the new items.  Previous versions of Python did not.
> 
> How do you grow a list while it's being pickled?  Must be another thread
> doing that, and if so that's mighty dubious.  If you're pickling to a
> genuine file object, then cPickle releases the GIL around its fwrite()
> calls, so that's one way.  But in the context of the patch:
> 
> +        p.persistent_id = lambda ob: None  # Stop recording references
>          p.dump(unmanaged)
>          s = outfile.getvalue()
>          event.addUnmanagedPersistentObjects(unmanaged)

Actually, the persistent_id() function was appending to the "unmanaged" 
list on each call.  Here is a simplified version of the mistake:


unmanaged = []

def persistent_id(ob):
    unmanaged.append(ob)
    return None

s = StringIO()
p = Pickler(s)
p.persistent_id = persistent_id
p.dump(unmanaged)


Python 2.2 apparently records the length of the list being pickled before
analyzing its contents.  This meant that my mistake went unnoticed.  (I
did not intend to modify "unmanaged" while it was being pickled!  There
are two passes in the actual code; the "unmanaged" list is built up in the
first pass and pickled in the second pass.)  Python 2.3, on the other
hand, picks up each new item as it is appended, resulting in a finite but
very long loop.

> Copying Python-Dev because this may reveal a new mountain of ways to crash
> the interpreter:  mutate objects in devious ways while they're getting
> cPickled.

I wouldn't be too concerned.  Only people who do crazy things with 
persistent_id() hooks will be affected. ;-)

Shane

From neal at metaslash.com  Wed Sep 17 09:13:54 2003
From: neal at metaslash.com (Neal Norwitz)
Date: Wed Sep 17 09:13:58 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
In-Reply-To: <3F67D078.1000106@ocf.berkeley.edu>
References: <3F669AF3.8050605@ocf.berkeley.edu>
	<16231.7058.987290.575924@montanaro.dyndns.org>
	<3F674015.8090801@ocf.berkeley.edu>
	<3F67D078.1000106@ocf.berkeley.edu>
Message-ID: <20030917131354.GC1211@epoch.metaslash.com>

On Tue, Sep 16, 2003 at 08:09:44PM -0700, Brett C. wrote:
> 
> And a question on patching configure.pre.in: is that all I have to patch 
> or do I need to run reconfigure and then also patch configure.in?

As Martin said, there is no configure.pre.in.  The file is configure.in
and you need to regenerate configure and pyconfig.h.in with autoconf
and autoheader (version 2.57).

Neal

From lomax at pumpichank.com  Wed Sep 17 09:18:26 2003
From: lomax at pumpichank.com (Frnak "Sealmbcrd Eggs" Lomax)
Date: Wed Sep 17 09:18:31 2003
Subject: [Python-Dev] Fsrit Propttoye: SCLABMRE
Message-ID: <16232.24354.366003.171152@gargle.gargle.HOWL>


I hvae jsut fhiseind rniedag a Shdlaost atrlcie from Mnoday, Sbepemetr
15th, aobut an ingtriniug sdtuy from Cgbdirmae Uinevtrsiy, dciinesrbg
how wodrs with thier itienrors selmrbacd can siltl be rdbaleae.
Bcaeuse of the tioeznme that I lvie in, an initettrment dfceet in my
clbae medom, and the lneiignrg atfer-ecfefts of my lsat "mohsroum
salad exntarvzagaa", I am aaylws two days bhneid on the Web.

So suentnd was I at the icoinatpimls of this study, taht I
cunlqtneesoy decedid to doetve my etrnie lfie and gtelray aeamssd
fnrtoue on cunitodcng ftrhuer rasreech and dnpeeelmvot into tihs
lifnsaevig tghlocnoey.  As a cross-eeyd dxsiyelc, I konw that teshe
cilmas of svotailan are niehetr hlroyepbe nor bmobaistc
over-daaomtiazitrn.

Pnseetred hree for your rveiew, aaovpprl and disiscsuon, is our
poyptotre sytsem, caelld SLABCRME for SeCren Rslmnebeig Aacrihc
Mnoaaluptiin of BetaoLd Ecuitolon.  I have scnie bugen a meosdt
sutartp based in Aceicdnt Maarnlyd, and we will soon be reeianlsg an
Olutook puglin, anolg with a sftorwae devmoeelpnt kit, and oethr
digintoasc tloos.  Smoe of our tlgeocnhoy wlil nallaruty be open
sruoce, but I do ietnnd and ecpext to frutehr etnxed my hoougmuns
wtaleh by sinlelg sftorawe lcseiens and srhink wrapped ratiel pocrudt.
Psaele cactnot us for more iomrfaiontn.

I'm aslo in enpommelyt duicnssioss wtih the Tmboit; I ecepxt that he
can bosot the pmecarrofne of our pooyptrte's itmpeanetolmin
salreolefvd, at wcihh piont we will be pthincig emdbeded ssmeyts to
the mfntacuuearrs of ruoters, eiltncreoc trprwyeiets, psnreoal dtaiigl
astsntsais, eorctinlec cepmutor coltlerond tersotas, maet prbeos, RFC
1149 digtail cemara pheons, and nxet gniaoetern wopehoe csionuhs.

hinpog-to-be-deotersyd-by-moroscift-ly y'rs,
-fnark

-------------- next part --------------
#! /usr/bin/env python
# Author: Frank "Topsy Turvy" Lomax <lomax@pumpichank.com>

"""Slmbcrae the letrtes in a word.

Tihs was seen on Soalhsdt, so it msut be ture:

Ardcnoicg to resraceh at Cgmdrabie Uesnirtivy, it dsoen't matter in what odrer
the lettres in a wrod are, the olny iotrmpant tnhig is taht the fsrit and lsat
lteter be at the rhigt pcale.

The rest can be a toatl mess and you can sltil raed it wuhoitt peolbrm.  This
is bseacue we do not raed eervy ltteer by it self but the wrod as a wohle.
Cireheo.
"""

import re
import sys
from random import shuffle

EMPTYSTRING = ''


def scrambleword(word):
    if len(word) <= 2:
        return word
    middle = list(word[1:-1])
    shuffle(middle)
    return word[0] + EMPTYSTRING.join(middle) + word[-1]


def main():
    text = sys.stdin.read()
    words = re.split(r'(\W+)', text)
    while words:
        sys.stdout.write(scrambleword(words.pop(0)))
        if words:
            sys.stdout.write(words.pop(0))


if __name__ == '__main__':
    main()
From jeremy at alum.mit.edu  Wed Sep 17 10:12:37 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed Sep 17 10:12:47 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEBGFPAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCGEBGFPAB.tim.one@comcast.net>
Message-ID: <1063807956.10299.31.camel@localhost.localdomain>

I've applied the latest patch to the head, 2.3, and 2.2 branches.  It
looks like it isn't necessary on the 2.1 branch.  While the
small-Unicode-keepalive code is still there, the length-one Unicode
strings aren't immortal.  Since there is no-special case code looking at
str[0], it appears unnecessary to initialize it.

Jeremy



From jepler at unpythonic.net  Wed Sep 17 10:59:11 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Wed Sep 17 10:59:26 2003
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <3F68166D.5030206@lemburg.com>
References: <LNBBLJKPBEHFEDALKOLCIEBMFPAB.tim.one@comcast.net>
	<3F68166D.5030206@lemburg.com>
Message-ID: <20030917145905.GF29089@unpythonic.net>

On Wed, Sep 17, 2003 at 10:08:13AM +0200, M.-A. Lemburg wrote:
> No question there, but wouldn't it be easier to test such a platform
> and then fallback to "unigned int" in case wchar_t is found to
> be a signed value ?

This inaccurate comment created by configure [from 2.3b1] implies that
sds/2 already expects wchar_t to be unsigned if it is to be usable:
  if test "$unicode_size" = "$ac_cv_sizeof_wchar_t"
  then
    PY_UNICODE_TYPE="wchar_t"
    AC_DEFINE(HAVE_USABLE_WCHAR_T, 1,
    [Define if you have a useable wchar_t type defined in wchar.h; useable
     means wchar_t must be 16-bit unsigned type. (see
     Include/unicodeobject.h).])
    AC_DEFINE(PY_UNICODE_TYPE,wchar_t)
... but configure doesn't actually check for signedness, and the "16-bit"
part is inaccurate. (it must be 16 bits for ucs-2, or 32-bits for ucs-4)

Jeff

From jeremy at alum.mit.edu  Wed Sep 17 11:05:27 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Wed Sep 17 11:05:31 2003
Subject: [Python-Dev] pending deprecation warning for Set.update
Message-ID: <1063811126.10299.42.camel@localhost.localdomain>

It looks like current CVS has grown a pending deprecation warning for
the update() method of Sets and that this change has been backported to
the 2.3 branch.  It seems to me to be bad form to add a warning in a
maintenance release.  The update() method was new in 2.3 and documented
in the 2.3 docs.  So people are going to write code using update() and
then update to 2.3.1 and get warnings?

I'm also not clear on the rationale for the warning.  update() seems to
match the semantics of dict.update(), and it less verbose than the
suggested alternative -- union_update().

Jeremy



From skip at pobox.com  Wed Sep 17 11:51:04 2003
From: skip at pobox.com (Skip Montanaro)
Date: Wed Sep 17 11:51:16 2003
Subject: [Python-Dev] pending deprecation warning for Set.update
In-Reply-To: <1063811126.10299.42.camel@localhost.localdomain>
References: <1063811126.10299.42.camel@localhost.localdomain>
Message-ID: <16232.33512.613600.989386@montanaro.dyndns.org>


    Jeremy> It looks like current CVS has grown a pending deprecation
    Jeremy> warning for the update() method of Sets ...

+1 on everything Jeremy wrote.

Skip

From python at rcn.com  Wed Sep 17 12:02:00 2003
From: python at rcn.com (Raymond Hettinger)
Date: Wed Sep 17 12:02:09 2003
Subject: [Python-Dev] pending deprecation warning for Set.update
References: <1063811126.10299.42.camel@localhost.localdomain>
Message-ID: <003401c37d35$0784fac0$9c3dc797@oemcomputer>

> It looks like current CVS has grown a pending deprecation warning for
> the update() method of Sets and that this change has been backported to
> the 2.3 branch.  It seems to me to be bad form to add a warning in a
> maintenance release.

To deprecate would be bad form.
To provide as much advanced warning as possible is good form.
That is much better than having 2.3.1 users spending the next year
developing code with update() and then having it deprecated in 2.4.
They might as well know right now.

> I'm also not clear on the rationale for the warning.  update() seems to
> match the semantics of dict.update(), and it less verbose than the
> suggested alternative -- union_update().

Part of the reason for initially implementing sets in Python instead of C
and for not having it as a builtin was was that the API experimental.
This change was based in part upon user feedback solicited from the
newsgroup.

The verbosity issue weighs less because there is the __ior__ operator
alternative:   a |= b.   That is succinct as you can get.  And the long
form is nothing compared to Set.symmetric_difference_update()    ;-)


Raymond




#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

From python at rcn.com  Wed Sep 17 12:18:15 2003
From: python at rcn.com (Raymond Hettinger)
Date: Wed Sep 17 12:18:21 2003
Subject: [Python-Dev] pending deprecation warning for Set.update
References: <1063811126.10299.42.camel@localhost.localdomain>
	<16232.33512.613600.989386@montanaro.dyndns.org>
Message-ID: <004e01c37d37$4cd979a0$9c3dc797@oemcomputer>

>     Jeremy> It looks like current CVS has grown a pending deprecation
>     Jeremy> warning for the update() method of Sets ...
[Skip] 
> +1 on everything Jeremy wrote.

One other thought:  having a PendingDeprecationWarning is somewhat
innocuous.  By default, those warnings are filtered out (set to Ignore).  
You have to ask to see them.    It would be much different for a 
DeprecationWarning.

It is pretty conservative to issue a pending warning now, a deprecation
warning in 2.4, and remove the duplicate method in 2.5.  This is doubly
true for a brand new module that has only a minimal existing code base.
Not bringing it up for 12 to18 months is, IMHO, much more impactful.


Raymond

From nas-python at python.ca  Wed Sep 17 12:44:35 2003
From: nas-python at python.ca (Neil Schemenauer)
Date: Wed Sep 17 12:37:16 2003
Subject: [Python-Dev] pending deprecation warning for Set.update
In-Reply-To: <003401c37d35$0784fac0$9c3dc797@oemcomputer>
References: <1063811126.10299.42.camel@localhost.localdomain>
	<003401c37d35$0784fac0$9c3dc797@oemcomputer>
Message-ID: <20030917164435.GA19007@mems-exchange.org>

On Wed, Sep 17, 2003 at 12:02:00PM -0400, Raymond Hettinger wrote:
> The verbosity issue weighs less because there is the __ior__ operator
> alternative:   a |= b.   That is succinct as you can get.

>>> from sets import Set
>>> s = Set()
>>> s |= [1, 2, 3]
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/www/plat/python2.3/lib/python2.3/sets.py", line 420, in __ior__
    self._binary_sanity_check(other)
  File "/www/plat/python2.3/lib/python2.3/sets.py", line 313, in _binary_sanity_check
    raise TypeError, "Binary operation only permitted between sets"
TypeError: Binary operation only permitted between sets
>>> s.update([1, 2, 3])
>>> s
Set([1, 2, 3])
>>> 

I quite a few instances of update() that will have to be changed to
union_update().

  Neil

From raymond.hettinger at verizon.net  Wed Sep 17 13:46:46 2003
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Wed Sep 17 13:46:55 2003
Subject: [Python-Dev] Py2.3.1
Message-ID: <00ce01c37d43$aa879940$9c3dc797@oemcomputer>

Are we on for doing a micro release this week?


Raymond Hettinger

From jepler at unpythonic.net  Wed Sep 17 15:40:29 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Wed Sep 17 15:40:33 2003
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <20030917145905.GF29089@unpythonic.net>
References: <LNBBLJKPBEHFEDALKOLCIEBMFPAB.tim.one@comcast.net>
	<3F68166D.5030206@lemburg.com>
	<20030917145905.GF29089@unpythonic.net>
Message-ID: <20030917194029.GI29089@unpythonic.net>

On Wed, Sep 17, 2003 at 09:59:11AM -0500, Jeff Epler wrote:
> This inaccurate comment created by configure [from 2.3b1] implies that
> _____ already expects wchar_t to be unsigned if it is to be usable:

urm, I meant to refer to Python, not a software product most of you are
lucky enough to have never heard of...

Jeff

From martin at v.loewis.de  Wed Sep 17 17:39:32 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Wed Sep 17 17:39:53 2003
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <3F68166D.5030206@lemburg.com>
References: <LNBBLJKPBEHFEDALKOLCIEBMFPAB.tim.one@comcast.net>
	<3F68166D.5030206@lemburg.com>
Message-ID: <m3r82fxffv.fsf@mira.informatik.hu-berlin.de>

"M.-A. Lemburg" <mal@lemburg.com> writes:

> No question there, but wouldn't it be easier to test such a platform
> and then fallback to "unigned int" in case wchar_t is found to
> be a signed value ?

Why is it important that Py_UNICODE is unsigned?

Regards,
Martin

From vinay_sajip at yahoo.co.uk  Wed Sep 17 17:43:58 2003
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed Sep 17 17:42:13 2003
Subject: [Python-Dev] Consistent logging in the standard library
Message-ID: <005101c37d64$d37270c0$652b6992@alpha>

A recent thread has discussed integrating logging with the rest of the
standard library. Among other things, a question has arisen about what
logging channel names standard library modules should use, and possible
clashes with channel names used by applications.

I'd like to propose that some portion of the namespace be reserved for use
by logging in the standard library. If acceptable, all that would need to be
done would be to update the documentation to indicate the namespace
reservation. The actual use of the namespace could be implemented as and
when each module is changed to use the logging module.

The scheme of having all standard library logging under one umbrella allows
applications to be configured so that all library-level messages can be
routed to specific handlers, filtered simply at one point, etc.

My preference for namespace reservation is that any logger name beginning
with "python." should be reserved for use by the standard distribution. I
think that each module should use an eponymous logging channel name rooted
in the hierarchy for the Python library - e.g. asyncore would use
"python.asyncore", etc.

Matthew Barnes, who kicked off the original thread, prefers "stdlib" as the
prefix to use, rather than "python".

Comments, please!


Vinay Sajip


From martin at v.loewis.de  Wed Sep 17 17:45:14 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Wed Sep 17 17:45:28 2003
Subject: [Python-Dev] pending deprecation warning for Set.update
In-Reply-To: <004e01c37d37$4cd979a0$9c3dc797@oemcomputer>
References: <1063811126.10299.42.camel@localhost.localdomain>
	<16232.33512.613600.989386@montanaro.dyndns.org>
	<004e01c37d37$4cd979a0$9c3dc797@oemcomputer>
Message-ID: <m3n0d3xf6d.fsf@mira.informatik.hu-berlin.de>

"Raymond Hettinger" <python@rcn.com> writes:

> It is pretty conservative to issue a pending warning now, a deprecation
> warning in 2.4, and remove the duplicate method in 2.5.

No, it is *not* conservative to add a warning in a maintenance branch,
pending or not. Even changing the documentation only would not be
conservative.

It might be sensible (which I think it is not), but it is not
conservative.

Regards,
Martin

From barry at python.org  Wed Sep 17 17:56:50 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Sep 17 17:56:41 2003
Subject: [Python-Dev] Consistent logging in the standard library
In-Reply-To: <005101c37d64$d37270c0$652b6992@alpha>
References: <005101c37d64$d37270c0$652b6992@alpha>
Message-ID: <1063835810.14316.20.camel@geddy>

On Wed, 2003-09-17 at 17:43, Vinay Sajip wrote:

> Matthew Barnes, who kicked off the original thread, prefers "stdlib" as the
> prefix to use, rather than "python".

What about _ (underscore)?

One of my half-baked totally unoriginal PEP ideas for Python 2.4 would
be a syntax for forcing imports from the standard library.  E.g. say I
have a foo package, and inside there I've got a foo/logging.py module. 
Say there's also a foo/bar.py module and it wants to import the global
logging package.  How can it do that?  "import logging" does a relative
import.

So the idea would be to be able to write "import _.logging" and
definitely get the global logging package.  Using underscore in the
logger would mirror this mnemonic for globalness.

-Barry



From azaidi at vsnl.com  Wed Sep 17 18:11:35 2003
From: azaidi at vsnl.com (Arsalan Zaidi)
Date: Wed Sep 17 18:12:53 2003
Subject: [Python-Dev] Consistent logging in the standard library
References: <005101c37d64$d37270c0$652b6992@alpha>
Message-ID: <016801c37d68$b2169100$31469cca@LocalHost>

Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
> I'd like to propose that some portion of the namespace be reserved
> for use by logging in the standard library. If acceptable, all that
> would need to be done would be to update the documentation to
> indicate the namespace reservation. The actual use of the namespace
> could be implemented as and when each module is changed to use the
> logging module.
>
> The scheme of having all standard library logging under one umbrella
> allows applications to be configured so that all library-level
> messages can be routed to specific handlers, filtered simply at one
> point, etc.

I agree with this and was infact about to propose this to Matthew Barnes a
short while ago. Good thing I read the list first. :-)

> My preference for namespace reservation is that any logger name
> beginning with "python." should be reserved for use by the standard
> distribution. I think that each module should use an eponymous
> logging channel name rooted in the hierarchy for the Python library -
> e.g. asyncore would use "python.asyncore", etc.
>
> Matthew Barnes, who kicked off the original thread, prefers "stdlib"
> as the prefix to use, rather than "python".

No strong preference either way, but if someone were to hold a gun to my
head, I'd scream... And then express my preference for 'python' as the root.
But that's just the Java talking :-)

--Arsalan


From azaidi at vsnl.com  Wed Sep 17 18:14:18 2003
From: azaidi at vsnl.com (Arsalan Zaidi)
Date: Wed Sep 17 18:15:32 2003
Subject: [Python-Dev] Consistent logging in the standard library
References: <005101c37d64$d37270c0$652b6992@alpha>
Message-ID: <017201c37d69$14898220$31469cca@LocalHost>

Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
> I'd like to propose that some portion of the namespace be reserved
> for use by logging in the standard library. If acceptable, all that
> would need to be done would be to update the documentation to
> indicate the namespace reservation. The actual use of the namespace
> could be implemented as and when each module is changed to use the
> logging module.
>
> The scheme of having all standard library logging under one umbrella
> allows applications to be configured so that all library-level
> messages can be routed to specific handlers, filtered simply at one
> point, etc.

I agree with this and was infact about to propose this to Matthew Barnes a
short while ago. Good thing I read the list first. :-)

> My preference for namespace reservation is that any logger name
> beginning with "python." should be reserved for use by the standard
> distribution. I think that each module should use an eponymous
> logging channel name rooted in the hierarchy for the Python library -
> e.g. asyncore would use "python.asyncore", etc.
>
> Matthew Barnes, who kicked off the original thread, prefers "stdlib"
> as the prefix to use, rather than "python".

No strong preference either way, but if someone were to hold a gun to my
head, I'd scream... And then express my preference for 'python' as the root.
But that's just the Java talking :-)

Someone proposed an '_'. I don't know. My gut feeling is that it's too
cryptic and Perl like. If you must have something, call it 'base' or
'python' or even 'stdlib'. But what do I know...

--Arsalan


From azaidi at vsnl.com  Wed Sep 17 18:24:26 2003
From: azaidi at vsnl.com (Arsalan Zaidi)
Date: Wed Sep 17 18:25:33 2003
Subject: [Python-Dev] Consistent logging in the standard library
References: <005101c37d64$d37270c0$652b6992@alpha>
	<016801c37d68$b2169100$31469cca@LocalHost>
Message-ID: <019501c37d6a$7bd57d20$31469cca@LocalHost>

I've accidently sent two slightly different emails on the same topic... Read
the one timestamped 6:15 am.

I know; my bad. No cookies for me for the next 24 hours...

--Arsalan


From skip at pobox.com  Wed Sep 17 18:29:45 2003
From: skip at pobox.com (Skip Montanaro)
Date: Wed Sep 17 18:29:59 2003
Subject: [Python-Dev] Consistent logging in the standard library
In-Reply-To: <1063835810.14316.20.camel@geddy>
References: <005101c37d64$d37270c0$652b6992@alpha>
	<1063835810.14316.20.camel@geddy>
Message-ID: <16232.57433.388130.573539@montanaro.dyndns.org>


    Barry> So the idea would be to be able to write "import _.logging" and
    Barry> definitely get the global logging package.  Using underscore in
    Barry> the logger would mirror this mnemonic for globalness.

The connection seems tenuous, at best.  If I log messages to syslog and and
that file is later viewed or summarized by a third party, "python" seems a
lot more useful than "_" or even "stdlib".

Skip

From cs1spw at bath.ac.uk  Wed Sep 17 19:57:43 2003
From: cs1spw at bath.ac.uk (Simon Willison)
Date: Wed Sep 17 19:56:58 2003
Subject: [Python-Dev] Improving the CGI module
Message-ID: <3F68F4F7.6020601@bath.ac.uk>

Hi all,

I've recently started using Python for server side web development, and 
I've been running in to some limitations in the CGI module. The most 
critical of these is that the module provides no method of determining 
whether form data originated in a query string (GET data) or a POSTed 
form. The module also provides very little in the way of support for 
common CGI related operations such as sending common headers and parsing 
and sending cookies; the Cookie module provides parsing capabilities but 
requires additional glue code to extract cookie data from the CGI 
environment.

I have previous experience with PHP, which has a number of extremely 
convenient mechanisms for dealing with form input and cookies that I 
sorely miss when working with Python. I have started work on porting 
some of these capabilities, but it struck me that several of the ideas 
represented would make valuable additions to the Python standard library 
itself.

I have been working with Python for over a year, but I have no real 
experience of the Python development process. Is this list the right 
place to discuss potential improvements to the CGI module, or should I 
work independently and construct a PEP? I noticed that PEP 222 already 
exists relating to "web library enhancements", but it has a 'Deferred' 
status and appears not to have been modified since December 2000.

Best regards,

Simon Willison
http://simon.incutio.com/categories/python/




From janssen at parc.com  Wed Sep 17 20:15:57 2003
From: janssen at parc.com (Bill Janssen)
Date: Wed Sep 17 20:16:25 2003
Subject: [Python-Dev] Improving the CGI module 
In-Reply-To: Your message of "Wed, 17 Sep 2003 16:57:43 PDT."
	<3F68F4F7.6020601@bath.ac.uk> 
Message-ID: <03Sep17.171602pdt."58611"@synergy1.parc.xerox.com>

Simon,

I sent the following out last week, and I'm currently working on a Web
page and sample charter for a Web-SIG devoted to improving the Python
Web support.  Let's get together and do it!

Bill

------------------------------------------
 Bill Janssen  <janssen@parc.com>     (650) 812-4763    FAX: (650) 812-4258
 Palo Alto Research Center, Inc., 3333 Coyote Hill Rd, Palo Alto, CA  94304


(From last week:)

Subject: Another try at a Python Web SIG?
From: Bill Janssen <janssen@parc.com>
Date: Thu, 11 Sep 2003 13:53:48 PDT
To: meta-sig@python.org

I know this was just discussed a few months ago, but I'm very
interested in getting a Web SIG together or reinvigorated or whatever
it takes.  The current Web support in Python is somewhat embarassingly
patchy, given the neatness of the rest of the system.

My specific thinking is about three things:

1)  Add SSL server-side support to the "socket" module.  This will
    include some way of managing certificates.  Probably modelled
    strongly after the way we did it in ILU, if I do it :-).

2)  Update the "httplib" module to include all of the client-side
    functionality available in the Linux "curl" program (a reasonable
    checklist, I think, but am open to better suggestions).

3)  Replace/extend the BaseHTTPServer and SimpleHTTPServer modules with
    an extended webserver module which provides at least the
    functionality in Medusa.

I'd be happy to work in #1 (starting with a PEP, I think) and provide
help with #2 and #3, if others would like to pitch in.  I don't want
to collide with others who are working on this already, and think a
SIG would be a good way to coordinate efforts.

Bill

From ping at zesty.ca  Wed Sep 17 20:21:59 2003
From: ping at zesty.ca (Ping)
Date: Wed Sep 17 20:22:09 2003
Subject: [Python-Dev] Improving the CGI module
In-Reply-To: <3F68F4F7.6020601@bath.ac.uk>
Message-ID: <Pine.LNX.4.33.0309171717540.8789-100000@server1.lfw.org>

On Thu, 18 Sep 2003, Simon Willison wrote:
> I've recently started using Python for server side web development, and
> I've been running in to some limitations in the CGI module. [...]
> The module also provides very little in the way of support for
> common CGI related operations ...

I think that improvements to the CGI module would be wonderful,
and we could do a lot of useful things ranging from the most basic
(e.g. friendly access to the CGI environment variables) to more
sophisticated things (e.g. constructing URLs that submit forms).

(I have recently been forced to use PHP instead of Python for some
major projects and constantly feel the pain of using PHP -- in my
opinion, it is full of inconsistencies and workarounds, and really
has the feel of a language that was not carefully designed.  It
would be nice for Python to be more competitive with PHP in its
support for simple Web scripting.)

I would be interested in helping with the CGI module project.


-- ?!ng


From tim.one at comcast.net  Wed Sep 17 20:22:13 2003
From: tim.one at comcast.net (Tim Peters)
Date: Wed Sep 17 20:22:22 2003
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <3F68166D.5030206@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEFOFPAB.tim.one@comcast.net>

[Tim]
>> Jeremy believed Py_UNICODE was already an unsigned type on his box,
>> and that was the box with the segfaults.  I don't know.  Comparison
>> with a signed int confused the issues to the point where we gave up
>> and just fixed it <wink>.

[M.-A. Lemburg]
> That sounds more like compiler bug to me.

It could be, but if so Jeremy is running on a mainstream Linux+gcc platform
and then it's something we can't really wish away.  Jeremy, can you tell us
what Py_UNICODE resolves to on your box, or give enough details so someone
else can figure it out?

As I read the C standard,

     unsigned_int < 256

has to use unsigned comparison, so it's a compiler bug, or I'm misreading
the standard, or Jeremy was mistaken in believing Py_UNICODE resolves to an
unsigned thingie on his box (we know for sure that the bit pattern
0xcdcdcdcd compared less than 256 on his box; that's obviously what it
should do if Py_UNICODE resolves to a signed 4-byte thing on his box, but
not otherwise).

> What's bothering me is that such compares are done in other places too,
> so a more general solution would be better.

I'd like to figure out what Jeremy's true problem was first -- we've got a
solution to his symptom now, but don't really know why it was necessary.


From mfb at lotusland.dyndns.org  Wed Sep 17 20:22:27 2003
From: mfb at lotusland.dyndns.org (Matthew F. Barnes)
Date: Wed Sep 17 20:36:43 2003
Subject: [Python-Dev] Consistent logging in the standard library
In-Reply-To: <16232.57433.388130.573539@montanaro.dyndns.org>
References: <005101c37d64$d37270c0$652b6992@alpha><1063835810.14316.20.camel@geddy
	> <16232.57433.388130.573539@montanaro.dyndns.org>
Message-ID: <57798.130.76.32.20.1063844547.squirrel@server.lotusland.dyndns.org>

Skip Montanaro said:
>
>     Barry> So the idea would be to be able to write "import _.logging" and
>     Barry> definitely get the global logging package.  Using underscore in
>     Barry> the logger would mirror this mnemonic for globalness.
>
> The connection seems tenuous, at best.  If I log messages to syslog and
> and
> that file is later viewed or summarized by a third party, "python" seems a
> lot more useful than "_" or even "stdlib".
>

I agree with Barry that a name should be given to the standard library
"package" and that the namespace reserved in the logging hierarchy for
standard library modules should match this name.

On the other hand, "_" is making me have unpleasant Perl flashbacks. 
*shiver*  So I also agree with Skip that something more verbose would be
more useful.

I'm okay with waiting to see what becomes of Barry's idea before deciding
on a logging namespace.

Barry:

Not to get too far off-topic, but I'm curious what this statement would do:

    from _ import *

And was this notation ever considered?

    import .logging

(Not that that would help us much with the logging namespace issue. :-)

Matthew Barnes

From jeremy at zope.com  Wed Sep 17 21:19:33 2003
From: jeremy at zope.com (Jeremy Hylton)
Date: Wed Sep 17 21:21:10 2003
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEFOFPAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCEFOFPAB.tim.one@comcast.net>
Message-ID: <1063847973.10299.76.camel@localhost.localdomain>

On Wed, 2003-09-17 at 20:22, Tim Peters wrote:
> It could be, but if so Jeremy is running on a mainstream Linux+gcc platform
> and then it's something we can't really wish away.  Jeremy, can you tell us
> what Py_UNICODE resolves to on your box, or give enough details so someone
> else can figure it out?
> 
> As I read the C standard,
> 
>      unsigned_int < 256
> 
> has to use unsigned comparison, so it's a compiler bug, or I'm misreading
> the standard, or Jeremy was mistaken in believing Py_UNICODE resolves to an
> unsigned thingie on his box (we know for sure that the bit pattern
> 0xcdcdcdcd compared less than 256 on his box; that's obviously what it
> should do if Py_UNICODE resolves to a signed 4-byte thing on his box, but
> not otherwise).

I was a little confused by the various UNICODE macros.  (Is there a
comment block somewhere that explains what they are for?)

gcc -E tells me:

typedef unsigned int Py_UCS4;
typedef wchar_t Py_UNICODE;
typedef long int wchar_t;

(not necessarily in that order)

I got Py_UCS4 and Py_UNICODE confused.  The detailed output confirms
that Py_UNICODE is a signed long int.

Jeremy



From sholden at holdenweb.com  Wed Sep 17 21:22:29 2003
From: sholden at holdenweb.com (Steve Holden)
Date: Wed Sep 17 21:26:07 2003
Subject: [Python-Dev] Improving the CGI module
In-Reply-To: <Pine.LNX.4.33.0309171717540.8789-100000@server1.lfw.org>
Message-ID: <CGECIJPNNHIFAJKHOLMAIELGHBAA.sholden@holdenweb.com>

[Ping]
> On Thu, 18 Sep 2003, Simon Willison wrote:
> > I've recently started using Python for server side web
> development, and
> > I've been running in to some limitations in the CGI module. [...]
> > The module also provides very little in the way of support for
> > common CGI related operations ...
>
> I think that improvements to the CGI module would be wonderful,
> and we could do a lot of useful things ranging from the most basic
> (e.g. friendly access to the CGI environment variables) to more
> sophisticated things (e.g. constructing URLs that submit forms).
>
I'd agree that there's lots of room for new web functionality, but
having looked in some detail at the CGI module, and with backward
compatibility in mind, I'd question whether changes to that module are
the best way to do this.

> (I have recently been forced to use PHP instead of Python for some
> major projects and constantly feel the pain of using PHP -- in my
> opinion, it is full of inconsistencies and workarounds, and really
> has the feel of a language that was not carefully designed.  It
> would be nice for Python to be more competitive with PHP in its
> support for simple Web scripting.)
>
Unfortunately the CGI modules itself appears to have "just growed" in a
rather organic way. It could be that my analytic skills simply aren't up
to a task which others will breeze through, but I suspect that it might
be better to start with a different framework, relieving the project of
the need to make the same mistakes that CGI did. New mistakes are always
more interesting ;-)

> I would be interested in helping with the CGI module project.
>
My time is pretty limited, but I'd like to monitor such a project and
snipe from the sidelines from time to time.

regards
--
Steve Holden                                 http://www.holdenweb.com/
Python Web Programming                http://pydish.holdenweb.com/pwp/
Interview with GvR August 14, 2003       http://www.onlamp.com/python/




From tim.one at comcast.net  Wed Sep 17 22:55:35 2003
From: tim.one at comcast.net (Tim Peters)
Date: Wed Sep 17 22:55:41 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <1063847973.10299.76.camel@localhost.localdomain>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEGLFPAB.tim.one@comcast.net>

[Jeremy Hylton]
> I was a little confused by the various UNICODE macros.  (Is there a
> comment block somewhere that explains what they are for?)

Not that I've found.  If someone writes one, don't forget the intended
difference between PY_UNICODE_TYPE and Py_UNICODE (hint:  there isn't a
difference <wink>).

> gcc -E tells me:
>
> typedef unsigned int Py_UCS4;
> typedef wchar_t Py_UNICODE;
> typedef long int wchar_t;
>
> (not necessarily in that order)
>
> I got Py_UCS4 and Py_UNICODE confused.  The detailed output confirms
> that Py_UNICODE is a signed long int.

So that puts an end to the claim that it's unlikely wchar_t will resolve to
a signed type.  Strangely, while char is a signed type under MSVC, wchar_t
is an unsigned type.  I expect both differ under gcc, then.  At least it's
consistent <wink>.

Anyway, everywhere the code may be doing

    a_Py_UNICODE  comparison  a_(signed)_int

is doing something unintended now on your box.  "The rules" for
mixed-signedness comparison are pretty much a nightmare, especially when
you're not sure how many bits are involved on both sides:

    http://yarchive.net/comp/ansic_broken_unsigned.html

MAL's idea of forcing PY_UNICODE_TYPE to resolve to an unsigned type may be
the easiest way out.


From barry at python.org  Thu Sep 18 00:52:02 2003
From: barry at python.org (Barry Warsaw)
Date: Thu Sep 18 00:52:10 2003
Subject: [Python-Dev] Consistent logging in the standard library
In-Reply-To: <57798.130.76.32.20.1063844547.squirrel@server.lotusland.dyndns.org>
References: <005101c37d64$d37270c0$652b6992@alpha>
	<1063835810.14316.20.camel@geddy >
	<16232.57433.388130.573539@montanaro.dyndns.org>
	<57798.130.76.32.20.1063844547.squirrel@server.lotusland.dyndns.org>
Message-ID: <1063860722.9847.2.camel@anthem>

On Wed, 2003-09-17 at 20:22, Matthew F. Barnes wrote:

> Not to get too far off-topic, but I'm curious what this statement would do:
> 
>     from _ import *

I don't know.  Perhaps Lib should have an __init__.py that would export
an __all__?

> And was this notation ever considered?
> 
>     import .logging

It just was! :)

> (Not that that would help us much with the logging namespace issue. :-)

No, but maybe they shouldn't be tied.  Unless they should <wink>.  I
just wanted to bring it up because it seemed somewhat related and I've
been meaning to write a PEP on forced global imports for Python 2.4.

-Barry



From barry at python.org  Thu Sep 18 00:58:59 2003
From: barry at python.org (Barry Warsaw)
Date: Thu Sep 18 00:59:09 2003
Subject: [Python-Dev] Improving the CGI module
In-Reply-To: <CGECIJPNNHIFAJKHOLMAIELGHBAA.sholden@holdenweb.com>
References: <CGECIJPNNHIFAJKHOLMAIELGHBAA.sholden@holdenweb.com>
Message-ID: <1063861139.9847.10.camel@anthem>

On Wed, 2003-09-17 at 21:22, Steve Holden wrote:
> >
> Unfortunately the CGI modules itself appears to have "just growed" in a
> rather organic way. It could be that my analytic skills simply aren't up
> to a task which others will breeze through, but I suspect that it might
> be better to start with a different framework, relieving the project of
> the need to make the same mistakes that CGI did. New mistakes are always
> more interesting ;-)

+1!
-Barry



From martin at v.loewis.de  Thu Sep 18 01:19:06 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Sep 18 01:19:21 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAEGLFPAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCAEGLFPAB.tim.one@comcast.net>
Message-ID: <m3r82ey8qd.fsf@mira.informatik.hu-berlin.de>

"Tim Peters" <tim.one@comcast.net> writes:

> So that puts an end to the claim that it's unlikely wchar_t will resolve to
> a signed type.  Strangely, while char is a signed type under MSVC, wchar_t
> is an unsigned type.  

I think people have learned, over time, that negative characters are
evil. So MS chose to make wchar_t unsigned, as, for 2-byte Unicode,
you might otherwise run into negative characters. For gcc, people
probably also agree that negative characters are evil, but using long
int causes no harm here: All assigned characters are still positive,
as Unicode goes only up to 2**21.

Regards,
Martin

From anthony at interlink.com.au  Thu Sep 18 01:23:52 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu Sep 18 01:25:32 2003
Subject: [Python-Dev] Py2.3.1 
In-Reply-To: <00ce01c37d43$aa879940$9c3dc797@oemcomputer> 
Message-ID: <200309180523.h8I5Nru7031988@localhost.localdomain>


>>> "Raymond Hettinger" wrote
> Are we on for doing a micro release this week?

I was planning on doing it mid-next week. (I sent an email about this
a couple of days ago - it doesn't seem to have gone through?)

Anthony

From bac at OCF.Berkeley.EDU  Thu Sep 18 02:52:47 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Sep 18 02:52:59 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
In-Reply-To: <20030917131354.GC1211@epoch.metaslash.com>
References: <3F669AF3.8050605@ocf.berkeley.edu>	<16231.7058.987290.575924@montanaro.dyndns.org>	<3F674015.8090801@ocf.berkeley.edu>	<3F67D078.1000106@ocf.berkeley.edu>
	<20030917131354.GC1211@epoch.metaslash.com>
Message-ID: <3F69563F.6090209@ocf.berkeley.edu>

Neal Norwitz wrote:

> On Tue, Sep 16, 2003 at 08:09:44PM -0700, Brett C. wrote:
> 
>>And a question on patching configure.pre.in: is that all I have to patch 
>>or do I need to run reconfigure and then also patch configure.in?
> 
> 
> As Martin said, there is no configure.pre.in.  The file is configure.in
> and you need to regenerate configure and pyconfig.h.in with autoconf
> and autoheader (version 2.57).
> 

OK.  Well, I ran autoreconf and autoheader.  Both configure.in and 
configure look fine and ready to be committed.

But pyconfig.h.in is another thing entirely.  I just ran ``cvs diff`` on 
it and the diff I get is (on both HEAD and release23-maint):

+/* Define this if you have tcl and TCL_UTF_MAX==6 */
+#undef HAVE_UCS4_TCL
+

Uh, why is that new?  Is this from Martin's patch to detect RH's custom 
Tcl?  Could this be from autoheader not being run after that patch (or 
someone else's)?

-Brett


From mal at lemburg.com  Thu Sep 18 03:23:44 2003
From: mal at lemburg.com (M.-A. Lemburg)
Date: Thu Sep 18 03:23:42 2003
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <m3r82fxffv.fsf@mira.informatik.hu-berlin.de>
References: <LNBBLJKPBEHFEDALKOLCIEBMFPAB.tim.one@comcast.net>	<3F68166D.5030206@lemburg.com>
	<m3r82fxffv.fsf@mira.informatik.hu-berlin.de>
Message-ID: <3F695D80.6020804@lemburg.com>

Martin v. L?wis wrote:
> "M.-A. Lemburg" <mal@lemburg.com> writes:
> 
>>No question there, but wouldn't it be easier to test such a platform
>>and then fallback to "unigned int" in case wchar_t is found to
>>be a signed value ?
> 
> Why is it important that Py_UNICODE is unsigned?

Because that's what was used as basis in the type implementation
as well as the codecs (internal and external). Comparisons simply
work differently when you're using a signed type which is also
why most compilers warn about this -- but you know that.

An signed type also doesn't make much sense for things like
character storage -- the sign information is useless and you
lose a bit for each character.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Sep 18 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::


From vinay_sajip at yahoo.co.uk  Thu Sep 18 03:45:30 2003
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu Sep 18 03:43:42 2003
Subject: [Python-Dev] Consistent logging in the standard library
References: <005101c37d64$d37270c0$652b6992@alpha>
	<1063835810.14316.20.camel@geddy>
	<16232.57433.388130.573539@montanaro.dyndns.org>
Message-ID: <00ad01c37db8$d8bc9d40$652b6992@alpha>

>     Barry> So the idea would be to be able to write "import _.logging" and
>     Barry> definitely get the global logging package.  Using underscore in
>     Barry> the logger would mirror this mnemonic for globalness.
>
Skip> The connection seems tenuous, at best.  If I log messages to syslog
and and
Skip> that file is later viewed or summarized by a third party, "python"
seems a
Skip> lot more useful than "_" or even "stdlib".

I agree. Readability in logs is the most important consideration here, and I
think a scheme such as "python." + <modulename> makes it very clear where an
event originated. Of course this would just be a prefix; a module could log
to any name under this namespace, e.g. "python.asyncore.socket" for
socket-related events in asyncore.

Vinay


From mwh at python.net  Thu Sep 18 06:59:03 2003
From: mwh at python.net (Michael Hudson)
Date: Thu Sep 18 06:58:37 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
In-Reply-To: <3F69563F.6090209@ocf.berkeley.edu> (Brett C.'s message of
	"Wed, 17 Sep 2003 23:52:47 -0700")
References: <3F669AF3.8050605@ocf.berkeley.edu>
	<16231.7058.987290.575924@montanaro.dyndns.org>
	<3F674015.8090801@ocf.berkeley.edu>
	<3F67D078.1000106@ocf.berkeley.edu>
	<20030917131354.GC1211@epoch.metaslash.com>
	<3F69563F.6090209@ocf.berkeley.edu>
Message-ID: <2misnq8irs.fsf@starship.python.net>

"Brett C." <bac@OCF.Berkeley.EDU> writes:

> But pyconfig.h.in is another thing entirely.  I just ran ``cvs diff``
> on it and the diff I get is (on both HEAD and release23-maint):
>
> +/* Define this if you have tcl and TCL_UTF_MAX==6 */
> +#undef HAVE_UCS4_TCL
> +
>
> Uh, why is that new?  Is this from Martin's patch to detect RH's
> custom Tcl?  Could this be from autoheader not being run after that
> patch (or someone else's)?

Looks like it (as in, I see that here too, and I don't have any local
changes to configure.in).

Cheers,
mwh

-- 
  > I'm a little confused.
  That's because you're Australian!  So all the blood flows to
  your head, away from the organ most normal guys think with.
                        -- Mark Hammond & Tim Peters, comp.lang.python

From mfb at lotusland.dyndns.org  Thu Sep 18 08:04:57 2003
From: mfb at lotusland.dyndns.org (Matthew F. Barnes)
Date: Thu Sep 18 08:19:06 2003
Subject: [Python-Dev] Consistent logging in the standard library
In-Reply-To: <00ad01c37db8$d8bc9d40$652b6992@alpha>
References: <005101c37d64$d37270c0$652b6992@alpha><1063835810.14316.20.camel@geddy
	><16232.57433.388130.573539@montanaro.dyndns.org> 
	<00ad01c37db8$d8bc9d40$652b6992@alpha>
Message-ID: <3122.192.168.1.1.1063886697.squirrel@server.lotusland.dyndns.org>

Vinay Sajip said:
> I agree. Readability in logs is the most important consideration here, and
> I think a scheme such as "python." + <modulename> makes it very clear where
> an event originated. Of course this would just be a prefix; a module could
> log to any name under this namespace, e.g. "python.asyncore.socket" for
> socket-related events in asyncore.
>

I'm convinced.  I'll work the "python" prefix into the proposal.

Matthew Barnes

From martin at v.loewis.de  Thu Sep 18 11:13:12 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Sep 18 11:13:24 2003
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <3F695D80.6020804@lemburg.com>
References: <LNBBLJKPBEHFEDALKOLCIEBMFPAB.tim.one@comcast.net>
	<3F68166D.5030206@lemburg.com>
	<m3r82fxffv.fsf@mira.informatik.hu-berlin.de>
	<3F695D80.6020804@lemburg.com>
Message-ID: <m3smmuxh87.fsf@mira.informatik.hu-berlin.de>

"M.-A. Lemburg" <mal@lemburg.com> writes:

> Because that's what was used as basis in the type implementation
> as well as the codecs (internal and external). Comparisons simply
> work differently when you're using a signed type which is also
> why most compilers warn about this -- but you know that.

Yes and no. It appears that you are assuming that Py_UNICODE is always
unsigned, however, it is (AFAICT) nowhere documented, and I don't see
any strong reason ("it used to be that way" is not a strong reason -
code does change over time).

> An signed type also doesn't make much sense for things like
> character storage -- the sign information is useless and you
> lose a bit for each character.

OTOH, using wchar_t where possible is also valuable. On all systems
with a signed wchar_t that we know of, that wchar_t has 32 bits -
losing the sign bit does not mean to lose character code points, as
Unicode has less than 2**17 code points, anyway.

Regards,
Martin


From tim.one at comcast.net  Thu Sep 18 11:18:43 2003
From: tim.one at comcast.net (Tim Peters)
Date: Thu Sep 18 11:18:47 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <m3r82ey8qd.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEJLFPAB.tim.one@comcast.net>

[martin@v.loewis.de]
> I think people have learned, over time, that negative characters are
> evil. So MS chose to make wchar_t unsigned, as, for 2-byte Unicode,
> you might otherwise run into negative characters. For gcc, people
> probably also agree that negative characters are evil, but using long
> int causes no harm here: All assigned characters are still positive,
> as Unicode goes only up to 2**21.

That's fine for gcc then, so long as we don't branch on the contents of
uninitialized memory, and we just fixed a bug of that sort.

If Py_UNICODE ever resolves to a signed 2-byte type, though, the sign bit is
still in play for legit contents.


From nick at nick.uklinux.net  Thu Sep 18 14:04:00 2003
From: nick at nick.uklinux.net (Nick Roberts)
Date: Thu Sep 18 14:09:32 2003
Subject: [Python-Dev] Enhancement to pdb in gud.el
Message-ID: <16233.62352.331254.344092@nick.uklinux.net>

Hi,

I've written a mode (gdb-ui.el) which is in the Emacs CVS repository
(http://savannah.gnu.org/projects/emacs) that displays breakpoint icons in the
display margin of Emacs when debugging with GDB. I can almost do the same
thing for pdb (the Emacs interface to the python debugger in gud.el).

Breakpoints in GUD can be set/cleared using the toolbar and it is easy to
write a lisp function to do that. However, they can also be set/cleared from
the GUD buffer with break/clear and gud-pdb-marker-filter is the best place to
detect this.

If I'm using pdb.py to debug fib.py and I type:

(Pdb) b 4

I get the message:

Breakpoint 1 at /home/nick/python/fib.py:4

and I can use gud-pdb-marker-filter to detect this string and insert a
breakpoint icon at the appropriate line in the source buffer.

Typing 'clear 1' gives Deleted breakpoint 1' and typing 'clear fib.py:4' gives
no message and can't be detected. If, in both cases, the the message is
modified to somthing like

Deleted breakpoint 1 at /home/nick/python/fib.py:4

then I would have enough information.

Anyone interested?


Nick


From martin at v.loewis.de  Thu Sep 18 14:54:25 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Sep 18 14:54:41 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEJLFPAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCGEJLFPAB.tim.one@comcast.net>
Message-ID: <m3k786x6zi.fsf@mira.informatik.hu-berlin.de>

"Tim Peters" <tim.one@comcast.net> writes:

> That's fine for gcc then, so long as we don't branch on the contents of
> uninitialized memory, and we just fixed a bug of that sort.
> 
> If Py_UNICODE ever resolves to a signed 2-byte type, though, the sign bit is
> still in play for legit contents.

Right. I agree that we don't want to use a signed 2-byte type. Given
that only MS uses a two-byte wchar_t (to my knowledge), and that
theirs is unsigned, having an autoconf test to detect that this
configuration is not supported is more than enough.

Regards,
Martin

From recht at netbsd.org  Thu Sep 18 18:44:24 2003
From: recht at netbsd.org (Marc Recht)
Date: Thu Sep 18 18:44:31 2003
Subject: [Python-Dev] Python and POSIX
In-Reply-To: <m3ekyg5x51.fsf@mira.informatik.hu-berlin.de>
References: <35230000.1063660079@leeloo.intern.geht.de>
	<m3llspwvvr.fsf@mira.informatik.hu-berlin.de>
	<46570000.1063709941@leeloo.intern.geht.de>
	<m3ekyg5x51.fsf@mira.informatik.hu-berlin.de>
Message-ID: <48950000.1063925064@leeloo.intern.geht.de>

> While that is true, I would hope that you come to the conclusion that
> these macros can potentially do much more than simply hiding library
> functions. Instead, their *primary* purpose is to request Single Unix
> behaviour in case a platform implements both the standard behaviour,
> and the traditional one (regardless of which tradition the system
> follows).
I'm fully aware of it. I say it's sometimes a PITA, too. At least if it 
comes to 3rd party modules.

> So I would be *very* careful to enable these macros only for a few
> translation units; I'm pretty sure that the "law" would be on the
> vendor's side if inconsistent usage of these defines would be found to
> cause problems, in Irix 27, or ValueBSD 3.
That's why I'm totaly in favor of "stricly POSIX conforming" for Python 
itself. It's IMHO an other situation when it comes to modules or 
extensions. (Also, the ones in the Python distribution.) IMHO the module 
implementors should define for themself which standard they need/support 
(if any). As you've pointed out in your last mail there are (major?) 
technical problems with stat (eg. struct stat) which need to be solved.

> If the change is only for 2.4, that would be fine - that code
> hopefully will get testing on several other platforms before 2.4 is
> released.
Totally agree. That would be a way to big change for 2.3.x and it needs 
much testing, too.

> Believes should not be trusted too much when it comes to
> portability. If you can think of an obscure interpretation of some
> standard, it is almost certain that there is some vendor out there who
> has chosen that interpretation.
And sometimes the standard itself is the problem...

Regards,
Marc

From bac at OCF.Berkeley.EDU  Thu Sep 18 20:38:57 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Sep 18 20:39:12 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
In-Reply-To: <2misnq8irs.fsf@starship.python.net>
References: <3F669AF3.8050605@ocf.berkeley.edu>	<16231.7058.987290.575924@montanaro.dyndns.org>	<3F674015.8090801@ocf.berkeley.edu>	<3F67D078.1000106@ocf.berkeley.edu>	<20030917131354.GC1211@epoch.metaslash.com>	<3F69563F.6090209@ocf.berkeley.edu>
	<2misnq8irs.fsf@starship.python.net>
Message-ID: <3F6A5021.1080100@ocf.berkeley.edu>

Michael Hudson wrote:

> "Brett C." <bac@OCF.Berkeley.EDU> writes:
> 
> 
>>But pyconfig.h.in is another thing entirely.  I just ran ``cvs diff``
>>on it and the diff I get is (on both HEAD and release23-maint):
>>
>>+/* Define this if you have tcl and TCL_UTF_MAX==6 */
>>+#undef HAVE_UCS4_TCL
>>+
>>
>>Uh, why is that new?  Is this from Martin's patch to detect RH's
>>custom Tcl?  Could this be from autoheader not being run after that
>>patch (or someone else's)?
> 
> 
> Looks like it (as in, I see that here too, and I don't have any local
> changes to configure.in).
> 

OK, cool.  Thanks for the info, Michael.

I am going to go ahead and patch HEAD and release23-maint for 
configure.in, configure, and pyconfig.h.in .

-Brett


From tim.one at comcast.net  Thu Sep 18 23:06:01 2003
From: tim.one at comcast.net (Tim Peters)
Date: Thu Sep 18 23:06:22 2003
Subject: [Python-Dev] Fun with 2.3 shutdown
Message-ID: <LNBBLJKPBEHFEDALKOLCOEOHFPAB.tim.one@comcast.net>

When the Zope3 tests are run under Python 2.3, after the test runner ends we
usually get treated to a long string of these things:

"""
Unhandled exception in thread started by
Error in sys.excepthook:
Original exception was:

"""

All that output is from Python, but it isn't particularly helpful <wink>.

Here's what I know:

For Zope readers:  these are indeed due to the _Checkpoint threads that are
left behind, sitting in

            self._event.wait(self._interval)

self._interval is 120 seconds (i.e., a relatively long time).


For Python readers:

Event.wait() with a timeout ends up in _Condition.wait(), where a lazy busy
loop wakes up about 20 times per second to see whether it can proceed.

For some reason an exception is getting raised in the wait() code.  I'm not
exactly sure what or why yet, but that will come soon enough.  For this
message, it's enough to know that it happens (an educated guess comes later
in this msg).

Python is in Py_Finalize, in its (new in 2.3) second call of PyGC_Collect():

	/* Collect garbage.  This may call finalizers; it's nice to
         call these before all modules are destroyed. */
	PyGC_Collect();

	/* Destroy all modules */
	PyImport_Cleanup();

	/* Collect final garbage.  This disposes of cycles created by
	   new-style class definitions, for example. */
	PyGC_Collect();    WE'RE IN THIS ONE

*Presumably* the _Condition.wait() code wakes up after modules have been
destroyed, and raises an exception due to referencing a None'ed-out
threading.py global.  A fact is that PyGC_Collect has called a __del__
method when the problem occurs, and that's why some other thread was able to
get the GIL and proceed (gc.collect() gave control back to Python code, via
a Python __del__ method).

*Presumably* this only causes a visible problem when running "at least most
of" the Zope3 tests because the Zope3 tests create a ton of containers and
gc takes significant time then.  Presumably if just a few tests are run, the
second call to PyGC_Collect() goes so fast that it's unlikely the
_Condition.wait() code will wake up and blow up before Python exits.

Back to for-sure stuff:  the thread does raise an exception, and we wind up
back in threadmodule.c t_bootstrap:

	if (res == NULL) {   IT IS NULL
		if (PyErr_ExceptionMatches(PyExc_SystemExit))
			PyErr_Clear();
		else {
			PyObject *file;
			PySys_WriteStderr(
				"Unhandled exception in thread started by ");
			file = PySys_GetObject("stderr");
			if (file)
				PyFile_WriteObject(boot->func, file, 0);
			else
				PyObject_Print(boot->func, stderr, 0);
			PySys_WriteStderr("\n");
			PyErr_PrintEx(0);
		}
	}

It's not PyExc_SystemExit, so we write

    Unhandled exception in thread started by

to C's stderr and then get sys.stderr.  Heh.  sys has been wiped by this
point, and file gets is Py_None.  Brrrrrrrr.  Everything after this is a
comedy of unintended errors.  file is true (so far as C is concerned), so we
do

		PyFile_WriteObject(boot->func, file, 0);

That tries some tests that don't pass, then does

	writer = PyObject_GetAttrString(f, "write");

None.write doesn't exist, so the exception we started with is lost and
replaced by an unwanted AttributeError.

PyFile_WriteObject returns -1 then, but t_bootstrap doesn't check for that.
Instead it writes a newline, and goes on to

			PyErr_PrintEx(0);

That eventually does

	hook = PySys_GetObject("excepthook");

and for a now-familiar reason, that also returns None.  Again hook looks
true to C, and

		PyObject *result = PyEval_CallObject(hook, args);

overwrites the unwanted AttributeError with a different unwanted exception.
PyErr_PrintEx notices that the call fails, and prints

    "Error in sys.excepthook:\n"

to C's stderr.  It then tries

			PyErr_Display(exception2, v2, tb2);

That starts with

	PyObject *f = PySys_GetObject("stderr");

and *again* that returns None from the ruined sys module, and again f looks
true to C.  It becomes something of a mess then, so suffice it to say that
PyErr_Display's attempts to write to file None via

    PyFile_WriteString(buf, f)

don't accomplish anything, and then PyErr_Display() ends with

	/* If an error happened here, don't show it.
	   XXX This is wrong, but too many callers rely on this behavior. */
	if (err != 0)
		PyErr_Clear();

So we come back with no exception set, and PyErr_PrintEx proceeds to write

    "\nOriginal exception was:\n"

to C's stderr.  Then it tries

			PyErr_Display(exception, v, tb);

and that goes thru the long dance of retrieving None from sys.stderr (etc)
all over again.


The best thing for the Zope3 tests would be to find ways not to leave
useless daemon threads hanging around.

I'm not sure what it says about Python's shutdown strategy.  If I find and
fix a None-module-global problem in _Condition.wait(), that's fine and
probably helpful (in 2.3.1), but in the context of Zope3 the run() methods
of these threads are going to trigger None-module-global problems of their
own.  That can also be fixed.

So a combination of changes to threading.py and Zope3 code can probably hide
the shutdown glitches.  Somehow, though, it just doesn't feel entirely
satisfactory <0.9 wink>.

seemingly-out-of-gordian-knot-cutters-for-this-week-ly y'rs  - tim


From martin at v.loewis.de  Fri Sep 19 00:59:00 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Fri Sep 19 00:59:18 2003
Subject: [Python-Dev] Python and POSIX
In-Reply-To: <48950000.1063925064@leeloo.intern.geht.de>
References: <35230000.1063660079@leeloo.intern.geht.de>
	<m3llspwvvr.fsf@mira.informatik.hu-berlin.de>
	<46570000.1063709941@leeloo.intern.geht.de>
	<m3ekyg5x51.fsf@mira.informatik.hu-berlin.de>
	<48950000.1063925064@leeloo.intern.geht.de>
Message-ID: <m3oexhgyqz.fsf@mira.informatik.hu-berlin.de>

Marc Recht <recht@netbsd.org> writes:

> That's why I'm totaly in favor of "stricly POSIX conforming" for
> Python itself. It's IMHO an other situation when it comes to modules
> or extensions. (Also, the ones in the Python distribution.) IMHO the
> module implementors should define for themself which standard they
> need/support (if any). As you've pointed out in your last mail there
> are (major?) technical problems with stat (eg. struct stat) which need
> to be solved.

Unfortunately, it might just be that this specific problem cannot be
solved, in a general way. Instead, the author of the code would have
to take the risk of breaking the in-process ABI. For example, if the
#defines would cause different libraries to be linked (rather than
different symbols of the same library being used), you might find that
mapping both libraries into the same process is just not supported,
and that the extension module would still get the POSIX version of the
function at run-time, and that the system vendor provides no support
for "mixed" processes.

So if you implement such a change, you should also make it easy for a
developer to avoid the danger of such undeliberate breakage, by having
a mode where extensions compile with the exact same settings as the
main interpreter. Perhaps that should even be the default, and the
do-it-yourself configuration would be activated only on explicit
request (e.g. a define before including Python.h).

Regards,
Martin


From anthony at interlink.com.au  Fri Sep 19 01:37:49 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Sep 19 01:39:26 2003
Subject: [Python-Dev] Fun with 2.3 shutdown 
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEOHFPAB.tim.one@comcast.net> 
Message-ID: <200309190537.h8J5bnIG015051@localhost.localdomain>


>>> "Tim Peters" wrote
> When the Zope3 tests are run under Python 2.3, after the test runner ends we
> usually get treated to a long string of these things:
> 
> """
> Unhandled exception in thread started by
> Error in sys.excepthook:
> Original exception was:
> 
> """

Yep. I remember mentioning these at the Melbourne sprint.

> I'm not sure what it says about Python's shutdown strategy.  If I find and
> fix a None-module-global problem in _Condition.wait(), that's fine and
> probably helpful (in 2.3.1), but in the context of Zope3 the run() methods
> of these threads are going to trigger None-module-global problems of their
> own.  That can also be fixed.

Hm. Is this serious enough a problem to hold off 2.3.1 until it's fixed?
My feeling is "no", because it only affects Z3 developers, and they're 
hardened to random pain by now, and it's also just a shutdown glitch. 

On the other hand, making the Z3 shutdown code os.kill(os.getpid(), 9) 
would also make the messages go away ;)

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From bac at OCF.Berkeley.EDU  Fri Sep 19 02:35:01 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Sep 19 02:35:02 2003
Subject: [Python-Dev] python-dev Summary for 2003-09-01 through 2003-09-15
	[draft]
Message-ID: <3F6AA395.9040500@ocf.berkeley.edu>

Not planning on rushing this summary out the door; most likely will send 
it out on Sunday.

So have a read and show me why I should not pass the writing test I have 
to take next month that is supposed to prove I am a competent writer.

---------------

python-dev Summary for 2003-09-01 through 2003-09-15
++++++++++++++++++++++++++++++++++++++++++++++++++++
This is a summary of traffic on the `python-dev mailing list`_ from 
September 1, 2003 through September 15, 2003.  It is intended to inform 
the wider Python community of on-going developments on the list.  To 
comment on anything mentioned here, just post to `comp.lang.python`_ (or 
email python-list@python.org which is a gateway to the newsgroup) with a 
subject line mentioning what you are discussing. All python-dev members 
are interested in seeing ideas discussed by the community, so don't 
hesitate to take a stance on something.  And if all of this really 
interests you then get involved and join `python-dev`_!

This is the twenty-fifth summary written by Brett Cannon (with school 
looming on the horizon).

All summaries are archived at http://www.python.org/dev/summary/ .

Please note that this summary is written using reStructuredText_ which 
can be found at http://docutils.sf.net/rst.html .  Any unfamiliar 
punctuation is probably markup for reST_ (otherwise it is probably 
regular expression syntax or a typo =); you can safely ignore it, 
although I suggest learning reST; its simple and is accepted for `PEP 
markup`_ and gives some perks for the HTML output.  Also, because of the 
wonders of programs that like to reformat text, I cannot guarantee you 
will be able to run the text version of this summary through Docutils_ 
as-is unless it is from the original text file.

.. _PEP Markup: http://www.python.org/peps/pep-0012.html

The in-development version of the documentation for Python can be found 
at http://www.python.org/dev/doc/devel/ and should be used when looking 
up any documentation on something mentioned here.  PEPs (Python 
Enhancement Proposals) are located at http://www.python.org/peps/ .  To 
view files in the Python CVS online, go to 
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/ .  Reported bugs 
and suggested patches can be found at the SourceForge_ project page.

.. _python-dev: http://www.python.org/dev/
.. _SourceForge: http://sourceforge.net/tracker/?group_id=5470
.. _python-dev mailing list: 
http://mail.python.org/mailman/listinfo/python-dev
.. _comp.lang.python: http://groups.google.com/groups?q=comp.lang.python
.. _Docutils: http://docutils.sf.net/
.. _reST:
.. _reStructuredText: http://docutils.sf.net/rst.html

.. contents::

.. _last summary: 
http://www.python.org/dev/summary/2003-08-16_2003-08-31.html


=====================
Summary Announcements
=====================
Summary is nice and short this time.  More people went on vacation 
(although Martin came back and so that helped to pick up the traffic a 
little).  I skipped covering some threads that are due for PEPs since 
those will be more in-depth then anything I write.

I should give fair warning that starting on the 22nd of September I will 
begin school.  I am hoping that it will not be difficult and I will have 
a carefree time in school.  But if my workload becomes a burden the 
Summaries might suffer.  I hope they don't, though, and I am going to do 
my best to make sure that doesn't happen.


=========
Summaries
=========
---------------------------------------
If It Isn't Documented, Is It a Secret?
---------------------------------------
Some undocumented methods in readline were discovered by Philip Eby.  He 
asked if this was on purpose and whether he should submit a bug report 
on the lack of documentation.  Guido told him to go ahead and submit the 
bug report.

If you ever find something undocumented in the Python source code and 
there is nothing suggesting the code is meant to be hidden from the user 
(it's for internal use, a comment says it is experimental, etc.), then 
please file a bug report!

Contributing threads:
   - `Undocumented functions in 'readline' module 
<http://mail.python.org/pipermail/python-dev/2003-September/037922.html>`__


-----------------------------------------------------------------------------------------
"You are more trouble than you are worth!", screamed the micro release 
to the new feature
-----------------------------------------------------------------------------------------
The topic of what should be backported for micro releases (with Python 
2.3.1 looming on the horizon) came up.  Originally minor improvements 
were allowed in if they didn't break backwards compatibility.  And of 
course bugfixes have been as well as long as they didn't break code that 
came to depend on the buggy behavior (really depends on how long the 
bugginess has been there and how well-known it is).

But then the point of OS X 10.3 possibly becoming the largest install 
base of Python of any version (it will be 2.3) came up.  With rough 
estimates being thrown around of 5 million installs in about a year's 
time, the point that making it difficult to run Python 2.3.x code on 
that size of an install base would be bad.  For instance, if some small 
new feature was added to 2.3.1 then any code using that feature would 
not be able to run on a virgin install of OS X 10.3 (and considering 
that this is Mac it should not be expected that most users will want to, 
let alone know how, to add a secondary install of Python since the 
original is used by the OS and thus should not be overwritten).  It 
looks like a more conservative patching scheme will be taken with micro 
releases.

Bytecode will also continue to work between releases.  Guido has always 
unofficially held that position but now it has been vocalized.

Contributing threads:
   - `Documenting branch policy 
<http://mail.python.org/pipermail/python-dev/2003-September/037955.html>`__


----------------------------------------------
PyPy "Berlin" sprint (29th Sep - 4th Oct 2003)
----------------------------------------------
Just what the title says: there is going to be a sprint_ for PyPy_ in 
Berlin from September 29 to October 4.  Read the email for more specific 
details on goals and such.

.. _sprint: http://www.python.org/cgi-bin/moinmoin/SprintPlan
.. _PyPy: http://codespeak.net/pypy/index.cgi?home

Contributing threads:
   - `PyPy "Berlin" sprint (29th Sep - 4th Oct 2003) 
<http://mail.python.org/pipermail/python-dev/2003-September/037961.html>`__


--------------------------------
Away with you ambiguous imports!
--------------------------------
A discussion on how to create a hierarchy of loggers in the standard 
library led to the idea of having a way to cause an import to come 
directly from the standard library and thus remove any ambiguity from a 
relative import grabbing a similarly named module from the local package.

Barry Warsaw he was thinking of writing a PEP on this idea.

Contributing threads:
   - `Consistent logging in the standard library 
<http://mail.python.org/pipermail/python-dev/2003-September/038000.html>`__


---------------------------------------------------
Quick: want to give a Python talk at Linuxworld NY?
---------------------------------------------------
If you are interested in what the title asks, then read the email for 
some details.

Contributing threads:
   - `Quick: want to give a Python talk at Linuxworld NY? 
<http://mail.python.org/pipermail/python-dev/2003-September/037993.html>`__


-------------------------------------------
Be careful about saying something is "easy"
-------------------------------------------
A word of advice about saying something is "easy" on python-dev: if you 
say that you should make sure you can back up that claim because 
otherwise you will be told to write the "easy" patch and that will be 
the end of the discussion.

Contributing threads:
   - `Making python C-API thread safe (try 2) 
<http://mail.python.org/pipermail/python-dev/2003-September/038007.html>`__


--------------------------
Parsing dates for datetime
--------------------------
There was some talk about coming up with some code to parse dates for 
the datetime module.  It was kind of hand-wavy since there are multiples 
chunks of code that can do that in the standard library.

If you have an opinion on this (or anything for that matter), make sure 
to make a post to `comp.lang.python`_ about it.

Contributing threads:
   - `datetime issues 
<http://mail.python.org/pipermail/python-dev/2003-September/038042.html>`__


From jafo-python-dev at tummy.com  Fri Sep 19 04:33:46 2003
From: jafo-python-dev at tummy.com (Sean Reifschneider)
Date: Fri Sep 19 04:33:55 2003
Subject: [Python-Dev] Discussion on adding rsplit() for strings and unicode
	objects.
Message-ID: <20030919083346.GA11541@tummy.com>

A few weeks ago I submitted a patch to add an rsplit() method to
string and unicode objects:

>Adding rsplit() to string and unicode objects. (2003-09-06)
>	http://python.org/sf/801847

loewis responded with:
>[...] I feel that there are already too many string methods, so I
>won't accept that patch. I'm not rejecting it, either, because I see
>that other maintainers might have a different opinion. In short, you
>should propose your change to python-dev, finding out what "a majority"
>of the maintainers thinks [...]

My understanding of why the patch was not accepted is that users can
implement it themselves.

My reason for spending the time implementing the patch was that I
noticed it was missing in some coding I was doing.  Instead of having
this tool available, I had to interrupt the flow of my coding and build
and test an rsplit implementation.  Not a huge undertaking, of course,
but still something that had to be dealt with.

I guess at least some maintainers wish for string objects to have fewer
methods.  I haven't heard of any alternative location to place it other
than placing a Python implementation in the Cookbook.  That alternative
seems strange to me -- requiring a user locate and maintain an
alternative implementation in their own code, likely at a fairly big
performance penalty (my rsplit is an adaptation of the existing split()
C code).

Obviously, I felt this was enough of a lack in the library that I spent
most of a Sunday afternoon tracking down and implementing it.  One of
Python's strengths is it's rich standard library (he says after spending
a few hours unsuccessfully trying to package a dozen Perl modules), and
I'd like to see it get even better.

I'd like to open this discussion, as loewis suggests, to the python-dev
folks.  Thoughts?

Thanks,
Sean
-- 
 Money is the root of all evil!  Man needs roots...
Sean Reifschneider, Member of Technical Staff <jafo@tummy.com>
tummy.com, ltd. - Linux Consulting since 1995.  Qmail, Python, SysAdmin
      Back off man. I'm a scientist.   http://HackingSociety.org/

From fincher.8 at osu.edu  Fri Sep 19 06:12:15 2003
From: fincher.8 at osu.edu (Jeremy Fincher)
Date: Fri Sep 19 05:13:20 2003
Subject: [Python-Dev] Discussion on adding rsplit() for strings and
	unicode objects.
In-Reply-To: <20030919083346.GA11541@tummy.com>
References: <20030919083346.GA11541@tummy.com>
Message-ID: <200309190612.15965.fincher.8@osu.edu>

On Friday 19 September 2003 04:33 am, Sean Reifschneider wrote:
> A few weeks ago I submitted a patch to add an rsplit() method to
>
> string and unicode objects:
> >Adding rsplit() to string and unicode objects. (2003-09-06)
> >	http://python.org/sf/801847

I'm all for it.  I've had to implement rsplit on my own as well, and always 
lamented (but never did anything, unfortunately) about its lack in the 
standard library.  That strings have rfind, rindex, rstrip, and rjust, but 
not rsplit always seemed somewhat less than complete to me.

> loewis responded with:
> >[...] I feel that there are already too many string methods, so I
> >won't accept that patch.

Python's string methods (and the fact that I can easily iterate over the 
characters in a string) enable me to write heavily string-manipulating code 
without resorting to the cryptic regular expressions my old Perl code was 
laden with.  As long as they're named appropriately, I can't see the harm in 
adding useful methods, especially one such as rsplit that completes a set of 
front-to-back/back-to-front methods.

Jeremy

From mal at lemburg.com  Fri Sep 19 07:10:15 2003
From: mal at lemburg.com (M.-A. Lemburg)
Date: Fri Sep 19 07:10:20 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 
	2.197, 2.198
In-Reply-To: <m3k786x6zi.fsf@mira.informatik.hu-berlin.de>
References: <LNBBLJKPBEHFEDALKOLCGEJLFPAB.tim.one@comcast.net>
	<m3k786x6zi.fsf@mira.informatik.hu-berlin.de>
Message-ID: <3F6AE417.2060301@lemburg.com>

Martin v. L?wis wrote:
> "Tim Peters" <tim.one@comcast.net> writes:
> 
>>That's fine for gcc then, so long as we don't branch on the contents of
>>uninitialized memory, and we just fixed a bug of that sort.
>>
>>If Py_UNICODE ever resolves to a signed 2-byte type, though, the sign bit is
>>still in play for legit contents.
> 
> Right. I agree that we don't want to use a signed 2-byte type. Given
> that only MS uses a two-byte wchar_t (to my knowledge), and that
> theirs is unsigned, having an autoconf test to detect that this
> configuration is not supported is more than enough.

Since wchar_t is the only case where a signed type can pop
up, why not extend the autoconf test to check for signedness
and then reject signed wchar_t value as not-usable (ie.
undefine HAVE_USABLE_WCHAR_T).

It looks to me as if this would resolve the problem once and
for all. Signed values simply cause too many problems for this
kind of application.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Sep 19 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::


From arigo at tunes.org  Fri Sep 19 08:01:00 2003
From: arigo at tunes.org (Armin Rigo)
Date: Fri Sep 19 08:06:55 2003
Subject: [Python-Dev] Fun with 2.3 shutdown
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEOHFPAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEOHFPAB.tim.one@comcast.net>
Message-ID: <20030919120100.GB23317@vicky.ecs.soton.ac.uk>

Hello Tim,

The behavior of Python regarding whether threads should continue to run after
the main one exited has never been too clear. Wouldn't it make sense to try to
control this aspect more precisely ? I have also experienced this kind of
shutdown glitches.

It definitely needs more thinking, but what about this one: Python would only
shutdown after all threads are finished; but exiting the main thread would by
default send a SystemExit exception to all current threads, if that can be
done cleanly (oh well, this problem again).

Another question, what was the rationale for setting module globals to None
instead of simply deleting them (and getting the expected AttributeErrors both
at C and at Python level) ?


Armin


From python-kbutler at sabaydi.com  Fri Sep 19 10:24:58 2003
From: python-kbutler at sabaydi.com (Kevin J. Butler)
Date: Fri Sep 19 10:25:22 2003
Subject: [Python-Dev] Enhancement to pdb in gud.el
In-Reply-To: <E1A0ErE-0003ue-8W@mail.python.org>
References: <E1A0ErE-0003ue-8W@mail.python.org>
Message-ID: <3F6B11BA.6000008@sabaydi.com>

From: Nick Roberts <nick@nick.uklinux.net>
> Typing 'clear 1' gives Deleted breakpoint 1' and typing 'clear fib.py:4' gives
> no message and can't be detected. If, in both cases, the the message is
> modified to somthing like
> 
> Deleted breakpoint 1 at /home/nick/python/fib.py:4
> 
> then I would have enough information.

The attached modifications do it for me.

A couple of issues:

- I completely replaced the Bdb.get_break method.  It was returning a 
boolean indicating if the breakpoint existed or not, which appeared to 
be an error (get_breaks returned the list of breakpoints).  The method 
was never used in bdb.py or pdb.py, so I replaced it with what I thought 
it should be.  :-)  That is, it now returns the breakpoint instance 
requested.  If there is a reason, I can easily make this a new method 
(get_breakpoint/get_breakpointinstance) rather than replacing get_break.

- I've duplicated logic from Bdb.clear_bpbynumber into a new method 
get_bpbynumber.  The logic differs only in the return error messages or 
None.  This seemed the simplest way to preserve the return value 
semantics convention of the Bdb class.

- I'm also calling 'get', then 'clear' which does the work of 'get' 
twice.  I did this to preserve the return value conventions of Bdb and 
make the breakpoint info available to the UI.  Shouldn't be a 
performance issue, right?  ;-)

Suggestions before I submit these as a patch to SF?

Life-doesn't-get-any-better-than-contributing-to-emacs-in-Python-ly y'rs,

kb

Sorry for the non-CVS diffs - I don't have a copy of the 2.3 repository 
right now:

*** bdb.py.1.42	Fri Sep 19 08:08:17 2003
--- bdb.py	Fri Sep 19 08:00:52 2003
***************
*** 266,274 ****
           self.breaks = {}

       def get_break(self, filename, lineno):
!         filename = self.canonic(filename)
!         return filename in self.breaks and \
!             lineno in self.breaks[filename]

       def get_breaks(self, filename, lineno):
           filename = self.canonic(filename)
--- 266,275 ----
           self.breaks = {}

       def get_break(self, filename, lineno):
!         bplist = self.get_breaks( filename, lineno )
!         if len( bplist ):
!             return bplist[-1]
!         return None

       def get_breaks(self, filename, lineno):
           filename = self.canonic(filename)
***************
*** 276,281 ****
--- 277,294 ----
               lineno in self.breaks[filename] and \
               Breakpoint.bplist[filename, lineno] or []

+     def get_bpbynumber(self, arg):
+         # duplicates logic from clear_bpbynumber
+         try:
+             number = int(arg)
+         except:
+             return None
+         try:
+             bp = Breakpoint.bpbynumber[number]
+         except IndexError:
+             return None
+         return bp
+
       def get_file_breaks(self, filename):
           filename = self.canonic(filename)
           if filename in self.breaks:


*** pdb.py.1.66	Fri Sep 19 08:06:01 2003
--- pdb.py	Fri Sep 19 08:07:23 2003
***************
*** 467,482 ****
               except:
                   err = "Invalid line number (%s)" % arg
               else:
                   err = self.clear_break(filename, lineno)
               if err: print '***', err
               return
           numberlist = arg.split()
           for i in numberlist:
               err = self.clear_bpbynumber(i)
               if err:
                   print '***', err
               else:
!                 print 'Deleted breakpoint %s ' % (i,)
       do_cl = do_clear # 'c' is already an abbreviation for 'continue'

       def do_where(self, arg):
--- 467,486 ----
               except:
                   err = "Invalid line number (%s)" % arg
               else:
+                 bp = self.get_break(filename, lineno)
                   err = self.clear_break(filename, lineno)
               if err: print '***', err
+             else:
+                 print 'Deleted breakpoint %s at %s:%s' % (i, bp.file, 
bp.line)
               return
           numberlist = arg.split()
           for i in numberlist:
+             bp = self.get_bpbynumber(i)
               err = self.clear_bpbynumber(i)
               if err:
                   print '***', err
               else:
!                 print 'Deleted breakpoint %s at %s:%s' % (i, bp.file, 
bp.line)
       do_cl = do_clear # 'c' is already an abbreviation for 'continue'

       def do_where(self, arg):



From hpk at trillke.net  Fri Sep 19 11:42:40 2003
From: hpk at trillke.net (Holger Krekel)
Date: Fri Sep 19 11:42:45 2003
Subject: [Python-Dev] pep 310 (reliable acquisition/release pairs)
Message-ID: <20030919174240.C31102@prim.han.de>

hello, 

admittedly i only followed Brett's very nice summaries (thanks!) 
in the last month so i may have missed some details. But here are a few
comments regarding "PEP 310 Reliable Acquisition/Release Pairs". 

The PEP actually is about interacting with the execution 
of a code block.  It allows to define (one-shot) interception points 
for entering and leaving a code block.  Now there are at least 
two interesting cases which the PEP does (quite explicitely) not cover:

- what to do with exceptions 

- what to do with yield 

IMHO introducing a new block statement at this stage in language 
development warrants an effort to tackle these cases (and maybe more
like e.g. allowing the handler to trigger looping). 

This is probably best done with trying to directly design a protocol between
the "interpreter-loop" and the - what i'd call - the "execution handler". 

And here it seems much more important to provide a concise definition 
of such a protocol (enter/leave/except/whatever) than to try to find 
an exact equivalence to plain python code.  "Execution events" are really 
occuring at several points/bytecodes in the eval-loop and a plain python 
construct can only go so far. 

for example, you *could* say something like 

    __enter__   called every time execution of the code block resumes
                (including when it is entered the first time)

but this is not easily expressed in plain Python code although it seems
implementable (and useful for resource-control when yielding).

So while i am not strictly against the proposal i'd humbly ask for not
hurrying into accepting the PEP as is. Python 2.4 is not closeby  so 
i hope there is still some time to discuss this. I have done some 
practical experiments ("x-python") which actually incorporate XML-syntax 
into Python using a concept of an execution handler (tags are execution
handlers and can additionally "catch" unassigned/dropped values that would
otherwise be POP_TOPed). While this is impossible to ever appear in 
official Python i'd like to think about and report some experiences - 
but not before Novemeber because the next PyPy sprint and other stuff 
is keeping me far too busy.  

thanks & regards,

    holger

From gherron at islandtraining.com  Fri Sep 19 11:50:47 2003
From: gherron at islandtraining.com (Gary Herron)
Date: Fri Sep 19 11:54:06 2003
Subject: [Python-Dev] Re: Fsrit Propttoye: SCLABMRE
In-Reply-To: <16232.24354.366003.171152@gargle.gargle.HOWL>
References: <16232.24354.366003.171152@gargle.gargle.HOWL>
Message-ID: <200309190850.47841.gherron@islandtraining.com>

Wow!  This otavbeiorsn is both aimznag and eaittnnnerig.
I was trohugh the Sjceubt: line and into the fisrt
senencte brefoe I even rleeiazd annhtiyg was odd.

Tahkns,
Gary Herron


On Wednesday 17 September 2003 06:18 am, Frnak "Sealmbcrd Eggs" Lomax wrote:
> I hvae jsut fhiseind rniedag a Shdlaost atrlcie from Mnoday, Sbepemetr
> 15th, aobut an ingtriniug sdtuy from Cgbdirmae Uinevtrsiy, dciinesrbg
> how wodrs with thier itienrors selmrbacd can siltl be rdbaleae.
> Bcaeuse of the tioeznme that I lvie in, an initettrment dfceet in my
> clbae medom, and the lneiignrg atfer-ecfefts of my lsat "mohsroum
> salad exntarvzagaa", I am aaylws two days bhneid on the Web.
>
> So suentnd was I at the icoinatpimls of this study, taht I
> cunlqtneesoy decedid to doetve my etrnie lfie and gtelray aeamssd
> fnrtoue on cunitodcng ftrhuer rasreech and dnpeeelmvot into tihs
> lifnsaevig tghlocnoey.  As a cross-eeyd dxsiyelc, I konw that teshe
> cilmas of svotailan are niehetr hlroyepbe nor bmobaistc
> over-daaomtiazitrn.
>
> Pnseetred hree for your rveiew, aaovpprl and disiscsuon, is our
> poyptotre sytsem, caelld SLABCRME for SeCren Rslmnebeig Aacrihc
> Mnoaaluptiin of BetaoLd Ecuitolon.  I have scnie bugen a meosdt
> sutartp based in Aceicdnt Maarnlyd, and we will soon be reeianlsg an
> Olutook puglin, anolg with a sftorwae devmoeelpnt kit, and oethr
> digintoasc tloos.  Smoe of our tlgeocnhoy wlil nallaruty be open
> sruoce, but I do ietnnd and ecpext to frutehr etnxed my hoougmuns
> wtaleh by sinlelg sftorawe lcseiens and srhink wrapped ratiel pocrudt.
> Psaele cactnot us for more iomrfaiontn.
>
> I'm aslo in enpommelyt duicnssioss wtih the Tmboit; I ecepxt that he
> can bosot the pmecarrofne of our pooyptrte's itmpeanetolmin
> salreolefvd, at wcihh piont we will be pthincig emdbeded ssmeyts to
> the mfntacuuearrs of ruoters, eiltncreoc trprwyeiets, psnreoal dtaiigl
> astsntsais, eorctinlec cepmutor coltlerond tersotas, maet prbeos, RFC
> 1149 digtail cemara pheons, and nxet gniaoetern wopehoe csionuhs.
>
> hinpog-to-be-deotersyd-by-moroscift-ly y'rs,
> -fnark



From python-kbutler at sabaydi.com  Fri Sep 19 12:43:38 2003
From: python-kbutler at sabaydi.com (Kevin J. Butler)
Date: Fri Sep 19 12:43:48 2003
Subject: [Python-Dev] Fsrit Propttoye: SCLABMRE
In-Reply-To: <E19zdmI-0007rK-Dd@mail.python.org>
References: <E19zdmI-0007rK-Dd@mail.python.org>
Message-ID: <3F6B323A.4070108@sabaydi.com>

I agilzoope bceasue I msiesd rnaiedg your msagese unitl I saw a rlepy.

I'm delgehitd taht you are cientarg a cnompay to exiplot tihs marouvles
pnmenoehon. If you are looikng for aiidndotal team mebmers for yuor work
snrlibcmag txet, I simbut this, the paorrgm I wtore when I haerd of
this ireesttnnig and ueufsl (or mbaye jsut ireeitsnntg) eefcft.

I hope it metes wtih yuor approval, tuhgoh wehn tarteed wtih ilestf
(and a maoterde aomunt of uesr ivtintorenen, it lsot sangtcniilfiy
more rlaaitebdiy than the visoern wtriten in the other 'P' luangage.
Stetrad wtih more, lsot more, raetenid mroe...).

In-serbmclad-txet-ly y'rs

kb

"""
Srbmlcae
by Kiven Beultr 2003 pthyon-kbuletr@sdabayi.com
Rlaseeed into pibulc dmioan wtih no wrtanray
Use at yuor own rsik
"""
import sys
import random
import re

def srbmlcae( wrod ):
    ieiedsns = list( wrod[1:-1] )
    random.shuffle( ieiedsns )
    return wrod[0] + ''.join( ieiedsns ) + wrod[-1]
    
def scRbaelmrx( match ):
    return srbmlcae( match.group() )

for lnie in sys.stdin:
    print re.sub( "\w{4,}", scRbaelmrx, lnie )[:-1]



From tjreedy at udel.edu  Fri Sep 19 13:44:09 2003
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri Sep 19 13:44:17 2003
Subject: [Python-Dev] Re: python-dev Summary for 2003-09-01 through
	2003-09-15[draft]
References: <3F6AA395.9040500@ocf.berkeley.edu>
Message-ID: <bkff99$rp3$1@sea.gmane.org>


"Brett C." <bac@OCF.Berkeley.EDU> wrote in message

> But then the point of OS X 10.3 possibly becoming the largest
install
> base of Python of any version (it will be 2.3) came up.

I an somehow amused at the prospect of Apple OX (Uni)X releases
influencing the timing of Python releases.

> With rough estimates being thrown around of 5 million installs
> in about a year's
> time, the point that making it difficult to run Python 2.3.x code on
> that size of an install base would be bad.

I've begun work on a 2.3 based project.  If I were to get users who
have received rather than installed Python, I would be more hesitant
to simply say 'upgrade', especially for minor features in a 2.3.x.

> that this is Mac it should not be expected that most users will want
to,
> let alone know how, to add a secondary install of Python since the
> original is used by the OS and thus should not be overwritten).

I presume a Mac installer would know how to add rather than overwrite.

I am curious whether anyone has ever suggested to PC makers that
Python + PyWin would be a useful addition to their Windows package -- 
both for setup/admin scripts (as done by RedHat. ...,  and now
Apple -- I don't know the specifics) and as a freebie addon (Basic for
the 21st Century)?

Terry J. Reedy




From bac at OCF.Berkeley.EDU  Fri Sep 19 13:59:07 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Sep 19 13:59:14 2003
Subject: [Python-Dev] Re: python-dev Summary for 2003-09-01
	through	2003-09-15[draft]
In-Reply-To: <bkff99$rp3$1@sea.gmane.org>
References: <3F6AA395.9040500@ocf.berkeley.edu> <bkff99$rp3$1@sea.gmane.org>
Message-ID: <3F6B43EB.1080308@ocf.berkeley.edu>

Terry Reedy wrote:

> "Brett C." <bac@OCF.Berkeley.EDU> wrote in message
> 
> 
>>But then the point of OS X 10.3 possibly becoming the largest
> 
> install
> 
>>base of Python of any version (it will be 2.3) came up.
> 
> 
> I an somehow amused at the prospect of Apple OX (Uni)X releases
> influencing the timing of Python releases.
> 

I personally find it cool, but I am an OSX user, so I am biased beyond 
belief.

> 
>>With rough estimates being thrown around of 5 million installs
>>in about a year's
>>time, the point that making it difficult to run Python 2.3.x code on
>>that size of an install base would be bad.
> 
> 
> I've begun work on a 2.3 based project.  If I were to get users who
> have received rather than installed Python, I would be more hesitant
> to simply say 'upgrade', especially for minor features in a 2.3.x.
> 
> 
>>that this is Mac it should not be expected that most users will want
> 
> to,
> 
>>let alone know how, to add a secondary install of Python since the
>>original is used by the OS and thus should not be overwritten).
> 
> 
> I presume a Mac installer would know how to add rather than overwrite.
> 

Depends on how the installer is set up.  Most OS X apps are fully 
self-contained within a directory (when you see an application icon it 
actually is just a directory that happens to follow a specific naming 
scheme and layout).  So Python apps that needs a different version of 
the interpreter could include their own copy with their app (perk of 
there only being one CPU family is you only need to compile once).

But for a site-wide install, that gets a little trickier.  It is 
definitely doable, but there is not much of a standard (I would assume 
it would be in either /Applications or ~/Applications , but no 
guarantees; could specify at install and store it somewhere, I guess) 
and there is always the UNIX way but that is not as much of a worry 
since if you are mucking with the UNIX side you should know how to 
handle this situation.

> I am curious whether anyone has ever suggested to PC makers that
> Python + PyWin would be a useful addition to their Windows package -- 
> both for setup/admin scripts (as done by RedHat. ...,  and now
> Apple -- I don't know the specifics) and as a freebie addon (Basic for
> the 21st Century)?
> 

HP/Compaq uses Python for something.  webmaster at python.org gets email 
every so often about an HP/Compaq user wanting to know what Python is 
and whether they can uninstall it safely.

As for OS X, they are using it apparently for driving PDF generation (or 
at least that was my understanding).

And I don't think MS wants competition for VB.NET in the form of Python. 
  But it would be nice.

-Brett


From nick at nick.uklinux.net  Fri Sep 19 14:11:28 2003
From: nick at nick.uklinux.net (Nick Roberts)
Date: Fri Sep 19 14:17:58 2003
Subject: [Python-Dev] Enhancement to pdb in gud.el
In-Reply-To: <3F6B11BA.6000008@sabaydi.com>
References: <E1A0ErE-0003ue-8W@mail.python.org> <3F6B11BA.6000008@sabaydi.com>
Message-ID: <16235.18128.422738.929390@nick.uklinux.net>

Kevin J. Butler writes:
 > 
 > A couple of issues:
 > 
 > ...

I know very little about python (!). I'm trying to extend what I have done in
Emacs to other languages. What's the relationship between bdb.py and pdb.py?
What message would your patch generate when a breakpoint is set/cleared?

Nick

From python-kbutler at sabaydi.com  Fri Sep 19 15:04:30 2003
From: python-kbutler at sabaydi.com (Kevin J. Butler)
Date: Fri Sep 19 15:04:56 2003
Subject: [Python-Dev] Enhancement to pdb in gud.el
In-Reply-To: <16235.18128.422738.929390@nick.uklinux.net>
References: <E1A0ErE-0003ue-8W@mail.python.org>	<3F6B11BA.6000008@sabaydi.com>
	<16235.18128.422738.929390@nick.uklinux.net>
Message-ID: <3F6B533E.5000403@sabaydi.com>



Nick Roberts wrote:

>Kevin J. Butler writes:
> > 
> > A couple of issues:
> > 
> > ...
>
>I know very little about python (!). I'm trying to extend what I have done in
>Emacs to other languages. What's the relationship between bdb.py and pdb.py?
>What message would your patch generate when a breakpoint is set/cleared?
>  
>
Sorry, bdb.py provides back-end features used by UI code in pdb.py - 
part of the fix went into each.

After my patch:
(Pdb) cl 3
Deleted breakpoint 3 at z:\work\wm\wm.py:136
(Pdb) cl wm.py:17
Deleted breakpoint 5 at z:\work\wm\wm.py:17

kb


From tim.one at comcast.net  Fri Sep 19 15:11:14 2003
From: tim.one at comcast.net (Tim Peters)
Date: Fri Sep 19 15:11:25 2003
Subject: [Python-Dev] Fun with 2.3 shutdown
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEOHFPAB.tim.one@comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEBGGAAB.tim.one@comcast.net>

[Tim]
> When the Zope3 tests are run under Python 2.3, after the test runner
> ends we usually get treated to a long string of these things:
>
> """
> Unhandled exception in thread started by
> Error in sys.excepthook:
> Original exception was:
>
> """

[bunch of analysis deleted]

> ...
> Event.wait() with a timeout ends up in _Condition.wait(), where a
> lazy busy loop wakes up about 20 times per second to see whether it
> can proceed.
>
> For some reason an exception is getting raised in the wait() code.
> I'm not exactly sure what or why yet, but that will come soon enough.

It didn't.  The primary effect of adding some vanilla debugging prints to
threading.py's _Condition.wait() was to make Python die with segfaults at
shutdown time instead.

If Python's *second* call to PyGC_collect() in Py_Finalize() is commented
out (the call that occurs after

	/* Destroy all modules */
	PyImport_Cleanup();

), all the problems go away, including the nonsense errors sprayed out at
the end of Zope3 test runs.

Recall that the nonsense errors are caused by a dozen stale daemon threads
trying to execute Python code after the interpreter has been severely torn
down, and they get the *chance* to do this because the second PyGC_collect()
finds trash with Python __del__ methods (so PyGC_collect() loses the GIL
when calling the __del__ methods, and all the daemon threads can proceed
then).  Note that this isn't a problem with code *in* __del__ methods!  That
makes it a different kind of shutdown glitch than we've usually wrestled
with.  It's a problem with Python code that has nothing to do with __del__;
__del__'s only contribution is to release the GIL.

Because the second PyGC_collect() *is* finding addtional finalizers to run,
it's unattractive to stop calling it (getting more user-defined finalizers
to run was the purpose of adding these PyGC_collect() shutdown calls to
2.3).

OTOH, any __del__ method that runs after PyImport_Cleanup() will be (AFAICT)
just as vulnerable to producing nonsense errors and segfaults as the code in
_Condition.wait() has proven to be (sys is useless by that point, and all
the Python internal code sucking basic objects out of sys isn't expecting to
get None back).

Maybe we should remove the second PyGC_collect() call before more apps run
into these mysteries.

Maybe we should delay tearing down sys as a special case (even more of a
special case than it is now).

Maybe the Zope3 tests should stop leaving an ever-growing number of daemon
threads around (which appears to be the only solution so long as they're run
under Python 2.3).


From martin at v.loewis.de  Fri Sep 19 16:43:46 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Fri Sep 19 16:44:10 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <3F6AE417.2060301@lemburg.com>
References: <LNBBLJKPBEHFEDALKOLCGEJLFPAB.tim.one@comcast.net>
	<m3k786x6zi.fsf@mira.informatik.hu-berlin.de>
	<3F6AE417.2060301@lemburg.com>
Message-ID: <m3ekycbjb1.fsf@mira.informatik.hu-berlin.de>

"M.-A. Lemburg" <mal@lemburg.com> writes:

> Since wchar_t is the only case where a signed type can pop
> up, why not extend the autoconf test to check for signedness
> and then reject signed wchar_t value as not-usable (ie.
> undefine HAVE_USABLE_WCHAR_T).

Because that would exclude a number of relevant systems where wchar_t
would be usable.

Regards,
Martin

From jvb at prairienet.org  Fri Sep 19 17:26:08 2003
From: jvb at prairienet.org (John Belmonte)
Date: Fri Sep 19 17:26:16 2003
Subject: [Python-Dev] Improving the CGI module
In-Reply-To: <3F68F4F7.6020601@bath.ac.uk>
References: <3F68F4F7.6020601@bath.ac.uk>
Message-ID: <3F6B7470.1060407@prairienet.org>

Simon Willison wrote:
> I've recently started using Python for server side web development, and 
> I've been running in to some limitations in the CGI module.
...
> I have previous experience with PHP, which has a number of extremely 
> convenient mechanisms for dealing with form input and cookies that I 
> sorely miss when working with Python. I have started work on porting 
> some of these capabilities, but it struck me that several of the ideas 
> represented would make valuable additions to the Python standard library 
> itself.

Recently I was also thinking there is a lot of room for improvement. 
Although the Python libraries have objects that represent HTTP request 
and responses on the client side, there is no such thing for the server 
side to use from a CGI script.  (There is a little of this functionality 
mixed in with the HTTP server modules however.)

I would like to see modules such as Quixote's http_request and 
http_response (which were taken from Zope) improved and incorporated 
into the standard library.  They do the essentials like manage headers, 
status, encoding, and cookies.  Something I found useful was that they 
can even parse headers such as Accept and give you a dictionary of MIME 
types that the client will accept.  The mod_python modules might also 
offer some ideas.

Also, in any improvements that are made, I would like to see things kept 
general enough to support the REST architecture style [1].  In other 
words, don't assume that the client is always a web browser, or that the 
body of a POST is always a form, etc.  HTTP and CGI are for more than 
just serving web pages.

Regards,
-John Belmonte


[1] http://internet.conveyor.com/RESTwiki/


-- 
http:// if   le.o  /


From ianb at colorstudy.com  Fri Sep 19 17:48:55 2003
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri Sep 19 17:51:47 2003
Subject: [Python-Dev] Improving the CGI module
In-Reply-To: <3F6B7470.1060407@prairienet.org>
Message-ID: <11097042-EAEB-11D7-A692-000393C2D67E@colorstudy.com>

On Friday, September 19, 2003, at 04:26 PM, John Belmonte wrote:
> Simon Willison wrote:
>> I've recently started using Python for server side web development, 
>> and I've been running in to some limitations in the CGI module.
> ...
>> I have previous experience with PHP, which has a number of extremely 
>> convenient mechanisms for dealing with form input and cookies that I 
>> sorely miss when working with Python. I have started work on porting 
>> some of these capabilities, but it struck me that several of the 
>> ideas represented would make valuable additions to the Python 
>> standard library itself.
>
> Recently I was also thinking there is a lot of room for improvement. 
> Although the Python libraries have objects that represent HTTP request 
> and responses on the client side, there is no such thing for the 
> server side to use from a CGI script.  (There is a little of this 
> functionality mixed in with the HTTP server modules however.)
>
> I would like to see modules such as Quixote's http_request and 
> http_response (which were taken from Zope) improved and incorporated 
> into the standard library.  They do the essentials like manage 
> headers, status, encoding, and cookies.  Something I found useful was 
> that they can even parse headers such as Accept and give you a 
> dictionary of MIME types that the client will accept.  The mod_python 
> modules might also offer some ideas.

I think this would be an excellent idea -- providing some (*any*) basic 
request and response implementation/interface would (hopefully) provide 
some sort of basic vocabulary for generic Python web packages (i.e., 
packages that aren't tied specifically to one framework).  Right now 
there isn't even a lowest common denominator, even though the 
differences in the implementation of these interfaces isn't that great. 
  If all a cgi (or web or whatever) module did was provide a request and 
response object, that'd be a big improvement.

With certain criteria, I guess: it should be easy to create from a CGI 
context, but not difficult from other contexts (i.e., given any 
file-like input and output, and a dictionary of CGI-environment-like 
variables).  It should also be general enough for most purposes, like 
allowing streamed output, you mentioned accepting any POST body, 
probably some other stuff.  But none of that is particular difficult to 
do.

> Also, in any improvements that are made, I would like to see things 
> kept general enough to support the REST architecture style [1].  In 
> other words, don't assume that the client is always a web browser, or 
> that the body of a POST is always a form, etc.  HTTP and CGI are for 
> more than just serving web pages.


From nick at nick.uklinux.net  Sat Sep 20 11:20:50 2003
From: nick at nick.uklinux.net (Nick Roberts)
Date: Sat Sep 20 11:27:38 2003
Subject: [Python-Dev] Enhancement to pdb in gud.el
In-Reply-To: <3F6B533E.5000403@sabaydi.com>
References: <E1A0ErE-0003ue-8W@mail.python.org> <3F6B11BA.6000008@sabaydi.com>
	<16235.18128.422738.929390@nick.uklinux.net>
	<3F6B533E.5000403@sabaydi.com>
Message-ID: <16236.28754.105629.565153@nick.uklinux.net>

Kevin J. Butler writes:
 > ...
 > 
 > After my patch:
 > (Pdb) cl 3
 > Deleted breakpoint 3 at z:\work\wm\wm.py:136
 > (Pdb) cl wm.py:17
 > Deleted breakpoint 5 at z:\work\wm\wm.py:17

I've applied your patch manually to my files bdb.py and pdb.py (I have 2.2)
and it appears to work. If you load the file below in Emacs (M-x eval-buffer
or M-x load-file), then type M-x pdb, the debugger (with your patch) should
work with breakpoint icons in the display margin. Emacs must be a version from
the CVS repository to work straight away. This will also give you buttons for
debugging if you turn the toolbar on (M-x tool-bar-mode).

I've tested it on GNU/Linux but it should work on Windows too if I've defined
the regexp correctly.

I've just noticed that you can enable/disable breakpoints in python. In my
mode for gdb, the breakpoints are greyed out when the breakpoints become
disabled. I could do this for python too if these commands were changed to
emit a message.

If your patches are accepted (by SF?) then I will integrate my changes into
gud.el in the Emacs CVS repository.

Nick

-----------------------------------------------------

(require 'gud)

(defun gud-sentinel (proc msg)
  (cond ((null (buffer-name (process-buffer proc)))
	 ;; buffer killed
	 ;; Stop displaying an arrow in a source file.
	 (setq overlay-arrow-position nil)
	 (set-process-buffer proc nil)
	 (if (memq gud-minor-mode-type '(gdba pdb))
	     (gdb-reset)
	   (gud-reset)))
	((memq (process-status proc) '(signal exit))
	 ;; Stop displaying an arrow in a source file.
	 (setq overlay-arrow-position nil)
	 (with-current-buffer gud-comint-buffer
	   (if (memq gud-minor-mode '(gdba pdb))
	       (gdb-reset)
	     (gud-reset)))
	 (let* ((obuf (current-buffer)))
	   ;; save-excursion isn't the right thing if
	   ;;  process-buffer is current-buffer
	   (unwind-protect
	       (progn
		 ;; Write something in *compilation* and hack its mode line,
		 (set-buffer (process-buffer proc))
		 ;; Fix the mode line.
		 (setq mode-line-process
		       (concat ":"
			       (symbol-name (process-status proc))))
		 (force-mode-line-update)
		 (if (eobp)
		     (insert ?\n mode-name " " msg)
		   (save-excursion
		     (goto-char (point-max))
		     (insert ?\n mode-name " " msg)))
		 ;; If buffer and mode line will show that the process
		 ;; is dead, we can delete it now.  Otherwise it
		 ;; will stay around until M-x list-processes.
		 (delete-process proc))
	     ;; Restore old buffer, but don't restore old point
	     ;; if obuf is the gud buffer.
	     (set-buffer obuf))))))

(defun gdb-reset ()
  "Exit a debugging session cleanly by killing the gdb buffers and resetting
 the source buffers."
  (dolist (buffer (buffer-list))
    (if (not (eq buffer gud-comint-buffer))
	(with-current-buffer buffer
	  (if (memq gud-minor-mode '(gdba pdb))
	      (if (string-match "^\*.+*$" (buffer-name))
		  (kill-buffer nil)
		(if (display-images-p)
		    (remove-images (point-min) (point-max))
		  (gdb-remove-strings (point-min) (point-max)))
		(setq left-margin-width 0)
		(setq gud-minor-mode nil)
		(kill-local-variable 'tool-bar-map)
		(setq gud-running nil)
		(if (get-buffer-window (current-buffer))
		    (set-window-margins (get-buffer-window
					 (current-buffer))
					left-margin-width
					right-margin-width))))))))

(defun gdb-put-string (putstring pos)
  "Put string PUTSTRING in front of POS in the current buffer.
PUTSTRING is displayed by putting an overlay into the current buffer with a
`before-string' STRING that has a `display' property whose value is
PUTSTRING."
  (let ((gdb-string "x")
	(buffer (current-buffer)))
    (let ((overlay (make-overlay pos pos buffer))
	  (prop (list (list 'margin 'left-margin) putstring)))
      (put-text-property 0 (length gdb-string) 'display prop gdb-string)
      (overlay-put overlay 'put-break t)
      (overlay-put overlay 'before-string gdb-string))))

(defun gdb-remove-strings (start end &optional buffer)
  "Remove strings between START and END in BUFFER.
Remove only strings that were put in BUFFER with calls to `put-string'.
BUFFER nil or omitted means use the current buffer."
  (unless buffer
    (setq buffer (current-buffer)))
  (let ((overlays (overlays-in start end)))
    (while overlays
      (let ((overlay (car overlays)))
	(when (overlay-get overlay 'put-break)
	  (delete-overlay overlay)))
      (setq overlays (cdr overlays)))))


(defconst breakpoint-xpm-data "/* XPM */
static char *magick[] = {
/* columns rows colors chars-per-pixel */
\"12 12 2 1\",
\"  c red\",
\"+ c None\",
/* pixels */
\"++++++++++++\",
\"+++      +++\",
\"++        ++\",
\"+          +\",
\"+          +\",
\"+          +\",
\"+          +\",
\"+          +\",
\"+          +\",
\"++        ++\",
\"+++      +++\",
\"++++++++++++\"
};"
  "XPM data used for breakpoint icon.")

(defconst breakpoint-enabled-pbm-data
"P1
12 12\",
0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 1 1 1 1 1 1 0 0 0
0 0 1 1 1 1 1 1 1 1 0 0
0 1 1 1 1 1 1 1 1 1 1 0
0 1 1 1 1 1 1 1 1 1 1 0
0 1 1 1 1 1 1 1 1 1 1 0
0 1 1 1 1 1 1 1 1 1 1 0
0 1 1 1 1 1 1 1 1 1 1 0
0 1 1 1 1 1 1 1 1 1 1 0
0 0 1 1 1 1 1 1 1 1 0 0
0 0 0 1 1 1 1 1 1 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0"
  "PBM data used for enabled breakpoint icon.")

(defvar breakpoint-enabled-icon
  (find-image `((:type xpm :data ,breakpoint-xpm-data)
		(:type pbm :data ,breakpoint-enabled-pbm-data)))
  "Icon for enabled breakpoint in display margin")

;; ======================================================================
;; pdb (Python debugger) functions

;; History of argument lists passed to pdb.
(defvar gud-pdb-history nil)

;; Last group is for return value, e.g. "> test.py(2)foo()->None"
;; Either file or function name may be omitted: "> <string>(0)?()"
(defvar gud-pdb-marker-regexp
  "^> \\([-a-zA-Z0-9_/.:\\]*\\|<string>\\)(\\([0-9]+\\))\\([a-zA-Z0-9_]*\\|\\?\\)()\\(->[^\n]*\\)?\n")
(defvar gud-pdb-marker-regexp-file-group 1)
(defvar gud-pdb-marker-regexp-line-group 2)
(defvar gud-pdb-marker-regexp-fnname-group 3)

(defvar gud-pdb-marker-regexp-start "^> ")

(defvar gud-pdb-marker-regexp-breakpoint 
  "reakpoint [0-9]+ at \\(\\([a-zA-Z]:\\)?[^:\n]*\\):\\([0-9]*\\)\n")

;; There's no guarantee that Emacs will hand the filter the entire
;; marker at once; it could be broken up across several strings.  We
;; might even receive a big chunk with several markers in it.  If we
;; receive a chunk of text which looks like it might contain the
;; beginning of a marker, we save it here between calls to the
;; filter.
(defun gud-pdb-marker-filter (string)
  (setq gud-marker-acc (concat gud-marker-acc string))
  (let ((output ""))

    ;; Process all the complete markers in this chunk.
    (while (string-match gud-pdb-marker-regexp gud-marker-acc)
      (setq

       ;; Extract the frame position from the marker.
       gud-last-frame
       (let ((file (match-string gud-pdb-marker-regexp-file-group
				 gud-marker-acc))
	     (line (string-to-int
		    (match-string gud-pdb-marker-regexp-line-group
				  gud-marker-acc))))
	 (if (string-equal file "<string>")
	     gud-last-frame
	   (cons file line)))

       ;; Output everything instead of the below
       output (concat output (substring gud-marker-acc 0 (match-end 0)))
;;	  ;; Append any text before the marker to the output we're going
;;	  ;; to return - we don't include the marker in this text.
;;	  output (concat output
;;		      (substring gud-marker-acc 0 (match-beginning 0)))
       ;; Set the accumulator to the remaining text.
       gud-marker-acc (substring gud-marker-acc (match-end 0))))

    (if (string-match (concat "B" gud-pdb-marker-regexp-breakpoint)
		      gud-marker-acc)
	(let ((file (match-string 1 gud-marker-acc))
	      (line (match-string 3 gud-marker-acc)))
	  (gud-pdb-insert-breakpoint file line)))	 
    (if (string-match (concat "Deleted b" gud-pdb-marker-regexp-breakpoint)
		      gud-marker-acc) 
	(let ((file (match-string 1 gud-marker-acc))
	      (line (match-string 3 gud-marker-acc)))
	  (gud-pdb-remove-breakpoint file line)))	 

    ;; Does the remaining text look like it might end with the
    ;; beginning of another marker?  If it does, then keep it in
    ;; gud-marker-acc until we receive the rest of it.	Since we
    ;; know the full marker regexp above failed, it's pretty simple to
    ;; test for marker starts.
    (if (string-match gud-pdb-marker-regexp-start gud-marker-acc)
	(progn
	  ;; Everything before the potential marker start can be output.
	  (setq output (concat output (substring gud-marker-acc
						 0 (match-beginning 0))))

	  ;; Everything after, we save, to combine with later input.
	  (setq gud-marker-acc
		(substring gud-marker-acc (match-beginning 0))))

      (setq output (concat output gud-marker-acc)
	    gud-marker-acc ""))

    output))

(defun gud-pdb-insert-breakpoint (file line)
  (with-current-buffer (find-file-noselect file)
    (save-current-buffer
      (set (make-local-variable 'gud-minor-mode) 'pdb)
      (set (make-local-variable 'tool-bar-map) gud-tool-bar-map)
      (setq left-margin-width 2)
      (if (get-buffer-window (current-buffer))
	  (set-window-margins (get-buffer-window (current-buffer))
			      left-margin-width right-margin-width)))
    (save-excursion
      (goto-line (string-to-number line))
      (let ((start (progn (beginning-of-line)
			  (- (point) 1)))
	    (end (progn (end-of-line) (+ (point) 1))))
	(if (display-images-p)
	    (progn
	      (remove-images start end)
	      (put-image breakpoint-enabled-icon
			 (+ start 1)
			 "breakpoint icon enabled"
			 'left-margin))
	  (gdb-remove-strings start end)
	      (gdb-put-string "B" (+ start 1)))))))

(defun gud-pdb-remove-breakpoint (file line)
  (with-current-buffer (find-file-noselect file)
    (save-excursion
      (goto-line (string-to-number line))
      (let ((start (progn (beginning-of-line)
			  (- (point) 1)))
	    (end (progn (end-of-line) (+ (point) 1))))
	(if (display-images-p)
	    (remove-images start end)
	  (gdb-remove-strings start end))))))

(defcustom gud-pdb-command-name "pdb"
  "File name for executing the Python debugger.
This should be an executable on your path, or an absolute file name."
  :type 'string
  :group 'gud)

(defun pdb (command-line)
  "Run pdb on program FILE in buffer `*gud-FILE*'.
The directory containing FILE becomes the initial working directory
and source-file directory for your debugger."
  (interactive
   (list (gud-query-cmdline 'pdb)))

  (gud-common-init command-line nil 'gud-pdb-marker-filter)
  (set (make-local-variable 'gud-minor-mode) 'pdb)

  (gud-def gud-break  "break %l"     "\C-b" "Set breakpoint at current line.")
  (gud-def gud-remove "clear %f:%l"  "\C-d" "Remove breakpoint at current line")
  (gud-def gud-step   "step"         "\C-s" "Step one source line with display.")
  (gud-def gud-next   "next"         "\C-n" "Step one line (skip functions).")
  (gud-def gud-cont   "continue"     "\C-r" "Continue with display.")
  (gud-def gud-finish "return"       "\C-f" "Finish executing current function.")
  (gud-def gud-up     "up"           "<" "Up one stack frame.")
  (gud-def gud-down   "down"         ">" "Down one stack frame.")
  (gud-def gud-print  "p %e"         "\C-p" "Evaluate Python expression at point.")
  ;; Is this right?
  (gud-def gud-statement "! %e"      "\C-e" "Execute Python statement at point.")

  ;; (setq comint-prompt-regexp "^(.*pdb[+]?) *")
  (setq comint-prompt-regexp "^(Pdb) *")
  (setq paragraph-start comint-prompt-regexp)
  (run-hooks 'pdb-mode-hook))





From public-kbutler at sabaydi.com  Sat Sep 20 12:06:33 2003
From: public-kbutler at sabaydi.com (Kevin J. Butler)
Date: Sat Sep 20 12:06:59 2003
Subject: [Python-Dev] Enhancement to pdb in gud.el
In-Reply-To: <16236.28754.105629.565153@nick.uklinux.net>
References: <E1A0ErE-0003ue-8W@mail.python.org>	<3F6B11BA.6000008@sabaydi.com>	<16235.18128.422738.929390@nick.uklinux.net>	<3F6B533E.5000403@sabaydi.com>
	<16236.28754.105629.565153@nick.uklinux.net>
Message-ID: <3F6C7B09.5060608@sabaydi.com>

Nick Roberts wrote:
> I've just noticed that you can enable/disable breakpoints in python. In my
> mode for gdb, the breakpoints are greyed out when the breakpoints become
> disabled. I could do this for python too if these commands were changed to
> emit a message.

Two additional lines will add that - see below.

> If your patches are accepted (by SF?) then I will integrate my changes into
> gud.el in the Emacs CVS repository.

OK, I'll go ahead and submit them...

However, what is the fallback behavior when emacs with the new gud.el 
encounters a pdb.py that doesn't have the patch?  That is, will users of 
existing versions of python lose any functionality?

kb

*** pdb.py.p1	Sat Sep 20 09:54:20 2003
--- pdb.py	Sat Sep 20 09:58:22 2003
***************
*** 388,393 ****
--- 388,394 ----
               bp = bdb.Breakpoint.bpbynumber[i]
               if bp:
                   bp.enable()
+                 print 'Enabled breakpoint %s at %s:%s' % (i, bp.file, 
bp.line)

       def do_disable(self, arg):
           args = arg.split()
***************
*** 405,410 ****
--- 406,412 ----
               bp = bdb.Breakpoint.bpbynumber[i]
               if bp:
                   bp.disable()
+                 print 'Disabled breakpoint %s at %s:%s' % (i, bp.file, 
bp.line)

       def do_condition(self, arg):
           # arg is breakpoint number and condition


From anthony at interlink.com.au  Sat Sep 20 12:16:35 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sat Sep 20 12:18:15 2003
Subject: [Python-Dev] Py2.3.1
Message-ID: <200309201616.h8KGGaf5021370@localhost.localdomain>


I'm planning on starting the dance of the seven release tarballs Tuesday 
morning AU time, which is more or less 0:00 Tuesday UTC. Assuming that
no-one objects to this timeframe, could people be _extremely_ conservative
with checkins on the branch for the 24 hours before this? I'm going to 
be starting to cut test release tarballs probably some time tomorrow and
building it on as many platforms as I can ssh to easily - I can make this
available to other people if they want to help test, or if they've got
some sort of strange version of Unix (Irix and HP/UX are always good
ones to check for platform oddities).

Anthony
--
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.


From nick at nick.uklinux.net  Sat Sep 20 17:33:56 2003
From: nick at nick.uklinux.net (Nick Roberts)
Date: Sat Sep 20 17:45:44 2003
Subject: [Python-Dev] Enhancement to pdb in gud.el
In-Reply-To: <3F6C7B09.5060608@sabaydi.com>
References: <E1A0ErE-0003ue-8W@mail.python.org> <3F6B11BA.6000008@sabaydi.com>
	<16235.18128.422738.929390@nick.uklinux.net>
	<3F6B533E.5000403@sabaydi.com>
	<16236.28754.105629.565153@nick.uklinux.net>
	<3F6C7B09.5060608@sabaydi.com>
Message-ID: <16236.51140.284372.526221@nick.uklinux.net>


 > However, what is the fallback behavior when emacs with the new gud.el 
 > encounters a pdb.py that doesn't have the patch?  That is, will users of 
 > existing versions of python lose any functionality?

The messages from clear, enable and disable will be new, so the existing
behaviour will hold for existing versions of python. However, the message from
break (and tbreak) will not have changed, but the way gud.el acts on it will.
So, break will place a breakpoint icon that clear can't remove.

However, these changes will go on the main trunk of the CVS version of Emacs
which is unlikely to be released this year. Emacs development is relatively
slow and the last non-bugfix release was Sepember 2001 (21.1). How often are
new versions of Python released? Providing the next release of Python is
before that of Emacs, I dont think there's a problem.

Nick

From tim.one at comcast.net  Sat Sep 20 18:49:53 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sat Sep 20 18:49:58 2003
Subject: [Python-Dev] Py2.3.1
In-Reply-To: <200309201616.h8KGGaf5021370@localhost.localdomain>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEGGGAAB.tim.one@comcast.net>

[Anthony Baxter]
> I'm planning on starting the dance of the seven release tarballs
> Tuesday morning AU time, which is more or less 0:00 Tuesday UTC.
> Assuming that no-one objects to this timeframe, could people be
> _extremely_ conservative with checkins on the branch for the 24 hours
> before this?

Since CVS doesn't store metadata about branches, whenever mentioning a
branch it's always helpful to name the exact branch tag you're talking
about.  Are you talking about the release23-maint branch here?

If so, please note that I just made a few checkins to repair Windows
breakage in the test suite, on the HEAD, but I don't expect to have time to
backport that stuff (it's not hard, I've just got too much else to do now
and don't even have a release23-maint checkout yet).

Good luck!


From tim.one at comcast.net  Sat Sep 20 19:15:08 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sat Sep 20 19:15:15 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <m3ekycbjb1.fsf@mira.informatik.hu-berlin.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEGHGAAB.tim.one@comcast.net>

[M.-A. Lemburg]
>> Since wchar_t is the only case where a signed type can pop
>> up, why not extend the autoconf test to check for signedness
>> and then reject signed wchar_t value as not-usable (ie.
>> undefine HAVE_USABLE_WCHAR_T).

[martin@v.loewis.de]
> Because that would exclude a number of relevant systems where wchar_t
> would be usable.

So what if MAL ammened his suggestion to

    reject signed 2-byte wchar_t value as not-usable
                 +++++++

?

From bac at OCF.Berkeley.EDU  Sat Sep 20 20:27:34 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sat Sep 20 20:27:37 2003
Subject: [Python-Dev] Enhancement to pdb in gud.el
In-Reply-To: <16236.51140.284372.526221@nick.uklinux.net>
References: <E1A0ErE-0003ue-8W@mail.python.org>
	<3F6B11BA.6000008@sabaydi.com>	<16235.18128.422738.929390@nick.uklinux.net>	<3F6B533E.5000403@sabaydi.com>	<16236.28754.105629.565153@nick.uklinux.net>	<3F6C7B09.5060608@sabaydi.com>
	<16236.51140.284372.526221@nick.uklinux.net>
Message-ID: <3F6CF076.30409@ocf.berkeley.edu>

Nick Roberts wrote:
>  > However, what is the fallback behavior when emacs with the new gud.el 
>  > encounters a pdb.py that doesn't have the patch?  That is, will users of 
>  > existing versions of python lose any functionality?
> 
> The messages from clear, enable and disable will be new, so the existing
> behaviour will hold for existing versions of python. However, the message from
> break (and tbreak) will not have changed, but the way gud.el acts on it will.
> So, break will place a breakpoint icon that clear can't remove.
> 
> However, these changes will go on the main trunk of the CVS version of Emacs
> which is unlikely to be released this year. Emacs development is relatively
> slow and the last non-bugfix release was Sepember 2001 (21.1). How often are
> new versions of Python released? Providing the next release of Python is
> before that of Emacs, I dont think there's a problem.
> 

The frequency of Python releases is actually being debated as of this 
moment.  Some have suggested having a release once a year (which seems 
to be winning unanimous support).  If we stick with the old way (I 
couldn't count on it) then who knows; probably year to a year and a half.

As for this thread, is this the proper place for this?  Wouldn't on the 
python-mode mailing list 
(http://mail.python.org/mailman/listinfo/python-mode) be a better place 
for Emacs-related stuff?  Or is my Vim bias influencing me too much?

-Brett


From theller at python.net  Fri Sep 19 15:35:31 2003
From: theller at python.net (Thomas Heller)
Date: Sat Sep 20 21:27:39 2003
Subject: [Python-Dev] 2.3.1 windows binaries
Message-ID: <znh0bmgs.fsf@python.net>

For the 2.3.1 Windows binaries, I've followed the instructions in
PCBuild\readme.txt and downloaded the sources for the additional
packages. I have

  bzip2-1.0.2.tar.gz
  db-4.1.25.NC.zip
  tcl843-src.zip
  tk843-src.zip
  openssl-0.9.7b.tar.gz

and zlib-1.1.4.

All these versions are exactly those the readme file mentions, with the
exception of openssl. Quote from the PCBuild\readme.txt:

    Get the latest source code for OpenSSL from
        http://www.openssl.org

    You (probably) don't want the "engine" code.  For example, get
        openssl-0.9.6g.tar.gz
    not
        openssl-engine-0.9.6g.tar.gz

    Unpack into the "dist" directory, retaining the folder name from
    the archive - for example, the latest stable OpenSSL will install as
        dist/openssl-0.9.6g

    You can (theoretically) use any version of OpenSSL you like - the
    build process will automatically select the latest version.

Is it important to use the same openssl version that Python 2.3.0 used,
or is the readme file correct in saying that the latest version is the
one to use? And does this have to be mentioned somewhere (Misc/NEWS, the
readme file for windows)?

Thomas


From tim.one at comcast.net  Sat Sep 20 21:35:27 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sat Sep 20 21:35:30 2003
Subject: [Python-Dev] 2.3.1 windows binaries
In-Reply-To: <znh0bmgs.fsf@python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEJFGAAB.tim.one@comcast.net>

[Thomas Heller]
> For the 2.3.1 Windows binaries, I've followed the instructions in
> PCBuild\readme.txt and downloaded the sources for the additional
> packages. I have
>
>   bzip2-1.0.2.tar.gz
>   db-4.1.25.NC.zip
>   tcl843-src.zip
>   tk843-src.zip
>   openssl-0.9.7b.tar.gz
>
> and zlib-1.1.4.
>
> All these versions are exactly those the readme file mentions, with
> the exception of openssl. Quote from the PCBuild\readme.txt:
>
>     Get the latest source code for OpenSSL from
>         http://www.openssl.org
>
>     You (probably) don't want the "engine" code.  For example, get
>         openssl-0.9.6g.tar.gz
>     not
>         openssl-engine-0.9.6g.tar.gz
>
>     Unpack into the "dist" directory, retaining the folder name from
>     the archive - for example, the latest stable OpenSSL will install
>         as dist/openssl-0.9.6g
>
>     You can (theoretically) use any version of OpenSSL you like - the
>     build process will automatically select the latest version.
>
> Is it important to use the same openssl version that Python 2.3.0
> used, or is the readme file correct in saying that the latest version
> is the one to use?

Nobody knows, because nobody (AFAIK) has ever tried building the Windows
Python with an OpenSSL release other than the one mentioned in README.txt.
Try it and see whether it works?  That would be a happier suggestion if the
Python test suite exercised more of the SSL code.

> And does this have to be mentioned somewhere (Misc/NEWS, the readme
> file for windows)?

PCbuild\readme.txt in a given release is intended to describe exactly how
that release was built, in detail sufficient so that someone other than the
person who built the release stands a good chance of reproducing the whole
bit.

I've also added a NEWS entry when moving to a different release of any of
the 3rd-party packages.  That's just full disclosure <wink>.


From public-kbutler at sabaydi.com  Sat Sep 20 23:27:51 2003
From: public-kbutler at sabaydi.com (Kevin J. Butler)
Date: Sat Sep 20 23:28:07 2003
Subject: [Python-Dev] Enhancement to pdb in gud.el
In-Reply-To: <3F6CF076.30409@ocf.berkeley.edu>
References: <E1A0ErE-0003ue-8W@mail.python.org>
	<3F6B11BA.6000008@sabaydi.com>	<16235.18128.422738.929390@nick.uklinux.net>	<3F6B533E.5000403@sabaydi.com>	<16236.28754.105629.565153@nick.uklinux.net>	<3F6C7B09.5060608@sabaydi.com>
	<16236.51140.284372.526221@nick.uklinux.net>
	<3F6CF076.30409@ocf.berkeley.edu>
Message-ID: <3F6D1AB7.1050608@sabaydi.com>

> Nick Roberts wrote:
> 
>>  > However, what is the fallback behavior when emacs with the new 
>> gud.el  > encounters a pdb.py that doesn't have the patch?  That is, 
>> will users of  > existing versions of python lose any functionality?
>>
>> The messages from clear, enable and disable will be new, so the existing
>> behaviour will hold for existing versions of python. However, the 
>> message from
>> break (and tbreak) will not have changed, but the way gud.el acts on 
>> it will.
>> So, break will place a breakpoint icon that clear can't remove.

>> However, these changes will go on the main trunk of the CVS version of 
>> Emacs
>> which is unlikely to be released this year. Emacs development is 
>> relatively
>> slow and the last non-bugfix release was Sepember 2001 (21.1). How 
>> often are
>> new versions of Python released? Providing the next release of Python is
>> before that of Emacs, I dont think there's a problem.

Well, older versions of Python tend to remain in usage long after newer 
versions are available - for instance, 1.5.2 is still the standard in 
some places, and there is virtually no chance of patching 1.5.2 for 
something like this.

It sounds like the emacs-side misbehavior will be cosmetic only, so 
probably not too bad.

Brett C. wrote:
> As for this thread, is this the proper place for this?  Wouldn't on the 
> python-mode mailing list 
> (http://mail.python.org/mailman/listinfo/python-mode) be a better place 
> for Emacs-related stuff?  Or is my Vim bias influencing me too much?

I think the Vim bias has is influencing you too much.  It has been known 
to do that.  ;-)  (Speaking as a recovering/apostate vi user).

I consider this more pdb-related than emacs-specific -
I'd expect that integration with, say, Eclipse would benefit from this 
enhancement as well, so I've been hoping to get some feedback from the 
list...

BTW, I've submitted the patch, it is 809887: 
http://sourceforge.net/tracker/index.php?func=detail&aid=809887&group_id=5470&atid=305470

kb


From martin at v.loewis.de  Sun Sep 21 03:38:03 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sun Sep 21 03:38:16 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEGHGAAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEGHGAAB.tim.one@comcast.net>
Message-ID: <m31xuamw10.fsf@mira.informatik.hu-berlin.de>

"Tim Peters" <tim.one@comcast.net> writes:

> So what if MAL ammened his suggestion to
> 
>     reject signed 2-byte wchar_t value as not-usable
>                  +++++++
> 
> ?

That would be a very sensible suggestion.

Regards,
Martin

From anthony at interlink.com.au  Sun Sep 21 03:45:15 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sun Sep 21 03:46:48 2003
Subject: [Python-Dev] Py2.3.1 
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEGGGAAB.tim.one@comcast.net> 
Message-ID: <200309210745.h8L7jGrU002249@localhost.localdomain>


>>> "Tim Peters" wrote
> Since CVS doesn't store metadata about branches, whenever mentioning a
> branch it's always helpful to name the exact branch tag you're talking
> about.  Are you talking about the release23-maint branch here?

Yep, that's the branch I was referring to..

> If so, please note that I just made a few checkins to repair Windows
> breakage in the test suite, on the HEAD, but I don't expect to have time to
> backport that stuff (it's not hard, I've just got too much else to do now
> and don't even have a release23-maint checkout yet).

I think Raymond's already done the backporting.

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From python at rcn.com  Sun Sep 21 04:37:01 2003
From: python at rcn.com (Raymond Hettinger)
Date: Sun Sep 21 04:37:19 2003
Subject: [Python-Dev] Py2.3.1 
References: <200309210745.h8L7jGrU002249@localhost.localdomain>
Message-ID: <005901c3801b$877d1620$e841fea9@oemcomputer>

> > If so, please note that I just made a few checkins to repair Windows
> > breakage in the test suite, on the HEAD, but I don't expect to have time to
> > backport that stuff (it's not hard, I've just got too much else to do now
> > and don't even have a release23-maint checkout yet).
> 
> I think Raymond's already done the backporting.

Yes.  There was only one that applied to Py2.3.1.

All tests are passing on Windows except for test_bsddb3
which fails for both 2.3.1 and 2.4.  I've sent Greg a note 
with the test output to see what he can make of it.


Raymond

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

From jim at zope.com  Sun Sep 21 06:06:45 2003
From: jim at zope.com (Jim Fulton)
Date: Sun Sep 21 06:07:29 2003
Subject: [Python-Dev] Fun with 2.3 shutdown
In-Reply-To: <20030919120100.GB23317@vicky.ecs.soton.ac.uk>
References: <LNBBLJKPBEHFEDALKOLCOEOHFPAB.tim.one@comcast.net>
	<20030919120100.GB23317@vicky.ecs.soton.ac.uk>
Message-ID: <3F6D7835.5080407@zope.com>

Armin Rigo wrote:

...

> Another question, what was the rationale for setting module globals to None
> instead of simply deleting them (and getting the expected AttributeErrors both
> at C and at Python level) ?

Or, given cyclic GC, why isn't it enough to clear sys.modules?
In theory, shouldn't that make all modules (and thier globals)
collectable unless there are threads hanging on to them?

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (703) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org


From mal at lemburg.com  Sun Sep 21 07:30:27 2003
From: mal at lemburg.com (M.-A. Lemburg)
Date: Sun Sep 21 07:30:22 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 
	2.197, 2.198
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEGHGAAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEGHGAAB.tim.one@comcast.net>
Message-ID: <3F6D8BD3.7090600@lemburg.com>

Tim Peters wrote:
> [M.-A. Lemburg]
> 
>>>Since wchar_t is the only case where a signed type can pop
>>>up, why not extend the autoconf test to check for signedness
>>>and then reject signed wchar_t value as not-usable (ie.
>>>undefine HAVE_USABLE_WCHAR_T).
> 
> [martin@v.loewis.de]
> 
>>Because that would exclude a number of relevant systems where wchar_t
>>would be usable.
> 
> So what if MAL ammened his suggestion to
> 
>     reject signed 2-byte wchar_t value as not-usable
>                  +++++++
> ?

That would not solve the problem.

Note that we have proper conversion routines that allow
converting between wchar_t and Py_UNICODE. These routines must
be used for conversions anyway (even if Py_UNICODE and wchar_t
happen to be the same type), so from a programmer perspective
changing Py_UNICODE to be unsigned won't be noticed and we
don't lose anything much.

Again, I don't see the point in using a signed type for data
that doesn't have any concept of signed values. It's just
bad design and we shouldn't try to go down the same route
if we don't have to.

The Unicode implementation has always defined Py_UNICODE to
be an unsigned type; see the Unicode PEP 100:

"""
Internal Format

     The internal format for Unicode objects should use a Python
     specific fixed format <PythonUnicode> implemented as 'unsigned
     short' (or another unsigned numeric type having 16 bits).  Byte
     order is platform dependent.

...

     The configure script should provide aid in deciding whether Python
     can use the native wchar_t type or not (it has to be a 16-bit
     unsigned type).
"""

Python can also deal with UCS4 now, but the concept remains the
same.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Sep 21 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::


From tim.one at comcast.net  Sun Sep 21 10:48:21 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Sep 21 10:48:29 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <3F6D8BD3.7090600@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEELMGAAB.tim.one@comcast.net>

[Tim]
>> So what if MAL ammened his suggestion to
>>
>>     reject signed 2-byte wchar_t value as not-usable
>> +++++++ ?

[M.-A. Lemburg]
> That would not solve the problem.

Then what is the problem, specifically?  I thought you agreed with Martin
that a signed 32-bit type doesn't hurt, since the sign bit remains clear
then in all cases of Unicode data.

> Note that we have proper conversion routines that allow
> converting between wchar_t and Py_UNICODE. These routines must
> be used for conversions anyway (even if Py_UNICODE and wchar_t
> happen to be the same type), so from a programmer perspective
> changing Py_UNICODE to be unsigned won't be noticed and we
> don't lose anything much.
>
> Again, I don't see the point in using a signed type for data
> that doesn't have any concept of signed values. It's just
> bad design and we shouldn't try to go down the same route
> if we don't have to.

I don't know why Martin favors wchar_t when possible.  The answer to that
isn't clear.  The answer to why there's an intractable problem if wchar_t
happens to be a signed type > 2 bytes also isn't clear.

> The Unicode implementation has always defined Py_UNICODE to
> be an unsigned type; see the Unicode PEP 100:
>
> """
> Internal Format
>
>      The internal format for Unicode objects should use a Python
>      specific fixed format <PythonUnicode> implemented as 'unsigned
>      short' (or another unsigned numeric type having 16 bits).  Byte
>      order is platform dependent.
>
> ...
>
>      The configure script should provide aid in deciding whether
>      Python can use the native wchar_t type or not (it has to be a
>      16-bit unsigned type).
> """
>
> Python can also deal with UCS4 now, but the concept remains the
> same.

Well, it doesn't have to be a 16-bit type either, even in a UCS2 build, and
we had a long argument about that one before, because a particular Cray
system didn't have any 16-bit type and the Unicode code wasn't working
there.  That got repaired when I rewrote the few bits of code that assumed
"exactly 16 bits" to live with the weaker "at least 16 bits".

In this iteration, Martin agreed that a signed 16-bit wchar_t can be
rejected.  The question remaining is what actual problem exists when there's
a signed wchar_t exceeding 16 bits.  Since Jeremy is running on exactly such
a system, and the tests pass for him, there's no *obvious* problem with it
(the segfault he experienced was due to reading uninitialized memory, and
that was a bug, and that's been fixed).


From jeremy at zope.com  Sun Sep 21 11:08:43 2003
From: jeremy at zope.com (Jeremy Hylton)
Date: Sun Sep 21 11:10:25 2003
Subject: [Python-Dev] Improving the CGI module
In-Reply-To: <11097042-EAEB-11D7-A692-000393C2D67E@colorstudy.com>
References: <11097042-EAEB-11D7-A692-000393C2D67E@colorstudy.com>
Message-ID: <1064156922.2074.37.camel@localhost.localdomain>

Do we have a SIG yet?  Nothing wrong with some discussion on python-dev,
but we ought to be capturing this stuff in a SIG archive and making it
part of the Plan.

Jeremy



From jeremy at alum.mit.edu  Sun Sep 21 11:13:26 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Sun Sep 21 11:13:44 2003
Subject: [Python-Dev] Fun with 2.3 shutdown
In-Reply-To: <200309190537.h8J5bnIG015051@localhost.localdomain>
References: <200309190537.h8J5bnIG015051@localhost.localdomain>
Message-ID: <1064157206.2074.39.camel@localhost.localdomain>

On Fri, 2003-09-19 at 01:37, Anthony Baxter wrote:
> Hm. Is this serious enough a problem to hold off 2.3.1 until it's fixed?
> My feeling is "no", because it only affects Z3 developers, and they're 
> hardened to random pain by now, and it's also just a shutdown glitch. 

I don't think it's worth waiting, because we don't have a plan for
fixing it.  Even if we did, it's arguable whether it belongs in a bug
fix release -- at least not without extensive developer testing with the
CVS head.

Jeremy



From tim.one at comcast.net  Sun Sep 21 11:34:29 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Sep 21 11:34:34 2003
Subject: [Python-Dev] Py2.3.1 
In-Reply-To: <005901c3801b$877d1620$e841fea9@oemcomputer>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEMBGAAB.tim.one@comcast.net>

[Raymond Hettinger]
> All tests are passing on Windows except for test_bsddb3
> which fails for both 2.3.1 and 2.4.

Did they also fail for you in 2.3?  They failed for me in 2.3, but only
under Win98SE, and IIRC you're running on the similar WinME.  The bsddb3
tests passed for me on Win2K in 2.3.

> I've sent Greg a note with the test output to see what he can make of
> it.

I'll attach what I see on Win98SE under current CVS Python (HEAD).  Looks a
lot like the bsddb3 failures I saw on Win98SE under 2.3, several

DBAgainError: (11, 'Resource temporarily unavailable -- unable to
                    join the environment')

complaints.
-------------- next part --------------
Capture file written by TCAP 3.1, TCAP Copyright(c) 1999 Tim Kannel

C:\Code\python\PCbuild>python  ../lib/test/regrtest.py -vv -u bsddb test_bsddb3
test_bsddb3
test01_associateWithDB (bsddb.test.test_associate.AssociateHashTestCase) ... ok
test02_associateAfterDB (bsddb.test.test_associate.AssociateHashTestCase) ... ok
test01_associateWithDB (bsddb.test.test_associate.AssociateBTreeTestCase) ... ok
test02_associateAfterDB (bsddb.test.test_associate.AssociateBTreeTestCase) ... ok
test01_associateWithDB (bsddb.test.test_associate.AssociateRecnoTestCase) ... ok
test02_associateAfterDB (bsddb.test.test_associate.AssociateRecnoTestCase) ... ok
test01_associateWithDB (bsddb.test.test_associate.ShelveAssociateHashTestCase) ... ok
test02_associateAfterDB (bsddb.test.test_associate.ShelveAssociateHashTestCase) ... ok
test01_associateWithDB (bsddb.test.test_associate.ShelveAssociateBTreeTestCase) ... ok
test02_associateAfterDB (bsddb.test.test_associate.ShelveAssociateBTreeTestCase) ... ok
test01_associateWithDB (bsddb.test.test_associate.ShelveAssociateRecnoTestCase) ... ok
test02_associateAfterDB (bsddb.test.test_associate.ShelveAssociateRecnoTestCase) ... ok
test01_associateWithDB (bsddb.test.test_associate.ThreadedAssociateHashTestCase) ... ok
test02_associateAfterDB (bsddb.test.test_associate.ThreadedAssociateHashTestCase) ... ok
test01_associateWithDB (bsddb.test.test_associate.ThreadedAssociateBTreeTestCase) ... ok
test02_associateAfterDB (bsddb.test.test_associate.ThreadedAssociateBTreeTestCase) ... ok
test01_associateWithDB (bsddb.test.test_associate.ThreadedAssociateRecnoTestCase) ... ok
test02_associateAfterDB (bsddb.test.test_associate.ThreadedAssociateRecnoTestCase) ... ok
test00_version (bsddb.test.test_basics.VersionTestCase) ... ok
test01_GetsAndPuts (bsddb.test.test_basics.BasicBTreeTestCase) ... ok
test02_DictionaryMethods (bsddb.test.test_basics.BasicBTreeTestCase) ... ok
test03_SimpleCursorStuff (bsddb.test.test_basics.BasicBTreeTestCase) ... ok
test03b_SimpleCursorWithoutGetReturnsNone0 (bsddb.test.test_basics.BasicBTreeTestCase) ... ok
test03c_SimpleCursorGetReturnsNone2 (bsddb.test.test_basics.BasicBTreeTestCase) ... ok
test04_PartialGetAndPut (bsddb.test.test_basics.BasicBTreeTestCase) ... ok
test05_GetSize (bsddb.test.test_basics.BasicBTreeTestCase) ... ok
test06_Truncate (bsddb.test.test_basics.BasicBTreeTestCase) ... ok
test01_GetsAndPuts (bsddb.test.test_basics.BasicHashTestCase) ... ok
test02_DictionaryMethods (bsddb.test.test_basics.BasicHashTestCase) ... ok
test03_SimpleCursorStuff (bsddb.test.test_basics.BasicHashTestCase) ... ok
test03b_SimpleCursorWithoutGetReturnsNone0 (bsddb.test.test_basics.BasicHashTestCase) ... ok
test03c_SimpleCursorGetReturnsNone2 (bsddb.test.test_basics.BasicHashTestCase) ... ok
test04_PartialGetAndPut (bsddb.test.test_basics.BasicHashTestCase) ... ok
test05_GetSize (bsddb.test.test_basics.BasicHashTestCase) ... ok
test06_Truncate (bsddb.test.test_basics.BasicHashTestCase) ... ok
test01_GetsAndPuts (bsddb.test.test_basics.BasicBTreeWithThreadFlagTestCase) ... ok
test02_DictionaryMethods (bsddb.test.test_basics.BasicBTreeWithThreadFlagTestCase) ... ok
test03_SimpleCursorStuff (bsddb.test.test_basics.BasicBTreeWithThreadFlagTestCase) ... ok
test03b_SimpleCursorWithoutGetReturnsNone0 (bsddb.test.test_basics.BasicBTreeWithThreadFlagTestCase) ... ok
test03c_SimpleCursorGetReturnsNone2 (bsddb.test.test_basics.BasicBTreeWithThreadFlagTestCase) ... ok
test04_PartialGetAndPut (bsddb.test.test_basics.BasicBTreeWithThreadFlagTestCase) ... ok
test05_GetSize (bsddb.test.test_basics.BasicBTreeWithThreadFlagTestCase) ... ok
test06_Truncate (bsddb.test.test_basics.BasicBTreeWithThreadFlagTestCase) ... ok
test01_GetsAndPuts (bsddb.test.test_basics.BasicHashWithThreadFlagTestCase) ... ok
test02_DictionaryMethods (bsddb.test.test_basics.BasicHashWithThreadFlagTestCase) ... ok
test03_SimpleCursorStuff (bsddb.test.test_basics.BasicHashWithThreadFlagTestCase) ... ok
test03b_SimpleCursorWithoutGetReturnsNone0 (bsddb.test.test_basics.BasicHashWithThreadFlagTestCase) ... ok
test03c_SimpleCursorGetReturnsNone2 (bsddb.test.test_basics.BasicHashWithThreadFlagTestCase) ... ok
test04_PartialGetAndPut (bsddb.test.test_basics.BasicHashWithThreadFlagTestCase) ... ok
test05_GetSize (bsddb.test.test_basics.BasicHashWithThreadFlagTestCase) ... ok
test06_Truncate (bsddb.test.test_basics.BasicHashWithThreadFlagTestCase) ... ok
test01_GetsAndPuts (bsddb.test.test_basics.BasicBTreeWithEnvTestCase) ... ok
test02_DictionaryMethods (bsddb.test.test_basics.BasicBTreeWithEnvTestCase) ... ok
test03_SimpleCursorStuff (bsddb.test.test_basics.BasicBTreeWithEnvTestCase) ... ok
test03b_SimpleCursorWithoutGetReturnsNone0 (bsddb.test.test_basics.BasicBTreeWithEnvTestCase) ... ok
test03c_SimpleCursorGetReturnsNone2 (bsddb.test.test_basics.BasicBTreeWithEnvTestCase) ... ok
test04_PartialGetAndPut (bsddb.test.test_basics.BasicBTreeWithEnvTestCase) ... ok
test05_GetSize (bsddb.test.test_basics.BasicBTreeWithEnvTestCase) ... ok
test06_Truncate (bsddb.test.test_basics.BasicBTreeWithEnvTestCase) ... ok
test01_GetsAndPuts (bsddb.test.test_basics.BasicHashWithEnvTestCase) ... ok
test02_DictionaryMethods (bsddb.test.test_basics.BasicHashWithEnvTestCase) ... ok
test03_SimpleCursorStuff (bsddb.test.test_basics.BasicHashWithEnvTestCase) ... ok
test03b_SimpleCursorWithoutGetReturnsNone0 (bsddb.test.test_basics.BasicHashWithEnvTestCase) ... ok
test03c_SimpleCursorGetReturnsNone2 (bsddb.test.test_basics.BasicHashWithEnvTestCase) ... ok
test04_PartialGetAndPut (bsddb.test.test_basics.BasicHashWithEnvTestCase) ... ok
test05_GetSize (bsddb.test.test_basics.BasicHashWithEnvTestCase) ... ok
test06_Truncate (bsddb.test.test_basics.BasicHashWithEnvTestCase) ... ok
test01_GetsAndPuts (bsddb.test.test_basics.BTreeTransactionTestCase) ... ok
test02_DictionaryMethods (bsddb.test.test_basics.BTreeTransactionTestCase) ... ok
test03_SimpleCursorStuff (bsddb.test.test_basics.BTreeTransactionTestCase) ... ok
test03b_SimpleCursorWithoutGetReturnsNone0 (bsddb.test.test_basics.BTreeTransactionTestCase) ... ok
test03c_SimpleCursorGetReturnsNone2 (bsddb.test.test_basics.BTreeTransactionTestCase) ... ok
test04_PartialGetAndPut (bsddb.test.test_basics.BTreeTransactionTestCase) ... ok
test05_GetSize (bsddb.test.test_basics.BTreeTransactionTestCase) ... ok
test06_Transactions (bsddb.test.test_basics.BTreeTransactionTestCase) ... ok
test06_Truncate (bsddb.test.test_basics.BTreeTransactionTestCase) ... ok
test07_TxnTruncate (bsddb.test.test_basics.BTreeTransactionTestCase) ... ok
test08_TxnLateUse (bsddb.test.test_basics.BTreeTransactionTestCase) ... ok
test01_GetsAndPuts (bsddb.test.test_basics.HashTransactionTestCase) ... ok
test02_DictionaryMethods (bsddb.test.test_basics.HashTransactionTestCase) ... ok
test03_SimpleCursorStuff (bsddb.test.test_basics.HashTransactionTestCase) ... ok
test03b_SimpleCursorWithoutGetReturnsNone0 (bsddb.test.test_basics.HashTransactionTestCase) ... ok
test03c_SimpleCursorGetReturnsNone2 (bsddb.test.test_basics.HashTransactionTestCase) ... ok
test04_PartialGetAndPut (bsddb.test.test_basics.HashTransactionTestCase) ... ok
test05_GetSize (bsddb.test.test_basics.HashTransactionTestCase) ... ok
test06_Transactions (bsddb.test.test_basics.HashTransactionTestCase) ... ok
test06_Truncate (bsddb.test.test_basics.HashTransactionTestCase) ... ok
test07_TxnTruncate (bsddb.test.test_basics.HashTransactionTestCase) ... ok
test08_TxnLateUse (bsddb.test.test_basics.HashTransactionTestCase) ... ok
test01_GetsAndPuts (bsddb.test.test_basics.BTreeRecnoTestCase) ... ok
test02_DictionaryMethods (bsddb.test.test_basics.BTreeRecnoTestCase) ... ok
test03_SimpleCursorStuff (bsddb.test.test_basics.BTreeRecnoTestCase) ... ok
test03b_SimpleCursorWithoutGetReturnsNone0 (bsddb.test.test_basics.BTreeRecnoTestCase) ... ok
test03c_SimpleCursorGetReturnsNone2 (bsddb.test.test_basics.BTreeRecnoTestCase) ... ok
test04_PartialGetAndPut (bsddb.test.test_basics.BTreeRecnoTestCase) ... ok
test05_GetSize (bsddb.test.test_basics.BTreeRecnoTestCase) ... ok
test06_Truncate (bsddb.test.test_basics.BTreeRecnoTestCase) ... ok
test07_RecnoInBTree (bsddb.test.test_basics.BTreeRecnoTestCase) ... ok
test01_GetsAndPuts (bsddb.test.test_basics.BTreeRecnoWithThreadFlagTestCase) ... ok
test02_DictionaryMethods (bsddb.test.test_basics.BTreeRecnoWithThreadFlagTestCase) ... ok
test03_SimpleCursorStuff (bsddb.test.test_basics.BTreeRecnoWithThreadFlagTestCase) ... ok
test03b_SimpleCursorWithoutGetReturnsNone0 (bsddb.test.test_basics.BTreeRecnoWithThreadFlagTestCase) ... ok
test03c_SimpleCursorGetReturnsNone2 (bsddb.test.test_basics.BTreeRecnoWithThreadFlagTestCase) ... ok
test04_PartialGetAndPut (bsddb.test.test_basics.BTreeRecnoWithThreadFlagTestCase) ... ok
test05_GetSize (bsddb.test.test_basics.BTreeRecnoWithThreadFlagTestCase) ... ok
test06_Truncate (bsddb.test.test_basics.BTreeRecnoWithThreadFlagTestCase) ... ok
test07_RecnoInBTree (bsddb.test.test_basics.BTreeRecnoWithThreadFlagTestCase) ... ok
test01_GetsAndPuts (bsddb.test.test_basics.BTreeDUPTestCase) ... ok
test02_DictionaryMethods (bsddb.test.test_basics.BTreeDUPTestCase) ... ok
test03_SimpleCursorStuff (bsddb.test.test_basics.BTreeDUPTestCase) ... ok
test03b_SimpleCursorWithoutGetReturnsNone0 (bsddb.test.test_basics.BTreeDUPTestCase) ... ok
test03c_SimpleCursorGetReturnsNone2 (bsddb.test.test_basics.BTreeDUPTestCase) ... ok
test04_PartialGetAndPut (bsddb.test.test_basics.BTreeDUPTestCase) ... ok
test05_GetSize (bsddb.test.test_basics.BTreeDUPTestCase) ... ok
test06_Truncate (bsddb.test.test_basics.BTreeDUPTestCase) ... ok
test08_DuplicateKeys (bsddb.test.test_basics.BTreeDUPTestCase) ... ok
test01_GetsAndPuts (bsddb.test.test_basics.HashDUPTestCase) ... ok
test02_DictionaryMethods (bsddb.test.test_basics.HashDUPTestCase) ... ok
test03_SimpleCursorStuff (bsddb.test.test_basics.HashDUPTestCase) ... ok
test03b_SimpleCursorWithoutGetReturnsNone0 (bsddb.test.test_basics.HashDUPTestCase) ... ok
test03c_SimpleCursorGetReturnsNone2 (bsddb.test.test_basics.HashDUPTestCase) ... ok
test04_PartialGetAndPut (bsddb.test.test_basics.HashDUPTestCase) ... ok
test05_GetSize (bsddb.test.test_basics.HashDUPTestCase) ... ok
test06_Truncate (bsddb.test.test_basics.HashDUPTestCase) ... ok
test08_DuplicateKeys (bsddb.test.test_basics.HashDUPTestCase) ... ok
test01_GetsAndPuts (bsddb.test.test_basics.BTreeDUPWithThreadTestCase) ... ok
test02_DictionaryMethods (bsddb.test.test_basics.BTreeDUPWithThreadTestCase) ... ok
test03_SimpleCursorStuff (bsddb.test.test_basics.BTreeDUPWithThreadTestCase) ... ok
test03b_SimpleCursorWithoutGetReturnsNone0 (bsddb.test.test_basics.BTreeDUPWithThreadTestCase) ... ok
test03c_SimpleCursorGetReturnsNone2 (bsddb.test.test_basics.BTreeDUPWithThreadTestCase) ... ok
test04_PartialGetAndPut (bsddb.test.test_basics.BTreeDUPWithThreadTestCase) ... ok
test05_GetSize (bsddb.test.test_basics.BTreeDUPWithThreadTestCase) ... ok
test06_Truncate (bsddb.test.test_basics.BTreeDUPWithThreadTestCase) ... ok
test08_DuplicateKeys (bsddb.test.test_basics.BTreeDUPWithThreadTestCase) ... ok
test01_GetsAndPuts (bsddb.test.test_basics.HashDUPWithThreadTestCase) ... ok
test02_DictionaryMethods (bsddb.test.test_basics.HashDUPWithThreadTestCase) ... ok
test03_SimpleCursorStuff (bsddb.test.test_basics.HashDUPWithThreadTestCase) ... ok
test03b_SimpleCursorWithoutGetReturnsNone0 (bsddb.test.test_basics.HashDUPWithThreadTestCase) ... ok
test03c_SimpleCursorGetReturnsNone2 (bsddb.test.test_basics.HashDUPWithThreadTestCase) ... ok
test04_PartialGetAndPut (bsddb.test.test_basics.HashDUPWithThreadTestCase) ... ok
test05_GetSize (bsddb.test.test_basics.HashDUPWithThreadTestCase) ... ok
test06_Truncate (bsddb.test.test_basics.HashDUPWithThreadTestCase) ... ok
test08_DuplicateKeys (bsddb.test.test_basics.HashDUPWithThreadTestCase) ... ok
test01_GetsAndPuts (bsddb.test.test_basics.BTreeMultiDBTestCase) ... ok
test02_DictionaryMethods (bsddb.test.test_basics.BTreeMultiDBTestCase) ... ok
test03_SimpleCursorStuff (bsddb.test.test_basics.BTreeMultiDBTestCase) ... ok
test03b_SimpleCursorWithoutGetReturnsNone0 (bsddb.test.test_basics.BTreeMultiDBTestCase) ... ok
test03c_SimpleCursorGetReturnsNone2 (bsddb.test.test_basics.BTreeMultiDBTestCase) ... ok
test04_PartialGetAndPut (bsddb.test.test_basics.BTreeMultiDBTestCase) ... ok
test05_GetSize (bsddb.test.test_basics.BTreeMultiDBTestCase) ... ok
test06_Truncate (bsddb.test.test_basics.BTreeMultiDBTestCase) ... ok
test09_MultiDB (bsddb.test.test_basics.BTreeMultiDBTestCase) ... ok
test01_GetsAndPuts (bsddb.test.test_basics.HashMultiDBTestCase) ... ok
test02_DictionaryMethods (bsddb.test.test_basics.HashMultiDBTestCase) ... ok
test03_SimpleCursorStuff (bsddb.test.test_basics.HashMultiDBTestCase) ... ok
test03b_SimpleCursorWithoutGetReturnsNone0 (bsddb.test.test_basics.HashMultiDBTestCase) ... ok
test03c_SimpleCursorGetReturnsNone2 (bsddb.test.test_basics.HashMultiDBTestCase) ... ok
test04_PartialGetAndPut (bsddb.test.test_basics.HashMultiDBTestCase) ... ok
test05_GetSize (bsddb.test.test_basics.HashMultiDBTestCase) ... ok
test06_Truncate (bsddb.test.test_basics.HashMultiDBTestCase) ... ok
test09_MultiDB (bsddb.test.test_basics.HashMultiDBTestCase) ... ok
test01_btopen (bsddb.test.test_compat.CompatibilityTestCase) ... ok
test02_hashopen (bsddb.test.test_compat.CompatibilityTestCase) ... ok
test03_rnopen (bsddb.test.test_compat.CompatibilityTestCase) ... ok
test04_n_flag (bsddb.test.test_compat.CompatibilityTestCase) ... ok
test01_both (bsddb.test.test_dbobj.dbobjTestCase) ... ok
test02_dbobj_dict_interface (bsddb.test.test_dbobj.dbobjTestCase) ... ok
test01_basics (bsddb.test.test_dbshelve.DBShelveTestCase) ... ok
test02_cursors (bsddb.test.test_dbshelve.DBShelveTestCase) ... ok
test01_basics (bsddb.test.test_dbshelve.BTreeShelveTestCase) ... ok
test02_cursors (bsddb.test.test_dbshelve.BTreeShelveTestCase) ... ok
test01_basics (bsddb.test.test_dbshelve.HashShelveTestCase) ... ok
test02_cursors (bsddb.test.test_dbshelve.HashShelveTestCase) ... ok
test01_basics (bsddb.test.test_dbshelve.ThreadBTreeShelveTestCase) ... ok
test02_cursors (bsddb.test.test_dbshelve.ThreadBTreeShelveTestCase) ... ok
test01_basics (bsddb.test.test_dbshelve.ThreadHashShelveTestCase) ... ok
test02_cursors (bsddb.test.test_dbshelve.ThreadHashShelveTestCase) ... ok
test01_basics (bsddb.test.test_dbshelve.EnvBTreeShelveTestCase) ... ERROR
test02_cursors (bsddb.test.test_dbshelve.EnvBTreeShelveTestCase) ... ok
test01_basics (bsddb.test.test_dbshelve.EnvHashShelveTestCase) ... ERROR
test02_cursors (bsddb.test.test_dbshelve.EnvHashShelveTestCase) ... ok
test01_basics (bsddb.test.test_dbshelve.EnvThreadBTreeShelveTestCase) ... ERROR
test02_cursors (bsddb.test.test_dbshelve.EnvThreadBTreeShelveTestCase) ... ok
test01_basics (bsddb.test.test_dbshelve.EnvThreadHashShelveTestCase) ... ERROR
test02_cursors (bsddb.test.test_dbshelve.EnvThreadHashShelveTestCase) ... ok
test01 (bsddb.test.test_dbtables.TableDBTestCase) ... ok
test02 (bsddb.test.test_dbtables.TableDBTestCase) ... ok
test03 (bsddb.test.test_dbtables.TableDBTestCase) ... ok
test04_MultiCondSelect (bsddb.test.test_dbtables.TableDBTestCase) ... ok
test_CondObjs (bsddb.test.test_dbtables.TableDBTestCase) ... ok
test_CreateOrExtend (bsddb.test.test_dbtables.TableDBTestCase) ... ok
test_Delete (bsddb.test.test_dbtables.TableDBTestCase) ... ok
test_Modify (bsddb.test.test_dbtables.TableDBTestCase) ... ok
test01_close_dbenv_before_db (bsddb.test.test_env_close.DBEnvClosedEarlyCrash) ... ok
test02_close_dbenv_delete_db_success (bsddb.test.test_env_close.DBEnvClosedEarlyCrash) ... ok
test01_get_returns_none (bsddb.test.test_get_none.GetReturnsNoneTestCase) ... ok
test02_get_raises_exception (bsddb.test.test_get_none.GetReturnsNoneTestCase) ... ok
test01_join (bsddb.test.test_join.JoinTestCase) ... ok
test01_simple (bsddb.test.test_lock.LockingTestCase) ... ok
test02_threaded (bsddb.test.test_lock.LockingTestCase) ... ok
test03_set_timeout (bsddb.test.test_lock.LockingTestCase) ... ok
test01_badpointer (bsddb.test.test_misc.MiscTestCase) ... ok
test02_db_home (bsddb.test.test_misc.MiscTestCase) ... ok
test01_basic (bsddb.test.test_queue.SimpleQueueTestCase) ... ok
test02_basicPost32 (bsddb.test.test_queue.SimpleQueueTestCase) ... ok
test01_basic (bsddb.test.test_recno.SimpleRecnoTestCase) ... ok
test02_WithSource (bsddb.test.test_recno.SimpleRecnoTestCase) ... ok
test03_FixedLength (bsddb.test.test_recno.SimpleRecnoTestCase) ... ok
test01_1WriterMultiReaders (bsddb.test.test_thread.BTreeConcurrentDataStore) ... ok
test01_1WriterMultiReaders (bsddb.test.test_thread.HashConcurrentDataStore) ... ok
test02_SimpleLocks (bsddb.test.test_thread.BTreeSimpleThreaded) ... ok
test02_SimpleLocks (bsddb.test.test_thread.HashSimpleThreaded) ... ok
test03_ThreadedTransactions (bsddb.test.test_thread.BTreeThreadedTransactions) ... ok
test03_ThreadedTransactions (bsddb.test.test_thread.HashThreadedTransactions) ... ok
test03_ThreadedTransactions (bsddb.test.test_thread.BTreeThreadedNoWaitTransactions) ... ok
test03_ThreadedTransactions (bsddb.test.test_thread.HashThreadedNoWaitTransactions) ... ok

======================================================================
ERROR: test01_basics (bsddb.test.test_dbshelve.EnvBTreeShelveTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\CODE\PYTHON\lib\bsddb\test\test_dbshelve.py", line 75, in test01_basics
    self.do_open()
  File "C:\CODE\PYTHON\lib\bsddb\test\test_dbshelve.py", line 238, in do_open
    self.env.open(homeDir, self.envflags | db.DB_INIT_MPOOL | db.DB_CREATE)
DBAgainError: (11, 'Resource temporarily unavailable -- unable to join the environment')

======================================================================
ERROR: test01_basics (bsddb.test.test_dbshelve.EnvHashShelveTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\CODE\PYTHON\lib\bsddb\test\test_dbshelve.py", line 75, in test01_basics
    self.do_open()
  File "C:\CODE\PYTHON\lib\bsddb\test\test_dbshelve.py", line 238, in do_open
    self.env.open(homeDir, self.envflags | db.DB_INIT_MPOOL | db.DB_CREATE)
DBAgainError: (11, 'Resource temporarily unavailable -- unable to join the environment')

======================================================================
ERROR: test01_basics (bsddb.test.test_dbshelve.EnvThreadBTreeShelveTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\CODE\PYTHON\lib\bsddb\test\test_dbshelve.py", line 75, in test01_basics
    self.do_open()
  File "C:\CODE\PYTHON\lib\bsddb\test\test_dbshelve.py", line 238, in do_open
    self.env.open(homeDir, self.envflags | db.DB_INIT_MPOOL | db.DB_CREATE)
DBAgainError: (11, 'Resource temporarily unavailable -- unable to join the environment')

======================================================================
ERROR: test01_basics (bsddb.test.test_dbshelve.EnvThreadHashShelveTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\CODE\PYTHON\lib\bsddb\test\test_dbshelve.py", line 75, in test01_basics
    self.do_open()
  File "C:\CODE\PYTHON\lib\bsddb\test\test_dbshelve.py", line 238, in do_open
    self.env.open(homeDir, self.envflags | db.DB_INIT_MPOOL | db.DB_CREATE)
DBAgainError: (11, 'Resource temporarily unavailable -- unable to join the environment')

----------------------------------------------------------------------
Ran 216 tests in 219.100s

FAILED (errors=4)
test test_bsddb3 failed -- errors occurred; run in verbose mode for details
1 test failed:
    test_bsddb3

C:\Code\python\PCbuild>tcap/u
From martin at v.loewis.de  Sun Sep 21 13:54:18 2003
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Sep 21 13:54:31 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 
	2.197, 2.198
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEELMGAAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCEELMGAAB.tim.one@comcast.net>
Message-ID: <3F6DE5CA.7080108@v.loewis.de>

Tim Peters wrote:
> I don't know why Martin favors wchar_t when possible.  The answer to that
> isn't clear.

If the wchar_t is "usable", some routines (notably PU_AsWideChar) are 
slightly more efficient, so I'd like to make wchar_t "usable" as much
as possible.

Regards,
Martin


From aleaxit at yahoo.com  Sun Sep 21 14:42:45 2003
From: aleaxit at yahoo.com (Alex Martelli)
Date: Sun Sep 21 14:42:51 2003
Subject: [Python-Dev] Discussion on adding rsplit() for strings and
	unicode objects.
In-Reply-To: <200309190612.15965.fincher.8@osu.edu>
References: <20030919083346.GA11541@tummy.com>
	<200309190612.15965.fincher.8@osu.edu>
Message-ID: <200309212042.45767.aleaxit@yahoo.com>

On Friday 19 September 2003 12:12 pm, Jeremy Fincher wrote:
> On Friday 19 September 2003 04:33 am, Sean Reifschneider wrote:
> > A few weeks ago I submitted a patch to add an rsplit() method to
> > string and unicode objects:
> > >Adding rsplit() to string and unicode objects. (2003-09-06)
> > >	http://python.org/sf/801847
>
> I'm all for it.  I've had to implement rsplit on my own as well, and always

Seconded (or thirded, whatever).  Actually had to do it just 3 days ago --
having a need for rsplit I in fact *wrote* my code as if it existed (didn't 
remember it wasn't there!!!), and it took a while to run tests and get the 
AttributeError, slap my forehead, and add a "def rsplit" function.  I think
it should be there by "principle of least surprise" -- experienced Python
coders *expect* it to be there (and I offer myself as an example;-).

> laden with.  As long as they're named appropriately, I can't see the harm
> in adding useful methods, especially one such as rsplit that completes a
> set of front-to-back/back-to-front methods.

Absolutely, but limited to those methods that "should" be there, for
completeness, and because they're expected.  rsplit is the outstanding
one that I can think of right now (because I just happened to need it
so recently, of course, and the coincidence of it being discussed now).


Alex


From barry at python.org  Sun Sep 21 15:24:02 2003
From: barry at python.org (Barry Warsaw)
Date: Sun Sep 21 15:24:08 2003
Subject: [Python-Dev] Enhancement to pdb in gud.el
In-Reply-To: <16233.62352.331254.344092@nick.uklinux.net>
References: <16233.62352.331254.344092@nick.uklinux.net>
Message-ID: <1064172242.4618.7.camel@anthem>

On Thu, 2003-09-18 at 14:04, Nick Roberts wrote:

> I've written a mode (gdb-ui.el) which is in the Emacs CVS repository
> (http://savannah.gnu.org/projects/emacs) that displays breakpoint icons in the
> display margin of Emacs when debugging with GDB. I can almost do the same
> thing for pdb (the Emacs interface to the python debugger in gud.el).

> Anyone interested?

Yes, but you might want to post to python-mode@python.org which is now a
full-fledged mailing list
<http://mail.python.org/mailman/listinfo/python-mode> and is where all
the python-moders tend to hang out.

-Barry



From tim.one at comcast.net  Sun Sep 21 19:02:24 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Sep 21 19:02:29 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <3F6DE5CA.7080108@v.loewis.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEPKGAAB.tim.one@comcast.net>

[Tim]
>> I don't know why Martin favors wchar_t when possible.  The answer to
>> that isn't clear.

[Martin v. L?wis]
> If the wchar_t is "usable", some routines (notably PU_AsWideChar) are
> slightly more efficient, so I'd like to make wchar_t "usable" as much
> as possible.

OK.  So is there an end to this thread <0.9 wink>?  At the moment, it
appears there's no identified reason to care about signedness of a
greater-than 16-bit type, good reason to insist that a 16-bit type is
unsigned, and that it's desirable for HAVE_USABLE_WCHAR_T to get defined
when possible.  What more does it take to bury this?  If it's Unixish config
chagnes, they won't be coming from me (the Windows build uses an unsigned
16-bit wchar_t).


From tim at zope.com  Sun Sep 21 19:50:42 2003
From: tim at zope.com (Tim Peters)
Date: Sun Sep 21 19:51:19 2003
Subject: [Python-Dev] Fun with 2.3 shutdown
In-Reply-To: <20030919120100.GB23317@vicky.ecs.soton.ac.uk>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEPOGAAB.tim@zope.com>

[Armin Rigo]
> The behavior of Python regarding whether threads should continue to
> run after the main one exited has never been too clear.  Wouldn't it
> make sense to try to control this aspect more precisely ?

threading.py already tried to.  Whether or not a native platform thread
outlives the main thread varies across platforms.  A Thread created via
threading.py prevents the main thread from exiting on all platforms so long
as the Thread is running, unless the user calls its setDaemon(True) method
before starting the thread.  Whether a daemon Thread can outlive the main
thread remains platform-dependent.  These platform dependencies exist
because there's no cross-platform way for Python to forcibly end a thread's
life from outside the thread, or to force a thread to continue running after
the main thread exits.  Python *can* wait for a Thread to end on its own in
a cross-platform way, and threading.py keeps the main thread alive until all
non-daemon Threads have stopped running via that portable method.  So it
does what it reasonably *could* do.

> I have also experienced this kind of shutdown glitches.

You mean the uselessly empty "Exception in thread" messages at shutdown?
I've only seen those when running Zope3 tests so far (I count *non*-empty
"Exception in thread" shutdown msgs as a different glitch, though).  I
expect every long-time Python programmer has seen the ever-popular "object
'None' has no attribute so-and-so" kinds of shutdown glitches, and we fix
more of those every release by purging __del__ methods, and methods invoked
by __del__ methods, of references to module globals.  The segfaults at
shutdown I saw later while trying to debug the Zope3 symptom were new to me.

> It definitely needs more thinking, but what about this one: Python
> would only shutdown after all threads are finished;

In the Zope3 case, there are about a dozen Threads still running, and
they'll never finish.  Nine are in an Event.wait() that will never trigger;
the rest are in an unbounded polling loop with a sleep(3) each time around.
They're all daemon Threads, though, so asked Python explicitly not to wait
for them to finish.

> but exiting the main thread would by default send a SystemExit
> exception to all current threads, if that can be done cleanly (oh
> well, this problem again).

I'm afraid that would be a major change in shutdown semantics for non-daemon
Threads, yes?  Right now the main thread will happily wait as long as it
takes for Threads to decide to stop on their own, and I've seen programs
rely on that (fire up a bunch of producer and consumer Threads, then let the
main thread fall off the end of the script).  It might help the Zope3
problem, though, if SystemExit got raised in daemon Threads when the main
thread wanted to exit.  But stuffing a pending exception on a Thread's todo
list is asynchronous, and in the case of the Zope3 Threads sitting in their
sleep(3) polling loops, the SystemExit won't be noticed until the sleep
expires.  Since 3 might actually be bigger than 3 in some apps <wink>,
there's really no limit on how long the main thread might have to wait to be
sure all the daemon Threads noticed their SystemExit.

> Another question, what was the rationale for setting module globals
> to None instead of simply deleting them (and getting the expected
> AttributeErrors both at C and at Python level) ?

I suspect it's because there's no "efficient" way to iterate over a dict and
simultaneously mutate it in a size-changing way; the C PyDict_Next()
protocol allows the caller to change the value associated with existing keys
(like setting them to None), but is unpredictable if the caller adds or
deletes keys during PyDict_Next() iteration.

Well, due to quirks of the current implementation, I suspect it actually
could delete all the keys without harm (the relevant quirk is that the
current implementation never resizes a dict in response to a key deletion;
it can resize only when a new key gets added; resizing can shuffle the keys
around in a new order, which is what makes PyDict_Next unpredictable if the
dict does get resized during iteration).


From jeremy at alum.mit.edu  Sun Sep 21 22:45:20 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Sun Sep 21 22:45:40 2003
Subject: [Python-Dev] latest bsddb3 test problems
Message-ID: <1064198720.2074.54.camel@localhost.localdomain>

With current cvs on the release23-maint branch, I get a bunch of
tracebacks printed to the console without having the test fail.
I haven't looked at the test at all, so I don't know what any of these
messages mean.  It seems, though, that the test should fail if a thread
fails with an exception.

Jeremy

test_bsddb3
Exception in thread reader 4:
Traceback (most recent call last):
  File "/home/jeremy/src/python23/Lib/threading.py", line 436, in
__bootstrap
    self.run()
  File "/home/jeremy/src/python23/Lib/threading.py", line 416, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/jeremy/src/python23/Lib/bsddb/test/test_thread.py", line
275, in readerThread
    rec = dbutils.DeadlockWrap(c.next, max_retries=10)
  File "/home/jeremy/src/python23/Lib/bsddb/dbutils.py", line 67, in
DeadlockWrap
    return function(*_args, **_kwargs)
DBLockDeadlockError: (-30996, 'DB_LOCK_DEADLOCK: Locker killed to
resolve a deadlock')
 
Exception in thread reader 3:
Traceback (most recent call last):
  File "/home/jeremy/src/python23/Lib/threading.py", line 436, in
__bootstrap
    self.run()
  File "/home/jeremy/src/python23/Lib/threading.py", line 416, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/jeremy/src/python23/Lib/bsddb/test/test_thread.py", line
275, in readerThread
    rec = dbutils.DeadlockWrap(c.next, max_retries=10)
  File "/home/jeremy/src/python23/Lib/bsddb/dbutils.py", line 67, in
DeadlockWrap
    return function(*_args, **_kwargs)
DBLockDeadlockError: (-30996, 'DB_LOCK_DEADLOCK: Locker killed to
resolve a deadlock')
 
Exception in thread reader 1:
Traceback (most recent call last):
  File "/home/jeremy/src/python23/Lib/threading.py", line 436, in
__bootstrap
    self.run()
  File "/home/jeremy/src/python23/Lib/threading.py", line 416, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/jeremy/src/python23/Lib/bsddb/test/test_thread.py", line
275, in readerThread
    rec = dbutils.DeadlockWrap(c.next, max_retries=10)
  File "/home/jeremy/src/python23/Lib/bsddb/dbutils.py", line 67, in
DeadlockWrap
    return function(*_args, **_kwargs)
DBLockDeadlockError: (-30996, 'DB_LOCK_DEADLOCK: Locker killed to
resolve a deadlock')
 
Exception in thread writer 1:
Traceback (most recent call last):
  File "/home/jeremy/src/python23/Lib/threading.py", line 436, in
__bootstrap
    self.run()
  File "/home/jeremy/src/python23/Lib/threading.py", line 416, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/jeremy/src/python23/Lib/bsddb/test/test_thread.py", line
239, in writerThread
    self.assertEqual(data, self.makeData(key))
  File "/home/jeremy/src/python23/Lib/unittest.py", line 302, in
failUnlessEqual    raise self.failureException, \
AssertionError: None != '1063-1063-1063-1063-1063'
 
Exception in thread writer 2:
Traceback (most recent call last):
  File "/home/jeremy/src/python23/Lib/threading.py", line 436, in
__bootstrap
    self.run()
  File "/home/jeremy/src/python23/Lib/threading.py", line 416, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/jeremy/src/python23/Lib/bsddb/test/test_thread.py", line
254, in writerThread
    self.assertEqual(data, self.makeData(key))
  File "/home/jeremy/src/python23/Lib/unittest.py", line 302, in
failUnlessEqual    raise self.failureException, \
AssertionError: None != '2000-2000-2000-2000-2000'
 
Exception in thread writer 0:
Traceback (most recent call last):
  File "/home/jeremy/src/python23/Lib/threading.py", line 436, in
__bootstrap
    self.run()
  File "/home/jeremy/src/python23/Lib/threading.py", line 416, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/jeremy/src/python23/Lib/bsddb/test/test_thread.py", line
254, in writerThread
    self.assertEqual(data, self.makeData(key))
  File "/home/jeremy/src/python23/Lib/unittest.py", line 302, in
failUnlessEqual    raise self.failureException, \
AssertionError: None != '0002-0002-0002-0002-0002'



From tim.one at comcast.net  Sun Sep 21 23:21:04 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Sep 21 23:21:10 2003
Subject: [Python-Dev] latest bsddb3 test problems
In-Reply-To: <1064198720.2074.54.camel@localhost.localdomain>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEAMGBAB.tim.one@comcast.net>

[python-dev-bounces@python.org]
> With current cvs on the release23-maint branch, I get a bunch of
> tracebacks printed to the console without having the test fail.
> I haven't looked at the test at all, so I don't know what any of these
> messages mean.  It seems, though, that the test should fail if a
> thread fails with an exception.

I've been told that:

  DBLockDeadlockError: (-30996, 'DB_LOCK_DEADLOCK: Locker killed to
  resolve a deadlock')

isn't an error, it's just something that's going to happen from time to
time.  You'll find DeadlockWrap in the traceback then, and the tests
typically do something like

            dbutils.DeadlockWrap(d.put, key, self.makeData(key),
                                 max_retries=12)


But the DeadlockWrap implementation suppresses DBLockDeadlockError until
max_retries attempts have been made, so I don't think the tests *expect*
ever to see this get raised.  It feels like the moral equivalent of our
ConcurrentUpdates ZEO tests that way.

WRT the rest, we've seen errors like

> AssertionError: None != '1063-1063-1063-1063-1063'
> AssertionError: None != '2000-2000-2000-2000-2000'
> AssertionError: None != '0002-0002-0002-0002-0002'

coming out of Lib/bsddb/test/test_thread.py before, on Windows and on Linux,
but rarely.  IIRC, no progress was made in tracking these down, and Barry
and I were dissuaded from pursuing it last time around.

These tests have multiple threads writing to a database, and the exceptions
are due to not finding the data in the database the threads thought they
wrote.  In that respect it's also reminiscent of earlier ConcurrentUpdates
ZEO tests.  However, these are clearly real test failures, and it's not good
that unittest thinks these tests pass despite that the threads barf.


From anthony at interlink.com.au  Sun Sep 21 23:40:51 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sun Sep 21 23:42:42 2003
Subject: [Python-Dev] latest bsddb3 test problems 
In-Reply-To: <1064198720.2074.54.camel@localhost.localdomain> 
Message-ID: <200309220340.h8M3eqju022753@localhost.localdomain>


>>> Jeremy Hylton wrote
> With current cvs on the release23-maint branch, I get a bunch of
> tracebacks printed to the console without having the test fail.
> I haven't looked at the test at all, so I don't know what any of these
> messages mean.  It seems, though, that the test should fail if a thread
> fails with an exception.

Argh. What platform/compiler/bsddb version?


-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From jeremy at alum.mit.edu  Mon Sep 22 00:05:17 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon Sep 22 00:05:38 2003
Subject: [Python-Dev] latest bsddb3 test problems
In-Reply-To: <200309220340.h8M3eqju022753@localhost.localdomain>
References: <200309220340.h8M3eqju022753@localhost.localdomain>
Message-ID: <1064203516.2074.102.camel@localhost.localdomain>

On Sun, 2003-09-21 at 23:40, Anthony Baxter wrote:
> >>> Jeremy Hylton wrote
> > With current cvs on the release23-maint branch, I get a bunch of
> > tracebacks printed to the console without having the test fail.
> > I haven't looked at the test at all, so I don't know what any of these
> > messages mean.  It seems, though, that the test should fail if a thread
> > fails with an exception.
> 
> Argh. What platform/compiler/bsddb version?

Python 2.3+ (#3, Sep 21 2003, 13:23:43)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import bsddb
>>> bsddb._bsddb.version()
(4, 1, 25)

RH 9.

Jeremy



From 2tiapuzxx at msn.com  Mon Sep 22 10:18:56 2003
From: 2tiapuzxx at msn.com (Phil Travis)
Date: Mon Sep 22 00:17:30 2003
Subject: [Python-Dev] Pheromone sex attractant f qnzncjejaj nji
Message-ID: <w4-uc$29zlp$n-3@bfry4o92>

An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20030922/7c413099/attachment-0001.html
From python at rcn.com  Mon Sep 22 00:52:42 2003
From: python at rcn.com (Raymond Hettinger)
Date: Mon Sep 22 00:53:01 2003
Subject: [Python-Dev] latest bsddb3 test problems
References: <1064198720.2074.54.camel@localhost.localdomain>
Message-ID: <00d201c380c5$5b6a1f60$0bb9958d@oemcomputer>

[Jeremy]
> With current cvs on the release23-maint branch, I get a bunch of
> tracebacks printed to the console without having the test fail.
> I haven't looked at the test at all, so I don't know what any of these
> messages mean.  It seems, though, that the test should fail if a thread
> fails with an exception.

I do get a test failure with the following diagnostic:


======================================================================
ERROR: test01_basics (bsddb.test.test_dbshelve.EnvThreadHashShelveTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\PY23\lib\bsddb\test\test_dbshelve.py", line 75, in test01_basics
    self.do_open()
  File "C:\PY23\lib\bsddb\test\test_dbshelve.py", line 238, in do_open
    self.env.open(homeDir, self.envflags | db.DB_INIT_MPOOL | db.DB_CREATE)
DBAgainError: (11, 'Resource temporarily unavailable -- unable to join the environment')



Raymond Hettinger

#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

From anthony at interlink.com.au  Mon Sep 22 03:04:23 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Sep 22 03:06:06 2003
Subject: [Python-Dev] latest bsddb3 test problems 
In-Reply-To: <1064198720.2074.54.camel@localhost.localdomain> 
Message-ID: <200309220704.h8M74NNk026354@localhost.localdomain>


I'm seeing this fail as well, on Redhat 9. Backing out Greg's changes over
the weekend doesn't help... the tests still mark as "passed", even though
there's the fairly horrible error/warnings. I'm guessing something that 
raises an error in a non-main thread doesn't get picked up by regrtest.py

test_bsddb3 is failing on pretty much every box I've tried it on here - 
Solaris and Redhat linux, most running the latest-n-greatest release of
DB4.1 (we use it pretty heavily in a variety of internal applications).

Solaris 8 (DB 4.1.25. gcc 2.96):
  DBAgainError: (11, 'Resource temporarily unavailable -- mmap: Resource temporarily unavailable')

Solaris 7 (DB 4.1.25, gcc 2.95.2)
  Crashes with a SIGBUS. The version of _bsddb.c from a week ago has a lot of
  errors, but doesn't crash.

Redhat 10beta/Rawhide (DB 4.1.25)
  DBNoMemoryError: (12, 'Cannot allocate memory -- Lock table is out of available locks')
  alternating with SIGBUS

Redhat 7.1,7.2,7.3 (DB 3.2.9)
  Crashes with a SIGBUS.


I'm going to check with Spambayes as an actual application and see whether 
the tests that are failing are at fault, or if we've got something that's
just busted. Can anyone report the bsddb3 tests as passing on their system?
Note that they're not run by default.

I'm at a loss as to whether to delay the release or not... 

Anthony


-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From greg at electricrain.com  Mon Sep 22 03:50:32 2003
From: greg at electricrain.com (Gregory P. Smith)
Date: Mon Sep 22 03:50:36 2003
Subject: [Python-Dev] latest bsddb3 test problems
In-Reply-To: <200309220704.h8M74NNk026354@localhost.localdomain>
References: <1064198720.2074.54.camel@localhost.localdomain>
	<200309220704.h8M74NNk026354@localhost.localdomain>
Message-ID: <20030922075032.GC16574@zot.electricrain.com>

> I'm seeing this fail as well, on Redhat 9. Backing out Greg's changes over
> the weekend doesn't help... the tests still mark as "passed", even though
> there's the fairly horrible error/warnings. I'm guessing something that 
> raises an error in a non-main thread doesn't get picked up by regrtest.py
> 
> test_bsddb3 is failing on pretty much every box I've tried it on here - 
> Solaris and Redhat linux, most running the latest-n-greatest release of
> DB4.1 (we use it pretty heavily in a variety of internal applications).
> 
> Solaris 8 (DB 4.1.25. gcc 2.96):
>   DBAgainError: (11, 'Resource temporarily unavailable -- mmap: Resource temporarily unavailable')
> 
> Solaris 7 (DB 4.1.25, gcc 2.95.2)
>   Crashes with a SIGBUS. The version of _bsddb.c from a week ago has a lot of
>   errors, but doesn't crash.

That's sounds like inconsistent behaviour.  Look at the diffs.  All of
the changes since the python 2.3 release to _bsddb.c have been trivial.
The _bsddb.c file version is not the culprit.

> Redhat 10beta/Rawhide (DB 4.1.25)
>   DBNoMemoryError: (12, 'Cannot allocate memory -- Lock table is out of available locks')
>   alternating with SIGBUS
> 
> Redhat 7.1,7.2,7.3 (DB 3.2.9)
>   Crashes with a SIGBUS.
> 
> 
> I'm going to check with Spambayes as an actual application and see whether 
> the tests that are failing are at fault, or if we've got something that's
> just busted. Can anyone report the bsddb3 tests as passing on their system?
> Note that they're not run by default.

It is passing on the systems that i've tested it on in the past few days:
(Using BerkeleyDB 4.0, 4.1 and a prerelease of 4.2).

  Linux/i686 Gentoo 1.4
  Linux/alpha Debian "testing"

I do see the occasional DBDeadLockError which can be ignored.  (Yes,
I agree, the test suite needs work to get rid of things like that.
if its not going to consider it an error it should at least catch with
a comment stating why to avoid the confusion)


NOTE:  A weakness of the bsddb tests that I have witnessed causing problems:

It often doesn't wipe its temporary directory before the tests (in theory
it should clean up on shutdown, but thats only if all goes well in the
tearDown methods).  If a previous run was aborted abnormally and left
a database environment & db around it can cause problems during future
test runs if the testsuite trys to use that environment and db.


> I'm at a loss as to whether to delay the release or not... 
> 
> Anthony

Do the same tests fail similarly when run on python 2.3?  I'd imagine so.
IMHO, I wouldn't hold python 2.3.1 up due to bsddb test suite issues.

-greg


From anthony at interlink.com.au  Mon Sep 22 04:11:31 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Sep 22 04:13:29 2003
Subject: [Python-Dev] latest bsddb3 test problems 
In-Reply-To: <20030922075032.GC16574@zot.electricrain.com> 
Message-ID: <200309220811.h8M8BVEN027031@localhost.localdomain>


>>> "Gregory P. Smith" wrote
> That's sounds like inconsistent behaviour.  Look at the diffs.  All of
> the changes since the python 2.3 release to _bsddb.c have been trivial.
> The _bsddb.c file version is not the culprit.

Yeah. Trying it repeatedly shows that it crashes, sometimes, on the old or
new code. 

> It is passing on the systems that i've tested it on in the past few days:
> (Using BerkeleyDB 4.0, 4.1 and a prerelease of 4.2).
> 
>   Linux/i686 Gentoo 1.4
>   Linux/alpha Debian "testing"

The only debian box I can get to easily is the sourceforge compile farm one,
and some charming individual appears to have fork-bombed it into uselessness.

> I do see the occasional DBDeadLockError which can be ignored.  (Yes,
> I agree, the test suite needs work to get rid of things like that.
> if its not going to consider it an error it should at least catch with
> a comment stating why to avoid the confusion)

That would be good. Note that I'm not just seeing DBDeadlockErrors, but a 
whole swag of different errors on the different machines.

> NOTE:  A weakness of the bsddb tests that I have witnessed causing problems:
> 
> It often doesn't wipe its temporary directory before the tests (in theory
> it should clean up on shutdown, but thats only if all goes well in the
> tearDown methods).  If a previous run was aborted abnormally and left
> a database environment & db around it can cause problems during future
> test runs if the testsuite trys to use that environment and db.

That shouldn't be a problem here, as I'm running the build and test processes
from different directories for each system.


> Do the same tests fail similarly when run on python 2.3?  I'd imagine so.
> IMHO, I wouldn't hold python 2.3.1 up due to bsddb test suite issues.

Going to try that next.


-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From mal at lemburg.com  Mon Sep 22 05:21:48 2003
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon Sep 22 05:21:52 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 
	2.197, 2.198
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEPKGAAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCMEPKGAAB.tim.one@comcast.net>
Message-ID: <3F6EBF2C.3040108@lemburg.com>

Tim Peters wrote:
> [Tim]
> 
>>>I don't know why Martin favors wchar_t when possible.  The answer to
>>>that isn't clear.
> 
> [Martin v. L?wis]
> 
>>If the wchar_t is "usable", some routines (notably PU_AsWideChar) are
>>slightly more efficient, so I'd like to make wchar_t "usable" as much
>>as possible.
> 
> OK.  So is there an end to this thread <0.9 wink>?  At the moment, it
> appears there's no identified reason to care about signedness of a
> greater-than 16-bit type, 

Sure there is: first of all, having a single type that can
be signed on some platforms and unsigned on others is a bad
thing per se and second the 32-bit signed wchar_t value was
what triggered this thread in the first place.

> good reason to insist that a 16-bit type is
> unsigned, and that it's desirable for HAVE_USABLE_WCHAR_T to get defined
> when possible.  What more does it take to bury this?  If it's Unixish config
> chagnes, they won't be coming from me (the Windows build uses an unsigned
> 16-bit wchar_t).

That's what it takes, right. I'll work on it.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Sep 22 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::


From mwh at python.net  Mon Sep 22 06:18:21 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Sep 22 06:17:45 2003
Subject: [Python-Dev] Py2.3.1
In-Reply-To: <200309201616.h8KGGaf5021370@localhost.localdomain> (Anthony
	Baxter's message of "Sun, 21 Sep 2003 02:16:35 +1000")
References: <200309201616.h8KGGaf5021370@localhost.localdomain>
Message-ID: <2misnl6s9e.fsf@starship.python.net>

Anthony Baxter <anthony@interlink.com.au> writes:

> I'm planning on starting the dance of the seven release tarballs Tuesday 
> morning AU time, which is more or less 0:00 Tuesday UTC. Assuming that
> no-one objects to this timeframe, could people be _extremely_ conservative
> with checkins on the branch for the 24 hours before this? I'm going to 
> be starting to cut test release tarballs probably some time tomorrow and
> building it on as many platforms as I can ssh to easily - I can make this
> available to other people if they want to help test, or if they've got
> some sort of strange version of Unix (Irix and HP/UX are always good
> ones to check for platform oddities).

Um.  Has someone done the 'reading through cvs logs' ritual for 2.3.1?
It's tedious, but valuable.  In particular, I'm not *certain* all the
results of the refleak fixing spree got backported, and some of them
should probably acquire NEWS entries.  I'm not sure I want to promise
being able to do this in the next twelve hours...

(If anyone wants to review the _hotshot.c patch attached to bug
#808756, that'd be grand, too).

Cheers,
mwh

-- 
  On the other hand, the following areas are subject to boycott
  in reaction to the rampant impurity of design or execution, as
  determined after a period of study, in no particular order:
    ...                              http://www.naggum.no/profile.html

From mal at lemburg.com  Mon Sep 22 07:18:32 2003
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon Sep 22 07:18:38 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 
	2.197, 2.198
In-Reply-To: <3F6EBF2C.3040108@lemburg.com>
References: <LNBBLJKPBEHFEDALKOLCMEPKGAAB.tim.one@comcast.net>
	<3F6EBF2C.3040108@lemburg.com>
Message-ID: <3F6EDA88.7050407@lemburg.com>

M.-A. Lemburg wrote:
>> good reason to insist that a 16-bit type is
>> unsigned, and that it's desirable for HAVE_USABLE_WCHAR_T to get defined
>> when possible.  What more does it take to bury this?  If it's Unixish 
>> config chagnes, they won't be coming from me (the Windows build uses an 
 >> unsigned 16-bit wchar_t).
> 
> That's what it takes, right. I'll work on it.

While working on the config changes, I noticed that Python now
defaults to UCS4 when it find a Tcl/Tk version that supports UCS4...

I can't say that I particularly like this, since the config script
now makes an implicit choice based on the third-party software
configuration with consequences that are not made obvious for the
user.

E.g. on platforms that happen to have Tcl/tk installed
with UCS4 configuration, Python will compile using UCS4 (regardeless
of whether the user wants to use Tcl/Tk or not), on system
that don't have such Tcl/tk installation, Python compiles using
UCS2.

I'd suggest to make the UCS4 choice explicit again.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Sep 22 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::


From mcherm at mcherm.com  Mon Sep 22 07:46:52 2003
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon Sep 22 07:46:34 2003
Subject: [Python-Dev] Re: Fsrit Propttoye: SCLABMRE
Message-ID: <1064231212.3f6ee12c8b2a8@mcherm.com>

Tihs drecvoisy souhld be supespserd imelidemtay... thnik waht 
it wlil do to SymaeapBs!

-- Mcaeihl Chdemisre


From mwh at python.net  Mon Sep 22 07:50:38 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Sep 22 07:50:07 2003
Subject: [Python-Dev] pep 310 (reliable acquisition/release pairs)
In-Reply-To: <20030919174240.C31102@prim.han.de> (Holger Krekel's message of
	"Fri, 19 Sep 2003 17:42:40 +0200")
References: <20030919174240.C31102@prim.han.de>
Message-ID: <2msmmp59f5.fsf@starship.python.net>

Holger Krekel <hpk@trillke.net> writes:

> hello, 
>
> admittedly i only followed Brett's very nice summaries (thanks!) 
> in the last month so i may have missed some details. But here are a few
> comments regarding "PEP 310 Reliable Acquisition/Release Pairs". 
>
> The PEP actually is about interacting with the execution 
> of a code block.  It allows to define (one-shot) interception points 
> for entering and leaving a code block.  Now there are at least 
> two interesting cases which the PEP does (quite explicitely) not cover:
>
> - what to do with exceptions 
>
> - what to do with yield 
>
> IMHO introducing a new block statement at this stage in language 
> development warrants an effort to tackle these cases (and maybe more
> like e.g. allowing the handler to trigger looping). 
>
> This is probably best done with trying to directly design a protocol between
> the "interpreter-loop" and the - what i'd call - the "execution handler". 

Well, in writing PEP 310 (as I suspect you know) I was aiming for a
simple, almost entirely syntactic way of shortening a common pattern.
You seem to be gunning for something far deeper here.

[...]
> So while i am not strictly against the proposal i'd humbly ask for
> not hurrying into accepting the PEP as is. Python 2.4 is not closeby
> so i hope there is still some time to discuss this.

While I agree and am in no hurry to rush, I'm not sure I see that
accepting PEP 310 as is necessarily hinders your more subtle aims...

Cheers,
mwh

-- 
  SCSI is not magic. There are fundamental technical reasons why it
  is necessary to sacrifice a young goat to your SCSI chain now and
  then.                                                  -- John Woods

From jepler at unpythonic.net  Mon Sep 22 08:04:38 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Mon Sep 22 08:05:00 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <3F6EDA88.7050407@lemburg.com>
References: <LNBBLJKPBEHFEDALKOLCMEPKGAAB.tim.one@comcast.net>
	<3F6EBF2C.3040108@lemburg.com> <3F6EDA88.7050407@lemburg.com>
Message-ID: <20030922120433.GB28986@unpythonic.net>

On Mon, Sep 22, 2003 at 01:18:32PM +0200, M.-A. Lemburg wrote:
> While working on the config changes, I noticed that Python now
> defaults to UCS4 when it find a Tcl/Tk version that supports UCS4...
> 
> I can't say that I particularly like this, since the config script
> now makes an implicit choice based on the third-party software
> configuration with consequences that are not made obvious for the
> user.
> 
> E.g. on platforms that happen to have Tcl/tk installed
> with UCS4 configuration, Python will compile using UCS4 (regardeless
> of whether the user wants to use Tcl/Tk or not), on system
> that don't have such Tcl/tk installation, Python compiles using
> UCS2.
> 
> I'd suggest to make the UCS4 choice explicit again.

See the (complete lack of) discussion on the bug at
http://python.org/sf/798202

I can only say that this is a convenient patch for me, and probably for
any redhat9 user who has not built his own tcl/tk.  On the other hand,
this patch is unlikely to actually affect anybody on a different
platform.

I don't think it's such a bad idea to have Python detect the
configuration it must use to make a very important, bundled extension
work properly.  Would you feel any different if a different patch was
added, which made --enable-unicode=tcl enable the behavior of this
patch?

Jeff

From mal at lemburg.com  Mon Sep 22 09:22:17 2003
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon Sep 22 09:22:23 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 
	2.197, 2.198
In-Reply-To: <20030922120433.GB28986@unpythonic.net>
References: <LNBBLJKPBEHFEDALKOLCMEPKGAAB.tim.one@comcast.net>	<3F6EBF2C.3040108@lemburg.com>
	<3F6EDA88.7050407@lemburg.com>
	<20030922120433.GB28986@unpythonic.net>
Message-ID: <3F6EF789.1070808@lemburg.com>

Jeff Epler wrote:
> On Mon, Sep 22, 2003 at 01:18:32PM +0200, M.-A. Lemburg wrote:
> 
>>While working on the config changes, I noticed that Python now
>>defaults to UCS4 when it find a Tcl/Tk version that supports UCS4...
>>
>>I can't say that I particularly like this, since the config script
>>now makes an implicit choice based on the third-party software
>>configuration with consequences that are not made obvious for the
>>user.
>>
>>E.g. on platforms that happen to have Tcl/tk installed
>>with UCS4 configuration, Python will compile using UCS4 (regardeless
>>of whether the user wants to use Tcl/Tk or not), on system
>>that don't have such Tcl/tk installation, Python compiles using
>>UCS2.
>>
>>I'd suggest to make the UCS4 choice explicit again.
> 
> See the (complete lack of) discussion on the bug at
> http://python.org/sf/798202
> 
> I can only say that this is a convenient patch for me, and probably for
> any redhat9 user who has not built his own tcl/tk.  On the other hand,
> this patch is unlikely to actually affect anybody on a different
> platform.
> 
> I don't think it's such a bad idea to have Python detect the
> configuration it must use to make a very important, bundled extension
> work properly.  Would you feel any different if a different patch was
> added, which made --enable-unicode=tcl enable the behavior of this
> patch?

Yes :-) ... "explicit is better than implicit"

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Sep 22 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::


From barry at python.org  Mon Sep 22 09:50:12 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Sep 22 09:50:45 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <20030922120433.GB28986@unpythonic.net>
References: <LNBBLJKPBEHFEDALKOLCMEPKGAAB.tim.one@comcast.net>
	<3F6EBF2C.3040108@lemburg.com> <3F6EDA88.7050407@lemburg.com>
	<20030922120433.GB28986@unpythonic.net>
Message-ID: <1064238611.1986.0.camel@anthem>

On Mon, 2003-09-22 at 08:04, Jeff Epler wrote:

> I don't think it's such a bad idea to have Python detect the
> configuration it must use to make a very important, bundled extension
> work properly.  Would you feel any different if a different patch was
> added, which made --enable-unicode=tcl enable the behavior of this
> patch?

Seems just as easy to document that on RH9 with the default tcl/tk, you
need to include --enable-unicode=ucs4.

-Barry



From anthony at interlink.com.au  Mon Sep 22 10:15:42 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Sep 22 10:17:24 2003
Subject: [Python-Dev] Py2.3.1 
In-Reply-To: <2misnl6s9e.fsf@starship.python.net> 
Message-ID: <200309221415.h8MEFgEv031731@localhost.localdomain>


>>> Michael Hudson wrote
> Um.  Has someone done the 'reading through cvs logs' ritual for 2.3.1?
> It's tedious, but valuable.  In particular, I'm not *certain* all the
> results of the refleak fixing spree got backported, and some of them
> should probably acquire NEWS entries.  I'm not sure I want to promise
> being able to do this in the next twelve hours...

I did a bit of a scan, and I think most of them are in. I'll have another
look tomorrow morning.

At this point I'd like to say "please hold off on changes to the 
release23-maint branch, unless you're involved in the release process."


-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From theller at python.net  Mon Sep 22 10:43:30 2003
From: theller at python.net (Thomas Heller)
Date: Mon Sep 22 10:43:38 2003
Subject: [Python-Dev] Py2.3.1
In-Reply-To: <200309221415.h8MEFgEv031731@localhost.localdomain> (Anthony
	Baxter's message of "Tue, 23 Sep 2003 00:15:42 +1000")
References: <200309221415.h8MEFgEv031731@localhost.localdomain>
Message-ID: <1xu8vq7h.fsf@python.net>

Anthony Baxter <anthony@interlink.com.au> writes:

>>>> Michael Hudson wrote
>> Um.  Has someone done the 'reading through cvs logs' ritual for 2.3.1?
>> It's tedious, but valuable.  In particular, I'm not *certain* all the
>> results of the refleak fixing spree got backported, and some of them
>> should probably acquire NEWS entries.  I'm not sure I want to promise
>> being able to do this in the next twelve hours...
>
> I did a bit of a scan, and I think most of them are in. I'll have another
> look tomorrow morning.
>
> At this point I'd like to say "please hold off on changes to the 
> release23-maint branch, unless you're involved in the release process."

I built a windows installer and started testing it under WinXP and
win98SE a few hours ago, there haven't been any checkins to the
release-23maint branch in the meantime except one to gopherlib.

On winXP everything worked (although test_largefile took ages to
complete on my notebook), on Win98SE (not the desktop) the testsuite
fails in test_winsound (do I need a soundcard for this?) and in
test_bsddb3.

Thomas


From skip at pobox.com  Mon Sep 22 12:19:32 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Sep 22 12:19:47 2003
Subject: [Python-Dev] 2.3.1 windows binaries
In-Reply-To: <znh0bmgs.fsf@python.net>
References: <znh0bmgs.fsf@python.net>
Message-ID: <16239.8468.552863.416218@montanaro.dyndns.org>


    Thomas> Is it important to use the same openssl version that Python
    Thomas> 2.3.0 used, or is the readme file correct in saying that the
    Thomas> latest version is the one to use? And does this have to be
    Thomas> mentioned somewhere (Misc/NEWS, the readme file for windows)?

I would use the latest version.  I believe there were security-related fixes
applied OpenSSL in the past month or so (so many security fixes have come
out recently, that I'm not sure if it's OpenSSL or OpenSSH -- or both --
which got patched).  It's probably worth a mention somewhere which versions
you used for all external packages.

Skip

From barry at barrys-emacs.org  Mon Sep 22 12:56:17 2003
From: barry at barrys-emacs.org (Barry Scott)
Date: Mon Sep 22 12:57:18 2003
Subject: [Python-Dev] Discussion on adding rsplit() for strings and
	unicode objects.
In-Reply-To: <200309190612.15965.fincher.8@osu.edu>
References: <20030919083346.GA11541@tummy.com>
	<200309190612.15965.fincher.8@osu.edu>
Message-ID: <6.0.0.22.0.20030922175257.020ecdd0@torment.chelsea.private>


>I'm all for it.  I've had to implement rsplit on my own as well, and always
>lamented (but never did anything, unfortunately) about its lack in the
>standard library.  That strings have rfind, rindex, rstrip, and rjust, but
>not rsplit always seemed somewhat less than complete to me.

These "r" are not a single family... some mean reverse some mean right.

BArry




From tim.one at comcast.net  Mon Sep 22 13:25:33 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Sep 22 13:26:08 2003
Subject: [Python-Dev] Py2.3.1
In-Reply-To: <1xu8vq7h.fsf@python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEFMGBAB.tim.one@comcast.net>

[Thomas Heller]
> I built a windows installer and started testing it under WinXP and
> win98SE a few hours ago, there haven't been any checkins to the
> release-23maint branch in the meantime except one to gopherlib.
>
> On winXP everything worked (although test_largefile took ages to
> complete on my notebook), on Win98SE (not the desktop) the testsuite
> fails in test_winsound (do I need a soundcard for this?) and in
> test_bsddb3.

See my msg yesterday:  test_bsddb3 always fails on Win98SE, and that's
nothing new (it failed the same ways under 2.3).

test_winsound recently started failing on my Win98SE box too.  Don't know
why, and ran out of time to dig into it yesterday.  I seem to recall it
failing the same way for Guido a long time ago.  Maybe some MS service pack
or update changed endcase behavior -- don't know.


From hpk at trillke.net  Mon Sep 22 13:41:37 2003
From: hpk at trillke.net (Holger Krekel)
Date: Mon Sep 22 13:41:45 2003
Subject: [Python-Dev] pep 310 (reliable acquisition/release pairs)
In-Reply-To: <2msmmp59f5.fsf@starship.python.net>;
	from mwh@python.net on Mon, Sep 22, 2003 at 12:50:38PM +0100
References: <20030919174240.C31102@prim.han.de>
	<2msmmp59f5.fsf@starship.python.net>
Message-ID: <20030922194137.C24546@prim.han.de>

Michael Hudson wrote:
> Holger Krekel <hpk@trillke.net> writes:
> > The PEP actually is about interacting with the execution 
> > of a code block.  It allows to define (one-shot) interception points 
> > for entering and leaving a code block.  Now there are at least 
> > two interesting cases which the PEP does (quite explicitely) not cover:
> >
> > - what to do with exceptions 
> >
> > - what to do with yield 
> >
> > IMHO introducing a new block statement at this stage in language 
> > development warrants an effort to tackle these cases (and maybe more
> > like e.g. allowing the handler to trigger looping). 
> >
> > This is probably best done with trying to directly design a protocol between
> > the "interpreter-loop" and the - what i'd call - the "execution handler". 
> 
> Well, in writing PEP 310 (as I suspect you know) I was aiming for a
> simple, almost entirely syntactic way of shortening a common pattern.
> You seem to be gunning for something far deeper here.

Hmmm, introducing a new block statement just for "shortening a common
pattern" doesn't sound like a sufficient reason to me.  OTOH i'd try to argue
that in order to add reliable resource synchronization an effort to get 
it working in the context of exceptions and generators would add more 
than just a macro :-) 

Also thinking more along the lines of Armin's alternative idea
(incorporate it in into the for-loop protocol) is interesting as
it avoids adding a new keyword which seems of great value to me. 

on a side note, it would be great if we could soon (after integration of the 
parser/compiler) branch off PyPy to do quick experiments with different
approaches.  IIRC Guido mentioned at EuroPython that this might
be an interesting use of PyPy (to easily test new language features). 
This way it's also easier for PyPy to keep up to date :-)

cheers,

    holger

From pedronis at bluewin.ch  Mon Sep 22 14:19:16 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Mon Sep 22 14:17:34 2003
Subject: [Python-Dev] pep 310 (reliable acquisition/release pairs)
In-Reply-To: <20030922194137.C24546@prim.han.de>
References: <2msmmp59f5.fsf@starship.python.net>
	<20030919174240.C31102@prim.han.de>
	<2msmmp59f5.fsf@starship.python.net>
Message-ID: <5.2.1.1.0.20030922200953.027ef7f8@pop.bluewin.ch>

At 19:41 22.09.2003 +0200, Holger Krekel wrote:
>Michael Hudson wrote:
> > Holger Krekel <hpk@trillke.net> writes:
> > > The PEP actually is about interacting with the execution
> > > of a code block.  It allows to define (one-shot) interception points
> > > for entering and leaving a code block.  Now there are at least
> > > two interesting cases which the PEP does (quite explicitely) not cover:
> > >
> > > - what to do with exceptions
> > >
> > > - what to do with yield
> > >
> > > IMHO introducing a new block statement at this stage in language
> > > development warrants an effort to tackle these cases (and maybe more
> > > like e.g. allowing the handler to trigger looping).
> > >
> > > This is probably best done with trying to directly design a protocol 
> between
> > > the "interpreter-loop" and the - what i'd call - the "execution 
> handler".
> >
> > Well, in writing PEP 310 (as I suspect you know) I was aiming for a
> > simple, almost entirely syntactic way of shortening a common pattern.
> > You seem to be gunning for something far deeper here.
>
>Hmmm, introducing a new block statement just for "shortening a common
>pattern" doesn't sound like a sufficient reason to me.  OTOH i'd try to argue
>that in order to add reliable resource synchronization an effort to get
>it working in the context of exceptions and generators would add more
>than just a macro :-)
>
>Also thinking more along the lines of Armin's alternative idea
>(incorporate it in into the for-loop protocol) is interesting as
>it avoids adding a new keyword which seems of great value to me.

can we avoid to have now the discussion that we don't wanted now.


>on a side note, it would be great if we could soon (after integration of the
>parser/compiler) branch off PyPy to do quick experiments with different
>approaches.  IIRC Guido mentioned at EuroPython that this might
>be an interesting use of PyPy (to easily test new language features).
>This way it's also easier for PyPy to keep up to date :-)

I won't say anything, see above, BUT ... 


From barry at python.org  Mon Sep 22 14:18:00 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Sep 22 14:18:07 2003
Subject: [Python-Dev] Re: latest bsddb3 test problems
In-Reply-To: <200309220704.h8M74NNk026354@localhost.localdomain>
References: <1064198720.2074.54.camel@localhost.localdomain>
	<200309220704.h8M74NNk026354@localhost.localdomain>
Message-ID: <3F6F3CD8.6010408@python.org>

Anthony Baxter wrote:
> 
> test_bsddb3 is failing on pretty much every box I've tried it on here - 
> Solaris and Redhat linux, most running the latest-n-greatest release of
> DB4.1 (we use it pretty heavily in a variety of internal applications).
> 
> Solaris 8 (DB 4.1.25. gcc 2.96):
>   DBAgainError: (11, 'Resource temporarily unavailable -- mmap: Resource temporarily unavailable')
> 
> Solaris 7 (DB 4.1.25, gcc 2.95.2)
>   Crashes with a SIGBUS. The version of _bsddb.c from a week ago has a lot of
>   errors, but doesn't crash.
> 
> Redhat 10beta/Rawhide (DB 4.1.25)
>   DBNoMemoryError: (12, 'Cannot allocate memory -- Lock table is out of available locks')
>   alternating with SIGBUS
> 
> Redhat 7.1,7.2,7.3 (DB 3.2.9)
>   Crashes with a SIGBUS.

FWIW, I just tested it on RH9, DB 4.1.25, release23-maint branch. 
Worked goodly for me.

-Barry


% ./python Lib/test/test_bsddb3.py
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Sleepycat Software: Berkeley DB 4.1.25: (December 19, 2002)
bsddb.db.version():   (4, 1, 25)
bsddb.db.__version__: 4.2.0
bsddb.db.cvsid:       $Id: _bsddb.c,v 1.17.6.2 2003/09/21 23:10:23 greg 
Exp $
python version:        2.3+ (#1, Sep 22 2003, 13:35:52)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)]
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
........................................................................................................................................................................................................................
----------------------------------------------------------------------
Ran 216 tests in 131.194s

OK


From tim.one at comcast.net  Mon Sep 22 14:25:26 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Sep 22 14:25:41 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 2.197, 2.198
In-Reply-To: <3F6EBF2C.3040108@lemburg.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEGBGBAB.tim.one@comcast.net>

[Tim]
>> At the moment, it appears there's no identified reason to care about
>> signedness of a greater-than 16-bit type,

[M.-A. Lemburg]
> Sure there is: first of all, having a single type that can
> be signed on some platforms and unsigned on others is a bad
> thing per se

We inherit that from C, though -- it's fine by C if wchar_t is signed or
unsigned, just as it refused to define the signedness of char.

> and second the 32-bit signed wchar_t value was what triggered this
> thread in the first place.

What triggered the thread originally was a segfault due to the code making a
branch based on the content of uninitialized memory.  The code clearly
didn't *think* it was reading up random heap bits, so that was a bug
regardless of wchar_t's signedness.  That wchar_t happened to be a signed
32-bit type on Jeremy's box is what uncovered the read-uninitialized-memory
bug.

If there's no other code vulnerable to bad behavior if wchar_t is a signed
32-bit type (nobody has identified another case), objections to it being
signed anyway seem technically groundless.  Martin did give a technical
reason (efficiency) for wanting to continue to use wchar_t on Jeremy's
system.


From barry at python.org  Mon Sep 22 14:18:00 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Sep 22 14:50:54 2003
Subject: [Python-Dev] Re: latest bsddb3 test problems
In-Reply-To: <200309220704.h8M74NNk026354@localhost.localdomain>
References: <1064198720.2074.54.camel@localhost.localdomain>
	<200309220704.h8M74NNk026354@localhost.localdomain>
Message-ID: <3F6F3CD8.6010408@python.org>

Anthony Baxter wrote:
> 
> test_bsddb3 is failing on pretty much every box I've tried it on here - 
> Solaris and Redhat linux, most running the latest-n-greatest release of
> DB4.1 (we use it pretty heavily in a variety of internal applications).
> 
> Solaris 8 (DB 4.1.25. gcc 2.96):
>   DBAgainError: (11, 'Resource temporarily unavailable -- mmap: Resource temporarily unavailable')
> 
> Solaris 7 (DB 4.1.25, gcc 2.95.2)
>   Crashes with a SIGBUS. The version of _bsddb.c from a week ago has a lot of
>   errors, but doesn't crash.
> 
> Redhat 10beta/Rawhide (DB 4.1.25)
>   DBNoMemoryError: (12, 'Cannot allocate memory -- Lock table is out of available locks')
>   alternating with SIGBUS
> 
> Redhat 7.1,7.2,7.3 (DB 3.2.9)
>   Crashes with a SIGBUS.

FWIW, I just tested it on RH9, DB 4.1.25, release23-maint branch. 
Worked goodly for me.

-Barry


% ./python Lib/test/test_bsddb3.py
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Sleepycat Software: Berkeley DB 4.1.25: (December 19, 2002)
bsddb.db.version():   (4, 1, 25)
bsddb.db.__version__: 4.2.0
bsddb.db.cvsid:       $Id: _bsddb.c,v 1.17.6.2 2003/09/21 23:10:23 greg 
Exp $
python version:        2.3+ (#1, Sep 22 2003, 13:35:52)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)]
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
........................................................................................................................................................................................................................
----------------------------------------------------------------------
Ran 216 tests in 131.194s

OK



From mal at lemburg.com  Mon Sep 22 15:13:26 2003
From: mal at lemburg.com (M.-A. Lemburg)
Date: Mon Sep 22 15:13:31 2003
Subject: [Python-Dev] Re:
	[Python-checkins]python/dist/src/Objects	unicodeobject.c, 
	2.197, 2.198
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEGBGBAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCEGBGBAB.tim.one@comcast.net>
Message-ID: <3F6F49D6.9050203@lemburg.com>

Tim Peters wrote:
> [Tim]
> 
>>>At the moment, it appears there's no identified reason to care about
>>>signedness of a greater-than 16-bit type,
> 
> 
> [M.-A. Lemburg]
> 
>>Sure there is: first of all, having a single type that can
>>be signed on some platforms and unsigned on others is a bad
>>thing per se
> 
> 
> We inherit that from C, though -- it's fine by C if wchar_t is signed or
> unsigned, just as it refused to define the signedness of char.

It maybe fine for C... it is not for the Unicode implementation
since that has always assumed Py_UNICODE to be unsigned. This
is fixed now.

>>and second the 32-bit signed wchar_t value was what triggered this
>>thread in the first place.
> 
> What triggered the thread originally was a segfault due to the code making a
> branch based on the content of uninitialized memory.  The code clearly
> didn't *think* it was reading up random heap bits, so that was a bug
> regardless of wchar_t's signedness. 

True, but the test (unicode->str[0] < 256) is what revealed a
second bug and that's what we've been discussing all along.

> That wchar_t happened to be a signed
> 32-bit type on Jeremy's box is what uncovered the read-uninitialized-memory
> bug.
>
> If there's no other code vulnerable to bad behavior if wchar_t is a signed
> 32-bit type (nobody has identified another case), objections to it being
> signed anyway seem technically groundless. 

There are more comparisons of the above type in the code and
even worse: it is documented that Py_UNICODE is unsigned,
so it's very likely that code external to the Python distribution
such as codec packages or applications talking to libraries
use that assumption as well.

> Martin did give a technical
> reason (efficiency) for wanting to continue to use wchar_t on Jeremy's
> system.

Python won't be using wchar_t on those systems anymore, so
the problem is solved and the original intent restored. If
efficiency matters programmers are always free to cast Py_UNICODE
to wchar_t on these systems for fast read-only access.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Sep 22 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::


From mfb at lotusland.dyndns.org  Mon Sep 22 20:36:36 2003
From: mfb at lotusland.dyndns.org (Matthew F. Barnes)
Date: Mon Sep 22 20:51:19 2003
Subject: [Python-Dev] Consistent logging in the standard library
In-Reply-To: <3122.192.168.1.1.1063886697.squirrel@server.lotusland.dyndns.org>
References: <005101c37d64$d37270c0$652b6992@alpha><1063835810.14316.20.camel@geddy
	><16232.57433.388130.573539@montanaro.dyndns.org> 
	<00ad01c37db8$d8bc9d40$652b6992@alpha> 
	<3122.192.168.1.1.1063886697.squirrel@server.lotusland.dyndns.org>
Message-ID: <2449.192.168.1.106.1064277396.squirrel@server.lotusland.dyndns.org>

To recap recent discussion on this thread, there seems to be consensus
among those I've spoken to that there should be a namespace in the logging
hierarchy named "python" that is reserved for the standard library, and
that standard library modules and packages should use loggers within this
namespace that are named after themselves (i.e. "python." + __name__). 
For example, the asyncore module should use a logger named
"python.asyncore".  The modules may also define child loggers as needed,
such as "python.asyncore.socket".


This week's topic: Configuration

I tend to think that, at least for Python 2.4, each logger should be
configured to mimic the behavior of the Python 2.3 version of its
corresponding module in terms of message format and destination (e.g.
log-file, sys.stdout, sys.stderr, etc.).

I think it would be nice for each "standard library logger" to default to
such a configuration automatically, while still allowing a configuration
file to customize them.

The issue is where this default configuration should reside and when it
should be applied.  My reference implementation currently has each module
configure its own logger at the end of the module, so that those
statements are executed when the module is imported.  But I'm not
confident that this is the best approach, or even a correct approach.  For
example, consider what would happen if a configuration file that defines a
section for the python.asyncore logger is processed *before* the asyncore
module is imported.

I'm looking for some helpful suggestions from the community.

- What's the best approach for configuring the "standard library loggers",
or should we at all?

- Should the standard library define its own logging configuration file,
and what would be the semantics around that?

- Perhaps the logging API should provide for a way to test whether a
particular logger is defined, so that the modules could check that before
applying their default settings?


Matthew Barnes


From fdrake01 at comcast.net  Tue Sep 23 01:39:53 2003
From: fdrake01 at comcast.net (fdrake01@comcast.net)
Date: Tue Sep 23 01:40:02 2003
Subject: [Python-Dev] Doc freeze for Python 2.3.1
Message-ID: <E1A1fu3-000486-3o@mail.python.org>

Python's Doc/ tree should be considered frozen on the release23-maint branch.
I've made the version number updates, but the machine I normally build the
documentation on is having some connectivity problems (which hopefully can be
resolved tomorrow morning).  Using what I have available to me now, I should
be able to build everything but the iSilo version of the docs; I don't think
a delay for those is needed; they can be added when available.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From aleaxit at yahoo.com  Tue Sep 23 03:46:18 2003
From: aleaxit at yahoo.com (Alex Martelli)
Date: Tue Sep 23 03:46:24 2003
Subject: [Python-Dev] Discussion on adding rsplit() for strings and
	unicode objects.
In-Reply-To: <6.0.0.22.0.20030922175257.020ecdd0@torment.chelsea.private>
References: <20030919083346.GA11541@tummy.com>
	<200309190612.15965.fincher.8@osu.edu>
	<6.0.0.22.0.20030922175257.020ecdd0@torment.chelsea.private>
Message-ID: <200309230946.18385.aleaxit@yahoo.com>

On Monday 22 September 2003 06:56 pm, Barry Scott wrote:
> >I'm all for it.  I've had to implement rsplit on my own as well, and
> > always lamented (but never did anything, unfortunately) about its lack in
> > the standard library.  That strings have rfind, rindex, rstrip, and
> > rjust, but not rsplit always seemed somewhat less than complete to me.
>
> These "r" are not a single family... some mean reverse some mean right.

???  They are: 

>>> [x for x in dir('') if x[:1]=='r']
['replace', 'rfind', 'rindex', 'rjust', 'rstrip']

'replace' is clearly distinct from the other (it's not an 'eplace' from the 
right:-).  All others can be read as "from the right" (reading them as
"reverse" is very obviously strained).  Find and Index the rightmost
occurrence, right-justify, strip characters from the right of the string.


Alex


From python-dev-list at pythonconsulting.com  Tue Sep 23 04:55:39 2003
From: python-dev-list at pythonconsulting.com (Steve Purcell)
Date: Tue Sep 23 04:55:44 2003
Subject: [Python-Dev] Merge unittest.py fixes to 2.3 maintenance branch?
Message-ID: <200309230955.39974.python-dev-list@pythonconsulting.com>

Hi all,

I recently checked into HEAD fixes for a couple of niggling issues with the 
unittest module, and I think it might be worth merging them into the 2.3 
maintenance branch. Any thumbs up or down on that? I see that a tag is set 
for 2.3.1, but it's not clear to me if that's final.

Most significant changes are:

- Fixed loading of tests by name when name refers to unbound
  method (PyUnit issue 563882, thanks to Alexandre Fayolle)
- Ignore non-callable attributes of classes when searching for test
  method names (PyUnit issue 769338, thanks to Seth Falcon)
- New assertTrue and assertFalse aliases for comfort of JUnit users
- Automatically discover 'runTest()' test methods (PyUnit issue 469444,
  thanks to Roeland Rengelink)
- Dropped Python 1.5.2 compatibility, merged appropriate shortcuts from
  Python CVS; should work with Python >= 2.1.
- Removed all references to string module by using string methods instead

Best wishes,

-Steve


-- 
Steve Purcell, Pythangelist
http://www.pythonconsulting.com/

From anthony at interlink.com.au  Tue Sep 23 05:23:33 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Sep 23 05:25:32 2003
Subject: [Python-Dev] latest bsddb3 test problems 
In-Reply-To: <200309220811.h8M8BVEN027031@localhost.localdomain> 
Message-ID: <200309230923.h8N9NZHf032580@localhost.localdomain>


On a friend's debian testing machine, the bsddb test suite crashes with 
a deadlock on _some_ occasions. On a Mac OS X machine, the test suite
hangs :-(

On the Mac OS X machine, the bsddb3 stuff doesn't get built under 2.3.

On the other hand, code using bsddb seems to work fine, so it's almost
certainly just that the test suite needs to be fixed to not be broken.

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From anthony at interlink.com.au  Tue Sep 23 05:27:03 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Sep 23 05:28:50 2003
Subject: [Python-Dev] Merge unittest.py fixes to 2.3 maintenance branch? 
In-Reply-To: <200309230955.39974.python-dev-list@pythonconsulting.com> 
Message-ID: <200309230927.h8N9R3I4032660@localhost.localdomain>


>>> Steve Purcell wrote
> Hi all,
> 
> I recently checked into HEAD fixes for a couple of niggling issues with the 
> unittest module, and I think it might be worth merging them into the 2.3 
> maintenance branch. Any thumbs up or down on that? I see that a tag is set 
> for 2.3.1, but it's not clear to me if that's final.

Pretty much. Thomas and I are doing a bunch of testing on the release candidate
tarball and exe, but unless we see something seriously busted, I don't think 
cutting a new release is going to happen. 

On the plus side, if the discipline to keep features out of the maintenance
releases continues, there's no reason to avoid a 2.3.2 somewhere down the
track.


-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From anthony at interlink.com.au  Tue Sep 23 05:43:12 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Sep 23 05:44:53 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
Message-ID: <200309230943.h8N9hCd1000446@localhost.localdomain>


At www.python.org/2.3.1 there's links to the release candidates for 
2.3.1 - Thomas and I are still running all sorts of tests against it,
but if someone else wants to give it a go, that'd be good. We're on
IRC #python-dev on irc.freenode.net.



From anthony at interlink.com.au  Tue Sep 23 05:46:15 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Sep 23 05:55:15 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go 
In-Reply-To: <200309230943.h8N9hCd1000446@localhost.localdomain> 
Message-ID: <200309230946.h8N9kFrP000507@localhost.localdomain>


>>> Anthony Baxter wrote
> 
> At www.python.org/2.3.1 there's links to the release candidates for 
> 2.3.1 - Thomas and I are still running all sorts of tests against it,
> but if someone else wants to give it a go, that'd be good. We're on
> IRC #python-dev on irc.freenode.net.

It's probably worth pointing out that this page isn't linked into the
site yet, nor is it in the sitemap.



From aahz at pythoncraft.com  Tue Sep 23 10:07:22 2003
From: aahz at pythoncraft.com (Aahz)
Date: Tue Sep 23 10:07:36 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <200309230946.h8N9kFrP000507@localhost.localdomain>
References: <200309230943.h8N9hCd1000446@localhost.localdomain>
	<200309230946.h8N9kFrP000507@localhost.localdomain>
Message-ID: <20030923140722.GB3988@panix.com>

On Tue, Sep 23, 2003, Anthony Baxter wrote:
>
> It's probably worth pointing out that this page isn't linked into the
> site yet, nor is it in the sitemap.

Don't worry, one of the webmasters (probably me, AMK, or Fred) will make
sure to tidy up.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

From aahz at pythoncraft.com  Tue Sep 23 10:09:31 2003
From: aahz at pythoncraft.com (Aahz)
Date: Tue Sep 23 10:09:43 2003
Subject: [Python-Dev] Merge unittest.py fixes to 2.3 maintenance branch?
In-Reply-To: <200309230955.39974.python-dev-list@pythonconsulting.com>
References: <200309230955.39974.python-dev-list@pythonconsulting.com>
Message-ID: <20030923140931.GC3988@panix.com>

On Tue, Sep 23, 2003, Steve Purcell wrote:
>
> Most significant changes are:
> 
> - Fixed loading of tests by name when name refers to unbound
>   method (PyUnit issue 563882, thanks to Alexandre Fayolle)
> - Ignore non-callable attributes of classes when searching for test
>   method names (PyUnit issue 769338, thanks to Seth Falcon)

These two are probably okay to port back (though too late for 2.3.1).

> - New assertTrue and assertFalse aliases for comfort of JUnit users
> - Automatically discover 'runTest()' test methods (PyUnit issue 469444,
>   thanks to Roeland Rengelink)
> - Dropped Python 1.5.2 compatibility, merged appropriate shortcuts from
>   Python CVS; should work with Python >= 2.1.
> - Removed all references to string module by using string methods instead

These should wait on 2.4.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

From jeremy at alum.mit.edu  Tue Sep 23 11:21:45 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue Sep 23 11:22:14 2003
Subject: [Python-Dev] latest bsddb3 test problems
In-Reply-To: <200309230923.h8N9NZHf032580@localhost.localdomain>
References: <200309230923.h8N9NZHf032580@localhost.localdomain>
Message-ID: <1064330505.2072.11.camel@localhost.localdomain>

On Tue, 2003-09-23 at 05:23, Anthony Baxter wrote:
> On a friend's debian testing machine, the bsddb test suite crashes with 
> a deadlock on _some_ occasions. On a Mac OS X machine, the test suite
> hangs :-(
> 
> On the Mac OS X machine, the bsddb3 stuff doesn't get built under 2.3.
> 
> On the other hand, code using bsddb seems to work fine, so it's almost
> certainly just that the test suite needs to be fixed to not be broken.

I think the question is whether the tests are doing something completely
artificial that fails, or if they are simply provoking failures that
would be particularly difficult to provoke in real life applications.

In Zope's BDBStorage, we've definitely seen some weird behavior.  For
example, we had a bug in our code where we tried to delete an entry from
a BTree that did not exist.  That operation failed with a
DBNotFoundError.  We caught that error can called txn.abort(); the abort
failed with a DBRunRecoveryError.  We didn't have any idea what the
abort would fail in that case.

It may be that we misunderstood how to use the Berkeley storage, but it
leaves me with the uneasy feeling that something isn't right.  The
failing tests make me feel the same way.

Jeremy



From arigo at tunes.org  Tue Sep 23 11:27:56 2003
From: arigo at tunes.org (Armin Rigo)
Date: Tue Sep 23 11:30:29 2003
Subject: [Python-Dev] Fun with 2.3 shutdown
In-Reply-To: <3F6D7835.5080407@zope.com>
References: <LNBBLJKPBEHFEDALKOLCOEOHFPAB.tim.one@comcast.net>
	<20030919120100.GB23317@vicky.ecs.soton.ac.uk>
	<3F6D7835.5080407@zope.com>
Message-ID: <20030923152756.GA20907@vicky.ecs.soton.ac.uk>

Hello Jim,

On Sun, Sep 21, 2003 at 06:06:45AM -0400, Jim Fulton wrote:
> Or, given cyclic GC, why isn't it enough to clear sys.modules?

Because the modules dictionary is stored in
PyThreadState_Get()->interp->modules, where the GC cannot see it, which
prevent the dictionary from being freed; in turn, it prevents any module from
being freed.

Still, it means (at a first sight) that there is actually no need to clear
each module's globals at all. Just removing them all from interp->modules is
fine. In practice there are probably a few issues that I'm not fully aware of,
like what occurs if we change module_dealloc() so that it no longer clears the
module's global dictionary -- a tweak that seems to come from a pre-GC
history.

Not speaking about threads in particular, and compatibility problems apart,
wouldn't it make sense to change sys.modules and interp->modules into a
WeakValueDictionary (or some equivalent refcount-based hack like the one for
interned strings)? This would give memory management at the module level, and
would allow a cleaner shutdown procedure as well. It would however change
Python's semantics of "a module is only loaded once unless you ask or you mess
with sys.modules explicitely".


A bientot,

Armin.


From python-dev-list at pythonconsulting.com  Tue Sep 23 11:42:45 2003
From: python-dev-list at pythonconsulting.com (Steve Purcell)
Date: Tue Sep 23 11:42:54 2003
Subject: [Python-Dev] Merge unittest.py fixes to 2.3 maintenance branch?
In-Reply-To: <20030923140931.GC3988@panix.com>
References: <200309230955.39974.python-dev-list@pythonconsulting.com>
	<20030923140931.GC3988@panix.com>
Message-ID: <200309231642.45495.python-dev-list@pythonconsulting.com>

On Tuesday 23 September 2003 15:09, Aahz wrote:
> On Tue, Sep 23, 2003, Steve Purcell wrote:
> > - Fixed loading of tests by name when name refers to unbound
> >   method (PyUnit issue 563882, thanks to Alexandre Fayolle)
> > - Ignore non-callable attributes of classes when searching for test
> >   method names (PyUnit issue 769338, thanks to Seth Falcon)
>
> These two are probably okay to port back (though too late for 2.3.1).

Great.

> > - New assertTrue and assertFalse aliases for comfort of JUnit users
> > - Automatically discover 'runTest()' test methods (PyUnit issue 469444,
> >   thanks to Roeland Rengelink)
> > - Dropped Python 1.5.2 compatibility, merged appropriate shortcuts from
> >   Python CVS; should work with Python >= 2.1.
> > - Removed all references to string module by using string methods instead
>
> These should wait on 2.4.

Yup, fair enough, though I might except the fix for issue 469444 from that, 
since it leads to unexpected behaviour when following examples in the std 
library docs. (i.e. TestCase classes that define only the method 'runTest()' 
will not be automatically run by unittest.main()).

Best wishes,

-Steve

-- 
Steve Purcell, Pythangelist
http://www.pythonconsulting.com/

From pje at telecommunity.com  Tue Sep 23 12:09:18 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Sep 23 12:09:23 2003
Subject: [Python-Dev] Fun with 2.3 shutdown
In-Reply-To: <20030923152756.GA20907@vicky.ecs.soton.ac.uk>
References: <3F6D7835.5080407@zope.com>
	<LNBBLJKPBEHFEDALKOLCOEOHFPAB.tim.one@comcast.net>
	<20030919120100.GB23317@vicky.ecs.soton.ac.uk>
	<3F6D7835.5080407@zope.com>
Message-ID: <5.1.0.14.0.20030923120249.025a0720@mail.telecommunity.com>

At 04:27 PM 9/23/03 +0100, Armin Rigo wrote:
>Not speaking about threads in particular, and compatibility problems apart,
>wouldn't it make sense to change sys.modules and interp->modules into a
>WeakValueDictionary (or some equivalent refcount-based hack like the one for
>interned strings)? This would give memory management at the module level, and
>would allow a cleaner shutdown procedure as well. It would however change
>Python's semantics of "a module is only loaded once unless you ask or you mess
>with sys.modules explicitely".

If I understand your suggestion correctly, the following code would break 
under such an arrangement:

Foo.py:
======
x = 1
def getX():
    return X

Bar.py:
=======
from Foo import getX  # returns getX() bound to one set of globals

import Foo   # imports a *new* Foo module, with new dictionary!
Foo.x = 2    # change the new Foo.x to 2

print getX()  # call the getX() with the old globals, and print 1


So, unless a module's dictionary were to reference the module (or functions 
were to reference the module rather than (or in addition to) the module 
dictionary), it seems the proposed semantics would lead to unexpected results.


From arigo at tunes.org  Tue Sep 23 12:49:09 2003
From: arigo at tunes.org (Armin Rigo)
Date: Tue Sep 23 12:51:43 2003
Subject: [Python-Dev] Fun with 2.3 shutdown
In-Reply-To: <5.1.0.14.0.20030923120249.025a0720@mail.telecommunity.com>
References: <3F6D7835.5080407@zope.com>
	<LNBBLJKPBEHFEDALKOLCOEOHFPAB.tim.one@comcast.net>
	<20030919120100.GB23317@vicky.ecs.soton.ac.uk>
	<3F6D7835.5080407@zope.com>
	<5.1.0.14.0.20030923120249.025a0720@mail.telecommunity.com>
Message-ID: <20030923164909.GC8415@vicky.ecs.soton.ac.uk>

Hello Phillip,

On Tue, Sep 23, 2003 at 12:09:18PM -0400, Phillip J. Eby wrote:
> So, unless a module's dictionary were to reference the module (or functions 
> were to reference the module rather than (or in addition to) the module 
> dictionary), it seems the proposed semantics would lead to unexpected 
> results.

Right.

I'm not sure I understand the reasons behind the current module/globals
relationship. As modules zap their globals with None when they are
deallocated, we observe the following behavior:

(foo.py)
def g():
    return 5
def f():
    return g()

>>> from foo import f
>>> import sys; del sys.modules['test4']
>>> f()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "foo.py", line 4, in f
    return g()
TypeError: 'NoneType' object is not callable

Possibly far-fetched, but I wouldn't be surprized to find large applications
that mess with sys.modules in some way. For example, in one case, to ensure
that a whole collection of interdependent modules will be reloaded on demand
after I detect a change in one of them, I'm simply removing them all from
sys.modules.


Armin


From pje at telecommunity.com  Tue Sep 23 13:36:10 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Sep 23 13:36:15 2003
Subject: [Python-Dev] Fun with 2.3 shutdown
In-Reply-To: <20030923164909.GC8415@vicky.ecs.soton.ac.uk>
References: <5.1.0.14.0.20030923120249.025a0720@mail.telecommunity.com>
	<3F6D7835.5080407@zope.com>
	<LNBBLJKPBEHFEDALKOLCOEOHFPAB.tim.one@comcast.net>
	<20030919120100.GB23317@vicky.ecs.soton.ac.uk>
	<3F6D7835.5080407@zope.com>
	<5.1.0.14.0.20030923120249.025a0720@mail.telecommunity.com>
Message-ID: <5.1.0.14.0.20030923132744.03e06560@mail.telecommunity.com>

At 05:49 PM 9/23/03 +0100, Armin Rigo wrote:
>Hello Phillip,
>
>On Tue, Sep 23, 2003 at 12:09:18PM -0400, Phillip J. Eby wrote:
> > So, unless a module's dictionary were to reference the module (or 
> functions
> > were to reference the module rather than (or in addition to) the module
> > dictionary), it seems the proposed semantics would lead to unexpected
> > results.
>
>Right.
>
>I'm not sure I understand the reasons behind the current module/globals
>relationship.As modules zap their globals with None when they are

Well, a function can have globals that aren't a module dictionary, so 
that's probably why functions don't refer to their modules directly.  As 
for the None thing, it's to break circular references.


>deallocated, we observe the following behavior:
>
>(foo.py)
>def g():
>     return 5
>def f():
>     return g()
>
> >>> from foo import f
> >>> import sys; del sys.modules['test4']
> >>> f()
>Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File "foo.py", line 4, in f
>     return g()
>TypeError: 'NoneType' object is not callable

Yeah, I've run into this (and weirder!) things before, when tracing down 
refcount bugs in C extensions.

But, I don't think anybody in their right mind would *rely* on this 
behavior, so your suggestion of e.g. clearing sys.modules and letting GC do 
the rest seems worthy of investigation.


>Possibly far-fetched, but I wouldn't be surprized to find large applications
>that mess with sys.modules in some way. For example, in one case, to ensure
>that a whole collection of interdependent modules will be reloaded on demand
>after I detect a change in one of them, I'm simply removing them all from
>sys.modules.

Of course, you have to also delete all modules and objects that reference 
the modules you want to delete.  And that would still be the case under 
your "clear sys.modules at shutdown" proposal.

OTOH, it's not clear what to do with uncollectable objects.  E.g., if you 
have a module global that is or contains an object with a __del__ method, 
and is part of a cycle back to that module.  I think you'd *still* be stuck 
going back and clearing the modules or setting their contents to None.


From nick at nick.uklinux.net  Tue Sep 23 14:25:33 2003
From: nick at nick.uklinux.net (Nick Roberts)
Date: Tue Sep 23 14:31:58 2003
Subject: [Python-Dev] Integration with Eclipse
In-Reply-To: <3F6D1AB7.1050608@sabaydi.com>
References: <E1A0ErE-0003ue-8W@mail.python.org> <3F6B11BA.6000008@sabaydi.com>
	<16235.18128.422738.929390@nick.uklinux.net>
	<3F6B533E.5000403@sabaydi.com>
	<16236.28754.105629.565153@nick.uklinux.net>
	<3F6C7B09.5060608@sabaydi.com>
	<16236.51140.284372.526221@nick.uklinux.net>
	<3F6CF076.30409@ocf.berkeley.edu> <3F6D1AB7.1050608@sabaydi.com>
Message-ID: <16240.36893.520437.974584@nick.uklinux.net>

Kevin J. Butler writes:
 > ...
 >
 > I consider this more pdb-related than emacs-specific -
 > I'd expect that integration with, say, Eclipse would benefit from this 
 > enhancement as well, so I've been hoping to get some feedback from the 
 > list...
 > 
 > BTW, I've submitted the patch, it is 809887: 
 > http://sourceforge.net/tracker/index.php?func=detail&aid=809887&group_id=5470&atid=305470
 > 
 > kb

The syntax of responses from pdb commands suggests to me that there is a fair
amount of GDB in the underlying code. The GDB developers have written an
interface called GDB/MI (MI = Machine Interface) for integration with an
IDE. Since the Eclipse project is using this interface to integrate with GDB
there may be some overlap for doing the same thing with with pdb.

Nick

From jeremy at zope.com  Tue Sep 23 16:48:07 2003
From: jeremy at zope.com (Jeremy Hylton)
Date: Tue Sep 23 16:48:52 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <200309230943.h8N9hCd1000446@localhost.localdomain>
References: <200309230943.h8N9hCd1000446@localhost.localdomain>
Message-ID: <1064350087.2072.66.camel@localhost.localdomain>

On Tue, 2003-09-23 at 05:43, Anthony Baxter wrote:
> At www.python.org/2.3.1 there's links to the release candidates for 
> 2.3.1 - Thomas and I are still running all sorts of tests against it,
> but if someone else wants to give it a go, that'd be good. We're on
> IRC #python-dev on irc.freenode.net.

You most post a note to python-dev when you think you've got the final
release files.  There are lots of things that can go wrong the first
time around, and we can do some last minute sanity checks.  I'd be happy
to test on RH9 and Win2k.

Jeremy



From theller at python.net  Tue Sep 23 17:02:42 2003
From: theller at python.net (Thomas Heller)
Date: Tue Sep 23 17:02:49 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <1064350087.2072.66.camel@localhost.localdomain> (Jeremy
	Hylton's message of "Tue, 23 Sep 2003 16:48:07 -0400")
References: <200309230943.h8N9hCd1000446@localhost.localdomain>
	<1064350087.2072.66.camel@localhost.localdomain>
Message-ID: <u173kykt.fsf@python.net>

Jeremy Hylton <jeremy@zope.com> writes:

> On Tue, 2003-09-23 at 05:43, Anthony Baxter wrote:
>> At www.python.org/2.3.1 there's links to the release candidates for 
>> 2.3.1 - Thomas and I are still running all sorts of tests against it,
>> but if someone else wants to give it a go, that'd be good. We're on
>> IRC #python-dev on irc.freenode.net.
>
> You most post a note to python-dev when you think you've got the final
> release files.  There are lots of things that can go wrong the first
> time around, and we can do some last minute sanity checks.  I'd be happy
> to test on RH9 and Win2k.

They are not yet ready. Tim Peters found that some Windows version
numbers have not been incremented, and I'm currently changing this.
It seems at least the source tarball and the windows installer have to
been rebuilt.

But since these are changes which should not affect the functionality
too much, testing can go on.

Thomas


From jeremy at zope.com  Tue Sep 23 17:07:57 2003
From: jeremy at zope.com (Jeremy Hylton)
Date: Tue Sep 23 17:09:23 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <u173kykt.fsf@python.net>
References: <200309230943.h8N9hCd1000446@localhost.localdomain>
	<1064350087.2072.66.camel@localhost.localdomain>
	<u173kykt.fsf@python.net>
Message-ID: <1064351277.2072.68.camel@localhost.localdomain>

On Tue, 2003-09-23 at 17:02, Thomas Heller wrote:
> > You most post a note to python-dev when you think you've got the final

(BTW this was supposed to say "You might post" but I got a little ost
happy.)

> > release files.  There are lots of things that can go wrong the first
> > time around, and we can do some last minute sanity checks.  I'd be happy
> > to test on RH9 and Win2k.
> 
> They are not yet ready. Tim Peters found that some Windows version
> numbers have not been incremented, and I'm currently changing this.
> It seems at least the source tarball and the windows installer have to
> been rebuilt.
> 
> But since these are changes which should not affect the functionality
> too much, testing can go on.

It can't go on unless you tell me where the files are <wink>.

Jeremy



From theller at python.net  Tue Sep 23 17:17:54 2003
From: theller at python.net (Thomas Heller)
Date: Tue Sep 23 17:18:00 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <1064351277.2072.68.camel@localhost.localdomain> (Jeremy
	Hylton's message of "Tue, 23 Sep 2003 17:07:57 -0400")
References: <200309230943.h8N9hCd1000446@localhost.localdomain>
	<1064350087.2072.66.camel@localhost.localdomain>
	<u173kykt.fsf@python.net>
	<1064351277.2072.68.camel@localhost.localdomain>
Message-ID: <oexbkxvh.fsf@python.net>

Jeremy Hylton <jeremy@zope.com> writes:

> It can't go on unless you tell me where the files are <wink>.

I thought you had seen this:

>>> Anthony Baxter wrote
> 
> At www.python.org/2.3.1 there's links to the release candidates for 
> 2.3.1 - Thomas and I are still running all sorts of tests against it,
> but if someone else wants to give it a go, that'd be good. We're on
> IRC #python-dev on irc.freenode.net.

Thomas


From doko at cs.tu-berlin.de  Tue Sep 23 17:32:27 2003
From: doko at cs.tu-berlin.de (Matthias Klose)
Date: Tue Sep 23 17:36:53 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <200309230943.h8N9hCd1000446@localhost.localdomain>
References: <200309230943.h8N9hCd1000446@localhost.localdomain>
Message-ID: <16240.48107.934445.199555@gargle.gargle.HOWL>

Anthony Baxter writes:
> 
> At www.python.org/2.3.1 there's links to the release candidates for 
> 2.3.1 - Thomas and I are still running all sorts of tests against it,
> but if someone else wants to give it a go, that'd be good. We're on
> IRC #python-dev on irc.freenode.net.

please could you consider

- either to hand-edit the generated configure script

- or rebuild configure with autoconf from current CVS

to enable a correct configuration on HPUX?

Please see
http://mail.python.org/pipermail/python-dev/2003-August/037648.html

I think the first solution should be more conservative at this point,
maybe using a nested ifdef instead of using the defined keyword.

If neither solution is appropriate, please add a note to the target
specific install section.

Thanks, Matthias

From zeddicus at satokar.com  Tue Sep 23 18:14:33 2003
From: zeddicus at satokar.com (Michael Bartl)
Date: Tue Sep 23 18:14:37 2003
Subject: [Python-Dev] Help offered
Message-ID: <20030923221433.GA1764@satokar>

Hi all!

I've sent my first patch to python sf.net and noticed that there are
still patches from ~2001! Is there a general patch review cycle or
something? I'd like to offer my help if there is interest.
(writing/reviewing patches, sorting out bugreports, ...)

My "python" background:

- Working as a software engineer in air traffic control.

- I suggested and embedded the python interpreter into our layout
  interpreter, which was kind of tricky due to the PowerPC platform.
  (python doesn't really like to be cross-compiled)

  This leads to the fact that some air traffic control centers around
  the world are now Python powered :)

Have fun,
    Bartl Michael

From aahz at pythoncraft.com  Tue Sep 23 20:19:09 2003
From: aahz at pythoncraft.com (Aahz)
Date: Tue Sep 23 20:19:15 2003
Subject: [Python-Dev] Help offered
In-Reply-To: <20030923221433.GA1764@satokar>
References: <20030923221433.GA1764@satokar>
Message-ID: <20030924001908.GB28288@panix.com>

On Wed, Sep 24, 2003, Michael Bartl wrote:
> 
> I've sent my first patch to python sf.net and noticed that there
> are still patches from ~2001! Is there a general patch review cycle
> or something? I'd like to offer my help if there is interest.
> (writing/reviewing patches, sorting out bugreports, ...)

Excellent!  Currently, it's rather ad-hoc.  Just pick a patch that
interests you and make a comment about its suitability.  Make sure to
check whether the patch comes with appropriate documentation.  [*]  If
you don't see activity on the patch within a week or two, feel free to
ask here.  Right now isn't the best time to poke people because we're
gearing up for the 2.3.1 release.

Read the Developer's Guide if you haven't yet (under "Python Project" on
the home page).

>   This leads to the fact that some air traffic control centers around
>   the world are now Python powered :)

Cool!
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

From anthony at interlink.com.au  Tue Sep 23 20:38:02 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Sep 23 20:39:53 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go 
In-Reply-To: <16240.48107.934445.199555@gargle.gargle.HOWL> 
Message-ID: <200309240038.h8O0c2VX012396@localhost.localdomain>


>>> Matthias Klose wrote
> please could you consider
> 
> - either to hand-edit the generated configure script
> 
> - or rebuild configure with autoconf from current CVS
> 
> to enable a correct configuration on HPUX?

Michael Hudson and I spent several painful hours on this last
night - at the moment, there's an entry in the bugs.html page
detailing the workaround (edit pyconfig.h after running 
configure). At this point in the cycle I'm extremely uncomfortable
trying to either hand-edit configure or running a new autoconf.

I'm recutting the tarball now, I'll add a note to the README
as well.

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From anthony at interlink.com.au  Tue Sep 23 21:29:51 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Sep 23 21:32:00 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go 
In-Reply-To: <u173kykt.fsf@python.net> 
Message-ID: <200309240129.h8O1Tqft013440@localhost.localdomain>



There's a new release candidate tarball/exe on www.python.org/2.3.1

Feedback welcome - either here or irc.freenode.net #python-dev.

Changes since yesterday: 

  Windows build numbers updated
  HP-UX README note added
  Misc/RPM notes updated


From jepler at unpythonic.net  Tue Sep 23 22:50:25 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Tue Sep 23 22:50:44 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <200309240129.h8O1Tqft013440@localhost.localdomain>
References: <u173kykt.fsf@python.net>
	<200309240129.h8O1Tqft013440@localhost.localdomain>
Message-ID: <20030924025019.GA1590@unpythonic.net>

As reported on IRC, I had good test results on the following machines
with the RC2 tarball:

FreeBSD zuul.dsndata.com 4.7-RELEASE FreeBSD 4.7-RELEASE #0: Wed Oct  9 15:08:34 GMT 2002

Linux parrot 2.4.20-9 #1 Wed Apr 2 13:42:50 EST 2003 i686 i686 i386 GNU/Linux
Red Hat Linux release 9 (Shrike)

Linux bald 2.4.7-10 #1 Thu Sep 6 17:21:28 EDT 2001 i586 unknown
Red Hat Linux release 7.2 (Enigma)

Tkinter works OOTB on the redhat machines without any configure option.
I don't think tcl/tk are on the bsd machine, so I didn't try.

(I just ran "make test", didn't enable any resources)

Jeff

From python at rcn.com  Tue Sep 23 23:46:52 2003
From: python at rcn.com (Raymond Hettinger)
Date: Tue Sep 23 23:47:12 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go 
In-Reply-To: <200309240129.h8O1Tqft013440@localhost.localdomain>
Message-ID: <000001c3824e$7edbcd60$04af958d@oemcomputer>

Tested without exception on WinME.

* ran regrtest -u all -r
* checked chm files
* ran unittests on own applications
* exercised IDLE


Raymond Hettinger



-----Original Message-----
From: python-dev-bounces@python.org
[mailto:python-dev-bounces@python.org] On Behalf Of Anthony Baxter
Sent: Tuesday, September 23, 2003 9:30 PM
To: Thomas Heller
Cc: jeremy@zope.com; python-dev@python.org
Subject: Re: [Python-Dev] 2.3.1 is (almost) a go 



There's a new release candidate tarball/exe on www.python.org/2.3.1

Feedback welcome - either here or irc.freenode.net #python-dev.

Changes since yesterday: 

  Windows build numbers updated
  HP-UX README note added
  Misc/RPM notes updated




#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

From bac at OCF.Berkeley.EDU  Wed Sep 24 02:25:34 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed Sep 24 02:25:54 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <200309240129.h8O1Tqft013440@localhost.localdomain>
References: <200309240129.h8O1Tqft013440@localhost.localdomain>
Message-ID: <3F7138DE.7080504@ocf.berkeley.edu>

Anthony Baxter wrote:
> 
> There's a new release candidate tarball/exe on www.python.org/2.3.1
> 
> Feedback welcome - either here or irc.freenode.net #python-dev.
> 

Ran ``./configure; make; make test`` and no failures on OS X 10.2.8 .

-Brett


From anthony at interlink.com.au  Wed Sep 24 03:15:37 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Sep 24 03:17:22 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go 
Message-ID: <200309240715.h8O7Fbj4024972@localhost.localdomain>


Last chance for feedback on python2.3.1 ... :)

Draft of announce message at www.python.org/2.3.1/announce.txt


-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From martin at v.loewis.de  Wed Sep 24 03:26:12 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Wed Sep 24 03:26:50 2003
Subject: [Python-Dev] Help offered
In-Reply-To: <20030924001908.GB28288@panix.com>
References: <20030923221433.GA1764@satokar> <20030924001908.GB28288@panix.com>
Message-ID: <m38yoe4pgr.fsf@mira.informatik.hu-berlin.de>

Aahz <aahz@pythoncraft.com> writes:

> Excellent!  Currently, it's rather ad-hoc.  Just pick a patch that
> interests you and make a comment about its suitability.  Make sure to
> check whether the patch comes with appropriate documentation.  [*]  If
> you don't see activity on the patch within a week or two, feel free to
> ask here.  Right now isn't the best time to poke people because we're
> gearing up for the 2.3.1 release.

Also, try to classify the patch somehow, indicating what most likely
the problem is for the patch not being reviewed/accepted:

- the patch might introduce a questionable feature, and nobody has
  dared to reject it yet, in order to not offend the submitter. In
  that case, voice an opinion (and record it in the patches tracker)
  on whether you are in favour of or opposed to the proposed feature
  (ideally giving a rationale on why you are). If there are enough
  voices showing opposition, the submitter might withdraw the patch.
  If there are enough voices in favour, a core developer might
  accept it.

- the patch might be incomplete. Ping the submitter. If the submitter
  is incomplete, either complete it yourself, or suggest rejection
  of the patch.

- the patch might be so involved that a quick review does not reveal
  whether it is correct. I personally find patches involving memory
  management bugs to be in that category. Review the patch excessively:
  perform tests, study all possible code paths to uncover cases that
  have been ignored. When done, record in the patch what kind of review
  you have done, and either indicate the problems you have found, or
  recommend acceptance.

- the patch might be in an area where the "core" developers have
  little expertise; you often find that trivial patches are ignored
  just because everybody thought not to be an expert in the subject.
  Perform a quick review of the patch to find out whether it meets the
  formal criteria (completeness), and whether it is perhaps obviously
  correct or obviously incorrect. If not, come up with a strategy
  to obtain a decision on the patch:
  * become an expert yourself in the subject area, and recommend
    acceptance or rejection afterwards. I have done that many times,
    and find it both time consuming and rewarding.
  * find an expert and have it review the patch
  * ask the submitter to clarify all questionable aspects of the
    patch, i.e. have him explain the patch to you. If the patch
    is for a little-used feature (e.g. for an obscure platform),
    it might be acceptable to incorporate incorrect patches, since
    nobody will notice, anyway - it might be enough that the submitter
    believes the patch is correct.

I personally find processing of patches more important: it takes often
less time per item, brings the project forward, and gives the
contributor the satisfaction of having contributed.

For bug reports, I would recommend the following classification:
- determine whether the report is complete: does the submitter
  indicate observed behaviour, expected behaviour? If relevant, OS
  version, Python version? Is a test case included (not mandatory,
  but desirable)?
- determine whether the report is reproducable: try reproducing
  it with the included test case, or come up with a test case
  yourself (and list it in the bug report). It might be:
  * unreproducable: you have the seemingly same setup as the
    the submitter, but you fail to reproduce it. Ask the submitter
    for clarification.
  * fixed: you can reproduce it for the version the submitter has
    used (or you are reasonably sure that the bug existed in that
    version), yet the problem disappears with the current version.
    Indicate that in the report, and recommend to close the
    report as fixed.
  * hard to reproduce: you cannot reproduce the bug because you
    lack some environment. Try to confirm presence of the bug
    by reviewing the code, or get yourself access to the missing
    environment.
- determine whether the report is invalid: is it really a bug?
  E.g. is the observed behaviour documented? Is the documentation
  silent on this aspect, and the observed behaviour surprising
  to experts? If the documented behaviour deviates from the
  observed behaviour (and perhaps both from the expected behaviour):
  which should be the correct one?

Once you have classified the report, and found it reports a proper
bug, consider asking the submitter for a patch (do so kindly, not
demanding).  If it is clear that the submitter is not going to provide
a patch, try developing one yourself.

Regards,
Martin

From anthony at interlink.com.au  Wed Sep 24 04:02:47 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Sep 24 04:04:44 2003
Subject: [Python-Dev] RELEASED Python 2.3.1
Message-ID: <200309240802.h8O82mqF026019@localhost.localdomain>

On behalf of the Python development team and the Python community, I'm
happy to announce the release of Python 2.3.1 (final).

Python 2.3.1 is a pure bug fix release of Python 2.3, released in late
July. A number of obscure crash-causing bugs have been fixed, various
memory leaks have been squished, but no new features have been added to
the language or to the library.

For more information on Python 2.3.1, including download links for
various platforms, release notes, and known issues, please see:

    http://www.python.org/2.3.1

Highlights of this new release include:

 - Bugfixes

 - The Windows installer now ships with documentation in searchable
   htmlhelp format, rather than individual HTML files.

Highlights of the previous major Python release (2.3) are available     
from the Python 2.3 page, at                                            

    http://www.python.org/2.3/highlights.html

This release was the work of a number of dedicated python-dev folks who
took the time to apply bugfixes back to the 2.3 code. The community owes
them many thanks for this work.

Enjoy the new release,
Anthony

Anthony Baxter
anthony@python.org
Python 2.3.1 Release Manager
(on behalf of the entire python-dev team)

From anthony at interlink.com.au  Wed Sep 24 04:06:14 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Sep 24 04:07:55 2003
Subject: [Python-Dev] RELEASED Python 2.3.1 
In-Reply-To: <200309240802.h8O82mqF026019@localhost.localdomain> 
Message-ID: <200309240806.h8O86Eti026111@localhost.localdomain>


>>> Anthony Baxter wrote
> On behalf of the Python development team and the Python community, I'm
> happy to announce the release of Python 2.3.1 (final).

Can someone who's an SF admin submit a News item on SF? It appears only
project admins can do this.


From anthony at interlink.com.au  Wed Sep 24 04:15:14 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Sep 24 04:17:03 2003
Subject: [Python-Dev] RELEASED Python 2.3.1 
In-Reply-To: <200309240802.h8O82mqF026019@localhost.localdomain> 
Message-ID: <200309240815.h8O8FIfo026231@localhost.localdomain>


Oh yeah - please hold off on the release23-maint branch for a bit,
in case Jack wants to cut a 2.3.1 release for the Mac... Jack?

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From zeddicus at satokar.com  Wed Sep 24 04:22:38 2003
From: zeddicus at satokar.com (Michael Bartl)
Date: Wed Sep 24 04:22:42 2003
Subject: [Python-Dev] Help offered
In-Reply-To: <m38yoe4pgr.fsf@mira.informatik.hu-berlin.de>
References: <20030923221433.GA1764@satokar> <20030924001908.GB28288@panix.com>
	<m38yoe4pgr.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20030924082238.GA11926@satokar>

On Wed, Sep 24, 2003 at 09:26:12AM +0200, Martin v. L?wis wrote:

[snip - probably the biggest list of recommendations ever seen :)]

First of all thanks for the fast and exhaustive answer.

A last question before I'm working on bugs/patches. I sent a patch
to fix a mkstemp() bug. The documentation and the implementation had
different opinions on how to return the path.

What has precedence? Nothing I guess. It's rather a bug in the doc or in
the code. Should I contact the original author(s) of tempfile.py, which
seem to be Guido and Tim Peters?

Best regards,
    Michael

From martin at v.loewis.de  Wed Sep 24 05:02:46 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Wed Sep 24 05:03:37 2003
Subject: [Python-Dev] RELEASED Python 2.3.1
In-Reply-To: <200309240806.h8O86Eti026111@localhost.localdomain>
References: <200309240806.h8O86Eti026111@localhost.localdomain>
Message-ID: <m33cem36fd.fsf@mira.informatik.hu-berlin.de>

Anthony Baxter <anthony@interlink.com.au> writes:

> Can someone who's an SF admin submit a News item on SF? It appears only
> project admins can do this.

Done!

Martin

From martin at v.loewis.de  Wed Sep 24 05:07:53 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Wed Sep 24 05:08:51 2003
Subject: [Python-Dev] Help offered
In-Reply-To: <20030924082238.GA11926@satokar>
References: <20030923221433.GA1764@satokar> <20030924001908.GB28288@panix.com>
	<m38yoe4pgr.fsf@mira.informatik.hu-berlin.de>
	<20030924082238.GA11926@satokar>
Message-ID: <m3y8we1rme.fsf@mira.informatik.hu-berlin.de>

Michael Bartl <zeddicus@satokar.com> writes:

> What has precedence? Nothing I guess. It's rather a bug in the doc or in
> the code. Should I contact the original author(s) of tempfile.py, which
> seem to be Guido and Tim Peters?

Not necessarily. If your common sense dictates that one is better than
the other, just explain it so. If you change the code, consider that
existing code may break. Don't worry about the potential breakage,
instead, document it - both in Misc/NEWS, and, if breakage is likely,
in Doc/whatsnew/whatsnew24.tex. Document the change itself in the
documentation, using \versionchanged.

[You might notice that I have abstained from actually thinking about
the change itself :-]

Regards,
Martin

From theller at python.net  Wed Sep 24 06:14:27 2003
From: theller at python.net (Thomas Heller)
Date: Wed Sep 24 06:14:34 2003
Subject: [Python-Dev] Editing peps with XEmacs
Message-ID: <he32ijcs.fsf@python.net>

Usually the M-q 'fill-paragraph or region' works nicely to format the
current paragraph.  It does not work when the paragraph has this form:

  ___ When the year changes, copyright legends need to be updated in
      many places, including the README and LICENSE files.

because of the '___' marker.
Is there any trick I do not know? A REST mode or something like that?

Thanks,

Thomas


From barry at python.org  Wed Sep 24 07:53:26 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Sep 24 07:53:31 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <200309240715.h8O7Fbj4024972@localhost.localdomain>
References: <200309240715.h8O7Fbj4024972@localhost.localdomain>
Message-ID: <1064404406.1958.71.camel@anthem>

On Wed, 2003-09-24 at 03:15, Anthony Baxter wrote:
> Last chance for feedback on python2.3.1 ... :)
> 
> Draft of announce message at www.python.org/2.3.1/announce.txt

I pulled down rc2 and ran the tests randomly on RH9, with "-u all -r". 
I see that test_mimetypes failed unexpectedly:

test test_mimetypes failed -- Traceback (most recent call last):
  File "/tmp/Python-2.3.1/Lib/test/test_mimetypes.py", line 52, in
test_guess_all_types
    eq(all, ['.bat', '.c', '.h', '.ksh', '.pl', '.txt'])
  File "/tmp/Python-2.3.1/Lib/unittest.py", line 302, in failUnlessEqual
    raise self.failureException, \
AssertionError: ['.asc', '.bat', '.c', '.h', '.ksh', '.pl', '.txt'] !=
['.bat', '.c', '.h', '.ksh', '.pl', '.txt']


test_bsddb3 produced two locker killed messages, which didn't happen
last time I tested from the HEAD.  Then again I wasn't running the tests
with -r last time either, but I also understand that these tracebacks
can happen and don't cause the tests to fail.

In the first run, test_queue also failed, with this output:

test_queue
test test_queue failed -- blocking function '<bound method Queue.get of
<Queue.Queue instance at 0x402471cc>>' appeared not to block

Of these, it seems like the mimetypes one should be fixed before the
release.  I doubt I'll have time today to debug it, but I could probably
arrange to run another test on RH9 if need be.

-Barry



From barry at python.org  Wed Sep 24 07:55:13 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Sep 24 07:55:16 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <1064404406.1958.71.camel@anthem>
References: <200309240715.h8O7Fbj4024972@localhost.localdomain>
	<1064404406.1958.71.camel@anthem>
Message-ID: <1064404512.1958.73.camel@anthem>

On Wed, 2003-09-24 at 07:53, Barry Warsaw wrote:

> Of these, it seems like the mimetypes one should be fixed before the
> release.  I doubt I'll have time today to debug it, but I could probably
> arrange to run another test on RH9 if need be.

Then again, I guess there's always 2.3.2...

stinkin'-timezones-ly y'rs,
-Barry



From barry at python.org  Wed Sep 24 07:59:43 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Sep 24 07:59:46 2003
Subject: [Python-Dev] Editing peps with XEmacs
In-Reply-To: <he32ijcs.fsf@python.net>
References: <he32ijcs.fsf@python.net>
Message-ID: <1064404783.1958.80.camel@anthem>

On Wed, 2003-09-24 at 06:14, Thomas Heller wrote:
> Usually the M-q 'fill-paragraph or region' works nicely to format the
> current paragraph.  It does not work when the paragraph has this form:
> 
>   ___ When the year changes, copyright legends need to be updated in
>       many places, including the README and LICENSE files.
> 
> because of the '___' marker.
> Is there any trick I do not know? A REST mode or something like that?

filladapt!

-Barry



From jepler at unpythonic.net  Wed Sep 24 08:10:51 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Wed Sep 24 08:11:01 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <1064404406.1958.71.camel@anthem>
References: <200309240715.h8O7Fbj4024972@localhost.localdomain>
	<1064404406.1958.71.camel@anthem>
Message-ID: <20030924121045.GB1590@unpythonic.net>

On Wed, Sep 24, 2003 at 07:53:26AM -0400, Barry Warsaw wrote:
> I pulled down rc2 and ran the tests randomly on RH9, with "-u all -r". 
> I see that test_mimetypes failed unexpectedly:
> 
> test test_mimetypes failed -- Traceback (most recent call last):
>   File "/tmp/Python-2.3.1/Lib/test/test_mimetypes.py", line 52, in
> test_guess_all_types
>     eq(all, ['.bat', '.c', '.h', '.ksh', '.pl', '.txt'])
>   File "/tmp/Python-2.3.1/Lib/unittest.py", line 302, in failUnlessEqual
>     raise self.failureException, \
> AssertionError: ['.asc', '.bat', '.c', '.h', '.ksh', '.pl', '.txt'] !=
> ['.bat', '.c', '.h', '.ksh', '.pl', '.txt']

This order fails:
[jepler@parrot Lib]$ ../python test/regrtest.py test_urllib2 test_mimetypes

This order doesn't:
[jepler@parrot Lib]$ ../python test/regrtest.py test_mimetypes test_urllib2

urllib2 calls mimetypes.guess_type without zeroing out
mimetypes.knownfiles.  I think this is the cause.  Adding this patch
makes both orders pass.

Perhaps a better way to fix it would be to test for a superset in
test_mimetypes.  assert set(all) >= set('.bat', ...)

--- test_urllib2.py.orig	2003-09-24 07:08:28.000000000 -0500
+++ test_urllib2.py	2003-09-24 07:07:31.000000000 -0500
@@ -11,6 +11,10 @@
 else:
     verify(0)
 
+import mimetypes
+mimetypes.knownfiles = []
+mimetypes.inited = False
+
 # XXX Name hacking to get this to work on Windows.
 fname = os.path.abspath(urllib2.__file__).replace('\\', '/')
 if fname[1:2] == ":":

From jepler at unpythonic.net  Wed Sep 24 08:19:20 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Wed Sep 24 08:19:30 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <20030924121045.GB1590@unpythonic.net>
References: <200309240715.h8O7Fbj4024972@localhost.localdomain>
	<1064404406.1958.71.camel@anthem>
	<20030924121045.GB1590@unpythonic.net>
Message-ID: <20030924121920.GC1590@unpythonic.net>

On Wed, Sep 24, 2003 at 07:10:51AM -0500, Jeff Epler wrote:
> Perhaps a better way to fix it would be to test for a superset in
> test_mimetypes.  assert set(all) >= set('.bat', ...)

The general idea is implemented in this patch:

--- test_mimetypes.py.orig	2003-09-24 07:11:12.000000000 -0500
+++ test_mimetypes.py	2003-09-24 07:18:58.000000000 -0500
@@ -1,18 +1,21 @@
 import mimetypes
 import StringIO
 import unittest
+import sets
 
 from test import test_support
 
-# Tell it we don't know about external files:
-mimetypes.knownfiles = []
-mimetypes.inited = False
-
-
 class MimeTypesTestCase(unittest.TestCase):
     def setUp(self):
         self.db = mimetypes.MimeTypes()
 
+    def assertSuperset(self, first, second, msg=None):
+        first = sets.Set(first)
+        second = sets.Set(second)
+        if not first >= second:
+            raise self.failureException, \
+                  (msg or '%r >= %r' % (first, second))
+
     def test_default_data(self):
         eq = self.assertEqual
         eq(self.db.guess_type("foo.html"), ("text/html", None))
@@ -45,15 +48,16 @@
         eq(self.db.guess_extension('image/jpg', strict=False), '.jpg')
 
     def test_guess_all_types(self):
+        ss = self.assertSuperset
         eq = self.assertEqual
         # First try strict
         all = self.db.guess_all_extensions('text/plain', strict=True)
         all.sort()
-        eq(all, ['.bat', '.c', '.h', '.ksh', '.pl', '.txt'])
+        ss(all, ['.bat', '.c', '.h', '.ksh', '.pl', '.txt'])
         # And now non-strict
         all = self.db.guess_all_extensions('image/jpg', strict=False)
         all.sort()
-        eq(all, ['.jpg'])
+        ss(all, ['.jpg'])
         # And now for no hits
         all = self.db.guess_all_extensions('image/jpg', strict=True)
         eq(all, [])

From anthony at interlink.com.au  Wed Sep 24 08:33:38 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Sep 24 08:35:24 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go 
In-Reply-To: <1064404406.1958.71.camel@anthem> 
Message-ID: <200309241233.h8OCXcbg028668@localhost.localdomain>


>>> Barry Warsaw wrote
> Of these, it seems like the mimetypes one should be fixed before the
> release.  I doubt I'll have time today to debug it, but I could probably
> arrange to run another test on RH9 if need be.

D'oh! Since it's only a failure of the test code (not the module itself)
I'll add a note to the bugs page in the test section.

Jeff Epler's fix seems appropriate. There's always 2.3.2 - since 2.4 is still
a fair way off, it seems likely that a 2.3.2 sometime around January would be
appropriate. Of course, if the bsddb3 weirdness does actually turn out to be
something more serious, it might be sooner.

who-uses-bsddb-anyway-just-those-zope-guys-<wink>,
Anthony 
-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From Paul.Moore at atosorigin.com  Wed Sep 24 08:41:42 2003
From: Paul.Moore at atosorigin.com (Moore, Paul)
Date: Wed Sep 24 08:42:42 2003
Subject: [Python-Dev] 2.3.1 - minor nit in HTML Help docs
Message-ID: <16E1010E4581B049ABC51D4975CEDB8802C09792@UKDCX001.uk.int.atosorigin.com>

The HTML Help documentation in the Windows installer has
"Release 2.4a0" in the footer for each page. I'm guessing
that this is a fairly minor tweak that might need adding
to the list of tasks for preparing a release...

Paul

From theller at python.net  Wed Sep 24 09:02:59 2003
From: theller at python.net (Thomas Heller)
Date: Wed Sep 24 09:03:04 2003
Subject: [Python-Dev] 2.3.1 - minor nit in HTML Help docs
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8802C09792@UKDCX001.uk.int.atosorigin.com>
	(Paul Moore's message of "Wed, 24 Sep 2003 13:41:42 +0100")
References: <16E1010E4581B049ABC51D4975CEDB8802C09792@UKDCX001.uk.int.atosorigin.com>
Message-ID: <8yoeibjw.fsf@python.net>

"Moore, Paul" <Paul.Moore@atosorigin.com> writes:

> The HTML Help documentation in the Windows installer has
> "Release 2.4a0" in the footer for each page.

The same is true for all the html docs online at
<http://www.python.org/doc/2.3.1/>, which isn't surprising.

Thomas


From anthony at interlink.com.au  Wed Sep 24 09:11:54 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Sep 24 09:13:53 2003
Subject: [Python-Dev] 2.3.1 - minor nit in HTML Help docs 
In-Reply-To: <8yoeibjw.fsf@python.net> 
Message-ID: <200309241311.h8ODBsS2029657@localhost.localdomain>


>>> Thomas Heller wrote
> "Moore, Paul" <Paul.Moore@atosorigin.com> writes:
> 
> > The HTML Help documentation in the Windows installer has
> > "Release 2.4a0" in the footer for each page.
> 
> The same is true for all the html docs online at
> <http://www.python.org/doc/2.3.1/>, which isn't surprising.

This would be part of the "Fred does Doc magic" section of the 
release PEPs ;)

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From barry at python.org  Wed Sep 24 09:20:27 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Sep 24 09:20:33 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <200309241233.h8OCXcbg028668@localhost.localdomain>
References: <200309241233.h8OCXcbg028668@localhost.localdomain>
Message-ID: <1064409627.1958.90.camel@anthem>

On Wed, 2003-09-24 at 08:33, Anthony Baxter wrote:

> D'oh! Since it's only a failure of the test code (not the module itself)
> I'll add a note to the bugs page in the test section.

+1

> Jeff Epler's fix seems appropriate. There's always 2.3.2 - since 2.4 is still
> a fair way off, it seems likely that a 2.3.2 sometime around January would be
> appropriate. Of course, if the bsddb3 weirdness does actually turn out to be
> something more serious, it might be sooner.

+1

> who-uses-bsddb-anyway-just-those-zope-guys-<wink>,

Just a bunch of anti-spam weirdos.
-Barry



From jeremy at zope.com  Wed Sep 24 09:39:48 2003
From: jeremy at zope.com (Jeremy Hylton)
Date: Wed Sep 24 09:40:44 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <1064404406.1958.71.camel@anthem>
References: <200309240715.h8O7Fbj4024972@localhost.localdomain>
	<1064404406.1958.71.camel@anthem>
Message-ID: <1064410788.2072.100.camel@localhost.localdomain>

On Wed, 2003-09-24 at 07:53, Barry Warsaw wrote:
> On Wed, 2003-09-24 at 03:15, Anthony Baxter wrote:
> > Last chance for feedback on python2.3.1 ... :)
> test test_mimetypes failed -- Traceback (most recent call last):
>   File "/tmp/Python-2.3.1/Lib/test/test_mimetypes.py", line 52, in
> test_guess_all_types
>     eq(all, ['.bat', '.c', '.h', '.ksh', '.pl', '.txt'])
>   File "/tmp/Python-2.3.1/Lib/unittest.py", line 302, in failUnlessEqual
>     raise self.failureException, \
> AssertionError: ['.asc', '.bat', '.c', '.h', '.ksh', '.pl', '.txt'] !=
> ['.bat', '.c', '.h', '.ksh', '.pl', '.txt']
> 

I've seen test_mimetypes fail frequently, depending on what order the
tests are run.  We worked on this a little for 2.3c1, but I didn't have
the patience to stick with it.  The problem is that the mimetypes module
initializes some global variables when it is first used.  test_mimetypes
avoids initializing the module, because it can't guess what extensions
will be known on a particular platform.  But some of the other tests
*use* mimetypes, so they cause it to be initialized.

> In the first run, test_queue also failed, with this output:
> 
> test_queue
> test test_queue failed -- blocking function '<bound method Queue.get of
> <Queue.Queue instance at 0x402471cc>>' appeared not to block

I see this from time-to-time, too.  The tests are timed based and
sometimes fail when there's not enough CPU time during the test
interval.

Jeremy



From skip at pobox.com  Wed Sep 24 10:54:42 2003
From: skip at pobox.com (Skip Montanaro)
Date: Wed Sep 24 10:55:02 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <1064409627.1958.90.camel@anthem>
References: <200309241233.h8OCXcbg028668@localhost.localdomain>
	<1064409627.1958.90.camel@anthem>
Message-ID: <16241.45106.296743.387428@montanaro.dyndns.org>


    >> who-uses-bsddb-anyway-just-those-zope-guys-<wink>,

    Barry> Just a bunch of anti-spam weirdos.

Many are deserting what at times appears to be a sinking bsddb ship in favor
of pickles...  For long-running SpamBayes apps like pop3proxy and the
Outlook plugin, pickles make sense if shared access to the scoring database
is not needed.

Skip


From python-kbutler at sabaydi.com  Wed Sep 24 11:12:38 2003
From: python-kbutler at sabaydi.com (Kevin J. Butler)
Date: Wed Sep 24 11:12:52 2003
Subject: [Python-Dev] Integration with Eclipse
In-Reply-To: <16240.36893.520437.974584@nick.uklinux.net>
References: <E1A0ErE-0003ue-8W@mail.python.org>	<3F6B11BA.6000008@sabaydi.com>	<16235.18128.422738.929390@nick.uklinux.net>	<3F6B533E.5000403@sabaydi.com>	<16236.28754.105629.565153@nick.uklinux.net>	<3F6C7B09.5060608@sabaydi.com>	<16236.51140.284372.526221@nick.uklinux.net>	<3F6CF076.30409@ocf.berkeley.edu>	<3F6D1AB7.1050608@sabaydi.com>
	<16240.36893.520437.974584@nick.uklinux.net>
Message-ID: <3F71B466.4040502@sabaydi.com>

Nick Roberts wrote:

>Kevin J. Butler writes:
> > ...
> >
> > I consider this more pdb-related than emacs-specific -
> > I'd expect that integration with, say, Eclipse would benefit from this 
> > enhancement as well, so I've been hoping to get some feedback from the 
> > list...
> > 
> > BTW, I've submitted the patch, it is 809887: 
> > http://sourceforge.net/tracker/index.php?func=detail&aid=809887&group_id=5470&atid=305470
> > 
> > kb
>
>The syntax of responses from pdb commands suggests to me that there is a fair
>amount of GDB in the underlying code. 
>
Nope - similarity of interface only.

pdb is pure python (and thus works in the Java implementation 'jython' 
as well), and I haven't heard that GDB has been incorporating python 
(yet  ;-)  ).

kb


From barry at python.org  Wed Sep 24 11:32:57 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Sep 24 11:33:07 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <16241.45106.296743.387428@montanaro.dyndns.org>
References: <200309241233.h8OCXcbg028668@localhost.localdomain>
	<1064409627.1958.90.camel@anthem>
	<16241.45106.296743.387428@montanaro.dyndns.org>
Message-ID: <1064417577.1082.0.camel@geddy>

On Wed, 2003-09-24 at 10:54, Skip Montanaro wrote:

> what at times appears to be a sinking bsddb ship

Dang.
-Barry



From anthony at interlink.com.au  Wed Sep 24 11:50:45 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Sep 24 11:53:27 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go 
In-Reply-To: <16241.45106.296743.387428@montanaro.dyndns.org> 
Message-ID: <200309241550.h8OFojlx031706@localhost.localdomain>


>>> Skip Montanaro wrote
> Many are deserting what at times appears to be a sinking bsddb ship in favor
> of pickles...  For long-running SpamBayes apps like pop3proxy and the
> Outlook plugin, pickles make sense if shared access to the scoring database
> is not needed.

:-(

I haven't been following the spambayes lists too closely. Are there concrete
problems with bsddb that are cropping up, or just a general wariness of it?

If there _is_ a problem with bsddb, it needs to be addressed. Too many things
depend on it.

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From tim.one at comcast.net  Wed Sep 24 11:54:05 2003
From: tim.one at comcast.net (Tim Peters)
Date: Wed Sep 24 11:54:14 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <1064404406.1958.71.camel@anthem>
Message-ID: <BIEJKCLHCIOIHAGOKOLHMEPNHEAA.tim.one@comcast.net>

[Barry Warsaw]
> ...
> In the first run, test_queue also failed, with this output:
>
> test_queue
> test test_queue failed -- blocking function '<bound method Queue.get
> of <Queue.Queue instance at 0x402471cc>>' appeared not to block

If there isn't already a bug report open on this, could you file one?  It's
a timing thing, and the test needs to be rewritten to avoid it (that doesn't
look easy, alas), or at least the message has to be changed to say up front
that sometimes it's just going to happen.


From anthony at interlink.com.au  Wed Sep 24 11:57:53 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Sep 24 11:59:39 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go 
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHMEPNHEAA.tim.one@comcast.net> 
Message-ID: <200309241557.h8OFvrNL031813@localhost.localdomain>


>>> "Tim Peters" wrote
> If there isn't already a bug report open on this, could you file one?  It's
> a timing thing, and the test needs to be rewritten to avoid it (that doesn't
> look easy, alas), or at least the message has to be changed to say up front
> that sometimes it's just going to happen.

If no-one does it soon, I plan to file a bunch of bug reports against the
various entries on 2.3.1's bugs.html page tomorrow.


-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From tim at zope.com  Wed Sep 24 12:01:04 2003
From: tim at zope.com (Tim Peters)
Date: Wed Sep 24 12:09:02 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go 
In-Reply-To: <200309241550.h8OFojlx031706@localhost.localdomain>
Message-ID: <BIEJKCLHCIOIHAGOKOLHGEPPHEAA.tim@zope.com>

Adding spambayes-dev in case anyone there has more reliable info.

[Anthony Baxter]
> I haven't been following the spambayes lists too closely. Are there
> concrete problems with bsddb that are cropping up, or just a general
> wariness of it?
>
> If there _is_ a problem with bsddb, it needs to be addressed. Too
> many things depend on it.

Reports of database corruption are common in spambayes.  At least one
knowledgable tester reported his problems went away after moving to a recent
Sleepycat release (4.1.25, IIRC).  I'm not sure any reports have come from
people using the Outlook client (I was happy using a dict, but switched all
3 of my Outlook classifiers to use Berkeley instead, *hoping* to provoke a
problem -- but no luck yet).  They seem to come from non-Outlook people
using Berkeley for the message info database.  Richie got a whittled down
threaded test that fails on Windows and Linux, and there's already a
(Python) bug report open on that; it's not thought to be relevant to how
spambayes uses Berkeley, though.


From skip at pobox.com  Wed Sep 24 12:41:28 2003
From: skip at pobox.com (Skip Montanaro)
Date: Wed Sep 24 12:45:29 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go 
In-Reply-To: <200309241550.h8OFojlx031706@localhost.localdomain>
References: <16241.45106.296743.387428@montanaro.dyndns.org>
	<200309241550.h8OFojlx031706@localhost.localdomain>
Message-ID: <16241.51512.389962.16881@montanaro.dyndns.org>


    >> Many are deserting what at times appears to be a sinking bsddb ship
    >> in favor of pickles...  For long-running SpamBayes apps like
    >> pop3proxy and the Outlook plugin, pickles make sense if shared access
    >> to the scoring database is not needed.

    Anthony> :-(

    Anthony> I haven't been following the spambayes lists too closely. Are
    Anthony> there concrete problems with bsddb that are cropping up, or
    Anthony> just a general wariness of it?

It's a combination of things.  First and foremost, using the pickle storage
is much faster assuming your database is not too huge and you have a
long-running spambayes app.  Second, people still use pre-2.3 Windows
versions of Python to run their spambayes apps.  By default, these people
get dumbdbm by default.  That's a disaster waiting to happen because of
storage size and speed concerns.  There were also bugs in dumbdbm (which
have since been fixed).

    Anthony> If there _is_ a problem with bsddb, it needs to be
    Anthony> addressed. Too many things depend on it.

I think it's more that bsddb is an innocent bystander in a drive-by
shooting, though some folks have been seeing DB_RUNRECOVERY (sp?) errors.
The anydbm-style access to Berkeley databases doesn't provide the necessary
locking for multiple process (or multiple thread?) access.

I agree if there are problems they should be fixed.  One step in the right
direction would probably be to create a default DBEnv for use by the
bsddb.*open() compatibility calls.  Also, the docs should mention threading
issues for people using the more featureful API so they don't unwittingly
shoot themselves in the foot.  (Maybe DBEnv=None should not be allowed in
open calls if threads are enabled?)

Skip

From skip at pobox.com  Wed Sep 24 13:15:14 2003
From: skip at pobox.com (Skip Montanaro)
Date: Wed Sep 24 13:32:29 2003
Subject: [Python-Dev] Python Glossary
Message-ID: <16241.53538.40615.455582@montanaro.dyndns.org>


Thanks to people who submitted content for the Python Glossary:

    http://manatee.mojam.com/python-glossary

I just completed a first pass at folding it into the Python tutorial.  It's
now a separate appendix to that document.  It will take awhile for the
changes to show up on the website and in distributions, but if you have
named CVS access to the Python distribution, you can get to it now.  People
with only anonymous CVS access or no CVS access at all will have to wait
awhile for SourceForge to update the server.  After a day or so it should be
available here:

    http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Doc/tut/

I would appreciate it if people could proofread it (if you're anxious to get
to work, mail me and I'll send you a copy of the glossary.tex file).  I made
some changes to the definitions, but was focused more on getting the content
into the tutorial than on wordsmithing.

As a side-effect, the tutorial now also has an index.  All but one or two of
the entries it contains are from the glossary.  The entire tutorial needs to
be "indexified".  If you would like to help with that, please feel free.  In
any case, any changes should be submitted as a documentation bug or patch
(if you have some proposed context diffs) and assigned to me (SF username:
montanaro).

Please don't feel shy about adding new entries to the existing Wiki-based
glossary either.  I will notice updates and migrate new or corrected terms
over as I have time.  I don't plan to dismantle it, at least not for the
forseeable future.  At some point I may migrate it to the Python.org Wiki
though.

Skip





From DavidA at ActiveState.com  Wed Sep 24 13:32:36 2003
From: DavidA at ActiveState.com (David Ascher)
Date: Wed Sep 24 13:54:15 2003
Subject: [Python-Dev] Announcement: ActiveState acquired by Sophos
Message-ID: <3F71D534.3010101@ActiveState.com>

To the Python community --

As some of you have heard by now, ActiveState was acquired
yesterday by Sophos Plc, a UK-based global anti-virus company, as
described in this press release:

	http://www.activestate.com/Corporate/Communications/Releases/Press1064364490.html

and in this open letter to ActiveState customers:

	http://www.activestate.com/corporate/letter/.

I figure that many of you who know about ActiveState's background in the
open source community probably have questions regarding what this means
for ActiveState's future, and the future of Python at ActiveState.

In short, there is no downside, only great opportunities.

What will not change:

    - Our line of tools, language distributions and enterprise
      support for open source programming languages will continue
      to be developed and evolve with customer needs.
    - Our involvement in the open source language communities
      will continue as before.
    - All of the ActiveState staff is retained in the acquisition.
    - The Vancouver, BC office is poised to expand, not shrink.
    - Email addresses, phone numbers, etc. will continue to
      function as before.

What will change:

    - ActiveState will now have a broader global impact and access
      to the resources of a larger company with a great deal of
      experience helping businesses secure their networks.
    - Our PureMessage anti-spam product will be integrated with
      the Sophos MailMonitor product and be sold as Sophos
      PureMessage, while the rest of our product line will
      still be sold under the ActiveState banner.
    - ActiveState is now a wholly-owned subsidiary of Sophos,
      with a focused mission to providing tools and services
      for users of open source programming languages.

I want to point out to this audience in particular that the acquisition
of ActiveState is a success story for open source programming
technologies.  It is in part thanks to our use of programming languages
like Perl, Python and Tcl _and_ to our involvement in the programming
communities around these languages that we've been able to quickly build
and sell enterprise-grade software such as PureMessage, as well as very
successful programming tools like Komodo, the Perl Dev Kit, etc.  I'm
glad to say that Sophos appreciates this and is looking to expand the
use of these technologies in the larger company, not shrink it.  Score
one for the good guys!

While I have to get back to work (on the usual things, like adding Python features to Komodo, help plan the next release of ActivePython, help plan for PyCon 2004, iron out legal issues for the PSF and a few other things like that), I'll be monitoring these lists, and welcome email questions as well -- send them to me at DavidA@ActiveState.com.

Cheers, and see you all at PyCon 2004!

-- David Ascher
   ActiveState, a division of Sophos





From Jack.Jansen at oratrix.com  Wed Sep 24 08:43:09 2003
From: Jack.Jansen at oratrix.com (Jack Jansen)
Date: Wed Sep 24 15:32:13 2003
Subject: [Python-Dev] RELEASED Python 2.3.1 
In-Reply-To: <200309240815.h8O8FIfo026231@localhost.localdomain>
Message-ID: <A731596A-EE8C-11D7-BDBC-003065517236@oratrix.com>


On woensdag, sep 24, 2003, at 10:15 Europe/Amsterdam, Anthony Baxter 
wrote:

>
> Oh yeah - please hold off on the release23-maint branch for a bit,
> in case Jack wants to cut a 2.3.1 release for the Mac... Jack?

I don't know whether I'll find the time, give me until the end of the 
week
to decide, please.

And off on a tangent: I think we need a python-dev-announce mailing 
list or
something of the sort. I've been busy with Real Work the last month, 
and I
only skim python-dev when I have time, so today is the first time I hear
about 2.3.1 (and, being on a train without internet-access right now
I can't even test it:-).

I'm not happy with this. For one thing, I haven't seen any indication 
that anyone
tried a framework build on MacOSX, and how such a build interacts with 
the
pre-installed 2.3 on Panther. Even though I wouldn't have had time to do
any fixes for MacOSX that are on my plate I would at least have tested
that 2.3.1 doesn't break anything...

I wouldn't mind a checklist of things that need to be tested (plus 
preferred
people responsible for them) before a release goes out.
--
- Jack Jansen        <Jack.Jansen@oratrix.com>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- Emma 
Goldman -


From barry at python.org  Wed Sep 24 15:47:38 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Sep 24 15:47:40 2003
Subject: [Python-Dev] RELEASED Python 2.3.1
In-Reply-To: <A731596A-EE8C-11D7-BDBC-003065517236@oratrix.com>
References: <A731596A-EE8C-11D7-BDBC-003065517236@oratrix.com>
Message-ID: <1064432857.1082.20.camel@geddy>

On Wed, 2003-09-24 at 08:43, Jack Jansen wrote:

> I wouldn't mind a checklist of things that need to be tested (plus 
> preferred
> people responsible for them) before a release goes out.

IMO, this needs to be added to PEP 102 and there needs to be a list of
people who will give a go, no-go on a new release.  There needs to be
timeouts though because we're all way too busy and it may just be that a
release needs to go out before a person on the critical path has a
chance to sign off on it.

-Barry



From zeddicus at satokar.com  Wed Sep 24 16:42:13 2003
From: zeddicus at satokar.com (Michael Bartl)
Date: Wed Sep 24 16:42:18 2003
Subject: [Python-Dev] Python in ATC (Air Traffic control)
Message-ID: <20030924204212.GA12794@satokar>

Hi all together!

Just to stop repeating myself I thought I'd post this to the python-dev
list.

In a foregoing post to the python-dev list I mentioned the use of python
in the field of Air Traffic control. It seems that many people are
interested in a description of where exactly python is used in this
high-demanding field, may it be for marketing or personal reasons. 

I'm going to write a "python success story" about it, if my
boss has no problem with that (which I assume).

I can't give you any other information except to wait until the
beginning of next week, when I will probably have finished the content
:o)

regards,
    Michael Bartl

From fperez at colorado.edu  Wed Sep 24 16:47:50 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Wed Sep 24 16:47:54 2003
Subject: [Python-Dev] Hi,
 and a question about filename completion via readline in Python 2.3
Message-ID: <3F7202F6.9050508@colorado.edu>

Hi all,

since this is my first post here, I figured I'd briefly introduce myself.  I'm 
the lead developer of ipython (http://ipython.scipy.org), a shell for 
interactive python work.  I've been a fan (and big advocate) of python over 
the last 2 years, mainly for scientific computing.  For those interested, you 
can find a brief overview of ipython at:

http://ipython.scipy.org/misc/ipython_scipy03.pdf

Besides thanking the python team for this fantastic language, I have a 
question concerning some changes apparently made to the C part of readline 
support for  python 2.3.  If I understood correctly this thread:

http://mail.python.org/pipermail/python-list/2002-February/085063.html

as of python 2.3, readline will NOT default anymore to providing filename 
completions when nothing matches in the available namespaces.  There is 
mention in that thread of making it a configurable parameter, but I failed to 
find any specifics on how to do that.

While perhaps I'm missing something, so far (after getting this reported as an 
ipython bug from 2.3 users) I've failed to be able to recover filename 
completion under 2.3.

While I'm sure you found good reasons for this change, under some environments 
it turns out to be really quite nasty.  One of the strengths of ipython is its 
tight integration with the underlying system environment, and its users rely 
constantly on filename/directory completion for navigating their filesystem as 
  they work.  I personally (and many others) use ipython as my main python 
shell, and I fell it provides a superior working environment to anything else 
I've seen out there (especially if you need to combine python work with 
manipulating underlying files).

So ultimately my concerns are:

- is there currently a way to re-enable readline filename completion, as 
suggested in that thread?

- if not, can such a change be made?

I hope if some of you test ipython, you'll find out that actually that kind of 
integration with the filesystem can make a very strong argument for 
python-based shells as fantastic tools for many data-driven environments.  I 
have already knowledge of several projects which are either using or 
considering ipython as the underlying shell for scientific data analysis 
environments, and this change will be a major setback for their usability.

Again, many thanks for all your hard work on python.

Best regards,

Fernando.


From zeddicus at satokar.com  Wed Sep 24 16:47:35 2003
From: zeddicus at satokar.com (Michael Bartl)
Date: Wed Sep 24 16:52:20 2003
Subject: [Python-Dev] PEP on patch processing
Message-ID: <20030924204734.GB12794@satokar>

Hi!

Martin v. L?wis has previously posted a guide on how to successfully
process patches on this mailinglist.

I just wanted to ask if there is interest in a PEP about that topic,
because I couldn't find any relevant information on python.org.

If you do not agree with me I still would recommend to put it on the
developers pages on python.org.

... just an idea

regards,
    Michael Bartl

From aahz at pythoncraft.com  Wed Sep 24 16:57:55 2003
From: aahz at pythoncraft.com (Aahz)
Date: Wed Sep 24 16:57:58 2003
Subject: [Python-Dev] PEP on patch processing
In-Reply-To: <20030924204734.GB12794@satokar>
References: <20030924204734.GB12794@satokar>
Message-ID: <20030924205755.GA26474@panix.com>

On Wed, Sep 24, 2003, Michael Bartl wrote:
> 
> Martin v. L?wis has previously posted a guide on how to successfully
> process patches on this mailinglist.
> 
> I just wanted to ask if there is interest in a PEP about that topic,
> because I couldn't find any relevant information on python.org.

Given the existence of PEP 3, this sounds like an excellent idea!
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

From jason at mastaler.com  Wed Sep 24 18:05:40 2003
From: jason at mastaler.com (Jason R. Mastaler)
Date: Wed Sep 24 18:05:55 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
References: <200309240802.h8O82mqF026019@localhost.localdomain>
Message-ID: <m2zngtx2ob.fsf@deacon-blues.mid.mastaler.com>

Oops.  See python.org/sf/812007

Python 2.3.1 (#1, Sep 24 2003, 15:55:24) 
[GCC 3.2 20020903 (Red Hat Linux 8.0 3.2-7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from os import fsync
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ImportError: cannot import name fsync
>>> 



From anthony at interlink.com.au  Wed Sep 24 23:20:10 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Sep 24 23:21:58 2003
Subject: [Python-Dev] RELEASED Python 2.3.1 
In-Reply-To: <A731596A-EE8C-11D7-BDBC-003065517236@oratrix.com> 
Message-ID: <200309250320.h8P3KAxs007608@localhost.localdomain>


Mea maxima culpa. I'll add a new section to the start of PEP101 and PEP102
about communicating with the appropriate people directly.

We had a bunch of testers try out the release candidates on various Mac OS X
releases - they reported no breakages. There's also been few/no checkins
to the Mac-specific chunks of the tree for the release23-maint branch.

I'm happy to store up checkins for the 23 branch for a couple of weeks to 
give you time to do a release if you so wish (consider it penance for my
forgetting to email you).

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From anthony at interlink.com.au  Wed Sep 24 23:21:43 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Sep 24 23:23:27 2003
Subject: [Python-Dev] RELEASED Python 2.3.1 
In-Reply-To: <1064432857.1082.20.camel@geddy> 
Message-ID: <200309250321.h8P3Lh7Z007641@localhost.localdomain>


>>> Barry Warsaw wrote
> IMO, this needs to be added to PEP 102 and there needs to be a list of
> people who will give a go, no-go on a new release.  There needs to be
> timeouts though because we're all way too busy and it may just be that a
> release needs to go out before a person on the critical path has a
> chance to sign off on it.

I'm re-doing PEP 101 and 102 to change the various names to roles. I'll add
a section to the top saying "You need to get a person signed up and agreed 
for each of the following roles. The people who have done each role in the
past are..."

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From anthony at interlink.com.au  Wed Sep 24 23:40:23 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Sep 24 23:42:04 2003
Subject: [Python-Dev] a small data point
Message-ID: <200309250340.h8P3eNrI008096@localhost.localdomain>


Of the people who bothered to email or otherwise contact me about 2.3.1,
the overwhelming response has been "thank you for not adding any new 
features". 

just a small data point for the features-vs-stability issue.

Anthony

From barry at python.org  Thu Sep 25 00:35:20 2003
From: barry at python.org (Barry Warsaw)
Date: Thu Sep 25 00:35:28 2003
Subject: [Python-Dev] RELEASED Python 2.3.1
In-Reply-To: <200309250321.h8P3Lh7Z007641@localhost.localdomain>
References: <200309250321.h8P3Lh7Z007641@localhost.localdomain>
Message-ID: <1064464520.12343.42.camel@anthem>

On Wed, 2003-09-24 at 23:21, Anthony Baxter wrote:

> I'm re-doing PEP 101 and 102 to change the various names to roles. I'll add
> a section to the top saying "You need to get a person signed up and agreed 
> for each of the following roles. The people who have done each role in the
> past are..."

+1
-B



From fdrake at acm.org  Thu Sep 25 01:52:00 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu Sep 25 01:52:14 2003
Subject: [Python-Dev] RELEASED Python 2.3.1 
In-Reply-To: <200309250321.h8P3Lh7Z007641@localhost.localdomain>
References: <1064432857.1082.20.camel@geddy>
	<200309250321.h8P3Lh7Z007641@localhost.localdomain>
Message-ID: <16242.33409.339583.936076@grendel.zope.com>


Anthony Baxter writes:
 > I'm re-doing PEP 101 and 102 to change the various names to roles. I'll add
 > a section to the top saying "You need to get a person signed up and agreed 
 > for each of the following roles. The people who have done each role in the
 > past are..."

Make sure you only list the people willing to do it again or at least
discuss the role with new volunteers.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From fdrake at acm.org  Thu Sep 25 02:12:30 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu Sep 25 02:12:42 2003
Subject: [Python-Dev] 2.3.1 - minor nit in HTML Help docs
In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8802C09792@UKDCX001.uk.int.atosorigin.com>
References: <16E1010E4581B049ABC51D4975CEDB8802C09792@UKDCX001.uk.int.atosorigin.com>
Message-ID: <16242.34638.614846.862608@grendel.zope.com>


Moore, Paul writes:
 > The HTML Help documentation in the Windows installer has
 > "Release 2.4a0" in the footer for each page. I'm guessing
 > that this is a fairly minor tweak that might need adding
 > to the list of tasks for preparing a release...

I'm pretty sure I know how this happened, and is actually a (fairly
minor) accident of how I built the docs this time combined with the
slow growth of the formatting tools from being a pile of scripts very
specific to building the Python documentation to tools that can format
the types of documents used in the Python documentation.

What I did this time that was new is this:  I used the formatting
tools from the head of the trunk instead of from the branch.  They
picked up all the right content files, but used the wrong "shared"
files from the Doc/texinputs/ directory.  (The trunk files were used
instead of the branch files.)

This is a strong indication that the conflation of content and style
files in Doc/texinputs/ is a bad thing; style should be provided by
the tool, and additional directories of content should be
configurable for the specific content.  For TeX-based tools,
additional directories of TeX source files are specified by the
TEXINPUTS environment variable, but the Doc/tools/mkhowto script
simply clobbers that currently.  There are a bunch of conventions
associated with path configuration in TeX (per the kpathsea library),
but implementing this for mkhowto shouldn't take long.

I expect to be able to handle this over the next few days, after which
I'll generate new documentation packages for Python 2.3.1.  The fixes
will be made on the trunk (since that's where the formatting tools are
being maintained); some changes to the Makefile and the organization
of the files in Doc/texinputs/ on the release23-maint branch will be
needed; not sure when these should be committed (in particular,
before/after a MacPython release).

Once I've got this done, I'll propose a more managable approach to
dealing with maintenance of the formatting tools.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From mwh at python.net  Thu Sep 25 05:54:50 2003
From: mwh at python.net (Michael Hudson)
Date: Thu Sep 25 05:54:12 2003
Subject: [Python-Dev] Hi, and a question about filename completion via
	readline in Python 2.3
In-Reply-To: <3F7202F6.9050508@colorado.edu> (Fernando Perez's message of
	"Wed, 24 Sep 2003 14:47:50 -0600")
References: <3F7202F6.9050508@colorado.edu>
Message-ID: <2mbrt942hh.fsf@starship.python.net>

Fernando Perez <fperez@colorado.edu> writes:

> Besides thanking the python team for this fantastic language, I have a
> question concerning some changes apparently made to the C part of
> readline support for  python 2.3.  If I understood correctly this
> thread:
>
> http://mail.python.org/pipermail/python-list/2002-February/085063.html
>
> as of python 2.3, readline will NOT default anymore to providing
> filename completions when nothing matches in the available namespaces.
> There is mention in that thread of making it a configurable parameter,
> but I failed to find any specifics on how to do that.
>
> While perhaps I'm missing something, so far (after getting this
> reported as an ipython bug from 2.3 users) I've failed to be able to
> recover filename completion under 2.3.
>
> While I'm sure you found good reasons for this change, under some
> environments it turns out to be really quite nasty.  One of the
> strengths of ipython is its tight integration with the underlying
> system environment, and its users rely constantly on
> filename/directory completion for navigating their filesystem as they
> work.  I personally (and many others) use ipython as my main python
> shell, and I fell it provides a superior working environment to
> anything else I've seen out there (especially if you need to combine
> python work with manipulating underlying files).

Bet pyrepl still canes it for multi-line work :-)

> So ultimately my concerns are:
>
> - is there currently a way to re-enable readline filename completion,
>   as suggested in that thread?

Yes.  Do it yourself, in your completer function.  Is it really that
hard?  I would have thought os.path + glob would make it a five line
or so excercise.

> - if not, can such a change be made?

Well, I'm not going to <wink>.  Feel free to submit a patch.

Cheers,
mwh

-- 
  You owe the Oracle a star-spangled dunce cap.
                 -- Internet Oracularity Internet Oracularity #1299-08

From anthony at interlink.com.au  Thu Sep 25 02:17:41 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu Sep 25 06:04:04 2003
Subject: [Python-Dev] RELEASED Python 2.3.1 
In-Reply-To: <16242.33409.339583.936076@grendel.zope.com> 
Message-ID: <200309250617.h8P6HgsA010531@localhost.localdomain>


>>> "Fred L. Drake, Jr." wrote
> Make sure you only list the people willing to do it again or at least
> discuss the role with new volunteers.  ;-)

Bah. That's no fun.

I'm-sure-Barry-said-he'd-build-the-windows-installer-<wink>,
Anthony

From mwh at python.net  Thu Sep 25 06:20:52 2003
From: mwh at python.net (Michael Hudson)
Date: Thu Sep 25 06:20:14 2003
Subject: [Python-Dev] RELEASED Python 2.3.1
In-Reply-To: <200309250320.h8P3KAxs007608@localhost.localdomain> (Anthony
	Baxter's message of "Thu, 25 Sep 2003 13:20:10 +1000")
References: <200309250320.h8P3KAxs007608@localhost.localdomain>
Message-ID: <2m7k3x41a3.fsf@starship.python.net>

Anthony Baxter <anthony@interlink.com.au> writes:

> Mea maxima culpa. I'll add a new section to the start of PEP101 and PEP102
> about communicating with the appropriate people directly.
>
> We had a bunch of testers try out the release candidates on various Mac OS X
> releases - they reported no breakages. There's also been few/no checkins
> to the Mac-specific chunks of the tree for the release23-maint branch.
> 
> I'm happy to store up checkins for the 23 branch for a couple of weeks to 
> give you time to do a release if you so wish (consider it penance for my
> forgetting to email you).

I think in future we should probably do the "2.3.2c1", wait a week, do
"2.3.2" thing.  It's more work, but there have been enough nits that I
think it would have been worthwhile (readline vs. no threads, fsync
and the docs issue spring to mind).

Cheers,
mwh

-- 
81. In computing, turning the obvious into the useful is a living
    definition of the word "frustration".
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html

From theller at python.net  Thu Sep 25 07:22:41 2003
From: theller at python.net (Thomas Heller)
Date: Thu Sep 25 07:22:52 2003
Subject: [Python-Dev] RELEASED Python 2.3.1
In-Reply-To: <2m7k3x41a3.fsf@starship.python.net> (Michael Hudson's message
	of "Thu, 25 Sep 2003 11:20:52 +0100")
References: <200309250320.h8P3KAxs007608@localhost.localdomain>
	<2m7k3x41a3.fsf@starship.python.net>
Message-ID: <y8wddse6.fsf@python.net>

Michael Hudson <mwh@python.net> writes:

> I think in future we should probably do the "2.3.2c1", wait a week, do
> "2.3.2" thing.  It's more work, but there have been enough nits that I
> think it would have been worthwhile (readline vs. no threads, fsync
> and the docs issue spring to mind).

+1

Thomas


From anthony at interlink.com.au  Thu Sep 25 07:43:46 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu Sep 25 07:45:31 2003
Subject: [Python-Dev] RELEASED Python 2.3.1 
In-Reply-To: <y8wddse6.fsf@python.net> 
Message-ID: <200309251143.h8PBhlKf019379@localhost.localdomain>


>>> Thomas Heller wrote
> Michael Hudson <mwh@python.net> writes:
> 
> > I think in future we should probably do the "2.3.2c1", wait a week, do
> > "2.3.2" thing.  It's more work, but there have been enough nits that I
> > think it would have been worthwhile (readline vs. no threads, fsync
> > and the docs issue spring to mind).
> 
> +1

+1 from me, also. I'll make sure to put words to the effect in the PEPs.


-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From zeddicus at satokar.com  Thu Sep 25 12:24:05 2003
From: zeddicus at satokar.com (Michael Bartl)
Date: Thu Sep 25 12:24:09 2003
Subject: [Python-Dev] PEP draft - Guidelines to processing patches
Message-ID: <20030925162405.GA3858@satokar>

Hi all!

I've written a draft for a PEP - Guidlines for processing patches.
Please have a look at it and give me some feedback. Private mail is
prefered so not to spam the list ;)

Please note that credits for the current content go to Martin v. L?wis.

regards,
    Michael Bartl
-------------- next part --------------
PEP: ---
Title: Guidelines for Processing Patches
Version: ---
Author: gnosticus@gmx.at (Michael Bartl)
Status: Draft
Type: Informational
Created: 25-Sep-2003
Post-History:

Introduction

    This PEP contains guidelines how to process patches for the
    Python project on SourceForge[1]. Still to be done is to collect
    a list of people willing to process patches and their areas of
    expertise.


Guidelines

    1.  Try to classify the patch somehow, indicating what most likely
        the problem is for the patch not being reviewed/accepted.

    1.1 The patch might introduce a questionable feature, and nobody has
        dared to reject it yet, in order to not offend the submitter. In
        that case, voice an opinion (and record it in the patches tracker)
        on whether you are in favour of or opposed to the proposed feature
        (ideally giving a rationale on why you are). If there are enough
        voices showing opposition, the submitter might withdraw the patch.
        If there are enough voices in favour, a core developer might
        accept it.

    1.2 The patch might be incomplete. Try to contact the submitter for a
        complete version. If the submitter is unreachable or does not want
        to finish the work, either complete it yourself, or suggest
        rejection of the patch.

    1.3 The patch might be so complex/involved that a quick review does not
        reveal whether it is correct. Review the patch excessively:
        Perform tests, study all possible code paths to uncover cases that
        have been ignored. When done, record in the patch what kind of review
        you have done, and either indicate the problems you have found, or
        recommend acceptance.

    2.  The patch might be in an area where the "core" developers have
        little expertise; you often find that trivial patches are ignored
        just because everybody thought not to be an expert in the subject.
        Perform a quick review of the patch to find out whether it meets the
        formal criteria (completeness), and whether it is perhaps obviously
        correct or obviously incorrect. If not, come up with a strategy
        to obtain a decision on the patch:

    2.1 Become an expert yourself in the subject area, and recommend
        acceptance or rejection afterwards. This is a both time consuming
        and rewarding effort.

    2.2 Find an expert and have it review the patch (have a look at the
        people posting to python-dev more than occassionally).

    2.3 Ask the submitter to clarify all questionable aspects of the
        patch, i.e. have him explain the patch to you. If the patch
        is for a little-used feature (e.g. for an obscure platform),
        it might be acceptable to incorporate incorrect patches, since
        nobody will notice, anyway - it might be enough that the submitter
        believes the patch is correct.

References

    [1] http://sourceforge.net/projects/python
From fperez at colorado.edu  Thu Sep 25 13:43:33 2003
From: fperez at colorado.edu (Fernando Perez)
Date: Thu Sep 25 13:43:37 2003
Subject: [Python-Dev] Hi, and a question about filename completion via
	readline in Python 2.3
In-Reply-To: <2mbrt942hh.fsf@starship.python.net>
References: <3F7202F6.9050508@colorado.edu>
	<2mbrt942hh.fsf@starship.python.net>
Message-ID: <3F732945.8020101@colorado.edu>

Michael Hudson wrote:
> Fernando Perez <fperez@colorado.edu> writes:

>>work.  I personally (and many others) use ipython as my main python
>>shell, and I fell it provides a superior working environment to
>>anything else I've seen out there (especially if you need to combine
>>python work with manipulating underlying files).
> 
> 
> Bet pyrepl still canes it for multi-line work :-)

It does for on-site multi-line work (though ipython now has an @edit command 
which is a reasonable substitute).  But ipython has a ton more features that 
make it a fuller environment.  As I said before (long ago), it would be cool 
if we could integrate the two a bit more.  Care to give it a go (off-list)?

>>So ultimately my concerns are:
>>
>>- is there currently a way to re-enable readline filename completion,
>>  as suggested in that thread?
> 
> 
> Yes.  Do it yourself, in your completer function.  Is it really that
> hard?  I would have thought os.path + glob would make it a five line
> or so excercise.

Oh well, I'll do it myself then.  I'm a bit bummed to see functionality 
disappear in a release, but I can live with it.  I have more and more 2.3 
users who aren't too happy about this change, so I'll just go ahead and 
implement it.

Cheers,

f.


From mwh at python.net  Thu Sep 25 13:50:10 2003
From: mwh at python.net (Michael Hudson)
Date: Thu Sep 25 13:49:36 2003
Subject: [Python-Dev] Hi, and a question about filename completion via
	readline in Python 2.3
In-Reply-To: <3F732945.8020101@colorado.edu> (Fernando Perez's message of
	"Thu, 25 Sep 2003 11:43:33 -0600")
References: <3F7202F6.9050508@colorado.edu>
	<2mbrt942hh.fsf@starship.python.net> <3F732945.8020101@colorado.edu>
Message-ID: <2my8wc3gh9.fsf@starship.python.net>

Fernando Perez <fperez@colorado.edu> writes:

> Michael Hudson wrote:
>> Fernando Perez <fperez@colorado.edu> writes:
>
>>>work.  I personally (and many others) use ipython as my main python
>>>shell, and I fell it provides a superior working environment to
>>>anything else I've seen out there (especially if you need to combine
>>>python work with manipulating underlying files).
>> Bet pyrepl still canes it for multi-line work :-)
>
> It does for on-site multi-line work (though ipython now has an @edit
> command which is a reasonable substitute).  But ipython has a ton more
> features that make it a fuller environment.  As I said before (long
> ago), it would be cool if we could integrate the two a bit more.  Care
> to give it a go (off-list)?

<sigh> not right now.  Too much else to do.  But one day, I promise!

>>>So ultimately my concerns are:
>>>
>>>- is there currently a way to re-enable readline filename completion,
>>>  as suggested in that thread?
>> Yes.  Do it yourself, in your completer function.  Is it really that
>> hard?  I would have thought os.path + glob would make it a five line
>> or so excercise.
>
> Oh well, I'll do it myself then.  I'm a bit bummed to see
> functionality disappear in a release, but I can live with it.

The problem before was that there was *no way* of preventing the files
from appearing in the completion list.  I don't think their appearence
there was ever intended.  I just don't know enough about readline to
know if implementing a 

    readline.set_filenames_in_completion()

function is feasible.  But that won't appease your users who have 2.3,
of course.

> I have more and more 2.3 users who aren't too happy about this
> change, so I'll just go ahead and implement it.

Didn't *any* of them try the betas?  This change is plenty old.

Cheers,
mwh

-- 
  The only problem with Microsoft is they just have no taste.
              -- Steve Jobs, (From _Triumph of the Nerds_ PBS special)
                         and quoted by Aahz Maruch on comp.lang.python

From jason at mastaler.com  Thu Sep 25 14:37:08 2003
From: jason at mastaler.com (Jason R. Mastaler)
Date: Thu Sep 25 14:37:14 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
References: <200309250320.h8P3KAxs007608@localhost.localdomain>
	<2m7k3x41a3.fsf@starship.python.net>
Message-ID: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>

Michael Hudson <mwh@python.net> writes:

> I think in future we should probably do the "2.3.2c1", wait a week,
> do "2.3.2" thing.  It's more work, but there have been enough nits
> that I think it would have been worthwhile (readline vs. no threads,
> fsync and the docs issue spring to mind).

Will there be a 2.3.2 release, or some other sort of bugfix release to
address the problems in 2.3.1?  I think the os.fsync() problem is
serious enough to warrant revoking 2.3.1 and making a new release
to replace it rather quickly.


From Jack.Jansen at cwi.nl  Thu Sep 25 16:10:37 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Thu Sep 25 16:10:47 2003
Subject: [Python-Dev] RELEASED Python 2.3.1 
In-Reply-To: <200309250320.h8P3KAxs007608@localhost.localdomain>
References: <200309250320.h8P3KAxs007608@localhost.localdomain>
Message-ID: <53F0B466-EF94-11D7-A80F-000A27B19B96@cwi.nl>


On 25-sep-03, at 5:20, Anthony Baxter wrote:

>
> Mea maxima culpa. I'll add a new section to the start of PEP101 and 
> PEP102
> about communicating with the appropriate people directly.

Don't worry: I assume there's no harm done, and otherwise I'll just 
lobby for
a quick 2.3.2.

> We had a bunch of testers try out the release candidates on various 
> Mac OS X
> releases - they reported no breakages. There's also been few/no 
> checkins
> to the Mac-specific chunks of the tree for the release23-maint branch.

There's a couple of version numbers that would have been nice to update,
in the various .plist files. In other words, if you build 2.3.1 from 
source
with a framework build then "Show Info" on the applications will show 
them
to be 2.3, not 2.3.1. Same for the Apple Help Book that is built in to 
the IDE.

> I'm happy to store up checkins for the 23 branch for a couple of weeks 
> to
> give you time to do a release if you so wish (consider it penance for 
> my
> forgetting to email you).

No, don't worry. I'm going to skip 2.3.1 for MacPython-OS9. I'll catch 
up again
with 2.3.2.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From martin at v.loewis.de  Thu Sep 25 18:47:35 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Sep 25 18:48:13 2003
Subject: [Python-Dev] RELEASED Python 2.3.1
In-Reply-To: <53F0B466-EF94-11D7-A80F-000A27B19B96@cwi.nl>
References: <200309250320.h8P3KAxs007608@localhost.localdomain>
	<53F0B466-EF94-11D7-A80F-000A27B19B96@cwi.nl>
Message-ID: <m3eky4v62g.fsf@mira.informatik.hu-berlin.de>

Jack Jansen <Jack.Jansen@cwi.nl> writes:

> There's a couple of version numbers that would have been nice to
> update, in the various .plist files. In other words, if you build
> 2.3.1 from source with a framework build then "Show Info" on the
> applications will show them to be 2.3, not 2.3.1. Same for the Apple
> Help Book that is built in to the IDE.

As a procedural note, I would suggest that suggest changes are done
immediately *after* the previous release, instead of shortly before
the next one. The release branch should be releasable at any time,
without having to ask anybody for permission. What gets done is done,
what doesn't get done has to wait for the next release. The less
coordination, the better.

Regards,
Martin


From martin at v.loewis.de  Thu Sep 25 18:49:34 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Sep 25 18:49:40 2003
Subject: [Python-Dev] PEP on patch processing
In-Reply-To: <20030924204734.GB12794@satokar>
References: <20030924204734.GB12794@satokar>
Message-ID: <m3ad8sv5z5.fsf@mira.informatik.hu-berlin.de>

Michael Bartl <zeddicus@satokar.com> writes:

> I just wanted to ask if there is interest in a PEP about that topic,
> because I couldn't find any relevant information on python.org.

A PEP would not be appropriate: it would indicate the need for BDFL
pronouncement on the procedures presented, or consensus within some
group, when I think neither consensus nor BDFL pronouncement is
necessary. Instead, a plain web page on python.org would be
sufficient.

Regards,
Martin


From martin at v.loewis.de  Thu Sep 25 18:50:47 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Sep 25 18:50:56 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <m2zngtx2ob.fsf@deacon-blues.mid.mastaler.com>
References: <200309240802.h8O82mqF026019@localhost.localdomain>
	<m2zngtx2ob.fsf@deacon-blues.mid.mastaler.com>
Message-ID: <m365jgv5x4.fsf@mira.informatik.hu-berlin.de>

"Jason R. Mastaler" <jason@mastaler.com> writes:

> Python 2.3.1 (#1, Sep 24 2003, 15:55:24) 
> [GCC 3.2 20020903 (Red Hat Linux 8.0 3.2-7)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from os import fsync
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> ImportError: cannot import name fsync
> >>> 

Why is that a bug? Your system just does not have a declared fsync
function.

Regards,
Martin


From aahz at pythoncraft.com  Thu Sep 25 19:46:31 2003
From: aahz at pythoncraft.com (Aahz)
Date: Thu Sep 25 19:46:35 2003
Subject: [Python-Dev] PEP on patch processing
In-Reply-To: <m3ad8sv5z5.fsf@mira.informatik.hu-berlin.de>
References: <20030924204734.GB12794@satokar>
	<m3ad8sv5z5.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20030925234631.GC5808@panix.com>

On Fri, Sep 26, 2003, Martin v. L?wis wrote:
> Michael Bartl <zeddicus@satokar.com> writes:
>> 
>> I just wanted to ask if there is interest in a PEP about that topic,
>> because I couldn't find any relevant information on python.org.
> 
> A PEP would not be appropriate: it would indicate the need for BDFL
> pronouncement on the procedures presented, or consensus within some
> group, when I think neither consensus nor BDFL pronouncement is
> necessary. Instead, a plain web page on python.org would be
> sufficient.

Hmmmm....  While I'd agree with you in the absence of anything else, it
seems to me that the existence of PEP 3 indicates that this subject is
PEP-worthy.  Unless you want to argue that PEP 3 should be removed?...
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

From pycon at python.org  Thu Sep 25 20:23:55 2003
From: pycon at python.org (PyCon Chair)
Date: Thu Sep 25 20:24:03 2003
Subject: [Python-Dev] PyCon Announcement: Mitch Kapor is opening keynote
	speaker
Message-ID: <20030926002355.GA10224@panix.com>

[Apologies to people who see this twice; there was a glitch in the
original sending.]


Mitch Kapor to Keynote PyCon DC 2004
------------------------------------
Washington, DC, September 24, 2003

The Python Software Foundation announced today that Mitch Kapor will be
the opening keynote speaker for their second US Python community
conference next March.

Mr. Kapor is well known as a co-founder of Lotus Development Corporation
and as a co-founder and the first chairman of the San Francisco-based
Electronic Freedom Foundation, and has been an active entrepreneur in
information technology for many years.

More recently he founded the Open Software Applications Foundation
(OSAF - see http://www.osafoundation.org/). The OSAF is using Python to
build Chandler, an open source Personal Information Manager.

PyCon is a community-oriented conference for developers of Python
applications and for those with an interest in the development of the
Python programming language.  It offers the opportunity to learn about
significant advances in Python's uses and development community, to
participate in a programming sprint with some of the leading minds in
the Open Source community, and to meet fellow developers from around the
world.  The organizers work to make the conference affordable and
accessible to all.

Pycon DC 2004 will be held March 24-26, 2004 in Washington, D.C. There
will be a four-day development sprint before the conference.

Volunteers interested in helping to run PyCon should subscribe to the
organizers mailing list at
http://mail.python.org/mailman/listinfo/pycon-organizers

Don't miss any PyCon announcements!  Subscribe at
http://mail.python.org/mailman/listinfo/pycon-announce

You can discuss PyCon with other interested people by subscribing at
http://mail.python.org/mailman/listinfo/pycon-interest

The central resource for PyCon DC 2004 is
http://www.python.org/pycon/dc2004/
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

From fdrake at acm.org  Thu Sep 25 22:28:03 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu Sep 25 22:28:17 2003
Subject: [Python-Dev] PEP on patch processing
In-Reply-To: <m3ad8sv5z5.fsf@mira.informatik.hu-berlin.de>
References: <20030924204734.GB12794@satokar>
	<m3ad8sv5z5.fsf@mira.informatik.hu-berlin.de>
Message-ID: <16243.42035.595883.668201@grendel.zope.com>


Martin v. L?wis writes:
 > A PEP would not be appropriate: it would indicate the need for BDFL
 > pronouncement on the procedures presented, or consensus within some
 > group, when I think neither consensus nor BDFL pronouncement is
 > necessary. Instead, a plain web page on python.org would be
 > sufficient.

If some4one writes up a reST document describing this, it would be
easy enough to add it to the dev/ area.

  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From bac at OCF.Berkeley.EDU  Thu Sep 25 23:23:20 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Sep 25 23:23:25 2003
Subject: [Python-Dev] PEP on patch processing
In-Reply-To: <16243.42035.595883.668201@grendel.zope.com>
References: <20030924204734.GB12794@satokar>	<m3ad8sv5z5.fsf@mira.informatik.hu-berlin.de>
	<16243.42035.595883.668201@grendel.zope.com>
Message-ID: <3F73B128.5020704@ocf.berkeley.edu>

Fred L. Drake, Jr. wrote:
> Martin v. L?wis writes:
>  > A PEP would not be appropriate: it would indicate the need for BDFL
>  > pronouncement on the procedures presented, or consensus within some
>  > group, when I think neither consensus nor BDFL pronouncement is
>  > necessary. Instead, a plain web page on python.org would be
>  > sufficient.
> 
> If some4one writes up a reST document describing this, it would be
> easy enough to add it to the dev/ area.
> 

Just so people don't go too far with this, I am going to be typing up 
something about the Python development process.  It is going to be my 
entry for a talk at PyCon this year.

So something that encompasses all of this will get written up by 
December.  If it isn't a total piece of literary trash I will add it to 
/dev on the site.

-Brett


From fdrake at acm.org  Thu Sep 25 23:35:05 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu Sep 25 23:35:34 2003
Subject: [Python-Dev] PEP on patch processing
In-Reply-To: <3F73B128.5020704@ocf.berkeley.edu>
References: <20030924204734.GB12794@satokar>
	<m3ad8sv5z5.fsf@mira.informatik.hu-berlin.de>
	<16243.42035.595883.668201@grendel.zope.com>
	<3F73B128.5020704@ocf.berkeley.edu>
Message-ID: <16243.46057.561208.909564@grendel.zope.com>


Brett C. writes:
 > So something that encompasses all of this will get written up by 
 > December.  If it isn't a total piece of literary trash I will add it to 
 > /dev on the site.

Sounds good to me!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From mwh at python.net  Fri Sep 26 04:46:48 2003
From: mwh at python.net (Michael Hudson)
Date: Fri Sep 26 04:46:10 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <m365jgv5x4.fsf@mira.informatik.hu-berlin.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "26 Sep 2003 00:50:47
	+0200")
References: <200309240802.h8O82mqF026019@localhost.localdomain>
	<m2zngtx2ob.fsf@deacon-blues.mid.mastaler.com>
	<m365jgv5x4.fsf@mira.informatik.hu-berlin.de>
Message-ID: <2mk77w2ayv.fsf@starship.python.net>

martin@v.loewis.de (Martin v. L?wis) writes:

> "Jason R. Mastaler" <jason@mastaler.com> writes:
>
>> Python 2.3.1 (#1, Sep 24 2003, 15:55:24) 
>> [GCC 3.2 20020903 (Red Hat Linux 8.0 3.2-7)] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> from os import fsync
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in ?
>> ImportError: cannot import name fsync
>> >>> 
>
> Why is that a bug? Your system just does not have a declared fsync
> function.

Err, no.  I assume you've read the rest of your mail by now...

Cheers,
mwh

-- 
  The Internet is full.  Go away.
                      -- http://www.disobey.com/devilshat/ds011101.htm

From mwh at python.net  Fri Sep 26 04:47:31 2003
From: mwh at python.net (Michael Hudson)
Date: Fri Sep 26 04:46:51 2003
Subject: [Python-Dev] RELEASED Python 2.3.1
In-Reply-To: <m3eky4v62g.fsf@mira.informatik.hu-berlin.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "26 Sep 2003 00:47:35
	+0200")
References: <200309250320.h8P3KAxs007608@localhost.localdomain>
	<53F0B466-EF94-11D7-A80F-000A27B19B96@cwi.nl>
	<m3eky4v62g.fsf@mira.informatik.hu-berlin.de>
Message-ID: <2mfzik2axo.fsf@starship.python.net>

martin@v.loewis.de (Martin v. L?wis) writes:

> Jack Jansen <Jack.Jansen@cwi.nl> writes:
>
>> There's a couple of version numbers that would have been nice to
>> update, in the various .plist files. In other words, if you build
>> 2.3.1 from source with a framework build then "Show Info" on the
>> applications will show them to be 2.3, not 2.3.1. Same for the Apple
>> Help Book that is built in to the IDE.
>
> As a procedural note, I would suggest that suggest changes are done
> immediately *after* the previous release, instead of shortly before
> the next one. The release branch should be releasable at any time,
> without having to ask anybody for permission. What gets done is done,
> what doesn't get done has to wait for the next release. The less
> coordination, the better.

+1

I guess this should be added to PEP 102, too.

Cheers,
mwh

-- 
  BUGS   Never use this function.  This function modifies its first
         argument.   The  identity  of  the delimiting character is
         lost.  This function cannot be used on constant strings.
                                    -- the glibc manpage for strtok(3)

From zeddicus at satokar.com  Fri Sep 26 05:34:38 2003
From: zeddicus at satokar.com (Michael Bartl)
Date: Fri Sep 26 05:34:42 2003
Subject: [Python-Dev] PEP on patch processing
In-Reply-To: <16243.46057.561208.909564@grendel.zope.com>
References: <20030924204734.GB12794@satokar>
	<m3ad8sv5z5.fsf@mira.informatik.hu-berlin.de>
	<16243.42035.595883.668201@grendel.zope.com>
	<3F73B128.5020704@ocf.berkeley.edu>
	<16243.46057.561208.909564@grendel.zope.com>
Message-ID: <20030926093438.GA2288@satokar>

On Thu, Sep 25, 2003 at 11:35:05PM -0400, Fred L. Drake, Jr. wrote:
> 
> Brett C. writes:
>  > So something that encompasses all of this will get written up by 
>  > December.  If it isn't a total piece of literary trash I will add it to 
>  > /dev on the site.
> 
> Sounds good to me!

+1

We could still put the content of the current PEP draft into a reST format
and put it up on /dev for now?

From anthony at interlink.com.au  Fri Sep 26 01:34:17 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Sep 26 06:37:17 2003
Subject: [Python-Dev] RELEASED Python 2.3.1 
In-Reply-To: <53F0B466-EF94-11D7-A80F-000A27B19B96@cwi.nl> 
Message-ID: <200309260534.h8Q5YHAb013833@localhost.localdomain>


>>> Jack Jansen wrote
> There's a couple of version numbers that would have been nice to update,
> in the various .plist files. In other words, if you build 2.3.1 from 
> source with a framework build then "Show Info" on the applications will show 
> them to be 2.3, not 2.3.1. Same for the Apple Help Book that is built in to 
> the IDE.

If you could point to the files, I'll add them to the PEPs.




-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From anthony at interlink.com.au  Fri Sep 26 01:24:46 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Sep 26 06:37:23 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1 
In-Reply-To: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com> 
Message-ID: <200309260524.h8Q5Ok1t013688@localhost.localdomain>


>>> "Jason R. Mastaler" wrote
> Will there be a 2.3.2 release, or some other sort of bugfix release to
> address the problems in 2.3.1?  I think the os.fsync() problem is
> serious enough to warrant revoking 2.3.1 and making a new release
> to replace it rather quickly.

There's a workaround on the 2.3.1 bugs page. The only problems I'm aware of
with 2.3.1 are the HP/UX build problems (workaround on bugs.html), the
documentation mistakenly identifying itself as '2.4.0alpha', and the os.fsync
problem.

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From amk at amk.ca  Fri Sep 26 07:46:39 2003
From: amk at amk.ca (A.M. Kuchling)
Date: Fri Sep 26 07:46:47 2003
Subject: [Python-Dev] Re: PEP on patch processing
References: <20030924204734.GB12794@satokar>	<m3ad8sv5z5.fsf@mira.informatik.hu-berlin.de>
	<16243.42035.595883.668201@grendel.zope.com>
	<3F73B128.5020704@ocf.berkeley.edu>
Message-ID: <oprv3uz1v6qmnqtt@news.gmane.org>

On Thu, 25 Sep 2003 20:23:20 -0700, Brett C. <bac@OCF.Berkeley.EDU> wrote:
> So something that encompasses all of this will get written up by 
> December.  If it isn't a total piece of literary trash I will add it to 
> /dev on the site.

Note that http://www.python.org/patches/ has some material on patch 
processing that also belongs in /dev/.

--amk


From amk at amk.ca  Fri Sep 26 07:50:07 2003
From: amk at amk.ca (A.M. Kuchling)
Date: Fri Sep 26 08:00:33 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
References: <200309250320.h8P3KAxs007608@localhost.localdomain>
	<2m7k3x41a3.fsf@starship.python.net>
	<m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>
Message-ID: <oprv3u5tykqmnqtt@news.gmane.org>

On Thu, 25 Sep 2003 12:37:08 -0600, Jason R. Mastaler <jason@mastaler.com> 
wrote:
> Will there be a 2.3.2 release, or some other sort of bugfix release to
> address the problems in 2.3.1?

Another minor nit: the 2.3.1 .tgz file contains the CVS/ subdirectories.  
This doesn't actually hurt anything, though, just cluttering up the tree a 
bit.

--amk


From barry at python.org  Fri Sep 26 08:21:41 2003
From: barry at python.org (Barry Warsaw)
Date: Fri Sep 26 08:21:44 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <oprv3u5tykqmnqtt@news.gmane.org>
Message-ID: <FBD384B8-F01B-11D7-9CCB-003065EEFAC8@python.org>

On Friday, September 26, 2003, at 07:50 AM, A.M. Kuchling wrote:

> On Thu, 25 Sep 2003 12:37:08 -0600, Jason R. Mastaler 
> <jason@mastaler.com> wrote:
>> Will there be a 2.3.2 release, or some other sort of bugfix release to
>> address the problems in 2.3.1?
>
> Another minor nit: the 2.3.1 .tgz file contains the CVS/ 
> subdirectories.  This doesn't actually hurt anything, though, just 
> cluttering up the tree a bit.

Was it not made with cvs export?
-Barry


From tjreedy at udel.edu  Fri Sep 26 09:28:03 2003
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri Sep 26 09:28:10 2003
Subject: [Python-Dev] Re: PEP on patch processing
References: <20030924204734.GB12794@satokar><m3ad8sv5z5.fsf@mira.informatik.hu-berlin.de><16243.42035.595883.668201@grendel.zope.com><3F73B128.5020704@ocf.berkeley.edu><16243.46057.561208.909564@grendel.zope.com>
	<20030926093438.GA2288@satokar>
Message-ID: <bl1et4$bs1$1@sea.gmane.org>


"Michael Bartl" <zeddicus@satokar.com> wrote in message
news:20030926093438.GA2288@satokar...
> On Thu, Sep 25, 2003 at 11:35:05PM -0400, Fred L. Drake, Jr. wrote:
> >
> > Brett C. writes:
> >  > So something that encompasses all of this will get written up
by
> >  > December.  If it isn't a total piece of literary trash I will
add it to
> >  > /dev on the site.
> >
> > Sounds good to me!
>
> +1
>
> We could still put the content of the current PEP draft into a reST
format
> and put it up on /dev for now?

Please do.  If there were good step-by-step instructions on how to
review a bug or patch submission, and 'official' encouragement to do
as much/many as one can, even if not all steps, then I might actually
try to do some -- and possibly before December.

Terry J. Reedy




From tjreedy at udel.edu  Fri Sep 26 09:52:15 2003
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri Sep 26 09:52:21 2003
Subject: [Python-Dev] Re: PEP on patch processing
References: <20030924204734.GB12794@satokar>	<m3ad8sv5z5.fsf@mira.informatik.hu-berlin.de><16243.42035.595883.668201@grendel.zope.com><3F73B128.5020704@ocf.berkeley.edu>
	<oprv3uz1v6qmnqtt@news.gmane.org>
Message-ID: <bl1gag$es1$1@sea.gmane.org>


"A.M. Kuchling" <amk@amk.ca> wrote in message
news:oprv3uz1v6qmnqtt@news.gmane.org...
> Note that http://www.python.org/patches/ has some material on patch
> processing that also belongs in /dev/.

I check all the links on ../dev/ I noticed and found no direct link to
/patches/.  I think there should be (Explanations list, bottom of
page)

The /devfaq/ has this entry:

3.2. How to submit a patch?
Please read the Patch Submission Guidelines at
http://www.python.org/patches
A recent copy can be found in the Appendix of this FAQ.

Q. Which of these two nearly identical pages is more recent?

Guess:Appendix A2 has "It's almost a year since Python 1.5.2 was
released".  /patches/ is undated (dating pages in one change I would
suggest for the site) but the differences suggest to me that it is
newer.  Please delete the older version.  Directing would-be new
submitters to both seems senseless and can only discourage, not
encourage.

Terry J. Reedy

cc webmaster@python.org




From skip at pobox.com  Fri Sep 26 10:24:19 2003
From: skip at pobox.com (Skip Montanaro)
Date: Fri Sep 26 10:24:34 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <m365jgv5x4.fsf@mira.informatik.hu-berlin.de>
References: <200309240802.h8O82mqF026019@localhost.localdomain>
	<m2zngtx2ob.fsf@deacon-blues.mid.mastaler.com>
	<m365jgv5x4.fsf@mira.informatik.hu-berlin.de>
Message-ID: <16244.19475.777510.550060@montanaro.dyndns.org>


    Martin> "Jason R. Mastaler" <jason@mastaler.com> writes:
    >> Python 2.3.1 (#1, Sep 24 2003, 15:55:24) 
    >> [GCC 3.2 20020903 (Red Hat Linux 8.0 3.2-7)] on linux2
    >> Type "help", "copyright", "credits" or "license" for more information.
    >> >>> from os import fsync
    >> Traceback (most recent call last):
    >> File "<stdin>", line 1, in ?
    >> ImportError: cannot import name fsync
    >> >>> 

    Martin> Why is that a bug? Your system just does not have a declared
    Martin> fsync function.

No, it was a typo in the configure scripts.  I checked in a fix for both the
head and release23-maint branches yesterday.  Red Hat 8.0 does have fsync:

    FSYNC(2)                   Linux Programmer's Manual                  FSYNC(2)

    NAME
           fsync,  fdatasync  -  synchronize  a file's complete in-core state with
           that on disk
    ...

That is from an 8.0 system.

Skip

From guido at python.org  Fri Sep 26 11:10:30 2003
From: guido at python.org (Guido van Rossum)
Date: Fri Sep 26 11:11:29 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: Your message of "Fri, 26 Sep 2003 08:21:41 EDT."
	<FBD384B8-F01B-11D7-9CCB-003065EEFAC8@python.org> 
References: <FBD384B8-F01B-11D7-9CCB-003065EEFAC8@python.org> 
Message-ID: <200309261510.h8QFAUb21375@12-236-84-31.client.attbi.com>

> > Another minor nit: the 2.3.1 .tgz file contains the CVS/ 
> > subdirectories.  This doesn't actually hurt anything, though, just 
> > cluttering up the tree a bit.
> 
> Was it not made with cvs export?

Ouch.  Isn't this in the PEP?  The CVS directories *do* hurt, IMO (by
confusing people).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Fri Sep 26 11:35:03 2003
From: barry at python.org (Barry Warsaw)
Date: Fri Sep 26 11:36:07 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <200309261510.h8QFAUb21375@12-236-84-31.client.attbi.com>
References: <FBD384B8-F01B-11D7-9CCB-003065EEFAC8@python.org>
	<200309261510.h8QFAUb21375@12-236-84-31.client.attbi.com>
Message-ID: <1064590503.30783.12.camel@anthem>

On Fri, 2003-09-26 at 11:10, Guido van Rossum wrote:
> > > Another minor nit: the 2.3.1 .tgz file contains the CVS/ 
> > > subdirectories.  This doesn't actually hurt anything, though, just 
> > > cluttering up the tree a bit.
> > 
> > Was it not made with cvs export?
> 
> Ouch.  Isn't this in the PEP?  The CVS directories *do* hurt, IMO (by
> confusing people).

It's in PEPs 101 and 102.
-Barry



From jason at mastaler.com  Fri Sep 26 11:47:14 2003
From: jason at mastaler.com (Jason R. Mastaler)
Date: Fri Sep 26 11:47:27 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
References: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>
	<200309260524.h8Q5Ok1t013688@localhost.localdomain>
Message-ID: <m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>

Anthony Baxter <anthony@interlink.com.au> writes:

>> Will there be a 2.3.2 release, or some other sort of bugfix release
>> to address the problems in 2.3.1?  I think the os.fsync() problem
>> is serious enough to warrant revoking 2.3.1 and making a new
>> release to replace it rather quickly.
>
> There's a workaround on the 2.3.1 bugs page. The only problems I'm
> aware of with 2.3.1 are the HP/UX build problems (workaround on
> bugs.html), the documentation mistakenly identifying itself as
> '2.4.0alpha', and the os.fsync problem.

There's a workaround, but the problem is that I think I'll be the one
bearing the brunt of this bug for who knows how long.  I've gotten
about a half dozen problem reports about it already on the TMDA lists.
TMDA uses os.fsync() in its mbox and Maildir delivery code, so
practically every user will find their TMDA broken as soon as they
install or upgrade to 2.3.1.  Since Python point releases are usually
so solid, they don't consider it might be a bug in the interpreter,
and assume it's a TMDA bug.  TMDA is just one example.  Repeat for the
other packages out there which rely on os.fsync().

The longer 2.3.1 is out there, the more it will be installed,
incorporated into OS package distributions, etc, and the harder this
breakage will be to shake.


From barry at python.org  Fri Sep 26 11:53:29 2003
From: barry at python.org (Barry Warsaw)
Date: Fri Sep 26 11:53:36 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
References: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>
	<200309260524.h8Q5Ok1t013688@localhost.localdomain>
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
Message-ID: <1064591608.30783.30.camel@anthem>

On Fri, 2003-09-26 at 11:47, Jason R.Mastaler wrote:

> The longer 2.3.1 is out there, the more it will be installed,
> incorporated into OS package distributions, etc, and the harder this
> breakage will be to shake.

It certainly seems reasonable to me to release 2.3.2 fairly quickly.

-Barry



From theller at python.net  Fri Sep 26 11:58:35 2003
From: theller at python.net (Thomas Heller)
Date: Fri Sep 26 12:00:40 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <1064591608.30783.30.camel@anthem> (Barry Warsaw's message of
	"Fri, 26 Sep 2003 11:53:29 -0400")
References: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>
	<200309260524.h8Q5Ok1t013688@localhost.localdomain>
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
	<1064591608.30783.30.camel@anthem>
Message-ID: <k77vh784.fsf@python.net>

Barry Warsaw <barry@python.org> writes:

> On Fri, 2003-09-26 at 11:47, Jason R.Mastaler wrote:
>
>> The longer 2.3.1 is out there, the more it will be installed,
>> incorporated into OS package distributions, etc, and the harder this
>> breakage will be to shake.
>
> It certainly seems reasonable to me to release 2.3.2 fairly quickly.

But not earlier than at least one week after 2.3.2rc1 ;-)

Thomas


From jason at mastaler.com  Fri Sep 26 12:07:29 2003
From: jason at mastaler.com (Jason R. Mastaler)
Date: Fri Sep 26 12:07:35 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
References: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>
	<200309260524.h8Q5Ok1t013688@localhost.localdomain>
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
	<1064591608.30783.30.camel@anthem> <k77vh784.fsf@python.net>
Message-ID: <m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com>

Thomas Heller <theller@python.net> writes:

>> It certainly seems reasonable to me to release 2.3.2 fairly quickly.
>
> But not earlier than at least one week after 2.3.2rc1 ;-)

Well, if you're going to wait that long, I think you might consider
adding some sort of warning to the 2.3.1 download area.  My fear is
that some large OS vendor (e.g, Redhat) will pick up 2.3.1 for a
future release.


From python at discworld.dyndns.org  Fri Sep 26 12:38:46 2003
From: python at discworld.dyndns.org (Charles Cazabon)
Date: Fri Sep 26 12:34:05 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com>;
	from jason@mastaler.com on Fri, Sep 26, 2003 at 10:07:29AM -0600
References: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>
	<200309260524.h8Q5Ok1t013688@localhost.localdomain>
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
	<1064591608.30783.30.camel@anthem> <k77vh784.fsf@python.net>
	<m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com>
Message-ID: <20030926103846.A26782@discworld.dyndns.org>

Jason R. Mastaler <jason@mastaler.com> wrote:
> Thomas Heller <theller@python.net> writes:
> 
> >> It certainly seems reasonable to me to release 2.3.2 fairly quickly.
> >
> > But not earlier than at least one week after 2.3.2rc1 ;-)
> 
> Well, if you're going to wait that long, I think you might consider
> adding some sort of warning to the 2.3.1 download area.  My fear is
> that some large OS vendor (e.g, Redhat) will pick up 2.3.1 for a
> future release.

Too late: 2.3.1 is in at least some versions of Debian already -- I'm getting
reports on the os.fsync() issue already from my users (it breaks getmail).
Could 2.3.1 be revoked or replaced ASAP? "2.3.1b" or similar with only this
change would suit me.

Charles
-- 
-----------------------------------------------------------------------
Charles Cazabon                           <python@discworld.dyndns.org>
GPL'ed software available at:     http://www.qcc.ca/~charlesc/software/
-----------------------------------------------------------------------

From guido at python.org  Fri Sep 26 12:40:12 2003
From: guido at python.org (Guido van Rossum)
Date: Fri Sep 26 12:41:03 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: Your message of "Fri, 26 Sep 2003 10:38:46 MDT."
	<20030926103846.A26782@discworld.dyndns.org> 
References: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>
	<200309260524.h8Q5Ok1t013688@localhost.localdomain>
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
	<1064591608.30783.30.camel@anthem> <k77vh784.fsf@python.net>
	<m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com> 
	<20030926103846.A26782@discworld.dyndns.org> 
Message-ID: <200309261640.h8QGeCb21645@12-236-84-31.client.attbi.com>

> Too late: 2.3.1 is in at least some versions of Debian already --
> I'm getting reports on the os.fsync() issue already from my users
> (it breaks getmail).  Could 2.3.1 be revoked or replaced ASAP? 
> "2.3.1b" or similar with only this change would suit me.

That doesn't fit in the numbering scheme, it would have to be 2.3.2.
Given the amount of traffic about this I'm tempted to agree that we
need to move quickly and issue a fix for this issue (only).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From anthony at interlink.com.au  Fri Sep 26 12:42:59 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Sep 26 12:45:01 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1 
In-Reply-To: <1064590503.30783.12.camel@anthem> 
Message-ID: <200309261643.h8QGh05H020344@localhost.localdomain>


>>> Barry Warsaw wrote
> > > Was it not made with cvs export?
> > 
> > Ouch.  Isn't this in the PEP?  The CVS directories *do* hurt, IMO (by
> > confusing people).
> 
> It's in PEPs 101 and 102.

I'm very very confused, since I _did_ use cvs export. Really. Repeatedly.

Ugh. Given this, I'm happy to look at a 2.3.2 in a couple of weeks - I'd
like very much to finish the PEP rewrite and get it reviewed first.

Arrrrrgh.
(no, not talk-like-a-pirate day, frustration)

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From anthony at interlink.com.au  Fri Sep 26 12:46:41 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Sep 26 12:48:30 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1 
In-Reply-To: <200309261640.h8QGeCb21645@12-236-84-31.client.attbi.com> 
Message-ID: <200309261646.h8QGkf4h020451@localhost.localdomain>


>>> Guido van Rossum wrote
> That doesn't fit in the numbering scheme, it would have to be 2.3.2.
> Given the amount of traffic about this I'm tempted to agree that we
> need to move quickly and issue a fix for this issue (only).

We really should fix the HP/UX build problem as well.

Anthony


From anthony at interlink.com.au  Fri Sep 26 12:59:28 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Sep 26 13:01:10 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1 
In-Reply-To: <m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com> 
Message-ID: <200309261659.h8QGxSTc020775@localhost.localdomain>


>>> "Jason R. Mastaler" wrote
> There's a workaround, but the problem is that I think I'll be the one
> bearing the brunt of this bug for who knows how long.  I've gotten
> about a half dozen problem reports about it already on the TMDA lists.
> TMDA uses os.fsync() in its mbox and Maildir delivery code, so
> practically every user will find their TMDA broken as soon as they
> install or upgrade to 2.3.1.  Since Python point releases are usually
> so solid, they don't consider it might be a bug in the interpreter,
> and assume it's a TMDA bug.  TMDA is just one example.  Repeat for the
> other packages out there which rely on os.fsync().

Hm. I ran the Zope2, Zope3, and Twisted test suites against the release
build before the release. I'm open to additional suggestions (I just 
grabbed tmda, but it doesn't have a test suite). In this case, the 
problem hopefully would've been picked up with a release candidate, but
I'm also happy to add additional slabs of test suites to run against 
code before it's released...

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From tim.one at comcast.net  Fri Sep 26 13:23:13 2003
From: tim.one at comcast.net (Tim Peters)
Date: Fri Sep 26 13:30:13 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1 
In-Reply-To: <200309261646.h8QGkf4h020451@localhost.localdomain>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEINGCAB.tim.one@comcast.net>

[Guido]
>> That doesn't fit in the numbering scheme, it would have to be 2.3.2.
>> Given the amount of traffic about this I'm tempted to agree that we
>> need to move quickly and issue a fix for this issue (only).

[Anthony Baxter]
> We really should fix the HP/UX build problem as well.

That sounds dangerous to me:  there's *always* an HP/UX build problem, if
you count (as I do) getting threads to work.  Every release, some HP/UX user
submits a patch that breaks the build for other HP/UX users.  So unless
we've got a bona fide HP/UX expert on tap now (do we? that would be way cool
<wink>), don't hold up solving a clear and widespread Linux problem by
waiting for one to appear.


From anthony at interlink.com.au  Fri Sep 26 13:42:45 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Sep 26 13:44:39 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1 
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEINGCAB.tim.one@comcast.net> 
Message-ID: <200309261742.h8QHgkHM029591@localhost.localdomain>


>>> "Tim Peters" wrote
> That sounds dangerous to me:  there's *always* an HP/UX build problem, if
> you count (as I do) getting threads to work.  Every release, some HP/UX user
> submits a patch that breaks the build for other HP/UX users.  So unless
> we've got a bona fide HP/UX expert on tap now (do we? that would be way cool
> <wink>), don't hold up solving a clear and widespread Linux problem by
> waiting for one to appear.

On the scale of "HP/UX build problems we've known and loved", this one's
a trivial one. 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=811160&group_id=5470
has a patch (from the gcc bugtracker - they ran into the exact same problem)

Now, the readline+no-threads=boom coredump, well, that involves threads, and
the original reporter even mentions HP/UX, but it's more a simple-ish one -
missing #ifdef WITH_THREAD calls, as far as I can see.
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=811844&group_id=5470

(Someone should probably do a scan of the code in Modules to make sure all
the threading API calls are wrapped in WITH_THREAD #ifdefs. Not me, now, 
though - way past bedtime).

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From jeremy at zope.com  Fri Sep 26 13:43:31 2003
From: jeremy at zope.com (Jeremy Hylton)
Date: Fri Sep 26 13:45:32 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <200309261640.h8QGeCb21645@12-236-84-31.client.attbi.com>
References: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>
	<200309260524.h8Q5Ok1t013688@localhost.localdomain>
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
	<1064591608.30783.30.camel@anthem> <k77vh784.fsf@python.net>
	<m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com>
	<20030926103846.A26782@discworld.dyndns.org>
	<200309261640.h8QGeCb21645@12-236-84-31.client.attbi.com>
Message-ID: <1064598210.19498.6.camel@localhost.localdomain>

On Fri, 2003-09-26 at 12:40, Guido van Rossum wrote:
> > Too late: 2.3.1 is in at least some versions of Debian already --
> > I'm getting reports on the os.fsync() issue already from my users
> > (it breaks getmail).  Could 2.3.1 be revoked or replaced ASAP? 
> > "2.3.1b" or similar with only this change would suit me.
> 
> That doesn't fit in the numbering scheme, it would have to be 2.3.2.
> Given the amount of traffic about this I'm tempted to agree that we
> need to move quickly and issue a fix for this i

+1.

We should treat 2.3.1 like 2.3.1rc1 and aim for Sep. 30 for a 2.3.2.

Jeremy



From anthony at interlink.com.au  Fri Sep 26 13:52:19 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Sep 26 13:54:10 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1 
In-Reply-To: <1064598210.19498.6.camel@localhost.localdomain> 
Message-ID: <200309261752.h8QHqJgD029823@localhost.localdomain>


>>> Jeremy Hylton wrote
> We should treat 2.3.1 like 2.3.1rc1 and aim for Sep. 30 for a 2.3.2.

Wouldn't it be better to cut a 2.3.2rc before the final 2.3.2?

once-bitten-ly,
Anthony


From jeremy at zope.com  Fri Sep 26 13:57:55 2003
From: jeremy at zope.com (Jeremy Hylton)
Date: Fri Sep 26 13:59:36 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <200309261752.h8QHqJgD029823@localhost.localdomain>
References: <200309261752.h8QHqJgD029823@localhost.localdomain>
Message-ID: <1064599075.19498.14.camel@localhost.localdomain>

On Fri, 2003-09-26 at 13:52, Anthony Baxter wrote:
> >>> Jeremy Hylton wrote
> > We should treat 2.3.1 like 2.3.1rc1 and aim for Sep. 30 for a 2.3.2.
> 
> Wouldn't it be better to cut a 2.3.2rc before the final 2.3.2?

If we only fix the few critical problems -- CVS directories, fsync
problem, Mac version numbers (?) -- then there doesn't seem to be much
risk.  But it wouldn't be too hard to convince me.  (I gave my all clear
on 2.3.1, after all.)

Jeremy



From tim.one at comcast.net  Fri Sep 26 14:24:27 2003
From: tim.one at comcast.net (Tim Peters)
Date: Fri Sep 26 14:24:33 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1 
In-Reply-To: <200309261752.h8QHqJgD029823@localhost.localdomain>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>

[Jeremy Hylton]
>> We should treat 2.3.1 like 2.3.1rc1 and aim for Sep. 30 for a 2.3.2.

[Anthony Baxter]
> Wouldn't it be better to cut a 2.3.2rc before the final 2.3.2?
>
> once-bitten-ly,

Well, Jeremy's "Sep. 30" is only 4 days away.  If 2.3.2 is restricted to
fixing just the obvious big problems already reported against 2.3.1, in
reality it's like Jeremy said:  2.3.1 was really 2.3.2c1 <0.5 wink>.

Don't get too gun-shy.  There were lots of changes in who did what for the
2.3.1 release, and some glitches were inevitable.  It *had* gotten so smooth
with the same people doing the same things all the time that, e.g., you must
have noticed that Guido stopped doing anything for releases.  I would have
liked to pay a lot more attention but just couldn't make time for it; I'm
sure the same is true of the other PythonLabs alumni (including Guido).  So
some things could have gone better.  It's not that big a deal -- next time
they will go better.  Remember that we pioneered all possible ways to screw
up a release already, and discovered that it's a finite set <wink>.


From barry at python.org  Fri Sep 26 14:30:33 2003
From: barry at python.org (Barry Warsaw)
Date: Fri Sep 26 14:30:44 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
Message-ID: <1064601033.31604.41.camel@anthem>

On Fri, 2003-09-26 at 14:24, Tim Peters wrote:

> Don't get too gun-shy.

Plus there's always 2.3.3 :)

-Barry



From nas at arctrix.com  Fri Sep 26 14:48:54 2003
From: nas at arctrix.com (Neil Schemenauer)
Date: Fri Sep 26 14:48:43 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
References: <200309261752.h8QHqJgD029823@localhost.localdomain>
	<LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
Message-ID: <20030926184853.GB22837@mems-exchange.org>

On Fri, Sep 26, 2003 at 02:24:27PM -0400, Tim Peters wrote:
> Don't get too gun-shy.  There were lots of changes in who did what for the
> 2.3.1 release, and some glitches were inevitable.

Some humble advice from someone who has done quite a few software
releases in the last few years (mostly internal).  The release
process should be automated as much as possible.  I think the Python
project could do a bit better in this regard.  Some ideas:

  * try to reduce the number of version numbers in different files need to
    be adjusted.   Also, try to automate the adjustment.

  * write scripts to generate the final package and also verify its
    sanity (e.g. check version numbers, dates, detect files that
    should not be included in the release)

  * write scripts to upload the package for distribution.  Ensure
    proper naming, permissions, etc. 

Python already has a release checklist.  That's good.  Be sure to
follow it when doing a release and make sure it's up-to-date.

Unfortunately Python doesn't do releases often enough for the
process to be really polished.  One effective way of improving the
process is to have a new person do it with a veteran looking on.
The new person should try to complete the release using only the
checklist as the guide.  The veteran is on hand to make sure nothing
goes wrong.  The veteran should also take notes; documenting any
errors or omissions in the release checklist or ideas on ways to
further automate the process.

  Neil

From barry at python.org  Fri Sep 26 15:08:54 2003
From: barry at python.org (Barry Warsaw)
Date: Fri Sep 26 15:09:09 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <20030926184853.GB22837@mems-exchange.org>
References: <200309261752.h8QHqJgD029823@localhost.localdomain>
	<LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
	<20030926184853.GB22837@mems-exchange.org>
Message-ID: <1064603333.31604.47.camel@anthem>

On Fri, 2003-09-26 at 14:48, Neil Schemenauer wrote:

> Some humble advice from someone who has done quite a few software
> releases in the last few years (mostly internal).  The release
> process should be automated as much as possible.  

I agree.  It's what I do for Mailman, although 1) the release process is
much simpler and 2) I don't have to coordinate with anyone else.  Both
are becoming less true though with each release.

-Barry



From theller at python.net  Fri Sep 26 15:52:03 2003
From: theller at python.net (Thomas Heller)
Date: Fri Sep 26 15:52:17 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <20030926184853.GB22837@mems-exchange.org> (Neil Schemenauer's
	message of "Fri, 26 Sep 2003 11:48:54 -0700")
References: <200309261752.h8QHqJgD029823@localhost.localdomain>
	<LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
	<20030926184853.GB22837@mems-exchange.org>
Message-ID: <he2zba58.fsf@python.net>

Neil Schemenauer <nas@arctrix.com> writes:

>   * try to reduce the number of version numbers in different files need to
>     be adjusted.

I suggest to update the version numbers in these files to the next
version *after* a release is done.  So, in the release23-maint branch
the version numbers could *now* be adjusted to 2.3.2.

And it would help if I could build the HTML docs myself from CVS. I did
manage to create the pdf files with TeTex under windows, but I didn't
succeed with the html pages so far.

This way, a Windows release could be built at any time, and I would not
only install and test it, but also use it as my default version.

Thomas


From python at rcn.com  Fri Sep 26 15:58:05 2003
From: python at rcn.com (Raymond Hettinger)
Date: Fri Sep 26 15:58:22 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <200309261640.h8QGeCb21645@12-236-84-31.client.attbi.com>
Message-ID: <001201c38468$8077b5a0$e841fea9@oemcomputer>

Should there be an announcement that 2.3.2 is forthcoming and hold off
on further downloads or distribution of 2.3.1?


Raymond



-----Original Message-----
From: python-dev-bounces@python.org
[mailto:python-dev-bounces@python.org] On Behalf Of Guido van Rossum
Sent: Friday, September 26, 2003 12:40 PM
To: Charles Cazabon
Cc: python-dev@python.org
Subject: Re: [Python-Dev] Re: RELEASED Python 2.3.1

> Too late: 2.3.1 is in at least some versions of Debian already --
> I'm getting reports on the os.fsync() issue already from my users
> (it breaks getmail).  Could 2.3.1 be revoked or replaced ASAP? 
> "2.3.1b" or similar with only this change would suit me.

That doesn't fit in the numbering scheme, it would have to be 2.3.2.
Given the amount of traffic about this I'm tempted to agree that we
need to move quickly and issue a fix for this issue (only).

--Guido van Rossum (home page: http://www.python.org/~guido/)

_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev


From jason at mastaler.com  Fri Sep 26 17:06:28 2003
From: jason at mastaler.com (Jason R. Mastaler)
Date: Fri Sep 26 17:06:33 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
References: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>
	<200309260524.h8Q5Ok1t013688@localhost.localdomain>
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
	<1064591608.30783.30.camel@anthem> <k77vh784.fsf@python.net>
	<m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com>
	<20030926103846.A26782@discworld.dyndns.org>
Message-ID: <m2d6dnntt7.fsf@deacon-blues.mid.mastaler.com>

Charles Cazabon <python@discworld.dyndns.org> writes:

> Too late: 2.3.1 is in at least some versions of Debian already

The Debian situation is a bit better because they can incorporate
patches and issue a point release of the package itself.  I contacted
the Debian/Python maintainer, and he plans to incorporate the
os.fsync() fix as soon as the sourceforge CVS delay catches up to him.
More troublesome are distros like Redhat whose packages are not
dynamically updateable.  They are sealed in stone once they become
part of a release.

> Could 2.3.1 be revoked or replaced ASAP? "2.3.1b" or similar with
> only this change would suit me.

I'd be in favour of revoking 2.3.1 as well because of the severity of
this bug.  I don't think we'll be completely rid of it otherwise.


From jason at mastaler.com  Fri Sep 26 17:09:07 2003
From: jason at mastaler.com (Jason R. Mastaler)
Date: Fri Sep 26 17:10:31 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
References: <200309261646.h8QGkf4h020451@localhost.localdomain>
	<LNBBLJKPBEHFEDALKOLCOEINGCAB.tim.one@comcast.net>
Message-ID: <m27k3vntos.fsf@deacon-blues.mid.mastaler.com>

"Tim Peters" <tim.one@comcast.net> writes:

> So unless we've got a bona fide HP/UX expert on tap now (do we? that
> would be way cool <wink>), don't hold up solving a clear and
> widespread Linux problem by waiting for one to appear.

Just to be clear, the os.fsync() problem is not Linux-only.  It's also
on FreeBSD (probably every Unix platform actually).


From skip at pobox.com  Fri Sep 26 17:14:13 2003
From: skip at pobox.com (Skip Montanaro)
Date: Fri Sep 26 17:14:22 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <m27k3vntos.fsf@deacon-blues.mid.mastaler.com>
References: <200309261646.h8QGkf4h020451@localhost.localdomain>
	<LNBBLJKPBEHFEDALKOLCOEINGCAB.tim.one@comcast.net>
	<m27k3vntos.fsf@deacon-blues.mid.mastaler.com>
Message-ID: <16244.44069.798860.174782@montanaro.dyndns.org>


    >> So unless we've got a bona fide HP/UX expert on tap now (do we? that
    >> would be way cool <wink>), don't hold up solving a clear and
    >> widespread Linux problem by waiting for one to appear.

    Jason> Just to be clear, the os.fsync() problem is not Linux-only.  It's
    Jason> also on FreeBSD (probably every Unix platform actually).

Correct.  There are two new C macros in pyconfig.h, HAVE_FSYNC and
HAVE_FDATASYNC.  When originally added, the former was mistyped as
HAVE_SYNC.

Skip


From jason at mastaler.com  Fri Sep 26 17:14:46 2003
From: jason at mastaler.com (Jason R. Mastaler)
Date: Fri Sep 26 17:20:31 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
References: <m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
	<200309261659.h8QGxSTc020775@localhost.localdomain>
Message-ID: <m21xu3ntfd.fsf@deacon-blues.mid.mastaler.com>

Anthony Baxter <anthony@interlink.com.au> writes:

> Hm. I ran the Zope2, Zope3, and Twisted test suites against the
> release build before the release. I'm open to additional suggestions
> (I just grabbed tmda, but it doesn't have a test suite).

TMDA doesn't have a test suite, but it really should.  It's on the
TODO list.  :-)

> In this case, the problem hopefully would've been picked up with a
> release candidate, but I'm also happy to add additional slabs of
> test suites to run against code before it's released...

I'll try to get more involved with testing release candidates from now
on.


From mwh at python.net  Fri Sep 26 17:34:41 2003
From: mwh at python.net (Michael Hudson)
Date: Fri Sep 26 17:34:01 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <he2zba58.fsf@python.net> (Thomas Heller's message of "Fri, 26
	Sep 2003 21:52:03 +0200")
References: <200309261752.h8QHqJgD029823@localhost.localdomain>
	<LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
	<20030926184853.GB22837@mems-exchange.org> <he2zba58.fsf@python.net>
Message-ID: <2mzngr1bf2.fsf@starship.python.net>

Thomas Heller <theller@python.net> writes:

> Neil Schemenauer <nas@arctrix.com> writes:
>
>>   * try to reduce the number of version numbers in different files need to
>>     be adjusted.
>
> I suggest to update the version numbers in these files to the next
> version *after* a release is done.  So, in the release23-maint branch
> the version numbers could *now* be adjusted to 2.3.2.
>
> And it would help if I could build the HTML docs myself from CVS. I did
> manage to create the pdf files with TeTex under windows, but I didn't
> succeed with the html pages so far.

It occurs to me that I don't know *why* Fred is so much the
documentation man; I've not had any trouble processing the docs into
HTML lately (haven't tried on Windows, admittedly, and I haven't
tried to make info ever).

What else needs to be done?  There must be quite a bit of mucking
about on creosote to do, I guess.

Finding a substitue Jack might be harder (though given Bob Ippolito's
work rate at the moment we could probably make him MacPython
maintainer without him noticing <wink>).

Cheers,
mwh

-- 
  In short, just business as usual in the wacky world of floating
  point <wink>.                        -- Tim Peters, comp.lang.python

From fdrake at acm.org  Fri Sep 26 18:17:37 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri Sep 26 18:17:54 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <200309261640.h8QGeCb21645@12-236-84-31.client.attbi.com>
References: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>
	<200309260524.h8Q5Ok1t013688@localhost.localdomain>
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
	<1064591608.30783.30.camel@anthem> <k77vh784.fsf@python.net>
	<m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com>
	<20030926103846.A26782@discworld.dyndns.org>
	<200309261640.h8QGeCb21645@12-236-84-31.client.attbi.com>
Message-ID: <16244.47873.343272.206145@grendel.zope.com>


Guido van Rossum writes:
 > That doesn't fit in the numbering scheme, it would have to be 2.3.2.
 > Given the amount of traffic about this I'm tempted to agree that we
 > need to move quickly and issue a fix for this issue (only).

I think the documentation versioning problem should be fixed as well.

I presume we can start making changes to the release23-maint branch
now?


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From greg at electricrain.com  Fri Sep 26 19:28:40 2003
From: greg at electricrain.com (Gregory P. Smith)
Date: Fri Sep 26 19:28:49 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <BIEJKCLHCIOIHAGOKOLHGEPPHEAA.tim@zope.com>
References: <200309241550.h8OFojlx031706@localhost.localdomain>
	<BIEJKCLHCIOIHAGOKOLHGEPPHEAA.tim@zope.com>
Message-ID: <20030926232840.GB17491@zot.electricrain.com>

On Wed, Sep 24, 2003 at 12:01:04PM -0400, Tim Peters wrote:
> [Anthony Baxter]
> > I haven't been following the spambayes lists too closely. Are there
> > concrete problems with bsddb that are cropping up, or just a general
> > wariness of it?
> >
> > If there _is_ a problem with bsddb, it needs to be addressed. Too
> > many things depend on it.
> 
> Reports of database corruption are common in spambayes.  At least one
> knowledgable tester reported his problems went away after moving to a recent
> Sleepycat release (4.1.25, IIRC).
...
>  They seem to come from non-Outlook people
> using Berkeley for the message info database.  Richie got a whittled down
> threaded test that fails on Windows and Linux, and there's already a
> (Python) bug report open on that; it's not thought to be relevant to how
> spambayes uses Berkeley, though.

Its been my impression that the sporatic bsddb testsuite failures
are BerkeleyDB related rather than anything the python module can be
responsible for (other than the previously mentioned needed improvements
in cleaning up the on disk "temporary" test environment before launching
tests).  For anyone reporting berkeleydb issues, its important to find
out the version of BerkeleyDB they're using and on what platform.

When I find time I want to take the simple test that Richie created,
try it in C and if it still fails there (it should) on the 4.2.xx release
canidate i'll submit it to sleepycat.

The full BerkeleyDB library is unfortunately complicated because it
supports so much more than most people need.

-g


From greg at electricrain.com  Fri Sep 26 19:39:59 2003
From: greg at electricrain.com (Gregory P. Smith)
Date: Fri Sep 26 19:40:44 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <16241.45106.296743.387428@montanaro.dyndns.org>
References: <200309241233.h8OCXcbg028668@localhost.localdomain>
	<1064409627.1958.90.camel@anthem>
	<16241.45106.296743.387428@montanaro.dyndns.org>
Message-ID: <20030926233959.GC17491@zot.electricrain.com>

> For long-running SpamBayes apps like pop3proxy and the
> Outlook plugin, pickles make sense if shared access to the scoring database
> is not needed.

Combine that with data event/change logging and you have effectively the
same thing as what many apps think they need to use a database library
for.  Replay the entire log of data changes internally on infrequent app
startup to generate your internal state.  when the log file is getting to
big, write a full state snapshot and continue logging changes from it.
no need to save the entire pickle everytime something changes.



From skip at pobox.com  Fri Sep 26 19:06:49 2003
From: skip at pobox.com (Skip Montanaro)
Date: Fri Sep 26 23:11:46 2003
Subject: [Python-Dev] How to test for stuff like fsync?
Message-ID: <16244.50825.588063.154695@montanaro.dyndns.org>


When I checked in the HAVE_SYNC -> HAVE_FSYNC change I thought briefly about
including a test to test_os.py.  I eventually decided against it because it
was clear that a simpleminded test for os.fsync's existence was useless.
Jason seems to have been bitten badly by it.  I doubt that fsync() is
available everywhere (otherwise why test for it in the configure script?),
so it seems that Jason should have been testing for its presence (though see
below) and worming around its abscence if he wanted TMDA to run on as many
platforms as possible.

This suggests the following questions:

* Is there a reasonable way to write tests which will detect this sort of
  error?

* Should there be special documentation about os-dependent functions which
  are not universally available?  The fsync() doc says nothing about it not
  always being available (thus Jason can be forgiven for not calling
  hasattr(os, "fsync")).

Skip




From anthony at interlink.com.au  Sat Sep 27 00:33:20 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sat Sep 27 00:35:16 2003
Subject: [Python-Dev] How to test for stuff like fsync? 
In-Reply-To: <16244.50825.588063.154695@montanaro.dyndns.org> 
Message-ID: <200309270433.h8R4XKwl006833@localhost.localdomain>


>>> Skip Montanaro wrote
> * Is there a reasonable way to write tests which will detect this sort of
>   error?

A thought that occurred to me a short time ago: write a script that outputs
the contents of each builtin module (the names in the module) to a file.

Run that under previous version of python. Run it under release candidate.
Compare differences. 

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From anthony at interlink.com.au  Sat Sep 27 00:37:59 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sat Sep 27 00:39:44 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1 
In-Reply-To: <m2d6dnntt7.fsf@deacon-blues.mid.mastaler.com> 
Message-ID: <200309270437.h8R4bxk0006927@localhost.localdomain>


>>> "Jason R. Mastaler" wrote
> > Could 2.3.1 be revoked or replaced ASAP? "2.3.1b" or similar with
> > only this change would suit me.
> 
> I'd be in favour of revoking 2.3.1 as well because of the severity of
> this bug.  I don't think we'll be completely rid of it otherwise.

I';m not sure what the term "revoke" would mean in this context. Replacing
it soon would work for me, and a big warning on the 2.3.1 page about the
fsync problem and the HP/UX build problem (these are the two that _must_ be
fixed in 2.3.2, as far as I'm concerned).

I can whack a bit of text on the 2.3.1 page saying "there will be a 2.3.2
release shortly - stay tuned" if people think it's appropriate.

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From anthony at interlink.com.au  Sat Sep 27 01:09:09 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sat Sep 27 01:10:55 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1 
In-Reply-To: <200309270437.h8R4bxk0006927@localhost.localdomain> 
Message-ID: <200309270509.h8R59ArD007367@localhost.localdomain>


>>> Anthony Baxter wrote
> I can whack a bit of text on the 2.3.1 page saying "there will be a 2.3.2
> release shortly - stay tuned" if people think it's appropriate.

This is now done. 

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From tjreedy at udel.edu  Sat Sep 27 01:16:40 2003
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat Sep 27 01:16:48 2003
Subject: [Python-Dev] Re: Re: RELEASED Python 2.3.1
References: <m2d6dnntt7.fsf@deacon-blues.mid.mastaler.com>
	<200309270437.h8R4bxk0006927@localhost.localdomain>
Message-ID: <bl36fo$gtc$1@sea.gmane.org>


"Anthony Baxter" <anthony@interlink.com.au> wrote in message
news:200309270437.h8R4bxk0006927@localhost.localdomain...
> I'm not sure what the term "revoke" would mean in this context.

Stop distribution by removing from download page .  Given that it
works fine for some platforms and most uses on the rest, a warning
message seems ok to me.

TJR




From anthony at interlink.com.au  Sat Sep 27 01:16:10 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sat Sep 27 01:18:18 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go 
In-Reply-To: <20030926232840.GB17491@zot.electricrain.com> 
Message-ID: <200309270516.h8R5GAT7007511@localhost.localdomain>


>>> "Gregory P. Smith" wrote
> Its been my impression that the sporatic bsddb testsuite failures
> are BerkeleyDB related rather than anything the python module can be
> responsible for (other than the previously mentioned needed improvements
> in cleaning up the on disk "temporary" test environment before launching
> tests).  For anyone reporting berkeleydb issues, its important to find
> out the version of BerkeleyDB they're using and on what platform.

I seem to recall some concern that using bsddb without a DBEnv being created
could be causing problems - if this is the case, should we consider putting a
deprecation warning into the code for people doing this?

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From anthony at interlink.com.au  Sat Sep 27 01:43:48 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sat Sep 27 01:46:11 2003
Subject: [Python-Dev] plans for 2.3.2
Message-ID: <200309270543.h8R5hmhH008225@localhost.localdomain>


Ok, here's my plans for 2.3.2.

Cut a release candidate on Tuesday (AU time, so Monday for merkins)

Cut a release Thursday (AU time).

I don't think the additional two days will hurt that badly, and I'd
strongly prefer to have a release candidate before the final build.

I don't know if the RC needs a windows installer built, as the problems
are in non-windows platforms.

Anthony


From martin at v.loewis.de  Sat Sep 27 02:55:33 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sat Sep 27 02:55:40 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <m2d6dnntt7.fsf@deacon-blues.mid.mastaler.com>
References: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>
	<200309260524.h8Q5Ok1t013688@localhost.localdomain>
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
	<1064591608.30783.30.camel@anthem> <k77vh784.fsf@python.net>
	<m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com>
	<20030926103846.A26782@discworld.dyndns.org>
	<m2d6dnntt7.fsf@deacon-blues.mid.mastaler.com>
Message-ID: <m3u16ybtzu.fsf@mira.informatik.hu-berlin.de>

"Jason R. Mastaler" <jason@mastaler.com> writes:

> I'd be in favour of revoking 2.3.1 as well because of the severity of
> this bug.

I don't think the problem is severe; os.fsync is rarely used. It is
just that people running into the problem are very vocal about it.

Regards,
Martin


From martin at v.loewis.de  Sat Sep 27 02:58:13 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sat Sep 27 02:58:44 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <200309270516.h8R5GAT7007511@localhost.localdomain>
References: <200309270516.h8R5GAT7007511@localhost.localdomain>
Message-ID: <m3pthmbtve.fsf@mira.informatik.hu-berlin.de>

Anthony Baxter <anthony@interlink.com.au> writes:

> I seem to recall some concern that using bsddb without a DBEnv being
> created could be causing problems - if this is the case, should we
> consider putting a deprecation warning into the code for people
> doing this?

Not until we have determined with certainty that this *is* the case.

Regards,
Martin

From martin at v.loewis.de  Sat Sep 27 03:02:00 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sat Sep 27 03:02:16 2003
Subject: [Python-Dev] How to test for stuff like fsync?
In-Reply-To: <200309270433.h8R4XKwl006833@localhost.localdomain>
References: <200309270433.h8R4XKwl006833@localhost.localdomain>
Message-ID: <m3llsabtp3.fsf@mira.informatik.hu-berlin.de>

Anthony Baxter <anthony@interlink.com.au> writes:

> A thought that occurred to me a short time ago: write a script that outputs
> the contents of each builtin module (the names in the module) to a file.
> 
> Run that under previous version of python. Run it under release candidate.
> Compare differences. 

In this specific case, it would have triggered a false alarm on some
systems. I rewrote the test for fsync to overcome a build problem on
some system, where fsync was present in the library, but not present
in the header files. Since we are taking fsync's address, this causes
a compile-time problem. So it *is* intentional that fsync is now gone
for systems on which it was previously (2.2) present. (More
specifically, fsync might disappear on systems that cannot stand
_XOPEN_SOURCE being defined)

Regards,
Martin

From jason at mastaler.com  Sat Sep 27 03:21:51 2003
From: jason at mastaler.com (Jason R. Mastaler)
Date: Sat Sep 27 03:21:58 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
References: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>
	<200309260524.h8Q5Ok1t013688@localhost.localdomain>
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
	<1064591608.30783.30.camel@anthem> <k77vh784.fsf@python.net>
	<m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com>
	<20030926103846.A26782@discworld.dyndns.org>
	<m2d6dnntt7.fsf@deacon-blues.mid.mastaler.com>
	<m3u16ybtzu.fsf@mira.informatik.hu-berlin.de>
Message-ID: <m2d6dmlmr4.fsf@deacon-blues.mid.mastaler.com>

martin@v.loewis.de (Martin v. L?wis) writes:

> I don't think the problem is severe; os.fsync is rarely used.

How did you arrive at this notion?


From anthony at interlink.com.au  Sat Sep 27 05:06:15 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sat Sep 27 05:11:54 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1 
In-Reply-To: <m2d6dmlmr4.fsf@deacon-blues.mid.mastaler.com> 
Message-ID: <200309270906.h8R96FCT010918@localhost.localdomain>


>>> "Jason R. Mastaler" wrote
> martin@v.loewis.de (Martin v. L?wis) writes:
> 
> > I don't think the problem is severe; os.fsync is rarely used.
> 
> How did you arrive at this notion?

Hm. The only usage of it I can find in the Zope2, Zope3 and Twisted codebases
is a single use inside zodb, and it's wrapped in a check that os.fsync exists 
(since it's not available on all platforms). I also did a grep over all the 
python packages I've downloaded or installed (which is a _lot_) and the TMDA 
code is the only package to use it.

I'm still planning on cutting a 2.3.2 to fix the problem, but I see no reason
to make any further steps to "revoke" 2.3.1 further than the warning message on
the release page - which is mostly aimed at people who might be planning to 
package the code.

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From martin at v.loewis.de  Sat Sep 27 06:30:53 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sat Sep 27 06:31:08 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <m2d6dmlmr4.fsf@deacon-blues.mid.mastaler.com>
References: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>
	<200309260524.h8Q5Ok1t013688@localhost.localdomain>
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
	<1064591608.30783.30.camel@anthem> <k77vh784.fsf@python.net>
	<m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com>
	<20030926103846.A26782@discworld.dyndns.org>
	<m2d6dnntt7.fsf@deacon-blues.mid.mastaler.com>
	<m3u16ybtzu.fsf@mira.informatik.hu-berlin.de>
	<m2d6dmlmr4.fsf@deacon-blues.mid.mastaler.com>
Message-ID: <m3n0cqa5gi.fsf@mira.informatik.hu-berlin.de>

"Jason R. Mastaler" <jason@mastaler.com> writes:

> > I don't think the problem is severe; os.fsync is rarely used.
> 
> How did you arrive at this notion?

>From four considerations:

1. It did not show up in any of the tests.
2. To use it, you have to use "bare" file descriptors; this is
   relatively un-pythonic.
3. To use it, you have to worry about data getting on the disk -
   this really requires some kind of expert application.
4. It is not universally available, so the expert application I had
   to assume in 3) would most likely test for presence of fsync, and
   fall back to just not use it (and perhaps invoke sync(1) instead).

Regards,
Martin

From python at discworld.dyndns.org  Sat Sep 27 12:09:40 2003
From: python at discworld.dyndns.org (Charles Cazabon)
Date: Sat Sep 27 12:04:55 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <m3n0cqa5gi.fsf@mira.informatik.hu-berlin.de>;
	from martin@v.loewis.de on Sat, Sep 27, 2003 at 12:30:53PM +0200
References: <200309260524.h8Q5Ok1t013688@localhost.localdomain>
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
	<1064591608.30783.30.camel@anthem> <k77vh784.fsf@python.net>
	<m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com>
	<20030926103846.A26782@discworld.dyndns.org>
	<m2d6dnntt7.fsf@deacon-blues.mid.mastaler.com>
	<m3u16ybtzu.fsf@mira.informatik.hu-berlin.de>
	<m2d6dmlmr4.fsf@deacon-blues.mid.mastaler.com>
	<m3n0cqa5gi.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20030927100940.E3923@discworld.dyndns.org>

Martin v. L?wis <martin@v.loewis.de> wrote:
> "Jason R. Mastaler" <jason@mastaler.com> writes:
> 
> > > I don't think the problem is severe; os.fsync is rarely used.
> > 
> > How did you arrive at this notion?
> 
> >From four considerations:
[...]
> 3. To use it, you have to worry about data getting on the disk -
>    this really requires some kind of expert application.

Yes, like delivering mail, or writing to a database, or doing any of a
thousand other tasks that use the filesystem to store and synchronize data.

os.fsync() breakage is a /big/ problem, particularly because the resulting
traceback makes users think the application is broken, not the interpreter.

Charles
-- 
-----------------------------------------------------------------------
Charles Cazabon                           <python@discworld.dyndns.org>
GPL'ed software available at:     http://www.qcc.ca/~charlesc/software/
-----------------------------------------------------------------------

From jason at mastaler.com  Sat Sep 27 12:07:18 2003
From: jason at mastaler.com (Jason R. Mastaler)
Date: Sat Sep 27 12:07:23 2003
Subject: [Python-Dev] Re: How to test for stuff like fsync?
References: <16244.50825.588063.154695@montanaro.dyndns.org>
Message-ID: <m27k3ukyfd.fsf@deacon-blues.mid.mastaler.com>

Skip Montanaro <skip@pobox.com> writes:

> I doubt that fsync() is available everywhere (otherwise why test for
> it in the configure script?), so it seems that Jason should have
> been testing for its presence (though see below) and worming around
> its abscence if he wanted TMDA to run on as many platforms as
> possible.

fsync() is POSIX, and TMDA assumes a POSIX-complaint UNIX.  Most mail
servers tend to be such, which is probably why I've never had a
report of an installation lacking fsync().

That's just me though, in general I see your point here.


From jason at mastaler.com  Sat Sep 27 12:14:48 2003
From: jason at mastaler.com (Jason R. Mastaler)
Date: Sat Sep 27 12:14:54 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
References: <m2d6dmlmr4.fsf@deacon-blues.mid.mastaler.com>
	<200309270906.h8R96FCT010918@localhost.localdomain>
Message-ID: <m21xu2ky2v.fsf@deacon-blues.mid.mastaler.com>

Anthony Baxter <anthony@interlink.com.au> writes:

> Hm. The only usage of it I can find in the Zope2, Zope3 and Twisted
> codebases is a single use inside zodb, and it's wrapped in a check
> that os.fsync exists (since it's not available on all platforms). I
> also did a grep over all the python packages I've downloaded or
> installed (which is a _lot_) and the TMDA code is the only package
> to use it.

Also, getmail, as Charles Cazabon mentioned earlier in this thread.
Also, Barry is considering using fsync() in the next release of
Mailman, albeit as an option turned off by default.  TMDA has
thousands of users, Mailman has many more.  So, this potentially
affects many individuals regardless of whether a large number of
packages use fsync() or not.

> I'm still planning on cutting a 2.3.2 to fix the problem, but I see
> no reason to make any further steps to "revoke" 2.3.1 further than
> the warning message on the release page - which is mostly aimed at
> people who might be planning to package the code.

Yeah, I think what you are doing is fine.  If 2.3.2 is available soon,
I think people will just tend to pass over 2.3.1.


From martin at v.loewis.de  Sat Sep 27 12:30:45 2003
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Sep 27 12:30:59 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <20030927100940.E3923@discworld.dyndns.org>
References: <200309260524.h8Q5Ok1t013688@localhost.localdomain>	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>	<1064591608.30783.30.camel@anthem>
	<k77vh784.fsf@python.net>	<m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com>	<20030926103846.A26782@discworld.dyndns.org>	<m2d6dnntt7.fsf@deacon-blues.mid.mastaler.com>	<m3u16ybtzu.fsf@mira.informatik.hu-berlin.de>	<m2d6dmlmr4.fsf@deacon-blues.mid.mastaler.com>	<m3n0cqa5gi.fsf@mira.informatik.hu-berlin.de>
	<20030927100940.E3923@discworld.dyndns.org>
Message-ID: <3F75BB35.8090106@v.loewis.de>

Charles Cazabon wrote:
> os.fsync() breakage is a /big/ problem, particularly because the resulting
> traceback makes users think the application is broken, not the interpreter.

It's not broken: it's absent. In a sense, the application /is/ broken, 
as it should have dealt with os.fsync being absent in the first place.

Applications should be prepared that POSIX functions are not available 
in Python even if the C library provides them on the system, as the 
system headers may fail to declare a prototype for the function (a 
problem which triggered that erroneous change in 2.3.1 in the first 
place). Python is taking the address of fsync, so merely having the 
function in the C library is not enough, nor is it to define fsync as a 
function-style macro. In either case, os.fsync will be rightfully absent.

FWIW, it was a Redhat system that failed to provide a fsync prototype.

Regards,
Martin


From guido at python.org  Sat Sep 27 12:47:03 2003
From: guido at python.org (Guido van Rossum)
Date: Sat Sep 27 12:47:43 2003
Subject: [Python-Dev] How to test for stuff like fsync?
In-Reply-To: Your message of "27 Sep 2003 09:02:00 +0200."
	<m3llsabtp3.fsf@mira.informatik.hu-berlin.de> 
References: <200309270433.h8R4XKwl006833@localhost.localdomain>  
	<m3llsabtp3.fsf@mira.informatik.hu-berlin.de> 
Message-ID: <200309271647.h8RGl3X23590@12-236-54-216.client.attbi.com>

> In this specific case, it would have triggered a false alarm on some
> systems. I rewrote the test for fsync to overcome a build problem on
> some system, where fsync was present in the library, but not present
> in the header files. Since we are taking fsync's address, this causes
> a compile-time problem. So it *is* intentional that fsync is now gone
> for systems on which it was previously (2.2) present. (More
> specifically, fsync might disappear on systems that cannot stand
> _XOPEN_SOURCE being defined)

I don't like this.  We're not taking fsync's address for a very good
reason, just to save some code duplication.  Removing fsync from
those platforms just because our code-saving trick doesn't work there
doesn't strike me as good service for those platforms.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Sat Sep 27 13:08:14 2003
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Sep 27 13:08:22 2003
Subject: [Python-Dev] How to test for stuff like fsync?
In-Reply-To: <200309271647.h8RGl3X23590@12-236-54-216.client.attbi.com>
References: <200309270433.h8R4XKwl006833@localhost.localdomain>
	<m3llsabtp3.fsf@mira.informatik.hu-berlin.de>
	<200309271647.h8RGl3X23590@12-236-54-216.client.attbi.com>
Message-ID: <3F75C3FE.2050408@v.loewis.de>

Guido van Rossum wrote:

> I don't like this.  We're not taking fsync's address for a very good
> reason, just to save some code duplication.  Removing fsync from
> those platforms just because our code-saving trick doesn't work there
> doesn't strike me as good service for those platforms.

It's not only fsync, though - there would be a lot of duplicated code if
we were to convert all function pointer passing back to function calls.

Also, if the system is lacking a function prototype, at the minimum, the
compiler will give a warning - at worst, we invoke the function 
incorrectly, causing a crash.

Regards,
Martin



From jeremy at zope.com  Sat Sep 27 13:28:39 2003
From: jeremy at zope.com (Jeremy Hylton)
Date: Sat Sep 27 13:29:48 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <m3n0cqa5gi.fsf@mira.informatik.hu-berlin.de>
References: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>
	<200309260524.h8Q5Ok1t013688@localhost.localdomain>
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
	<1064591608.30783.30.camel@anthem> <k77vh784.fsf@python.net>
	<m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com>
	<20030926103846.A26782@discworld.dyndns.org>
	<m2d6dnntt7.fsf@deacon-blues.mid.mastaler.com>
	<m3u16ybtzu.fsf@mira.informatik.hu-berlin.de>
	<m2d6dmlmr4.fsf@deacon-blues.mid.mastaler.com>
	<m3n0cqa5gi.fsf@mira.informatik.hu-berlin.de>
Message-ID: <1064683719.19498.68.camel@localhost.localdomain>

On Sat, 2003-09-27 at 06:30, Martin v. L?wis wrote:
> "Jason R. Mastaler" <jason@mastaler.com> writes:
> 
> > > I don't think the problem is severe; os.fsync is rarely used.
> > 
> > How did you arrive at this notion?
> 
> >From four considerations:
> 
> 1. It did not show up in any of the tests.
> 2. To use it, you have to use "bare" file descriptors; this is
>    relatively un-pythonic.
> 3. To use it, you have to worry about data getting on the disk -
>    this really requires some kind of expert application.
> 4. It is not universally available, so the expert application I had
>    to assume in 3) would most likely test for presence of fsync, and
>    fall back to just not use it (and perhaps invoke sync(1) instead).

I suppose ZODB is such an expert application.  It has to cope with
systems that do not provide fsync(), but it provides degraded service on
such platforms.  It is very important for the database to call fsync()
when it commits a transaction.

Jeremy



From jason at mastaler.com  Sat Sep 27 13:17:32 2003
From: jason at mastaler.com (Jason R. Mastaler)
Date: Sat Sep 27 13:34:48 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
References: <200309260524.h8Q5Ok1t013688@localhost.localdomain>
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
	<1064591608.30783.30.camel@anthem> <k77vh784.fsf@python.net>
	<m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com>
	<20030926103846.A26782@discworld.dyndns.org>
	<m2d6dnntt7.fsf@deacon-blues.mid.mastaler.com>
	<m3u16ybtzu.fsf@mira.informatik.hu-berlin.de>
	<m2d6dmlmr4.fsf@deacon-blues.mid.mastaler.com>
	<m3n0cqa5gi.fsf@mira.informatik.hu-berlin.de>
	<20030927100940.E3923@discworld.dyndns.org>
	<3F75BB35.8090106@v.loewis.de>
Message-ID: <m2k77ujglv.fsf@deacon-blues.mid.mastaler.com>

"Martin v. L?wis" <martin@v.loewis.de> writes:

> It's not broken: it's absent.

I'm not sure I see the distinction.  The system provides fsync(), and
the Python documentation says os.fsync() is available, but it isn't.
Python therefore doesn't work as advertised.  This is broken to me.


From martin at v.loewis.de  Sat Sep 27 13:48:48 2003
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Sep 27 13:49:00 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <1064683719.19498.68.camel@localhost.localdomain>
References: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>	
	<200309260524.h8Q5Ok1t013688@localhost.localdomain>	
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>	
	<1064591608.30783.30.camel@anthem> <k77vh784.fsf@python.net>	
	<m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com>	
	<20030926103846.A26782@discworld.dyndns.org>	
	<m2d6dnntt7.fsf@deacon-blues.mid.mastaler.com>	
	<m3u16ybtzu.fsf@mira.informatik.hu-berlin.de>	
	<m2d6dmlmr4.fsf@deacon-blues.mid.mastaler.com>	
	<m3n0cqa5gi.fsf@mira.informatik.hu-berlin.de>
	<1064683719.19498.68.camel@localhost.localdomain>
Message-ID: <3F75CD80.3080008@v.loewis.de>

Jeremy Hylton wrote:

> I suppose ZODB is such an expert application.  It has to cope with
> systems that do not provide fsync(), but it provides degraded service on
> such platforms.  It is very important for the database to call fsync()
> when it commits a transaction.

I mostly agree: ZODB is indeed advanced, and it is indeed a good idea
to check for presence of os.fsync before using it.

While this is OT, I'd still like to question the usefulness of fsync(2)
in the first place, for applications like ZODB. I assume fsync is used
as a write barrier, to make sure old modifications are on disk, before
making new changes. There are several cases in which this might be relevant:
1. The application will crash soon after performing fsync, leaving data
    potentially in an inconsistent state. Here, using fsync is not
    necessary, as the system will still perform all modifications on
    disk, even though the process has long terminated.
2. The application will be kill(2)ed soon after fsync completes (or
    even while fsync completes). Like 1), fsync is not needed.
3. There is an operating system crash (kernel panic or similar).
    fsync does not help, as, for a buggy kernel, anything might have
    happened to the data before.
4. There is a disk failure. fsync does not help, as the data on disk
    might not be recoverable.
5. There is a power outage. This is the case where fsync should help:
    everything up to the write barrier is on disk. Of course, if the
    disk drive itself has write caching, fsync may have completed
    without the data being on the disk (this would be an fsync bug, but
    I believe Linux suffers from this particular bug).

So in short, fsync(2) helps only in case of a power outage; for normal
operation, it is not needed. In the case of a power outage, it is
doubtful whether it has the desired effect.

Slightly more on-topic: os.fsync is even worse, as it cannot be
used to implement a write barrier in case of multiple threads. It
runs with the GIL released, so while fsync is running, other
threads might change the file. The semantics of fsync in this case
is pretty unclear, but it is likely not a write barrier.

Regards,
Martin



From skip at pobox.com  Sat Sep 27 14:11:07 2003
From: skip at pobox.com (Skip Montanaro)
Date: Sat Sep 27 14:12:23 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <m3pthmbtve.fsf@mira.informatik.hu-berlin.de>
References: <200309270516.h8R5GAT7007511@localhost.localdomain>
	<m3pthmbtve.fsf@mira.informatik.hu-berlin.de>
Message-ID: <16245.53947.33728.86658@montanaro.dyndns.org>


    Martin> Anthony Baxter <anthony@interlink.com.au> writes:
    >> I seem to recall some concern that using bsddb without a DBEnv being
    >> created could be causing problems - if this is the case, should we
    >> consider putting a deprecation warning into the code for people doing
    >> this?

    Martin> Not until we have determined with certainty that this *is* the
    Martin> case.

According to info I got from Sleepycat, a DBEnv is required for using in a
multi-threaded environment.  Since the legacy bsddb module API is widely
used and use of threading has increased in the past couple years, I think we
need to figure out how to solve that problem.  This is just a wild-ass guess
(I think I posted something like it before), but maybe all that's needed is
to extend the bsddb.(bt|hash|rn)open functions to accept a dbenv arg, define
a module-level default environment in bsddb/__init__.py which is used as the
dbenv arg if the caller doesn't provide one.  The __init__.py code would
look like this:

    _env = db.DBEnv()

    def hashopen(file, flag='c', mode=0666, pgsize=None, ffactor=None, nelem=None,
                cachesize=None, lorder=None, hflags=0, dbenv=_env):

        flags = _checkflag(flag)
        d = db.DB(dbenv)
        ...

    def btopen(file, flag='c', mode=0666,
                btflags=0, cachesize=None, maxkeypage=None, minkeypage=None,
                pgsize=None, lorder=None, dbenv=_env):

        flags = _checkflag(flag)
        d = db.DB(dbenv)
        ...

    def rnopen(file, flag='c', mode=0666,
                rnflags=0, cachesize=None, pgsize=None, lorder=None,
                rlen=None, delim=None, source=None, pad=None, dbenv=_env):

        flags = _checkflag(flag)
        d = db.DB(dbenv)
        ...

My reading of the bsddb3 docs at
<http://pybsddb.sourceforge.net/bsddb3.html> suggests that should be
sufficient (though certain args may need to be passed to the DBEnv() call).
See <http://python.org/sf/775414>.

Attached is a modified version of the hammer.py script which seems to not
fail for me on either Windows run from IDLE (Python 2.3, BDB 4.1.6) or Mac
OS X (Python CVS, BDB 4.2.1).  The original script failed for me on Windows
but not Mac OS X.  Can some other people for whom the original script fails
please try it?  (I also attached it to bug #775414.)

Skip

-------------- next part --------------
A non-text attachment was scrubbed...
Name: studly_hammer.py
Type: application/octet-stream
Size: 1818 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20030927/b92fcc07/studly_hammer-0001.obj
From theller at python.net  Sat Sep 27 15:02:44 2003
From: theller at python.net (Thomas Heller)
Date: Sat Sep 27 15:03:01 2003
Subject: [Python-Dev] Re: plans for 2.3.2
In-Reply-To: <200309270543.h8R5hmhH008225@localhost.localdomain> (Anthony
	Baxter's message of "Sat, 27 Sep 2003 15:43:48 +1000")
References: <200309270543.h8R5hmhH008225@localhost.localdomain>
Message-ID: <1xu2vyuj.fsf@python.net>

Anthony Baxter <anthony@interlink.com.au> writes:

> Ok, here's my plans for 2.3.2.
>
> Cut a release candidate on Tuesday (AU time, so Monday for merkins)
>
> Cut a release Thursday (AU time).
>
> I don't think the additional two days will hurt that badly, and I'd
> strongly prefer to have a release candidate before the final build.

Will the release candidate have the same version number as the final
version?

> I don't know if the RC needs a windows installer built, as the problems
> are in non-windows platforms.

I don't mind building one.

Thomas


From fdrake at acm.org  Sat Sep 27 15:10:52 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sat Sep 27 15:11:08 2003
Subject: [Python-Dev] Re: plans for 2.3.2
In-Reply-To: <1xu2vyuj.fsf@python.net>
References: <200309270543.h8R5hmhH008225@localhost.localdomain>
	<1xu2vyuj.fsf@python.net>
Message-ID: <16245.57532.609456.554472@grendel.zope.com>


Thomas Heller writes:
 > Will the release candidate have the same version number as the final
 > version?

The typical case has been that the candidate has a distinct version
number.  I think 2.3.1 has taught us to be very careful, so we should
keep the distinction between the candidate and the actual release.

My opinion:  What's "candidate" isn't the actual file we post to
python.org, but the bits in CVS that form the implementation and build
process.  The actual packages are a byproduct of that.

 > > I don't know if the RC needs a windows installer built, as the problems
 > > are in non-windows platforms.
 > 
 > I don't mind building one.

I think there should be one.  They've paid off in the past.

I'm planning to post the 2.3.2c1 documentation late Sunday night or
early Monday morning.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From theller at python.net  Sat Sep 27 15:16:50 2003
From: theller at python.net (Thomas Heller)
Date: Sat Sep 27 15:17:06 2003
Subject: [Python-Dev] [development doc updates]
In-Reply-To: <20030927190724.DDC8818F3E6@grendel.zope.com> (Fred L. Drake's
	message of "Sat, 27 Sep 2003 15:07:24 -0400 (EDT)")
References: <20030927190724.DDC8818F3E6@grendel.zope.com>
Message-ID: <vfreujml.fsf@python.net>

"Fred L. Drake" <fdrake@acm.org> writes:

> The development version of the documentation has been updated:
>
>     http://www.python.org/dev/doc/devel/
>
> The new glossary in the tutorial is really nice!

Sure, but the hyperlinks don't work ;-)

Thomas


From theller at python.net  Sat Sep 27 15:17:46 2003
From: theller at python.net (Thomas Heller)
Date: Sat Sep 27 15:17:59 2003
Subject: [Python-Dev] Re: plans for 2.3.2
In-Reply-To: <1xu2vyuj.fsf@python.net> (Thomas Heller's message of "Sat, 27
	Sep 2003 21:02:44 +0200")
References: <200309270543.h8R5hmhH008225@localhost.localdomain>
	<1xu2vyuj.fsf@python.net>
Message-ID: <r822ujl1.fsf@python.net>

Thomas Heller <theller@python.net> writes:

> Will the release candidate have the same version number as the final
> version?

Never mind, I just missed Fred's checkin to ptachlevel.

Thomas


From jafo at tummy.com  Sat Sep 27 15:19:06 2003
From: jafo at tummy.com (Sean Reifschneider)
Date: Sat Sep 27 15:19:37 2003
Subject: [Python-Dev] Re: plans for 2.3.2
In-Reply-To: <200309270543.h8R5hmhH008225@localhost.localdomain>
References: <200309270543.h8R5hmhH008225@localhost.localdomain>
Message-ID: <20030927191906.GB17740@tummy.com>

On Sat, Sep 27, 2003 at 03:43:48PM +1000, Anthony Baxter wrote:
>Ok, here's my plans for 2.3.2.
>Cut a release candidate on Tuesday (AU time, so Monday for merkins)
>Cut a release Thursday (AU time).

That sounds fine, I can do a test build and put up sample RPMs for the
RC, and get the final pushed up on Thursday-ish.

Sean
-- 
 Men are from Mars. Women are from Venus. Computers are from hell.
Sean Reifschneider, Member of Technical Staff <jafo@tummy.com>
tummy.com, ltd. - Linux Consulting since 1995.  Qmail, Python, SysAdmin
      Back off man. I'm a scientist.   http://HackingSociety.org/

From theller at python.net  Sat Sep 27 15:21:26 2003
From: theller at python.net (Thomas Heller)
Date: Sat Sep 27 15:21:40 2003
Subject: [Python-Dev] Re: plans for 2.3.2
In-Reply-To: <16245.57532.609456.554472@grendel.zope.com> (Fred L. Drake,
	Jr.'s message of "Sat, 27 Sep 2003 15:10:52 -0400")
References: <200309270543.h8R5hmhH008225@localhost.localdomain>
	<1xu2vyuj.fsf@python.net> <16245.57532.609456.554472@grendel.zope.com>
Message-ID: <llsaujex.fsf@python.net>

"Fred L. Drake, Jr." <fdrake@acm.org> writes:

> Thomas Heller writes:
>  > Will the release candidate have the same version number as the final
>  > version?
>
> The typical case has been that the candidate has a distinct version
> number.  I think 2.3.1 has taught us to be very careful, so we should
> keep the distinction between the candidate and the actual release.
>
> My opinion:  What's "candidate" isn't the actual file we post to
> python.org, but the bits in CVS that form the implementation and build
> process.  The actual packages are a byproduct of that.

We've seen with 2.3.1 that this is only theory, at least in part ;-)

>  > > I don't know if the RC needs a windows installer built, as the problems
>  > > are in non-windows platforms.
>  > 
>  > I don't mind building one.
>
> I think there should be one.  They've paid off in the past.

Thomas


From martin at v.loewis.de  Sat Sep 27 16:39:32 2003
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Sep 27 16:39:49 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <16245.53947.33728.86658@montanaro.dyndns.org>
References: <200309270516.h8R5GAT7007511@localhost.localdomain>
	<m3pthmbtve.fsf@mira.informatik.hu-berlin.de>
	<16245.53947.33728.86658@montanaro.dyndns.org>
Message-ID: <3F75F584.8040207@v.loewis.de>

Skip Montanaro wrote:

> According to info I got from Sleepycat, a DBEnv is required for using in a
> multi-threaded environment.  Since the legacy bsddb module API is widely
> used and use of threading has increased in the past couple years, I think we
> need to figure out how to solve that problem.  This is just a wild-ass guess
> (I think I posted something like it before), but maybe all that's needed is
> to extend the bsddb.(bt|hash|rn)open functions to accept a dbenv arg, define
> a module-level default environment in bsddb/__init__.py which is used as the
> dbenv arg if the caller doesn't provide one.  The __init__.py code would
> look like this:
> 
>     _env = db.DBEnv()

I think we should define a set of flags for the "common case". In 
general, multiple threads may write to the same database, as might 
multiple instances of the application. IOW, we might want to create
the environment with DB_INIT_CDB|DB_INIT_MPOOL|DB_THREAD.

Applications that don't want to suffer from the possible
serialization of CDB would need to use their own environment.

Regards,
Martin



From theller at python.net  Sat Sep 27 16:40:28 2003
From: theller at python.net (Thomas Heller)
Date: Sat Sep 27 16:40:42 2003
Subject: [Python-Dev] PC/python_nt.rc
Message-ID: <eky2ufr7.fsf@python.net>

The file PC/python_nt.rc defines the version information which will be
embedded as resource in pythonXX.dll.

Because of limitations in MS resource compiler, the info cannot be
constructed from patchlevel.h at compile time, instead it must be
changed manually. There is a python script PC/field3.py which helps, but
it cannot be used by the build process because python is then not
available.

I'm working on a way to automate this:

A new project is inserted into the MSVC project file, and the pythoncore
project (which builds the python dll) will depend on it, so this new
project is built first.

This new project builds a small exe which will create (as custom build
step) an include file for python_nt.rc, so no manual step is needed any
more.

Does this sound like a sensible approach, or are there any objections?

Thomas


From guido at python.org  Sat Sep 27 16:49:54 2003
From: guido at python.org (Guido van Rossum)
Date: Sat Sep 27 16:50:36 2003
Subject: [Python-Dev] How to test for stuff like fsync?
In-Reply-To: Your message of "Sat, 27 Sep 2003 19:08:14 +0200."
	<3F75C3FE.2050408@v.loewis.de> 
References: <200309270433.h8R4XKwl006833@localhost.localdomain>
	<m3llsabtp3.fsf@mira.informatik.hu-berlin.de>
	<200309271647.h8RGl3X23590@12-236-54-216.client.attbi.com> 
	<3F75C3FE.2050408@v.loewis.de> 
Message-ID: <200309272049.h8RKnso23818@12-236-54-216.client.attbi.com>

> Guido van Rossum wrote:
> > I don't like this.  We're not taking fsync's address for a very good
> > reason, just to save some code duplication.  Removing fsync from
> > those platforms just because our code-saving trick doesn't work there
> > doesn't strike me as good service for those platforms.

[MvL]
> It's not only fsync, though - there would be a lot of duplicated code if
> we were to convert all function pointer passing back to function calls.

Well, too bad.  We could use macros instead.  I believe I started
taking the function address many years ago, and I just did it as a
trick to save some typing, not to save code size.  I think it's wrong
to use it as an argument against including a function on a platform
because the platform doesn't support that trick.

> Also, if the system is lacking a function prototype, at the minimum, the
> compiler will give a warning - at worst, we invoke the function 
> incorrectly, causing a crash.

For fsync(), invoking it wrongly sounds unlikely.  I'm sure the users
of fsync() would gladly take the warning if they can have the
function.

Is there really no function prototype in the system headers for
fsync() on those systems, or don't we see it because we're using
feature selection macros (isn't that the right term?) that hide it?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at electricrain.com  Sat Sep 27 16:51:47 2003
From: greg at electricrain.com (Gregory P. Smith)
Date: Sat Sep 27 16:51:51 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <3F75F584.8040207@v.loewis.de>
References: <200309270516.h8R5GAT7007511@localhost.localdomain>
	<m3pthmbtve.fsf@mira.informatik.hu-berlin.de>
	<16245.53947.33728.86658@montanaro.dyndns.org>
	<3F75F584.8040207@v.loewis.de>
Message-ID: <20030927205147.GH17491@zot.electricrain.com>

> >According to info I got from Sleepycat, a DBEnv is required for using in a
> >multi-threaded environment.  Since the legacy bsddb module API is widely
> >used and use of threading has increased in the past couple years, I think 
> >we
> >need to figure out how to solve that problem.  This is just a wild-ass 
> >guess
> >(I think I posted something like it before), but maybe all that's needed is
> >to extend the bsddb.(bt|hash|rn)open functions to accept a dbenv arg, 
> >define
> >a module-level default environment in bsddb/__init__.py which is used as 
> >the
> >dbenv arg if the caller doesn't provide one.  The __init__.py code would
> >look like this:
> >
> >    _env = db.DBEnv()
> 
> I think we should define a set of flags for the "common case". In 
> general, multiple threads may write to the same database, as might 
> multiple instances of the application. IOW, we might want to create
> the environment with DB_INIT_CDB|DB_INIT_MPOOL|DB_THREAD.

It is worth noting that using a DBEnv in this manner will create a bunch
of auxilary DBEnv related files on the disk.  That has the potential of
confusing people who expect only the database file.  It also means that
separate unrelated databases cannot exist in the same directory without
being part of the same DBEnv (which BerkeleyDB multiprocess access should
handle just fine; but it might not be what people expect).

> Applications that don't want to suffer from the possible
> serialization of CDB would need to use their own environment.

An alternative would be to say that applications that want to use bsddb
with threading need to use the *real* BerkeleyDB API rather than the
ancient compatibility interface.  (it'd be easy to check that bsddb
doesn't get used by multiple threads, raising an exception if it does)

greg


From theller at python.net  Sat Sep 27 16:51:52 2003
From: theller at python.net (Thomas Heller)
Date: Sat Sep 27 16:52:07 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <2mzngr1bf2.fsf@starship.python.net> (Michael Hudson's message
	of "Fri, 26 Sep 2003 22:34:41 +0100")
References: <200309261752.h8QHqJgD029823@localhost.localdomain>
	<LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
	<20030926184853.GB22837@mems-exchange.org> <he2zba58.fsf@python.net>
	<2mzngr1bf2.fsf@starship.python.net>
Message-ID: <8yoauf87.fsf@python.net>

[cc to webmaster@starship.python.net]

Michael Hudson <mwh@python.net> writes:

> Thomas Heller <theller@python.net> writes:
>
>> And it would help if I could build the HTML docs myself from CVS. I did
>> manage to create the pdf files with TeTex under windows, but I didn't
>> succeed with the html pages so far.
>
> It occurs to me that I don't know *why* Fred is so much the
> documentation man; I've not had any trouble processing the docs into
> HTML lately (haven't tried on Windows, admittedly, and I haven't
> tried to make info ever).

I could do it myself if the tools needed to build the docs would be
available on the starship. Last time I checked, neither Tex nor
latex2html were installed.

So, I hereby kindly request: Could the tools to build the Python docs
from source please be installed on the starship?

Thanks,

Thomas


From martin at v.loewis.de  Sat Sep 27 16:57:36 2003
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Sep 27 16:57:49 2003
Subject: [Python-Dev] PC/python_nt.rc
In-Reply-To: <eky2ufr7.fsf@python.net>
References: <eky2ufr7.fsf@python.net>
Message-ID: <3F75F9C0.8040504@v.loewis.de>

Thomas Heller wrote:
> Does this sound like a sensible approach, or are there any objections?

Would it be possible to use VBScript instead? That should be available
at any phase of the build process.

While we are at it: At what point will we switch to VC7.1 for the 
Windows builds?

Regards,
Martin


From martin at v.loewis.de  Sat Sep 27 17:12:17 2003
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Sep 27 17:12:39 2003
Subject: [Python-Dev] How to test for stuff like fsync?
In-Reply-To: <200309272049.h8RKnso23818@12-236-54-216.client.attbi.com>
References: <200309270433.h8R4XKwl006833@localhost.localdomain>	<m3llsabtp3.fsf@mira.informatik.hu-berlin.de>	<200309271647.h8RGl3X23590@12-236-54-216.client.attbi.com>
	<3F75C3FE.2050408@v.loewis.de>
	<200309272049.h8RKnso23818@12-236-54-216.client.attbi.com>
Message-ID: <3F75FD31.4010103@v.loewis.de>

Guido van Rossum wrote:

> Is there really no function prototype in the system headers for
> fsync() on those systems, or don't we see it because we're using
> feature selection macros (isn't that the right term?) that hide it?

I really can't answer that for all systems that lost fsync; I only
know about a single one. I earlier said it was Redhat, but I recalled
that incorrectly; it was "a 2-year-old Linux box running slackware"
(no idea what slackware version). It is also unclear what specific
version of the C library headers where installed on the system; the
compiler is gcc 2.7.2.

On that system, fsync is only available if __USE_BSD is defined,
see python.org/sf/800710. So on the surface, it looks like a feature
selection macro which we fail to define (_BSD_SOURCE), instead of
a feature selection macro that we do define. OTOH, it might be
that the definition of _XOPEN_SOURCE on that system would cause
__USE_BSD not to be defined - I don't have a copy of features.h
of the particular system.

In recent glibc versions, fsync is declared if either __USE_BSD or 
__USE_XOPEN is defined.

Regards,
Martin




From theller at python.net  Sat Sep 27 17:15:11 2003
From: theller at python.net (Thomas Heller)
Date: Sat Sep 27 17:15:26 2003
Subject: [Python-Dev] PC/python_nt.rc
In-Reply-To: <3F75F9C0.8040504@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Sat,
	27 Sep 2003 22:57:36 +0200")
References: <eky2ufr7.fsf@python.net> <3F75F9C0.8040504@v.loewis.de>
Message-ID: <1xu2ue5c.fsf@python.net>

"Martin v. L?wis" <martin@v.loewis.de> writes:

> Thomas Heller wrote:
>> Does this sound like a sensible approach, or are there any objections?
>
> Would it be possible to use VBScript instead? That should be available
> at any phase of the build process.

I'm much more comfortable to write the following C program than to parse
patchlevel.h in VB.  What would the advantage of VB be?

#include <stdio.h>
#include "patchlevel.h"

int main(int argc, char **argv)
{
    printf("#define FIELD3 %d\n",
        PY_MICRO_VERSION*1000 + PY_RELEASE_LEVEL*10
        + PY_RELEASE_SERIAL);
    /* some more fields needed */
    return 0;
}


>
> While we are at it: At what point will we switch to VC7.1 for the
> Windows builds?

Hm, for 2.4, maybe.

Or should there 7.1 compiled .msi installers in parallel for the 2.3 line?

Thomas


From martin at v.loewis.de  Sat Sep 27 17:17:46 2003
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Sep 27 17:17:57 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <20030927205147.GH17491@zot.electricrain.com>
References: <200309270516.h8R5GAT7007511@localhost.localdomain>
	<m3pthmbtve.fsf@mira.informatik.hu-berlin.de>
	<16245.53947.33728.86658@montanaro.dyndns.org>
	<3F75F584.8040207@v.loewis.de>
	<20030927205147.GH17491@zot.electricrain.com>
Message-ID: <3F75FE7A.6070800@v.loewis.de>

Gregory P. Smith wrote:

> It is worth noting that using a DBEnv in this manner will create a bunch
> of auxilary DBEnv related files on the disk.  That has the potential of
> confusing people who expect only the database file.  It also means that
> separate unrelated databases cannot exist in the same directory without
> being part of the same DBEnv (which BerkeleyDB multiprocess access should
> handle just fine; but it might not be what people expect).

I see. I could accept that it is confusing; would it also be backwards
compatible (i.e. would BerkeleyDB create those files on demand, and
would old Python installation still be able to read the database even
if those files where around)?

> An alternative would be to say that applications that want to use bsddb
> with threading need to use the *real* BerkeleyDB API rather than the
> ancient compatibility interface.  (it'd be easy to check that bsddb
> doesn't get used by multiple threads, raising an exception if it does)

But is it also easy to detect that multiple applications try to use
the same database?

Regards,
Martin



From greg at electricrain.com  Sat Sep 27 17:33:32 2003
From: greg at electricrain.com (Gregory P. Smith)
Date: Sat Sep 27 17:33:40 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <3F75FE7A.6070800@v.loewis.de>
References: <200309270516.h8R5GAT7007511@localhost.localdomain>
	<m3pthmbtve.fsf@mira.informatik.hu-berlin.de>
	<16245.53947.33728.86658@montanaro.dyndns.org>
	<3F75F584.8040207@v.loewis.de>
	<20030927205147.GH17491@zot.electricrain.com>
	<3F75FE7A.6070800@v.loewis.de>
Message-ID: <20030927213332.GI17491@zot.electricrain.com>

On Sat, Sep 27, 2003 at 11:17:46PM +0200, "Martin v. L?wis" wrote:
> Gregory P. Smith wrote:
> 
> >It is worth noting that using a DBEnv in this manner will create a bunch
> >of auxilary DBEnv related files on the disk.  That has the potential of
> >confusing people who expect only the database file.  It also means that
> >separate unrelated databases cannot exist in the same directory without
> >being part of the same DBEnv (which BerkeleyDB multiprocess access should
> >handle just fine; but it might not be what people expect).
> 
> I see. I could accept that it is confusing; would it also be backwards
> compatible (i.e. would BerkeleyDB create those files on demand, and
> would old Python installation still be able to read the database even
> if those files where around)?

It would not.  Its already not backwards compatible.  Thats what the
bsddb185 module is for.

The BerkeleyDB file format changed between 3.1, 3.2 and 4.0 even.
BerkeleyDB can upgrade from an older format to the current one (using
the DB.upgrade method).

> >An alternative would be to say that applications that want to use bsddb
> >with threading need to use the *real* BerkeleyDB API rather than the
> >ancient compatibility interface.  (it'd be easy to check that bsddb
> >doesn't get used by multiple threads, raising an exception if it does)
> 
> But is it also easy to detect that multiple applications try to use
> the same database?

No.  But it is easy to document.  The old bsddb module never allowed
it either.  I only suggest detecting multithreaded access because it
is possible to do.  Anyone who wants multiprocess access should use the
real DBEnv+DB interface and know what they're doing.

-g


From skip at pobox.com  Sat Sep 27 15:50:49 2003
From: skip at pobox.com (Skip Montanaro)
Date: Sat Sep 27 17:55:18 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <m21xu2ky2v.fsf@deacon-blues.mid.mastaler.com>
References: <m2d6dmlmr4.fsf@deacon-blues.mid.mastaler.com>
	<200309270906.h8R96FCT010918@localhost.localdomain>
	<m21xu2ky2v.fsf@deacon-blues.mid.mastaler.com>
Message-ID: <16245.59929.264386.827541@montanaro.dyndns.org>


    Jason> Also, Barry is considering using fsync() in the next release of
    Jason> Mailman, albeit as an option turned off by default.  

Something tells me Barry will probably guard its use with a hasattr(). ;-)

Skip

From skip at pobox.com  Sat Sep 27 18:07:57 2003
From: skip at pobox.com (Skip Montanaro)
Date: Sat Sep 27 18:08:11 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <3F75F584.8040207@v.loewis.de>
References: <200309270516.h8R5GAT7007511@localhost.localdomain>
	<m3pthmbtve.fsf@mira.informatik.hu-berlin.de>
	<16245.53947.33728.86658@montanaro.dyndns.org>
	<3F75F584.8040207@v.loewis.de>
Message-ID: <16246.2621.472392.671642@montanaro.dyndns.org>


    Martin> Applications that don't want to suffer from the possible
    Martin> serialization of CDB would need to use their own environment.

I've given that a little more thought.  Instead of encumbering the factory
functions which implement the old API with an optional dbenv argument, I
think if people want to provide their own environment it's reasonable to 
expect them to use the new bsddb3 API.  All we should do with the old API is
make it work in a multi-threaded environment if possible.

Skip

From skip at pobox.com  Sat Sep 27 18:09:37 2003
From: skip at pobox.com (Skip Montanaro)
Date: Sat Sep 27 18:09:49 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <20030927205147.GH17491@zot.electricrain.com>
References: <200309270516.h8R5GAT7007511@localhost.localdomain>
	<m3pthmbtve.fsf@mira.informatik.hu-berlin.de>
	<16245.53947.33728.86658@montanaro.dyndns.org>
	<3F75F584.8040207@v.loewis.de>
	<20030927205147.GH17491@zot.electricrain.com>
Message-ID: <16246.2721.881433.68068@montanaro.dyndns.org>


    Greg> An alternative would be to say that applications that want to use
    Greg> bsddb with threading need to use the *real* BerkeleyDB API rather
    Greg> than the ancient compatibility interface.  (it'd be easy to check
    Greg> that bsddb doesn't get used by multiple threads, raising an
    Greg> exception if it does)

Well, perhaps a warning.  I doubt you can tell if the programmer has
provided his own locks around db accesses.

Skip


From greg at electricrain.com  Sat Sep 27 19:09:07 2003
From: greg at electricrain.com (Gregory P. Smith)
Date: Sat Sep 27 19:09:11 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <16246.2621.472392.671642@montanaro.dyndns.org>
References: <200309270516.h8R5GAT7007511@localhost.localdomain>
	<m3pthmbtve.fsf@mira.informatik.hu-berlin.de>
	<16245.53947.33728.86658@montanaro.dyndns.org>
	<3F75F584.8040207@v.loewis.de>
	<16246.2621.472392.671642@montanaro.dyndns.org>
Message-ID: <20030927230907.GJ17491@zot.electricrain.com>

On Sat, Sep 27, 2003 at 05:07:57PM -0500, Skip Montanaro wrote:
> 
>     Martin> Applications that don't want to suffer from the possible
>     Martin> serialization of CDB would need to use their own environment.
> 
> I've given that a little more thought.  Instead of encumbering the factory
> functions which implement the old API with an optional dbenv argument, I
> think if people want to provide their own environment it's reasonable to 
> expect them to use the new bsddb3 API.  All we should do with the old API is
> make it work in a multi-threaded environment if possible.
> 
> Skip

Agreed, that sounds like a good option.

In my previous email I had forgotten about the DB_PRIVATE flag to
DBEnv objects; that prevents them from writing extra DBEnv files to the
filesystem for use when multi-process access is not needed.

I just committed the small change needed to bsddb/__init__.py for it to
use a DBEnv allowing multithreaded access.  The original hammer.py from
bug 775414 has been running for 15 minutes without problems on my alpha
with BerkeleyDB 4.1.25.

-g


From tim.one at comcast.net  Sat Sep 27 21:47:05 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sat Sep 27 21:53:13 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <1064683719.19498.68.camel@localhost.localdomain>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEPBGCAB.tim.one@comcast.net>

[Jeremy Hylton]
> I suppose ZODB is such an expert application.  It has to cope with
> systems that do not provide fsync(), but it provides degraded service
> on such platforms.  It is very important for the database to call
> fsync() when it commits a transaction.

I'm not sure that ZODB *intends* to run on any system w/o fsync anymore.  It
had to before 2.3 because os.fsync() didn't exist on Windows, and ZODB has
to run on Windows.  I implemented os.fysnc() for Windows in 2.3 precisely so
that ZODB would work better on Windows.

More mundanely, as I recall, we were having a specific problem wherein a
backup script wasn't getting the correct size for a ZODB Data.fs file on
Windows, and file.flush()'ing Data.fs didn't help enough -- backup kept
screwing up until FileStorage's

            if fsync is not None: fsync(file.fileno())

actually did something on Windows.


From guido at python.org  Sat Sep 27 22:03:38 2003
From: guido at python.org (Guido van Rossum)
Date: Sat Sep 27 22:04:17 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: Your message of "Sat, 27 Sep 2003 21:47:05 EDT."
	<LNBBLJKPBEHFEDALKOLCOEPBGCAB.tim.one@comcast.net> 
References: <LNBBLJKPBEHFEDALKOLCOEPBGCAB.tim.one@comcast.net> 
Message-ID: <200309280203.h8S23cA24143@12-236-54-216.client.attbi.com>

> [Jeremy Hylton]
> > I suppose ZODB is such an expert application.  It has to cope with
> > systems that do not provide fsync(), but it provides degraded service
> > on such platforms.  It is very important for the database to call
> > fsync() when it commits a transaction.

[Tim]
> I'm not sure that ZODB *intends* to run on any system w/o fsync anymore.  It
> had to before 2.3 because os.fsync() didn't exist on Windows, and ZODB has
> to run on Windows.  I implemented os.fysnc() for Windows in 2.3 precisely so
> that ZODB would work better on Windows.
> 
> More mundanely, as I recall, we were having a specific problem wherein a
> backup script wasn't getting the correct size for a ZODB Data.fs file on
> Windows, and file.flush()'ing Data.fs didn't help enough -- backup kept
> screwing up until FileStorage's
> 
>             if fsync is not None: fsync(file.fileno())
> 
> actually did something on Windows.

Bizarre.  On Unix, fsync()'s only point is to do something that is
only ever noticeable if there's an OS or hardware failure.  But on
Windows, it was needed to get the OS to synchronize the directory
contents so that another process or thread would see the right thing.

Anyway, I think that on Unix, that problem doesn't exist, and there
the only problem of not having fsync() is the minuscule risk of
hardware or OS failure right after you wrote the data but before it
actually was written to the media.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From tim.one at comcast.net  Sat Sep 27 22:08:31 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sat Sep 27 22:08:38 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <3F75CD80.3080008@v.loewis.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEPDGCAB.tim.one@comcast.net>

[Martin v. L?wis]
> I mostly agree: ZODB is indeed advanced, and it is indeed a good idea
> to check for presence of os.fsync before using it.
>
> While this is OT, I'd still like to question the usefulness of
> fsync(2) in the first place, for applications like ZODB. I assume
> fsync is used as a write barrier, to make sure old modifications are
> on disk, before making new changes.

That's important, but not primarily for the catastrophic error-recovery
scenarios you go on to sketch.

The most important error-recovery procedure is preventative, running backups
against a ZODB database while the database is active (Tools/repozo.py in a
recent ZODB distribution is the right tool for this).  Since a ZODB process
may run for months, it's not practical to say that backups require shutting
ZODB down.

Without both flushing and fsync'ing, the backup process can't get at all the
data that's "really" in the file.  Here's a little Python driver:

import os
fp = file('test.dat', 'wb')
guts = 'x' * 1000000

n = 0
while 1:
    fp.write(guts)
    fp.flush()
    os.fsync(fp.fileno())
    n += len(guts)
    proceed = raw_input("wrote %d so far; hit key" % n)

At least on Windows, both the flush and the fsync are necessary to see one
million bytes (via a different process) at the first prompt, two million at
the second, and so on.  With neither, another process typically sees 0 bytes
before the file gets huge.  With just one of them, it seems hard to predict,
ranging from 0 to "almost" a million additional bytes per prompt.

ZODB does a flush and an fsync after each transaction, so that the backup
script (or any other distinct process) sees the most recent data available.

Besides missing large gobs of newer data, without the fsync the backup
script may see incomplete data for the most recent transaction that managed
to wind up on disk, and erroneously conclude that the database is corrupted.

In short, fsync is necessary to support ZODB best practice.


From barry at python.org  Sun Sep 28 00:42:30 2003
From: barry at python.org (Barry Warsaw)
Date: Sun Sep 28 00:42:38 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <16246.2621.472392.671642@montanaro.dyndns.org>
References: <200309270516.h8R5GAT7007511@localhost.localdomain>
	<m3pthmbtve.fsf@mira.informatik.hu-berlin.de>
	<16245.53947.33728.86658@montanaro.dyndns.org>
	<3F75F584.8040207@v.loewis.de>
	<16246.2621.472392.671642@montanaro.dyndns.org>
Message-ID: <1064724149.31604.83.camel@anthem>

On Sat, 2003-09-27 at 18:07, Skip Montanaro wrote:
>     Martin> Applications that don't want to suffer from the possible
>     Martin> serialization of CDB would need to use their own environment.
> 
> I've given that a little more thought.  Instead of encumbering the factory
> functions which implement the old API with an optional dbenv argument, I
> think if people want to provide their own environment it's reasonable to 
> expect them to use the new bsddb3 API.  All we should do with the old API is
> make it work in a multi-threaded environment if possible.

+1
-Barry



From itamar at itamarst.org  Sun Sep 28 00:52:52 2003
From: itamar at itamarst.org (Itamar Shtull-Trauring)
Date: Sun Sep 28 00:53:03 2003
Subject: [Python-Dev] Procedures for submitting patches to pybsddb
Message-ID: <20030928005252.6fd1b4b9.itamar@itamarst.org>

I have a patch (DBCursor.get_current_size(), returns size of data for
current entry) which I'd like to submit. This involves changes to
pybsddb cvs as well as python cvs, from what I can tell (for tests and
docs in the pybsddb repo).

While I have developer access to pybsddb, I don't have it for Python.
Submitting patches for two different repositories seems cumbersome, so
where should I add it? Python SF tracker?

-- 
Itamar Shtull-Trauring    http://itamarst.org/
Available for Python & Twisted consulting

From barry at python.org  Sun Sep 28 01:01:19 2003
From: barry at python.org (Barry Warsaw)
Date: Sun Sep 28 01:01:27 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <m21xu2ky2v.fsf@deacon-blues.mid.mastaler.com>
References: <m2d6dmlmr4.fsf@deacon-blues.mid.mastaler.com>
	<200309270906.h8R96FCT010918@localhost.localdomain>
	<m21xu2ky2v.fsf@deacon-blues.mid.mastaler.com>
Message-ID: <1064725279.31604.97.camel@anthem>

On Sat, 2003-09-27 at 12:14, Jason R.Mastaler wrote:

> Also, Barry is considering using fsync() in the next release of
> Mailman, albeit as an option turned off by default. 

It'll be in Mailman 2.1.3, but turned off by default.  I plan on adding
something to the release notes about it, warning that it may not be
available on every system (although we don't have to worry about Windows
<wink>).  Its use shouldn't be enabled if it's not available.

-Barry



From anthony at interlink.com.au  Sun Sep 28 01:03:39 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sun Sep 28 01:05:55 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go 
In-Reply-To: <1064724149.31604.83.camel@anthem> 
Message-ID: <200309280503.h8S53e3P005335@localhost.localdomain>


>>> Barry Warsaw wrote
> On Sat, 2003-09-27 at 18:07, Skip Montanaro wrote:
> > I've given that a little more thought.  Instead of encumbering the factory
> > functions which implement the old API with an optional dbenv argument, I
> > think if people want to provide their own environment it's reasonable to 
> > expect them to use the new bsddb3 API.  All we should do with the old API is
> > make it work in a multi-threaded environment if possible.
> 
> +1
> -Barry

I'm +1 on this as well, but with the caveat that if we _can't_ make it work
reliably in a MT environment, we should either decline the opportunity to
mangle the user's data, or at _least_ issue a warning.

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From barry at python.org  Sun Sep 28 01:06:55 2003
From: barry at python.org (Barry Warsaw)
Date: Sun Sep 28 01:07:03 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <16245.59929.264386.827541@montanaro.dyndns.org>
References: <m2d6dmlmr4.fsf@deacon-blues.mid.mastaler.com>
	<200309270906.h8R96FCT010918@localhost.localdomain>
	<m21xu2ky2v.fsf@deacon-blues.mid.mastaler.com>
	<16245.59929.264386.827541@montanaro.dyndns.org>
Message-ID: <1064725614.31604.99.camel@anthem>

On Sat, 2003-09-27 at 15:50, Skip Montanaro wrote:
>     Jason> Also, Barry is considering using fsync() in the next release of
>     Jason> Mailman, albeit as an option turned off by default.  
> 
> Something tells me Barry will probably guard its use with a hasattr(). ;-)

Actually, no, but as of right now, it has to be explicitly enabled, so
it shouldn't be when there's no fsync <fwink>.
-Barry



From barry at python.org  Sun Sep 28 01:24:27 2003
From: barry at python.org (Barry Warsaw)
Date: Sun Sep 28 01:24:35 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <3F75CD80.3080008@v.loewis.de>
References: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>
	<200309260524.h8Q5Ok1t013688@localhost.localdomain>
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
	<1064591608.30783.30.camel@anthem> <k77vh784.fsf@python.net>
	<m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com>
	<20030926103846.A26782@discworld.dyndns.org>
	<m2d6dnntt7.fsf@deacon-blues.mid.mastaler.com>
	<m3u16ybtzu.fsf@mira.informatik.hu-berlin.de>
	<m2d6dmlmr4.fsf@deacon-blues.mid.mastaler.com>
	<m3n0cqa5gi.fsf@mira.informatik.hu-berlin.de>
	<1064683719.19498.68.camel@localhost.localdomain>
	<3F75CD80.3080008@v.loewis.de>
Message-ID: <1064726667.31604.105.camel@anthem>

On Sat, 2003-09-27 at 13:48, "Martin v. L?wis" wrote:

> 5. There is a power outage. This is the case where fsync should help:
>     everything up to the write barrier is on disk. Of course, if the
>     disk drive itself has write caching, fsync may have completed
>     without the data being on the disk (this would be an fsync bug, but
>     I believe Linux suffers from this particular bug).
> 
> So in short, fsync(2) helps only in case of a power outage; for normal
> operation, it is not needed. In the case of a power outage, it is
> doubtful whether it has the desired effect.

This was the situation that folks reported causing message loss in
Mailman 2.1.2.  Those folks also reported that adding the fsync helped
them (I guess a UPS would have helped too <wink>).  I've added it to
MM2.1.3, but I haven't enabled it by default.  I'm still not sure the
specific problem can be fixed in better ways (e.g. running on a more
reliable host, running it on a sync'ing filesystem, or maybe opening the
file with O_SYNC).  Also, I did some benchmarking with fsync's added and
on the system I tested I saw a 97% performance hit when always
fsync'ing.  That made me nervous.

Before I enable it unconditionally, I'd want to be more certain that
it's necessary, or useful for the majority of sites, and that the
performance hit is either worth it, not as bad as my benchmarks showed,
or at least not as observably bad under real-world conditions.

-Barry



From barry at python.org  Sun Sep 28 01:26:46 2003
From: barry at python.org (Barry Warsaw)
Date: Sun Sep 28 01:26:56 2003
Subject: [Python-Dev] Procedures for submitting patches to pybsddb
In-Reply-To: <20030928005252.6fd1b4b9.itamar@itamarst.org>
References: <20030928005252.6fd1b4b9.itamar@itamarst.org>
Message-ID: <1064726806.31604.107.camel@anthem>

On Sun, 2003-09-28 at 00:52, Itamar Shtull-Trauring wrote:

> While I have developer access to pybsddb, I don't have it for Python.
> Submitting patches for two different repositories seems cumbersome, so
> where should I add it? Python SF tracker?

Yes, because the pybsddb project weaves the actual bsddb files from the
Python project.  pybsddb is now just a distutils/documentation shell
project.

-Barry



From pf_moore at yahoo.co.uk  Sun Sep 28 07:47:31 2003
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Sun Sep 28 07:47:32 2003
Subject: [Python-Dev] Re: PC/python_nt.rc
References: <eky2ufr7.fsf@python.net> <3F75F9C0.8040504@v.loewis.de>
	<1xu2ue5c.fsf@python.net>
Message-ID: <brt5rv70.fsf@yahoo.co.uk>

Thomas Heller <theller@python.net> writes:

>> While we are at it: At what point will we switch to VC7.1 for the
>> Windows builds?
>
> Hm, for 2.4, maybe.

That sounds reasonable. The currently released mingw is close to being
able to build MSVC7.1 compatible DLLs, but not quite there yet (it
needs a workaround). I'd expect it to be there by the time 2.4 is
released.

> Or should there 7.1 compiled .msi installers in parallel for the 2.3 line?

That would be extremely useful - my view on mingw above is based on
import table inspection only. It would be nice to have a way of
testing genuine Python extensions built with mingw.

Of course, a 7.1-compiled installer would need to allow for
installation in parallel with the 6.0-compiled one, because I can't
imagine many people wanting to use it as their production version
(most extensions will only be distributed as 6.0-compiled versions, so
wouldn't be usable with the 7.1-compiled Python).

Paul.
-- 
This signature intentionally left blank


From skip at manatee.mojam.com  Sun Sep 28 08:00:36 2003
From: skip at manatee.mojam.com (Skip Montanaro)
Date: Sun Sep 28 08:00:50 2003
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200309281200.h8SC0aAS011740@manatee.mojam.com>


Bug/Patch Summary
-----------------

507 open / 4175 total bugs (+25)
201 open / 2395 total patches (+10)

New Bugs
--------

exit() raises exception (2003-09-21)
	http://python.org/sf/810214
mkstemp doesn't return abspath (2003-09-21)
	http://python.org/sf/810408
nested variables in 'class:' statements (2003-09-22)
	http://python.org/sf/810714
Support for non-string data in ConfigParser unclear/buggy (2003-09-22)
	http://python.org/sf/810843
shutil.copyfile fails when dst exists read-only (2003-09-22)
	http://python.org/sf/810879
os.stat returning a time = -1 (2003-09-22)
	http://python.org/sf/810887
2.3.1 configure bug (2003-09-23)
	http://python.org/sf/811028
Py2.2.3: Problem with Expat/XML/Zope on MacOSX 10.2.8 (2003-09-23)
	http://python.org/sf/811070
test_tempfile fails on windows if space in install path (2003-09-23)
	http://python.org/sf/811082
HP/UX vs configure (2003-09-23)
	http://python.org/sf/811160
dis.disassemble_string() broken (2003-09-23)
	http://python.org/sf/811294
ntohs on Solaris can return negative values (2003-09-23)
	http://python.org/sf/811295
"string".split behaviour for empty strings (2003-09-24)
	http://python.org/sf/811604
readline+no threads = core dump (2003-09-24)
	http://python.org/sf/811844
int ("ffffffd3", 16) gives error (2003-09-24)
	http://python.org/sf/811898
webbrowser.open_new() opens in an existing browser window (2003-09-24)
	http://python.org/sf/812089
randint is always even (2003-09-24)
	http://python.org/sf/812202
tarfile violates bufsize (2003-09-25)
	http://python.org/sf/812325
BasHTTPServer IE Mac 5.1 size problem (2003-09-25)
	http://python.org/sf/812340
a Py_DECREF() too much (2003-09-25)
	http://python.org/sf/812353
Problem with ftplib on HP-UX11i (2003-09-25)
	http://python.org/sf/812376
OSA support for properties broken (2003-09-25)
	http://python.org/sf/812750
2.3.1 re_test causes sigsegv (2003-09-25)
	http://python.org/sf/812804
turtle.py deferres exec of stmnts with tracer(0) (2003-09-26)
	http://python.org/sf/812986
Migrate PEP material to docs (2003-09-26)
	http://python.org/sf/813198
-Qnew switch doesn't work (2003-09-26)
	http://python.org/sf/813342
locale.getdefaultlocale doesnt handle all locales gracefully (2003-09-27)
	http://python.org/sf/813449
restrictions in _tkinter built with threaded tk undocumented (2003-09-27)
	http://python.org/sf/813453

New Patches
-----------

Fix many doc typos (2003-09-22)
	http://python.org/sf/810751
socket.ssl should check certificates (2003-09-22)
	http://python.org/sf/810754
fix for mkstemp with relative paths (bug #810408) (2003-09-22)
	http://python.org/sf/810914
More precise realm parsing in AbstractBasicAuthHandler (2003-09-25)
	http://python.org/sf/812285
module shutdown procedure based on GC (2003-09-25)
	http://python.org/sf/812369
normalize whitespace (2003-09-25)
	http://python.org/sf/812378
Fix test_tempfile: space in Win32 install path bug #811082 (2003-09-26)
	http://python.org/sf/813200
_sre stack overflow on FreeBSD/amd64 and /sparc64 (2003-09-26)
	http://python.org/sf/813391
Scalable zipfile extension (2003-09-27)
	http://python.org/sf/813436
Add IPPROTO_IPV6 option to the socketmodule (2003-09-27)
	http://python.org/sf/813445
entry size for cursors (2003-09-27)
	http://python.org/sf/813877

Closed Bugs
-----------

In 2.2, types are inheritable, &c (2001-08-21)
	http://python.org/sf/453684
Minor Python Intro update (2003-09-13)
	http://python.org/sf/805788
WinCVs & Python 2.3 (2003-09-16)
	http://python.org/sf/807253
Max and min are incorrect with one argument (2003-09-17)
	http://python.org/sf/807771
leak on lambda with duplicate arguments error (2003-09-18)
	http://python.org/sf/808594

Closed Patches
--------------

Adding rsplit() to string and unicode objects. (2003-09-06)
	http://python.org/sf/801847

From Scott.Daniels at Acm.Org  Sun Sep 28 19:00:53 2003
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Sun Sep 28 11:24:53 2003
Subject: [Python-Dev] Re:coping with a missing fsync
In-Reply-To: <E1A3QzJ-0007rx-AV@mail.python.org>
References: <E1A3QzJ-0007rx-AV@mail.python.org>
Message-ID: <3F776825.6060302@Acm.Org>

After seeing:

>backup kept screwing up until FileStorage's
>
>            if fsync is not None: fsync(file.fileno())
>
>actually did something on Windows.
>
I suspect making the change where you obtain fsync, rather than where 
you call it
makes the code clearer.  Just after import os, you can do the 
appropriate bit of
the following code:

        try:
            fsync = os.fsync
        except AttributeError:
            fsync = lambda filenumber: None   # Unfortunately nothing to 
call
-------- or --------
        fsync = os.fsync or lambda filenumber: None # if we cannot sync, 
do nothing


This (at least to me) emphasizes the lack of a system facility.  The 
existing
code emphasizes local solutions on a case-by-case basis.  If the 
solution is
always to skip the call, I'd find this method clearer.

-Scott Daid Daniels
Scott.Daniels@Acm.Org



From guido at python.org  Sun Sep 28 12:12:29 2003
From: guido at python.org (Guido van Rossum)
Date: Sun Sep 28 12:13:17 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: Your message of "Sat, 27 Sep 2003 22:08:31 EDT."
	<LNBBLJKPBEHFEDALKOLCCEPDGCAB.tim.one@comcast.net> 
References: <LNBBLJKPBEHFEDALKOLCCEPDGCAB.tim.one@comcast.net> 
Message-ID: <200309281612.h8SGCTS00510@12-236-54-216.client.attbi.com>

> [Martin v. Löwis]
> > I mostly agree: ZODB is indeed advanced, and it is indeed a good idea
> > to check for presence of os.fsync before using it.
> >
> > While this is OT, I'd still like to question the usefulness of
> > fsync(2) in the first place, for applications like ZODB. I assume
> > fsync is used as a write barrier, to make sure old modifications are
> > on disk, before making new changes.

[Tim]
> That's important, but not primarily for the catastrophic error-recovery
> scenarios you go on to sketch.

Tim, trust me.  The scenario you go on to sketch is unique to Windows.

fsync() was invented on Unix.  On that platform, there's *never* a
need to do fsync() to allow another process to see the right data.
fsync()'s effect is just not visible at the abstraction level
presented by the kernel.  fsync() is needed because sometimes
(e.g. through a power or kernel failure) the kernel's abstraction is
broken.  Everybody except you who is arguing for fsync()'s importance
does so because they have experienced (or fear to experience) crashes.

It's a bug in the Windows implementation of Unix-style I/O that a
commit() call (which is what fsync() is aliased to in posixmodule.c on
Windows) is needed in order to ensure that other processes see what a
process wrote.

Given that fsync() on Windows doesn't have the configuration problem
that started all this, I'm not sure how *your* argument for fsync()
can help convince Martin that we should do a 2.3.2 release ASAP.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Sun Sep 28 15:18:08 2003
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Sep 28 15:18:28 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <20030927213332.GI17491@zot.electricrain.com>
References: <200309270516.h8R5GAT7007511@localhost.localdomain>
	<m3pthmbtve.fsf@mira.informatik.hu-berlin.de>
	<16245.53947.33728.86658@montanaro.dyndns.org>
	<3F75F584.8040207@v.loewis.de>
	<20030927205147.GH17491@zot.electricrain.com>
	<3F75FE7A.6070800@v.loewis.de>
	<20030927213332.GI17491@zot.electricrain.com>
Message-ID: <3F7733F0.6060803@v.loewis.de>

Gregory P. Smith wrote:
>>I see. I could accept that it is confusing; would it also be backwards
>>compatible (i.e. would BerkeleyDB create those files on demand, and
>>would old Python installation still be able to read the database even
>>if those files where around)?
> 
> 
> It would not.  Its already not backwards compatible.  Thats what the
> bsddb185 module is for.

I should be more specific: If CDB is activated by default in 2.3.1,
would that compatible with files created in 2.3.0 (which are not
1.85 files, but some 4.x files).

>>But is it also easy to detect that multiple applications try to use
>>the same database?
> 
> 
> No.  But it is easy to document.  The old bsddb module never allowed
> it either.

I doubt many users are aware of that restriction (I, myself, wasn't).

Regards,
Martin


From gward at python.net  Sun Sep 28 15:18:28 2003
From: gward at python.net (Greg Ward)
Date: Sun Sep 28 15:18:42 2003
Subject: [Starship] Re: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <8yoauf87.fsf@python.net>
References: <200309261752.h8QHqJgD029823@localhost.localdomain>
	<LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
	<20030926184853.GB22837@mems-exchange.org>
	<he2zba58.fsf@python.net> <2mzngr1bf2.fsf@starship.python.net>
	<8yoauf87.fsf@python.net>
Message-ID: <20030928191828.GA2852@cthulhu.gerg.ca>

On 27 September 2003, Thomas Heller said:
> I could do it myself if the tools needed to build the docs would be
> available on the starship. Last time I checked, neither Tex nor
> latex2html were installed.
> 
> So, I hereby kindly request: Could the tools to build the Python docs
> from source please be installed on the starship?

OK, I'm installing tetex-base and latex2html now.  No guarantees if the
version of latex2html in Debian testing will actually work, of course.
;-)

(No, wait, it should work: I have the same version (2000-beta1-8) on my
PC, and the last time I did Python doc work, I *think* I was able to
produce HTML output.)

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
A closed mouth gathers no foot.

From martin at v.loewis.de  Sun Sep 28 15:34:00 2003
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Sep 28 15:34:18 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <16246.2621.472392.671642@montanaro.dyndns.org>
References: <200309270516.h8R5GAT7007511@localhost.localdomain>
	<m3pthmbtve.fsf@mira.informatik.hu-berlin.de>
	<16245.53947.33728.86658@montanaro.dyndns.org>
	<3F75F584.8040207@v.loewis.de>
	<16246.2621.472392.671642@montanaro.dyndns.org>
Message-ID: <3F7737A8.8010400@v.loewis.de>

Skip Montanaro wrote:

> I've given that a little more thought.  Instead of encumbering the factory
> functions which implement the old API with an optional dbenv argument, I
> think if people want to provide their own environment it's reasonable to 
> expect them to use the new bsddb3 API.  All we should do with the old API is
> make it work in a multi-threaded environment if possible.

But wouldn't that precisely involve creating environments? If you merely 
propose that the environment is not a parameter - +1.

Regards,
Martin



From martin at v.loewis.de  Sun Sep 28 15:38:14 2003
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Sep 28 15:38:39 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEPDGCAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCEPDGCAB.tim.one@comcast.net>
Message-ID: <3F7738A6.5020004@v.loewis.de>

Tim Peters wrote:

> At least on Windows, both the flush and the fsync are necessary to see one
> million bytes (via a different process) at the first prompt, two million at
> the second, and so on.  With neither, another process typically sees 0 bytes
> before the file gets huge.  With just one of them, it seems hard to predict,
> ranging from 0 to "almost" a million additional bytes per prompt.

As Guido explains, fsync is not necessary for that kind of application 
on a POSIX system. Once write(2) has completed, all other processes 
immediately see the changed data.

Regards,
Martin



From martin at v.loewis.de  Sun Sep 28 15:52:44 2003
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Sep 28 15:52:48 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <1064726667.31604.105.camel@anthem>
References: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>	
	<200309260524.h8Q5Ok1t013688@localhost.localdomain>	
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>	
	<1064591608.30783.30.camel@anthem> <k77vh784.fsf@python.net>	
	<m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com>	
	<20030926103846.A26782@discworld.dyndns.org>	
	<m2d6dnntt7.fsf@deacon-blues.mid.mastaler.com>	
	<m3u16ybtzu.fsf@mira.informatik.hu-berlin.de>	
	<m2d6dmlmr4.fsf@deacon-blues.mid.mastaler.com>	
	<m3n0cqa5gi.fsf@mira.informatik.hu-berlin.de>	
	<1064683719.19498.68.camel@localhost.localdomain>	
	<3F75CD80.3080008@v.loewis.de> <1064726667.31604.105.camel@anthem>
Message-ID: <3F773C0C.7060102@v.loewis.de>

Barry Warsaw wrote:

> Before I enable it unconditionally, I'd want to be more certain that
> it's necessary, or useful for the majority of sites, and that the
> performance hit is either worth it, not as bad as my benchmarks showed,
> or at least not as observably bad under real-world conditions.

You should first understand why message loss occurs if there is no 
fsync. For example, if the MTA stores incoming messages on disk, then 
invokes mailman, message loss may occur if the MTA deletes the message 
from the spool before mailman has fsync'ed its copy, and the power goes 
out at this moment.

If so, users might be better of using a powerful filesystem. I *think*
that, e.g. on Linux, ext3fs with ordered data writes might be 
sufficient, as deletion of the file should only occur after the data 
have been written (OTOH, it might be that ext3 in journalled mode is
needed, if data and metadata changes are not mutually subject to the
ordering). This should be less expensive than fsync, as the journalled
file system will still perform lazy writes - just in the right order.

Also notice that some file system drivers are incapable of implementing 
fsync correctly, so they fall back to treat fsync(2) like sync(2), which 
is quite expensive.

Regards,
Martin


From Jack.Jansen at cwi.nl  Sun Sep 28 16:52:42 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Sun Sep 28 16:52:48 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <2mzngr1bf2.fsf@starship.python.net>
References: <200309261752.h8QHqJgD029823@localhost.localdomain>
	<LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
	<20030926184853.GB22837@mems-exchange.org>
	<he2zba58.fsf@python.net> <2mzngr1bf2.fsf@starship.python.net>
Message-ID: <B46ED142-F1F5-11D7-AA57-000A27B19B96@cwi.nl>


On 26-sep-03, at 23:34, Michael Hudson wrote:
> Finding a substitue Jack might be harder (though given Bob Ippolito's
> work rate at the moment we could probably make him MacPython
> maintainer without him noticing <wink>).

Not that I'm planning to get run over by a tram shortly, but just in
case: I think Just or Ronald or yourself or any various other 
python-devvers
would make a better substitute *at this point in time*. Bob does some
truly wonderful things at terrific speed, but that has some drawbacks
for maintainability, IMHO. Being MacPython maintainer is 10% doing fun
stuff and 90% taking care of all the nitty-gritty details, making 
everything work
in as many situations as possible, documenting, etc etc etc.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From tim.one at comcast.net  Sun Sep 28 17:50:30 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Sep 28 17:50:34 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <3F7738A6.5020004@v.loewis.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEBGGDAB.tim.one@comcast.net>

[Martin v. L?wis]
> As Guido explains, fsync is not necessary for that kind of application
> on a POSIX system. Once write(2) has completed, all other processes
> immediately see the changed data.

But we're not calling write(2) -- Python's file.write() (which my little
driver used, and also what ZODB uses) calls the buffered fwrite.

Ignoring that, some systems aren't "pure".  I note that, on my Windows box,
some Cygwin utilties "see" the hoped-for file sizes while running my driver
without the fsync under Cygwin Python, but Windows utilities don't.  I was
using Cygwin 2.2.2; I don't know whether 2.3.1 under Cygwin thinks fsync has
gone missing.


From jeremy at zope.com  Sun Sep 28 17:58:30 2003
From: jeremy at zope.com (Jeremy Hylton)
Date: Sun Sep 28 17:59:29 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <3F75CD80.3080008@v.loewis.de>
References: <m2isngvhnv.fsf@deacon-blues.mid.mastaler.com>
	<200309260524.h8Q5Ok1t013688@localhost.localdomain>
	<m2n0crpn5p.fsf@deacon-blues.mid.mastaler.com>
	<1064591608.30783.30.camel@anthem> <k77vh784.fsf@python.net>
	<m24qyzpm7y.fsf@deacon-blues.mid.mastaler.com>
	<20030926103846.A26782@discworld.dyndns.org>
	<m2d6dnntt7.fsf@deacon-blues.mid.mastaler.com>
	<m3u16ybtzu.fsf@mira.informatik.hu-berlin.de>
	<m2d6dmlmr4.fsf@deacon-blues.mid.mastaler.com>
	<m3n0cqa5gi.fsf@mira.informatik.hu-berlin.de>
	<1064683719.19498.68.camel@localhost.localdomain>
	<3F75CD80.3080008@v.loewis.de>
Message-ID: <1064786309.19498.93.camel@localhost.localdomain>

On Sat, 2003-09-27 at 13:48, "Martin v. L?wis" wrote:
> 3. There is an operating system crash (kernel panic or similar).
>     fsync does not help, as, for a buggy kernel, anything might have
>     happened to the data before.

This is a vacuously true statement :-).  Anything might have happened;
among other things, the kernel buffers could have been flushed to disk
before the kernel panic.  I imagine that there are many bugs that could
cause a kernel panic or deadlock that would not also corrupt data on its
way to disk.  That is, we can't avoid bugs that are going to cause
problems writing to disk, but we can do our best to minimize the effects
of other bugs.

> 5. There is a power outage. This is the case where fsync should help:
>     everything up to the write barrier is on disk. Of course, if the
>     disk drive itself has write caching, fsync may have completed
>     without the data being on the disk (this would be an fsync bug, but
>     I believe Linux suffers from this particular bug).

(There are OSes other than Linux.)

> So in short, fsync(2) helps only in case of a power outage; for normal
> operation, it is not needed. In the case of a power outage, it is
> doubtful whether it has the desired effect.

I see two cases where it improves the changes that data is on persistent
storage before the database reports to the client that the write
succeeded.

> Slightly more on-topic: os.fsync is even worse, as it cannot be
> used to implement a write barrier in case of multiple threads. It
> runs with the GIL released, so while fsync is running, other
> threads might change the file. The semantics of fsync in this case
> is pretty unclear, but it is likely not a write barrier.

Sure.  ZODB only allows a single thread to write at a time.

Jeremy



From tim.one at comcast.net  Sun Sep 28 18:11:41 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Sep 28 18:12:41 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <200309281612.h8SGCTS00510@12-236-54-216.client.attbi.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIECBGDAB.tim.one@comcast.net>

[Guido]
> Tim, trust me.  The scenario you go on to sketch is unique to Windows.

I would rather trust the test program, but I don't see much point to the
argument (if that's what this is):  if os.fsync has gone missing on
platforms that "used to" support it, of course that's a serious regression.

> fsync() was invented on Unix.  On that platform, there's *never* a
> need to do fsync() to allow another process to see the right data.
> fsync()'s effect is just not visible at the abstraction level
> presented by the kernel.  fsync() is needed because sometimes
> (e.g. through a power or kernel failure) the kernel's abstraction is
> broken.  Everybody except you who is arguing for fsync()'s importance
> does so because they have experienced (or fear to experience) crashes.
>
> It's a bug in the Windows implementation of Unix-style I/O that a
> commit() call (which is what fsync() is aliased to in posixmodule.c on
> Windows) is needed in order to ensure that other processes see what a
> process wrote.

The test driver (and ZODB) use buffered I/O, not unbuffered POSIX I/O.
Assuring that writes done to buffered streams are visible to other processes
is trickier than Martin's reliance on the write(2) man page, since write(2)
isn't involved in the chain.

> Given that fsync() on Windows doesn't have the configuration problem
> that started all this,

I don't know whether Cygwin Python suffers the same problem.  If it does,
then people running ZODB under Cygwin 2.3.1 (which I don't believe has been
released yet) are in for nasty surprises if they run repozo.py under the
"native" Windows Python.  I've tried this under Cygwin 2.2.2 (see other
email).  Since that did have os.fsync(), but the native Windows Python
didn't at that time, my best guess is that Cygwin Python is affected.

> I'm not sure how *your* argument for fsync() can help convince Martin
> that we should do a 2.3.2 release ASAP.

It's enough for me that a function that used to be there has gone away
without good reason.  It's easier to cut a new release than argue about how
much trouble the missing os.fsync should cause, as people have already
poured more energy into complaining about it than would be needed to cut a
new release.


From tim.one at comcast.net  Sun Sep 28 18:32:51 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Sep 28 18:32:55 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <20030926233959.GC17491@zot.electricrain.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAECDGDAB.tim.one@comcast.net>

[Skip]
>> For long-running SpamBayes apps like pop3proxy and the
>> Outlook plugin, pickles make sense if shared access to the scoring
>> database is not needed.

[Gregory P. Smith]
> Combine that with data event/change logging and you have effectively
> the same thing as what many apps think they need to use a database
> library for.  Replay the entire log of data changes internally on
> infrequent app startup to generate your internal state.  when the log
> file is getting to big, write a full state snapshot and continue
> logging changes from it. no need to save the entire pickle everytime
> something changes.

That may be OK for pop3proxy users.  Alas, startup time and memory burden
are keen topics for Outlook plugin users, and, e.g., the startup time to
unpickle a large dict (let alone replay logs since the last snapshot) is
unbearable for people with large training sets.  When you start Outlook, it
typically starts fetching email right away, and you typically want to see
your email right away then.  The relatively zippy startup time of a Berkeley
DB is highly welcome for that reason.


From guido at python.org  Sun Sep 28 19:43:43 2003
From: guido at python.org (Guido van Rossum)
Date: Sun Sep 28 19:44:36 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: Your message of "Sun, 28 Sep 2003 18:11:41 EDT."
	<LNBBLJKPBEHFEDALKOLCIECBGDAB.tim.one@comcast.net> 
References: <LNBBLJKPBEHFEDALKOLCIECBGDAB.tim.one@comcast.net> 
Message-ID: <200309282343.h8SNhht00773@12-236-54-216.client.attbi.com>

> The test driver (and ZODB) use buffered I/O, not unbuffered POSIX I/O.
> Assuring that writes done to buffered streams are visible to other processes
> is trickier than Martin's reliance on the write(2) man page, since write(2)
> isn't involved in the chain.

All you need to do is call f.flush().  This calls write(2) internally
(even on Windows).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.one at comcast.net  Sun Sep 28 20:33:05 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Sep 28 20:33:10 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <200309282343.h8SNhht00773@12-236-54-216.client.attbi.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCIECKGDAB.tim.one@comcast.net>

[Tim]
>> The test driver (and ZODB) use buffered I/O, not unbuffered POSIX
>> I/O. Assuring that writes done to buffered streams are visible to
>> other processes is trickier than Martin's reliance on the write(2)
>> man page, since write(2) isn't involved in the chain.

[Guido]
> All you need to do is call f.flush().  This calls write(2) internally
> (even on Windows).

Sorry, I don't understand the point of this Unixish chest-puffing <0.8
wink>.

POSIX doesn't mandate that fflush() call write(), although it requires, as
an extension to the C standard, pretty much the same end result.

Nevertheless, calling f.flush() isn't enough on Windows, regardless of
whether the native Windows Python or Cygwin Python is doing it.  Without the
os.fsync(f.fileno()) too, native Windows utilities and the Windows Python
see a wrong current file size.  This gets pretty bizarre too:  after letting
Cygwin Python write and flush a million bytes (i.e., at the first prompt in
the test driver), the Cygwin Python (in a different process)
os.path.getsize() returns a million, but the Windows Python returns 0.  Put
in the fsync too and everyone is happy.  This can't work if fsynch goes
missing in the Cygwin Python.


From tim.one at comcast.net  Sun Sep 28 21:38:44 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Sep 28 21:38:48 2003
Subject: [Python-Dev] 2.3.1 is (almost) a go
In-Reply-To: <16245.53947.33728.86658@montanaro.dyndns.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCCECOGDAB.tim.one@comcast.net>

[Skip Montanaro]
> ...
> Attached is a modified version of the hammer.py script which seems to
> not fail for me on either Windows run from IDLE (Python 2.3, BDB
> 4.1.6) or Mac OS X (Python CVS, BDB 4.2.1).  The original script
> failed for me on Windows but not Mac OS X.  Can some other people for
> whom the original script fails please try it?  (I also attached it to
> bug #775414.)

On Win98SE with current Python 2.3.1, it doesn't fail, but it never seemed
to finish for me either.  Staring at WinTop showed that the Python process
stopped accumulating cycles.  Can't be killed with Ctrl+C (no visible
effect).  Can be killed with Ctrl+Break.

Dumping

        print "%s %s" % (thread.get_ident(), i)

at the top of the hammer loop showed that the threads get through several
hundred iterations, then all printing stops.

Attaching to a debug-build Python from the debugger when a freeze occurs
isn't terribly illuminating.  One thread's stack shows

_BSDDB_D! __db_win32_mutex_lock + 134 bytes
_BSDDB_D! __lock_get + 2264 bytes
_BSDDB_D! __lock_get + 197 bytes
_BSDDB_D! __ham_get_meta + 120 bytes
_BSDDB_D! __ham_c_dup + 4201 bytes
_BSDDB_D! __db_c_put + 2544 bytes
_BSDDB_D! __db_put + 507 bytes
_DB_put(DBObject * 0x016cff88, __db_txn * 0x016d0000, __db_dbt * 0x016cc000,
__db_dbt * 0x50d751fe, int 0) line 562 + 35 bytes

The main thread's stack shows

_BSDDB_D! __db_win32_mutex_lock + 134 bytes
_BSDDB_D! __lock_get + 2264 bytes
_BSDDB_D! __lock_get + 197 bytes
_BSDDB_D! __db_lget + 365 bytes
_BSDDB_D! __ham_lock_bucket + 105 bytes
_BSDDB_D! __ham_get_cpage + 195 bytes
_BSDDB_D! __ham_item_next + 25 bytes
_BSDDB_D! __ham_call_hash + 2479 bytes
_BSDDB_D! __ham_c_dup + 4307 bytes
_BSDDB_D! __db_c_put + 2544 bytes
_BSDDB_D! __db_put + 507 bytes
_DB_put(DBObject * 0x008fe2e8, __db_txn * 0x00000000, __db_dbt * 0x0062f230,
__db_dbt * 0x0062f248, int 0) line 562 + 35 bytes
DB_ass_sub(DBObject * 0x008fe2e8, _object * 0x00b83178, _object *
0x00b83370) line 2330 + 23 bytes
PyObject_SetItem(_object * 0x008fe2e8, _object * 0x00b83178, _object *
0x00b83370) line 123 + 18 bytes
eval_frame(_frame * 0x00984948) line 1448 + 17 bytes
...

The other threads are somewhere in the OS kernel and don't have useful
tracebacks.  This varies from run to run, but all threads with a useful
stack are always stuck at the same place in __db_win32_mutex_lock.

All in all, looks like it's simply deadlocked.

Running the original hammer.py under current CVS Python freezes in the same
way now.

I added this info to the bug report:

    http://www.python.org/sf/775414


From bac at OCF.Berkeley.EDU  Sun Sep 28 22:40:38 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Sep 28 22:40:36 2003
Subject: [Python-Dev] Good way of finding out what C functions we have?
Message-ID: <3F779BA6.6050407@ocf.berkeley.edu>

I just did a bunch of research into trying to find out if using strdup() 
in a patch was reasonable.  I finally grepped enough to find strdup in 
pyconfig.h and configure.in .  But that leads to a Brett Newbie Question.

Does there happen to be a better way to find out what functions we check 
for beyond grepping configure.in?  The entire reason I bothered to do so 
much research is I thought we were only supposed to assume ANSI C with 
POSIX (which, from what I can tell, strdup() is not).  But with more 
Googling and grepping I discovered the meaning of AC_REPLACE_FUNCS() 
which strdup was being called in configure.in .  I would never have 
known strdup was available to me for code had I not randomly come across 
this patch and done all of this research.

Because of this I would like to list in the Python development guide I 
am writing up all of the places one can check for info; what 
configure.in checks for or handles and pyport.h come to mind.  It would 
be nice to have a place that lists all the places one can check for C 
functionality that are not in the Python/C API directly.  If people have 
other places in the code that one should check please let me know so I 
can make sure it makes it into the doc and prevents me from having to 
come up with another Brett Newbie Question.  =)

-Brett


From martin at v.loewis.de  Mon Sep 29 02:03:39 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Mon Sep 29 02:04:00 2003
Subject: [Python-Dev] Good way of finding out what C functions we have?
In-Reply-To: <3F779BA6.6050407@ocf.berkeley.edu>
References: <3F779BA6.6050407@ocf.berkeley.edu>
Message-ID: <m3r820p1vo.fsf@mira.informatik.hu-berlin.de>

"Brett C." <bac@OCF.Berkeley.EDU> writes:

> Does there happen to be a better way to find out what functions we
> check for beyond grepping configure.in?  The entire reason I bothered
> to do so much research is I thought we were only supposed to assume
> ANSI C with POSIX (which, from what I can tell, strdup() is not).  

Actually, strdup is in POSIX:

http://www.opengroup.org/onlinepubs/007904975/functions/strdup.html

If we still check for it, it is because it might not have been
available on some system, at some point in time.

> Because of this I would like to list in the Python development guide I
> am writing up all of the places one can check for info; what
> configure.in checks for or handles and pyport.h come to mind.  It
> would be nice to have a place that lists all the places one can check
> for C functionality that are not in the Python/C API directly.  If
> people have other places in the code that one should check please let
> me know so I can make sure it makes it into the doc and prevents me
> from having to come up with another Brett Newbie Question.  =)

I would look in pyconfig.h.in as the first thing. There is a pitfall,
though: presence of some HAVE_foo in pyconfig.h does not mean that
Python gracefully deals with the absence of foo. When converting
configure to autoconf 2.5x, I checked all macros to determine whether
they were still in use, but some may have become unused/not processed
correctly since. That would be a bug, of course.

Regards,
Martin

From sjoerd at acm.org  Mon Sep 29 05:12:33 2003
From: sjoerd at acm.org (Sjoerd Mullender)
Date: Mon Sep 29 05:12:43 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <20030926184853.GB22837@mems-exchange.org>
References: <200309261752.h8QHqJgD029823@localhost.localdomain>	<LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
	<20030926184853.GB22837@mems-exchange.org>
Message-ID: <3F77F781.1020803@acm.org>

Neil Schemenauer wrote:
>   * try to reduce the number of version numbers in different files need to
>     be adjusted.   Also, try to automate the adjustment.

Check out the package vertooo (on SourceForge) for this.  It does just 
this (automating version numbers in lots of files).

It's even written in Python.  :-)


From anthony at interlink.com.au  Mon Sep 29 04:05:34 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Sep 29 06:08:20 2003
Subject: [Python-Dev] Python2.3.2 and release23-maint branch
Message-ID: <200309290805.h8T85ZxC025742@localhost.localdomain>


I've not heard any complaints, so please consider the release23-maint branch
frozen.

Anthony

From mwh at python.net  Mon Sep 29 07:00:46 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Sep 29 07:00:05 2003
Subject: [Starship] Re: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <20030928191828.GA2852@cthulhu.gerg.ca> (Greg Ward's message of
	"Sun, 28 Sep 2003 15:18:28 -0400")
References: <200309261752.h8QHqJgD029823@localhost.localdomain>
	<LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
	<20030926184853.GB22837@mems-exchange.org> <he2zba58.fsf@python.net>
	<2mzngr1bf2.fsf@starship.python.net> <8yoauf87.fsf@python.net>
	<20030928191828.GA2852@cthulhu.gerg.ca>
Message-ID: <2mvfrb271d.fsf@starship.python.net>

Greg Ward <gward@python.net> writes:

> On 27 September 2003, Thomas Heller said:
>> I could do it myself if the tools needed to build the docs would be
>> available on the starship. Last time I checked, neither Tex nor
>> latex2html were installed.
>> 
>> So, I hereby kindly request: Could the tools to build the Python docs
>> from source please be installed on the starship?
>
> OK, I'm installing tetex-base and latex2html now.  No guarantees if the
> version of latex2html in Debian testing will actually work, of course.
> ;-)

Somehow this seems to have avoided installing some important
executables -- like, e.g., "latex".  Are there some other packages
that need to be installed too?

Cheers,
mwh

-- 
58. Fools ignore complexity. Pragmatists suffer it. Some can avoid
    it. Geniuses remove it.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html

From Jack.Jansen at cwi.nl  Mon Sep 29 08:35:33 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Mon Sep 29 08:35:15 2003
Subject: [Python-Dev] Python2.3.2 and release23-maint branch
In-Reply-To: <200309290805.h8T85ZxC025742@localhost.localdomain>
Message-ID: <6B24E828-F279-11D7-BFE6-0030655234CE@cwi.nl>


On Monday, September 29, 2003, at 10:05 AM, Anthony Baxter wrote:

>
> I've not heard any complaints, so please consider the release23-maint 
> branch
> frozen.

Uhm... I was busy doing all the version numbers and testing for 
MacPython, I thought
I had most of today (Monday, european) for that??!?

I won't check anything in unless you explicitly give me permission,
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From andrew-pythondev at puzzling.org  Mon Sep 29 09:03:40 2003
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Mon Sep 29 09:03:45 2003
Subject: [Starship] Re: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <2mvfrb271d.fsf@starship.python.net>
References: <200309261752.h8QHqJgD029823@localhost.localdomain>
	<LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
	<20030926184853.GB22837@mems-exchange.org>
	<he2zba58.fsf@python.net> <2mzngr1bf2.fsf@starship.python.net>
	<8yoauf87.fsf@python.net> <20030928191828.GA2852@cthulhu.gerg.ca>
	<2mvfrb271d.fsf@starship.python.net>
Message-ID: <20030929130340.GF21156@frobozz>

Michael Hudson wrote:
> Greg Ward <gward@python.net> writes:
> 
> > OK, I'm installing tetex-base and latex2html now.  No guarantees if the
> > version of latex2html in Debian testing will actually work, of course.
> > ;-)
> 
> Somehow this seems to have avoided installing some important
> executables -- like, e.g., "latex".  Are there some other packages
> that need to be installed too?

Looks like you need tetex-bin, judging from my debian (unstable) system:

  bash-2.05b$ dpkg -S `which latex`
  tetex-bin: /usr/bin/latex

It's probably worth grabbing tetex-extra, too; its description says:
  ...
  Together with tetex-bin and tetex-base, this will give you a
  complete teTeX installation.

-Andrew.


From anthony at interlink.com.au  Mon Sep 29 09:08:22 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Sep 29 09:10:35 2003
Subject: [Python-Dev] Python2.3.2 and release23-maint branch 
In-Reply-To: <6B24E828-F279-11D7-BFE6-0030655234CE@cwi.nl> 
Message-ID: <200309291308.h8TD8N8O030399@localhost.localdomain>


>>> Jack Jansen wrote
> Uhm... I was busy doing all the version numbers and testing for 
> MacPython, I thought
> I had most of today (Monday, european) for that??!?
> 
> I won't check anything in unless you explicitly give me permission,

My bad - I was going from the release PEPs. All checkins, _other_ than
those required for the release. ;)

I was looking through the .plist files, and it seems like there's about
8 or so different places featuring the string "2.3". Is there a way to
make this number less? (Obviously, not for 2.3.2)...

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From Jack.Jansen at cwi.nl  Mon Sep 29 10:44:39 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Mon Sep 29 10:44:16 2003
Subject: [Python-Dev] Python2.3.2 and release23-maint branch 
In-Reply-To: <200309291308.h8TD8N8O030399@localhost.localdomain>
Message-ID: <746AF574-F28B-11D7-BFE6-0030655234CE@cwi.nl>


On Monday, September 29, 2003, at 03:08 PM, Anthony Baxter wrote:

>
>>>> Jack Jansen wrote
>> Uhm... I was busy doing all the version numbers and testing for
>> MacPython, I thought
>> I had most of today (Monday, european) for that??!?
>>
>> I won't check anything in unless you explicitly give me permission,
>
> My bad - I was going from the release PEPs. All checkins, _other_ than
> those required for the release. ;)

Sigh... Don't wait for me to update the Mac-specific files: I've 
apparently
messed up the developer tools on my system. It's now a mix between 
gcc3.3
and gcc3, and I keep running into problems. Reverting to gcc3.1 also
doesn't seem to work. My home system has a virgin 10.2 partition, but 
I'm
away tonight so I won't have the chance to build and test anything
(and I assume tomorrow-morning-European is too late, right?). I'm 99%
sure that just changing the version numbers in the .plist and .strings 
files
won't break anything, but without testing I don't want to chance it.

> I was looking through the .plist files, and it seems like there's about
> 8 or so different places featuring the string "2.3". Is there a way to
> make this number less? (Obviously, not for 2.3.2)...

Eventually these files need to be generated (we have a plistlib module
for that), and the same goes for the accompanying .strings files.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From anthony at interlink.com.au  Mon Sep 29 10:45:05 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Sep 29 10:46:55 2003
Subject: [Python-Dev] 2.3.2 and bsddb
Message-ID: <200309291445.h8TEj5hc002980@localhost.localdomain>


For those of you not following every bug in the SF tracker closely, 
in http://www.python.org/sf/775414 it's been suggested that the docs
for 2.3.2 include a warning about using the old-style interface to 
bsddb (without a DBEnv) with multi-threaded applications. This seems
like a prudent suggestion - does someone want to supply some words?

Anthony

From anthony at interlink.com.au  Mon Sep 29 10:47:54 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Sep 29 10:49:43 2003
Subject: [Python-Dev] Python2.3.2 and release23-maint branch 
In-Reply-To: <746AF574-F28B-11D7-BFE6-0030655234CE@cwi.nl> 
Message-ID: <200309291447.h8TElsJG003047@localhost.localdomain>


>>> Jack Jansen wrote
> Sigh... Don't wait for me to update the Mac-specific files: I've 
> apparently
> messed up the developer tools on my system. It's now a mix between 
> gcc3.3
> and gcc3, and I keep running into problems. Reverting to gcc3.1 also
> doesn't seem to work. My home system has a virgin 10.2 partition, but 
> I'm
> away tonight so I won't have the chance to build and test anything
> (and I assume tomorrow-morning-European is too late, right?). I'm 99%
> sure that just changing the version numbers in the .plist and .strings 
> files
> won't break anything, but without testing I don't want to chance it.

I'm cutting a release candidate tomorrow morning, but the release itself
on Friday AU time. If it's just a matter of updating the version numbers in
the .plist files, I can do this. Assuming, of course, that just editing 
the .plist files with vi to s/2.3/2.3.2/ is going to Do The Right Thing.

I know of a couple of folks with Macs who could test the framework build - 
aside from "./configure ; make ; make testall", what else do they need to
test?

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From fdrake at acm.org  Mon Sep 29 10:31:08 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon Sep 29 11:03:50 2003
Subject: [Python-Dev] PC/python_nt.rc
In-Reply-To: <eky2ufr7.fsf@python.net>
References: <eky2ufr7.fsf@python.net>
Message-ID: <16248.16940.996691.656779@grendel.zope.com>


Thomas Heller writes:
 > Does this sound like a sensible approach, or are there any objections?

Not from me!  If Tim doesn't have any objections, then you should
proceed.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From Jack.Jansen at cwi.nl  Mon Sep 29 11:13:31 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Mon Sep 29 11:13:08 2003
Subject: [Python-Dev] Python2.3.2 and release23-maint branch 
In-Reply-To: <200309291447.h8TElsJG003047@localhost.localdomain>
Message-ID: <7C78AF06-F28F-11D7-BFE6-0030655234CE@cwi.nl>


On Monday, September 29, 2003, at 04:47 PM, Anthony Baxter wrote:
> I'm cutting a release candidate tomorrow morning, but the release 
> itself
> on Friday AU time. If it's just a matter of updating the version 
> numbers in
> the .plist files, I can do this. Assuming, of course, that just editing
> the .plist files with vi to s/2.3/2.3.2/ is going to Do The Right 
> Thing.

Nevermind, I just did the checkins. I got my build system in enough of
a shape that I've built everything and I'm confident it'll work.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From theller at python.net  Mon Sep 29 11:37:47 2003
From: theller at python.net (Thomas Heller)
Date: Mon Sep 29 11:37:59 2003
Subject: [Python-Dev] Python2.3.2 and release23-maint branch
In-Reply-To: <200309290805.h8T85ZxC025742@localhost.localdomain> (Anthony
	Baxter's message of "Mon, 29 Sep 2003 18:05:34 +1000")
References: <200309290805.h8T85ZxC025742@localhost.localdomain>
Message-ID: <3cef7ghg.fsf@python.net>

Anthony Baxter <anthony@interlink.com.au> writes:

> I've not heard any complaints, so please consider the release23-maint branch
> frozen.

When will you tag the branch?

And I'm still unable to create the html docs myself ;-(

Thomas


From skip at pobox.com  Mon Sep 29 11:37:45 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Sep 29 11:38:06 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEBGGDAB.tim.one@comcast.net>
References: <3F7738A6.5020004@v.loewis.de>
	<LNBBLJKPBEHFEDALKOLCGEBGGDAB.tim.one@comcast.net>
Message-ID: <16248.20937.685975.429148@montanaro.dyndns.org>


    Tim> [Martin v. L?wis]
    >> As Guido explains, fsync is not necessary for that kind of
    >> application on a POSIX system. Once write(2) has completed, all other
    >> processes immediately see the changed data.

    Tim> But we're not calling write(2) -- Python's file.write() (which my
    Tim> little driver used, and also what ZODB uses) calls the buffered
    Tim> fwrite.

Then shouldn't a file.flush() call be sufficient to force a call to write(2)
on POSIX systems?

    Tim> Ignoring that, some systems aren't "pure".  I note that, on my
    Tim> Windows box, some Cygwin utilties "see" the hoped-for file sizes
    Tim> while running my driver without the fsync under Cygwin Python, but
    Tim> Windows utilities don't.  I was using Cygwin 2.2.2; I don't know
    Tim> whether 2.3.1 under Cygwin thinks fsync has gone missing.

It's been established that os.fsync() is required for Windows in at least
some circumstances, so you have to assume it's required under all
circumstances, correct?

Skip

From tim.one at comcast.net  Mon Sep 29 11:58:43 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Sep 29 11:58:54 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <16248.20937.685975.429148@montanaro.dyndns.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEGKGDAB.tim.one@comcast.net>

[Skip Montanaro]
> Then shouldn't a file.flush() call be sufficient to force a call to
> write(2) on POSIX systems?

The POSIX spec doesn't say that, but it does say enough that it shouldn't
matter whether or not an fflush implementation happens to call write under
the covers -- POSIX fflush is required to write data "to the file"
regardless of how it's implemented.

A difficulty with Windows is that a file ain't an inode -- a file doesn't
exist except in a directory, and the directory structure records the current
length of the files in the directory.  Writing data to a file doesn't
necessarily update the file length stored in the file's directory entry, and
flushing doesn't necessary update it either; _commit() does.  Native Windows
utilities get their idea of a file's length from reading the file's
directory entry.

> ...
> It's been established that os.fsync() is required for Windows in at
> least some circumstances, so you have to assume it's required under
> all circumstances, correct?

Sorry, I didn't follow the question.


From tim.one at comcast.net  Mon Sep 29 12:05:03 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Sep 29 12:05:13 2003
Subject: [Python-Dev] PC/python_nt.rc
In-Reply-To: <16248.16940.996691.656779@grendel.zope.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEGLGDAB.tim.one@comcast.net>

[Thomas Heller]
>> Does this sound like a sensible approach, or are there any
>> objections?

[Fred L. Drake, Jr.]
> Not from me!  If Tim doesn't have any objections, then you should
> proceed.

Too much context got snipped here.  If this was about eliminating the need
for the field3.py dance in building Windows Python, ya, Thomas's idea
sounded fine to me.  Silence is assent <0.9 wink>.


From skip at pobox.com  Mon Sep 29 12:20:11 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Sep 29 12:20:21 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEGKGDAB.tim.one@comcast.net>
References: <16248.20937.685975.429148@montanaro.dyndns.org>
	<LNBBLJKPBEHFEDALKOLCMEGKGDAB.tim.one@comcast.net>
Message-ID: <16248.23483.916661.489555@montanaro.dyndns.org>


    >> ...  It's been established that os.fsync() is required for Windows in
    >> at least some circumstances, so you have to assume it's required
    >> under all circumstances, correct?

    Tim> Sorry, I didn't follow the question.

We've already established that os.fsync() is required in at least some cases
on Windows.  It seemed to me that you were trying to construct some other
cases (possibly involving Cygwin) where that wasn't necessary.  I probably
just didn't follow your intent.

Skip


From tim.one at comcast.net  Mon Sep 29 12:38:19 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Sep 29 12:38:28 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <16248.23483.916661.489555@montanaro.dyndns.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEHBGDAB.tim.one@comcast.net>

[Skip Montanaro]
> We've already established that os.fsync() is required in at least
> some cases on Windows.  It seemed to me that you were trying to
> construct some other cases (possibly involving Cygwin) where that
> wasn't necessary.

Cygwin doesn't appear to need fsync() so long as you stick purely to Cygwin
tools in all processes -- Cygwin appears POSIX-like in that respect.

But Cygwin runs on Windows, and fsync() is still needed in Cygwin Python
programs if native Windows utilities (including the native Windows Python)
are to see correct current file sizes too.

If os.fsync() is missing in Cygwin 2.3.1 (I don't know whether it is, but
suspect that it is), then problems follow from that.  A specific example is
ZODB running under Cygwin, while the ZODB live-backup script repozo.py runs
under native Windows Python.  If the former can't do an fsync(), the latter
can't know the correct current file size.


From fdrake at acm.org  Mon Sep 29 13:00:38 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon Sep 29 13:00:55 2003
Subject: [Python-Dev] Python 2.3.2c1 documentation
Message-ID: <16248.25910.512604.157482@grendel.zope.com>


The Python 2.3.2c1 documentation packages are now available on the FTP
site:

    ftp://ftp.python.org/pub/python/doc/2.3.2c1/


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From shane at zope.com  Mon Sep 29 16:57:49 2003
From: shane at zope.com (Shane Hathaway)
Date: Mon Sep 29 16:58:19 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <20030926184853.GB22837@mems-exchange.org>
References: <LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
	<20030926184853.GB22837@mems-exchange.org>
Message-ID: <3F789CCD.1010707@zope.com>

Neil Schemenauer wrote:
>   * write scripts to generate the final package and also verify its
>     sanity (e.g. check version numbers, dates, detect files that
>     should not be included in the release)

FWIW, one thing I like to do is compare the list of files in the new 
release with the list of files in the previous release.  It's amazing 
what can creep in.  I've attached the script I use--it's ugly but it works.

Shane
-------------- next part --------------
#!/bin/sh

# This utility compares the directory listing of two tar files.
# It detects .gz or .bz2 extensions automatically and acts accordingly.

if [ "$1" == "" ] || [ "$2" == "" ]; then
  echo usage: $0 tarfile1 tarfile2
  exit 1
fi

f1=`mktemp /tmp/difftar1.XXXXXX`
f2=`mktemp /tmp/difftar2.XXXXXX`

if echo $1 | grep '\.gz' > /dev/null; then
  flags='tfz'
elif echo $1 | grep '\.bz2' > /dev/null; then
  flags='tfj'
else
  flags='tf'
fi
tar $flags $1 | sort > $f1

if echo $2 | grep '\.gz' > /dev/null; then
  flags='tfz'
elif echo $2 | grep '\.bz2' > /dev/null; then
  flags='tfj'
else
  flags='tf'
fi
tar $flags $2 | sort > $f2

diff -u $f1 $f2
rm -f $f1 $f2

From jeremy at zope.com  Mon Sep 29 17:07:39 2003
From: jeremy at zope.com (Jeremy Hylton)
Date: Mon Sep 29 17:09:18 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <3F789CCD.1010707@zope.com>
References: <LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
	<20030926184853.GB22837@mems-exchange.org> <3F789CCD.1010707@zope.com>
Message-ID: <1064869658.19498.183.camel@localhost.localdomain>

On Mon, 2003-09-29 at 16:57, Shane Hathaway wrote:
> Neil Schemenauer wrote:
> >   * write scripts to generate the final package and also verify its
> >     sanity (e.g. check version numbers, dates, detect files that
> >     should not be included in the release)
> 
> FWIW, one thing I like to do is compare the list of files in the new 
> release with the list of files in the previous release.  It's amazing 
> what can creep in.  I've attached the script I use--it's ugly but it works.

Sure is!  This is a little OT, but distutils MANIFEST files are very
useful for this purpose.

Jeremy



From gward at python.net  Mon Sep 29 21:25:56 2003
From: gward at python.net (Greg Ward)
Date: Mon Sep 29 21:26:00 2003
Subject: [Starship] Re: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <2mvfrb271d.fsf@starship.python.net>
References: <200309261752.h8QHqJgD029823@localhost.localdomain>
	<LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
	<20030926184853.GB22837@mems-exchange.org>
	<he2zba58.fsf@python.net> <2mzngr1bf2.fsf@starship.python.net>
	<8yoauf87.fsf@python.net> <20030928191828.GA2852@cthulhu.gerg.ca>
	<2mvfrb271d.fsf@starship.python.net>
Message-ID: <20030930012556.GA6451@cthulhu.gerg.ca>

On 29 September 2003, Michael Hudson said:
> Somehow this seems to have avoided installing some important
> executables -- like, e.g., "latex".  Are there some other packages
> that need to be installed too?

Argh.  Installing tetex-bin (and -doc, -extra, -lib just for fun) now.

        Greg
-- 
Greg Ward <gward@python.net>                         http://www.gerg.ca/
A man without religion is like a fish without a bicycle.

From bac at OCF.Berkeley.EDU  Mon Sep 29 22:07:58 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Sep 29 22:08:00 2003
Subject: [Python-Dev] Good way of finding out what C functions we have?
In-Reply-To: <m3r820p1vo.fsf@mira.informatik.hu-berlin.de>
References: <3F779BA6.6050407@ocf.berkeley.edu>
	<m3r820p1vo.fsf@mira.informatik.hu-berlin.de>
Message-ID: <3F78E57E.8090403@ocf.berkeley.edu>

Martin v. L?wis wrote:

> "Brett C." <bac@OCF.Berkeley.EDU> writes:
> 
> 
>>Does there happen to be a better way to find out what functions we
>>check for beyond grepping configure.in?  The entire reason I bothered
>>to do so much research is I thought we were only supposed to assume
>>ANSI C with POSIX (which, from what I can tell, strdup() is not).  
> 
> 
> Actually, strdup is in POSIX:
> 
> http://www.opengroup.org/onlinepubs/007904975/functions/strdup.html
> 

Shows how much I know; I found that last night but since it didn't say 
POSIX anywhere I thought it was part of another standard.  I guess the 
"Issue 5" mention means that BASE is POSIX?  Or is IEEE 1003.1 basically 
POSIX?

> If we still check for it, it is because it might not have been
> available on some system, at some point in time.
> 

We don't only check for it, but we have a replacement implementation 
(the comment for Python/strdup.c seems to suggest it is needed by the 
parser).

> 
>>Because of this I would like to list in the Python development guide I
>>am writing up all of the places one can check for info; what
>>configure.in checks for or handles and pyport.h come to mind.  It
>>would be nice to have a place that lists all the places one can check
>>for C functionality that are not in the Python/C API directly.  If
>>people have other places in the code that one should check please let
>>me know so I can make sure it makes it into the doc and prevents me
>>from having to come up with another Brett Newbie Question.  =)
> 
> 
> I would look in pyconfig.h.in as the first thing. There is a pitfall,
> though: presence of some HAVE_foo in pyconfig.h does not mean that
> Python gracefully deals with the absence of foo. When converting
> configure to autoconf 2.5x, I checked all macros to determine whether
> they were still in use, but some may have become unused/not processed
> correctly since. That would be a bug, of course.
> 

What does happen if a HAVE_foo is actually required?  Does the build 
fail or will configure halt and say that the build will fail if you proceed?

And if no one objects I will mention that looking in pyconfig.h.in is a 
good place to look to see what C functions that are non-POSIX and 
non-ANSI C we check for and pyport.h has helpful stuff as well.

-Brett



From anthony at interlink.com.au  Tue Sep 30 03:01:30 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Sep 30 03:03:25 2003
Subject: [Python-Dev] Lib/idlelib/NEWS.txt
Message-ID: <200309300701.h8U71VG7017578@localhost.localdomain>


Idle still has it's own NEWS.txt file, and it's own version number
(for instance, 2.3.1's version of idle is "1.0.1". Does this make 
sense? What should happen for 2.3.2, where no idle changes have been
made?

Anthony

From anthony at python.org  Tue Sep 30 06:37:16 2003
From: anthony at python.org (Anthony Baxter)
Date: Tue Sep 30 06:39:13 2003
Subject: [Python-Dev] RELEASED Python 2.3.2, release candidate 1
Message-ID: <200309301037.h8UAbGhY029290@localhost.localdomain>


On behalf of the Python development team and the Python community, I'm
happy to announce the release of Python 2.3.2 (release candidate 1).

Python 2.3.2 is a bug-fix release, to repair a couple of build problems
and packaging errors in Python 2.3.1. Assuming no major problems crop up,
a final release of Python 2.3.2 will follow later this week.

For more information on Python 2.3.2, including download links for
various platforms, release notes, and known issues, please see:

    http://www.python.org/2.3.2

Highlights of this new release include:

  - A bug in autoconf that broke building on HP/UX systems is fixed.

  - A bug in the Python configure script that meant os.fsync() was 
    never available is fixed.

Highlights of the previous major Python release (2.3) are available     
from the Python 2.3 page, at                                            

    http://www.python.org/2.3/highlights.html

Many apologies for the flaws in 2.3.1 - the final 2.3.2 release should
address the problems.

Enjoy the new release,
Anthony

Anthony Baxter
anthony@python.org
Python 2.3.2 Release Manager
(on behalf of the entire python-dev team)

From skip at pobox.com  Tue Sep 30 09:12:09 2003
From: skip at pobox.com (Skip Montanaro)
Date: Tue Sep 30 09:12:26 2003
Subject: [Python-Dev] Good way of finding out what C functions we have?
In-Reply-To: <3F78E57E.8090403@ocf.berkeley.edu>
References: <3F779BA6.6050407@ocf.berkeley.edu>
	<m3r820p1vo.fsf@mira.informatik.hu-berlin.de>
	<3F78E57E.8090403@ocf.berkeley.edu>
Message-ID: <16249.33065.797263.217111@montanaro.dyndns.org>


    >> I would look in pyconfig.h.in as the first thing. There is a pitfall,
    >> though: presence of some HAVE_foo in pyconfig.h does not mean that
    >> Python gracefully deals with the absence of foo. When converting
    >> configure to autoconf 2.5x, I checked all macros to determine whether
    >> they were still in use, but some may have become unused/not processed
    >> correctly since. That would be a bug, of course.

    Brett> What does happen if a HAVE_foo is actually required?  Does the
    Brett> build fail or will configure halt and say that the build will
    Brett> fail if you proceed?

It's not exactly clear what you mean by "required".  All the HAVE_foo macros
are supposed to indicate the presence (when defined) or absence (when not
defined) of a particular "feature" which is not felt by one or more of the
authors to be universally available.  As far as I know, a HAVE_foo macro
should never be required to be defined, though I suppose if we had an
infinite amount of time on our hands be might code a configure test for
HAVE_WRITE.  HAVE_* macros should only be used in a conditional way like

    #ifdef HAVE_FOO
       /* assume foo is available */
    else
       /* assume foo is missing and code around its absence */
    endif

or as in the case of fsync() in posixmodule.c

    #ifdef HAVE_FSYNC
       /* define wrapper for fsync()
    endif

The lack of a HAVE_foo macro definition might make an entire module
unavailable.  More often it causes a single module function to not be
defined or for a messier or less efficient implementation to be used.

A rather extreme case was strptime().  It's always been known to not be
available everywhere, and in cases where it was available it didn't always
present the same semantics to the Python programmer.  You can to the rescue
and removed the need for that macro altogether (I think it's too ill-defined
for the interpreter itself to rely on a consistent definition).

Skip

From tim.one at comcast.net  Tue Sep 30 09:45:18 2003
From: tim.one at comcast.net (Tim Peters)
Date: Tue Sep 30 09:45:25 2003
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Tools/scripts
	md5sum.py, 1.3, 1.3.8.1
In-Reply-To: <16249.34285.353517.140250@montanaro.dyndns.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEMFGDAB.tim.one@comcast.net>

>     ! -b   : read files in binary mode (default)
>     ! -t   : read files in text mode (you almost certainly don't want
> this!)
>
> When is -t useful?

I don't know that it ever is.  md5sum.py before 2.3 didn't have such an
option, and always used binary mode.  md5sum.py in 2.3 suddenly grew this
option, and *defaulted* to text mode, which is a disaster for normal usage
on Windows.  On the CVS head I switched the default back to binary mode, but
left -t in since we had released a version supporting a -t option (and as
the default, no less).

I can invent a use case for -t (a text file checksum independent of platform
line-end convention), but it's strained (it's never been a real use case in
my protracted life <wink>).


From skip at pobox.com  Tue Sep 30 09:53:56 2003
From: skip at pobox.com (Skip Montanaro)
Date: Tue Sep 30 09:56:21 2003
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Tools/scripts
 md5sum.py, 1.3, 1.3.8.1
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEMFGDAB.tim.one@comcast.net>
References: <16249.34285.353517.140250@montanaro.dyndns.org>
	<LNBBLJKPBEHFEDALKOLCMEMFGDAB.tim.one@comcast.net>
Message-ID: <16249.35572.260244.137333@montanaro.dyndns.org>


    Tim> ... (it's never been a real use case in my protracted life <wink>).

On the other hand, you never said you were good at geometry.  (Sorry for the
bad pun.)

Skip


From guido at python.org  Tue Sep 30 10:11:52 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Sep 30 10:12:53 2003
Subject: [Python-Dev] Lib/idlelib/NEWS.txt
In-Reply-To: Your message of "Tue, 30 Sep 2003 17:01:30 +1000."
	<200309300701.h8U71VG7017578@localhost.localdomain> 
References: <200309300701.h8U71VG7017578@localhost.localdomain> 
Message-ID: <200309301411.h8UEBqv03072@12-236-54-216.client.attbi.com>

> Idle still has it's own NEWS.txt file, and it's own version number
> (for instance, 2.3.1's version of idle is "1.0.1". Does this make 
> sense? What should happen for 2.3.2, where no idle changes have been
> made?

I thin it's appropriate for IDLE to have its own version, news etc.
Many modules in the stdlib have their own __version__ and docstring.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From anthony at interlink.com.au  Tue Sep 30 10:39:36 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Sep 30 10:41:37 2003
Subject: [Python-Dev] RELEASED Python 2.3.2, release candidate 1 
In-Reply-To: <200309301037.h8UAbGhY029290@localhost.localdomain> 
Message-ID: <200309301439.h8UEdaJv009639@localhost.localdomain>


As previously mentioned, I plan to do the 2.3.2 final release on Friday morning
AU time. Thomas will probably be doing the windows installer on Thursday night
(EU time), so please test test and test again and get your feedback in before 
then.

I've opened bugs against most of the problems I've found on the HP testdrive 
systems (which covers a lot of systems).

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From theller at python.net  Tue Sep 30 12:29:06 2003
From: theller at python.net (Thomas Heller)
Date: Tue Sep 30 12:29:33 2003
Subject: [Python-Dev] OpenSSL vulnerability
Message-ID: <7k3q8ckt.fsf@python.net>

Is this enough reason to use OpenSSL version 0.9.7c instead of 0.9.7b
for the 2.3.2 final windows installer, or should the release candidate
remain unchanged?

<http://www.openssl.org/news/secadv_20030930.txt>

Thomas



From skip at pobox.com  Tue Sep 30 12:58:39 2003
From: skip at pobox.com (Skip Montanaro)
Date: Tue Sep 30 12:59:00 2003
Subject: [Python-Dev] OpenSSL vulnerability
In-Reply-To: <7k3q8ckt.fsf@python.net>
References: <7k3q8ckt.fsf@python.net>
Message-ID: <16249.46655.874245.937560@montanaro.dyndns.org>


    Thomas> Is this enough reason to use OpenSSL version 0.9.7c instead of
    Thomas> 0.9.7b for the 2.3.2 final windows installer, or should the
    Thomas> release candidate remain unchanged?

    Thomas> <http://www.openssl.org/news/secadv_20030930.txt>

At this point I'm inclined to let it go.  There are many other vulnerable
SS[LH targets out there, and you can't wait forever until the OpenSS[LH]
folks stop emitting patches.

Skip


From anthony at interlink.com.au  Tue Sep 30 13:12:00 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Sep 30 13:14:06 2003
Subject: [Python-Dev] OpenSSL vulnerability 
In-Reply-To: <7k3q8ckt.fsf@python.net> 
Message-ID: <200309301712.h8UHC0sJ012578@localhost.localdomain>


>>> Thomas Heller wrote
> Is this enough reason to use OpenSSL version 0.9.7c instead of 0.9.7b
> for the 2.3.2 final windows installer, or should the release candidate
> remain unchanged?
> 
> <http://www.openssl.org/news/secadv_20030930.txt>

I'd say build with the patched libraries, but only if you can test them
before then. OTOH, most of the advisory seems to be about server-side 
problems, and the inbuilt SSL stuff in python is client stuff.

go-not-to-australians-for-answers-for-they-will-answer-both-yes-and-no,
Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From fdrake at acm.org  Tue Sep 30 17:10:38 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue Sep 30 17:10:59 2003
Subject: [Python-Dev] Documentation packages
Message-ID: <16249.61774.732640.328478@grendel.zope.com>


After a brief discussion on the Doc-SIG, it looks like I can
reasonably drop the .tar.gz packaging for the documentation, leaving
only .zip and .tar.bz2 formats.

Are there any strong objections to this change?

(If this flies here, I'll ask the larger community, but I wanted to
get and idea of the reaction here first.)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From bac at OCF.Berkeley.EDU  Tue Sep 30 17:41:12 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Sep 30 17:41:18 2003
Subject: [Python-Dev] Good way of finding out what C functions we have?
In-Reply-To: <16249.33065.797263.217111@montanaro.dyndns.org>
References: <3F779BA6.6050407@ocf.berkeley.edu>
	<m3r820p1vo.fsf@mira.informatik.hu-berlin.de>
	<3F78E57E.8090403@ocf.berkeley.edu>
	<16249.33065.797263.217111@montanaro.dyndns.org>
Message-ID: <3F79F878.4070805@ocf.berkeley.edu>

Skip Montanaro wrote:
>     >> I would look in pyconfig.h.in as the first thing. There is a pitfall,
>     >> though: presence of some HAVE_foo in pyconfig.h does not mean that
>     >> Python gracefully deals with the absence of foo. When converting
>     >> configure to autoconf 2.5x, I checked all macros to determine whether
>     >> they were still in use, but some may have become unused/not processed
>     >> correctly since. That would be a bug, of course.
> 
>     Brett> What does happen if a HAVE_foo is actually required?  Does the
>     Brett> build fail or will configure halt and say that the build will
>     Brett> fail if you proceed?
> 
> It's not exactly clear what you mean by "required".

As in Python will not be at all functional without the functionality 
being required.

>  All the HAVE_foo macros
> are supposed to indicate the presence (when defined) or absence (when not
> defined) of a particular "feature" which is not felt by one or more of the
> authors to be universally available.  As far as I know, a HAVE_foo macro
> should never be required to be defined, though I suppose if we had an
> infinite amount of time on our hands be might code a configure test for
> HAVE_WRITE.  HAVE_* macros should only be used in a conditional way like
> 
>     #ifdef HAVE_FOO
>        /* assume foo is available */
>     else
>        /* assume foo is missing and code around its absence */
>     endif
> 
> or as in the case of fsync() in posixmodule.c
> 
>     #ifdef HAVE_FSYNC
>        /* define wrapper for fsync()
>     endif
> 
> The lack of a HAVE_foo macro definition might make an entire module
> unavailable.  More often it causes a single module function to not be
> defined or for a messier or less efficient implementation to be used.
> 

Right.  I was wondering if there are any checks in configure.in that if 
something was not available that Python itself would not compile *at 
all*.  I would suspect not since that is what the ANSI C/POSIX coding 
requirement is supposed to handle, right?

-Brett


From martin at v.loewis.de  Tue Sep 30 17:58:07 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Sep 30 17:58:18 2003
Subject: [Python-Dev] Good way of finding out what C functions we have?
In-Reply-To: <3F78E57E.8090403@ocf.berkeley.edu>
References: <3F779BA6.6050407@ocf.berkeley.edu>
	<m3r820p1vo.fsf@mira.informatik.hu-berlin.de>
	<3F78E57E.8090403@ocf.berkeley.edu>
Message-ID: <m3ad8mdjm8.fsf@mira.informatik.hu-berlin.de>

"Brett C." <bac@OCF.Berkeley.EDU> writes:

> Shows how much I know; I found that last night but since it didn't say
> POSIX anywhere I thought it was part of another standard.  I guess the
> "Issue 5" mention means that BASE is POSIX?  Or is IEEE 1003.1
> basically POSIX?

In the past, the various standards different slightly. Today, IEEE
1003.1 is the same as ISO/IEC 9945:2003 (parts 1..4), is the same as
Single Unix version 3, is the same as XPG/5. It contains the
OpenGroup's XBD, XSH, XCU, and XRAT documents. The UNIX product
standard (which apparently hasn't been revised since '98) also covers
XCURSES, XTI, and a few others; UNIX98 workstation also supports X11
and CDE.

> What does happen if a HAVE_foo is actually required?  Does the build
> fail or will configure halt and say that the build will fail if you
> proceed?

configure only checks for the feature. If the feature is absent, yet
we use the feature without an #ifdef, the build just fails; that would
be a bug.

Regards,
Martin

From fdrake at acm.org  Tue Sep 30 18:11:50 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue Sep 30 18:13:04 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <2mzngr1bf2.fsf@starship.python.net>
References: <200309261752.h8QHqJgD029823@localhost.localdomain>
	<LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
	<20030926184853.GB22837@mems-exchange.org>
	<he2zba58.fsf@python.net> <2mzngr1bf2.fsf@starship.python.net>
Message-ID: <16249.65446.699521.103706@grendel.zope.com>


Michael Hudson writes:
 > It occurs to me that I don't know *why* Fred is so much the
 > documentation man; I've not had any trouble processing the docs into
 > HTML lately (haven't tried on Windows, admittedly, and I haven't
 > tried to make info ever).

It's certainly gotten easier to deal with the documentation on modern
Linux distributions.  At CNRI, we used mostly Solaris boxes, and I
have to build my own teTeX installations from source, and hand-select
a version of LaTeX2HTML that worked for me.

At this point, all the software that I can't just install from a
RedHat CD is part of what gets pulled down from CVS.  I've been able
to build the docs on Cygwin as well, though I've not tried lately.
A lot of what it takes to build the docs is written into Doc/Makefile,
but it does require a solid make (it even uses $(shell ...) now, so
maybe only GNU make will do; not sure).

 > What else needs to be done?  There must be quite a bit of mucking
 > about on creosote to do, I guess.

There's a bit, but that's getting easier and easier as I've gone
through it a few times now.  I updated PEP 101 the other evening so
anyone can do what's needed to build the packages and get them in the
download locations.  There's more to be written to explain what else
needs to be updated on the site.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From fdrake at acm.org  Tue Sep 30 18:15:51 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue Sep 30 18:16:04 2003
Subject: [Python-Dev] Re: RELEASED Python 2.3.1
In-Reply-To: <he2zba58.fsf@python.net>
References: <200309261752.h8QHqJgD029823@localhost.localdomain>
	<LNBBLJKPBEHFEDALKOLCEEJHGCAB.tim.one@comcast.net>
	<20030926184853.GB22837@mems-exchange.org>
	<he2zba58.fsf@python.net>
Message-ID: <16250.151.309386.606310@grendel.zope.com>


Thomas Heller writes:
 > I suggest to update the version numbers in these files to the next
 > version *after* a release is done.  So, in the release23-maint branch
 > the version numbers could *now* be adjusted to 2.3.2.

The Doc/ tree no longer depends on version numbers stored in the Doc/
tree; the makefile causes the script Doc/tools/getversioninfo to pull
the version numbers from Include/patchlevel.h.

 > And it would help if I could build the HTML docs myself from CVS. I did
 > manage to create the pdf files with TeTex under windows, but I didn't
 > succeed with the html pages so far.

Are you using Cygwin?  What problems did you encounter?  I'll help if
I can; I have a Windows machine (sometimes), but don't know anything
about non-Cygwin Windows TeX systems (and I don't have Cygwin
installed most of the time).


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From tree at basistech.com  Tue Sep 30 18:33:50 2003
From: tree at basistech.com (Tom Emerson)
Date: Tue Sep 30 18:36:25 2003
Subject: [Python-Dev] imp.findmodule and zip files
Message-ID: <16250.1230.305462.233362@magrathea.basistech.com>

Should imp.find_module() work for modules that are packaged in a zip
file in 2.3.x? I'm seeing that this doesn't, and before I dive in to
figure out why, I want to see if this is the intent.

-- 
Tom Emerson                                          Basis Technology Corp.
Software Architect                                 http://www.basistech.com
  "Beware the lollipop of mediocrity: lick it once and you suck forever"