From tim.hochberg at ieee.org  Wed Dec  1 06:35:30 2004
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Wed Dec  1 06:35:39 2004
Subject: [Python-Dev] Roster Deadline
Message-ID: <41AD5822.2000501@ieee.org>

Hi Larry,


FYI: I asked EB about the roster deadline and she says that she doesn't 
know when it is either. Checking on the Lei Out web page didn't help 
much either.

So, you are no wiser now than at the start of this message.

-tim

From fdrake at acm.org  Wed Dec  1 07:34:51 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed Dec  1 07:35:12 2004
Subject: [Python-Dev] Python.org current docs
In-Reply-To: <200411301612.37180.fdrake@acm.org>
References: <8y8jrufo.fsf@python.net> <200411301612.37180.fdrake@acm.org>
Message-ID: <200412010134.51160.fdrake@acm.org>

On Tuesday 30 November 2004 02:46 pm, Thomas Heller wrote:
  > http://www.python.org/doc/current/
  > and
  > http://docs.python.org/
  >
 >  > still point to 2.3.4 docs.

I think everything is properly updated now.  Please let me know if I've missed 
anything.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>

From mal at egenix.com  Wed Dec  1 10:23:16 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Dec  1 10:23:12 2004
Subject: [Python-Dev] File encodings
In-Reply-To: <20041130130934.GC6610@burma.localdomain>
References: <20041129190448.GA23399@burma.localdomain>	<41AC34D2.1060909@egenix.com>
	<20041130130934.GC6610@burma.localdomain>
Message-ID: <41AD8D84.7070903@egenix.com>

Gustavo Niemeyer wrote:
> [...]
> 
>>The idiom presented by Bob is the right way to go: wrap
>>sys.stdout with a StreamWriter.
> 
> I don't see that as a good solution, since every Python software
> that is internationalizaed will have do figure out this wrapping,
> introducing extra overhead unnecessarily.

I don't see any unnecessary overhead and using the wrappers
is really easy, e.g.:

#
# Application uses Latin-1 for I/O, terminal uses UTF-8
#
import codecs, sys

# Make stdout translate Latin-1 output into UTF-8 output
sys.stdout = codecs.EncodedFile(sys.stdout, 'latin-1', 'utf-8')

# Have stdin translate Latin-1 input into UTF-8 input
sys.stdin = codecs.EncodedFile(sys.stdin, 'utf-8', 'latin-1')


We should probably extend the support in StreamRecoder (which
is used by the above EncodedFile helper) to also support Unicode
input to .write() and have a special codec 'unicode' that converts
Unicode to Unicode, so that you can request the EncodedFile object
to return Unicode for .read().

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 01 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From kiko at async.com.br  Wed Dec  1 14:08:25 2004
From: kiko at async.com.br (Christian Robottom Reis)
Date: Wed Dec  1 14:08:34 2004
Subject: [Python-Dev] Difflib modifications [reposted]
Message-ID: <20041201130825.GQ4847@async.com.br>


[Reposted to python-dev!]

Hello there,

    We've has done some customizations to difflib to make it work well
with pagetests we are running on a project at Canonical, and we are
looking for some guidance as to what's the best way to do them. There
are some tricky bits that have to do with how the class inheritance is
put together, and since we would want to avoid duplicating difflib I
figured we'd ask and see if some grand ideas come up.

A [rough first cut of the] patch is inlined below. Essentially, it does:

    - Implements a custom Differ.fancy_compare function that supports
      ellipsis and omits equal content

    - Hacks _fancy_replace to skip ellipsis as well.

    - Hacks best_ratio and cutoff. I'm a bit fuzzy on why this was
      changed, to be honest, and Celso's travelling today, but IIRC it
      had to do with how difflib grouped changes.

Essentially, what we aim for is:

    - Ignoring ellipsisized(!) content
    - Omitting content which is equal

I initially thought the best way to do this would be to inherit from
SequenceMatcher and make it not return opcodes for ellipsis. However,
there is no easy way to replace the class short of rewriting major bits
of Differ. I suspect this could be easily changed to use a class
attribute that we could override, but let me know what you think of the
whole thing.

--- /usr/lib/python2.3/difflib.py	2004-11-18 20:05:38.720109040 -0200
+++ difflib.py	2004-11-18 20:24:06.731665680 -0200
@@ -885,6 +885,45 @@
             for line in g:
                 yield line
 
+    def fancy_compare(self, a, b):
+        """
+        >>> import difflib
+        >>> engine = difflib.Differ()
+        >>> got = ['World is Cruel', 'Dudes are Cool']
+        >>> want = ['World ... Cruel', 'Dudes ... Cool']
+        >>> list(engine.fancy_compare(want, got))
+        []
+         
+        """
+        cruncher = SequenceMatcher(self.linejunk, a, b)
+        for tag, alo, ahi, blo, bhi in cruncher.get_opcodes():
+
+            if tag == 'replace':
+                ## replace single line
+                if a[alo:ahi][0].rstrip() == '...' and ((ahi - alo) == 1):   
+                    g = None
+                ## two lines replaced  
+                elif a[alo:ahi][0].rstrip() == '...' and ((ahi - alo) > 1):   
+                    g = self._fancy_replace(a, (ahi - 1), ahi,
+                                            b, (bhi - 1), bhi)
+                ## common
+                else:
+                    g = self._fancy_replace(a, alo, ahi, b, blo, bhi)
+            elif tag == 'delete':
+                g = self._dump('-', a, alo, ahi)
+            elif tag == 'insert':
+                g = self._dump('+', b, blo, bhi)
+            elif tag == 'equal':
+                # do not show anything
+                g = None
+            else:
+                raise ValueError, 'unknown tag ' + `tag`
+
+            if g:
+                for line in g:
+                    yield line
+        
+
     def _dump(self, tag, x, lo, hi):
         """Generate comparison results for a same-tagged range."""
         for i in xrange(lo, hi):
@@ -926,7 +965,13 @@
 
         # don't synch up unless the lines have a similarity score of at
         # least cutoff; best_ratio tracks the best score seen so far
-        best_ratio, cutoff = 0.74, 0.75
+        #best_ratio, cutoff = 0.74, 0.75
+
+        ## reduce the cutoff to have enough similarity
+        ## between '<something> ... <something>' and '<a> blabla </a>'
+        ## for example 
+        best_ratio, cutoff = 0.009, 0.01
+
         cruncher = SequenceMatcher(self.charjunk)
         eqi, eqj = None, None   # 1st indices of equal lines (if any)
 
@@ -981,7 +1026,11 @@
             cruncher.set_seqs(aelt, belt)
             for tag, ai1, ai2, bj1, bj2 in cruncher.get_opcodes():
                 la, lb = ai2 - ai1, bj2 - bj1
-                if tag == 'replace':
+
+                if aelt[ai1:ai2] == '...':
+                    return
+
+                if tag == 'replace':                    
                     atags += '^' * la
                     btags += '^' * lb
                 elif tag == 'delete':

Take care,
--
Christian Robottom Reis | http://async.com.br/~kiko/ | [+55 16] 3361 2331
From anthony at interlink.com.au  Wed Dec  1 14:32:07 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Dec  1 14:33:13 2004
Subject: [Python-Dev] Re: Small subprocess patch
In-Reply-To: <Pine.GSO.4.51L2.0411302255440.21491@koeberg.lysator.liu.se>
References: <Pine.GSO.4.51L2.0411302123220.18754@koeberg.lysator.liu.se>
	<Pine.GSO.4.51L2.0411302255440.21491@koeberg.lysator.liu.se>
Message-ID: <200412020032.07794.anthony@interlink.com.au>

On Wednesday 01 December 2004 09:00, Peter Astrand wrote:
> I'm also wondering if patch 1071755 and 1071764 should go into
> release24-maint:
>
> * 1071755 makes subprocess raise TypeError if Popen is called with a
> bufsize that is not an integer.

Since this isn't changing anything that's user facing (just making the
error handling more explicit) this is suitable for the maint branch.

> * 1071764 adds a new, small utility function.

Please read PEP 6. Maintenance branches are not the place for new
features. For an example why, consult almost any code that requires
Python 2.2 or later. Chances are you'll find, all over the place, code 
like:

try:
    True, False
except NameError:
    True, False = 1, 0

From mal at egenix.com  Wed Dec  1 16:10:10 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed Dec  1 16:10:15 2004
Subject: [Python-Dev] MS VC compiler versions
Message-ID: <41ADDED2.30707@egenix.com>

Preparing for the distutils patch to allow building extensions
using the .NET SDK compilers, I am compiling a list of version
numbers and MS compiler logo outputs in order to use these to
identify the correct compiler to use for the extensions.

These are the compilers I have found so far:

* MS VC6 (German version; optimizing VC6 compiler):

Optimierender Microsoft (R) 32-Bit C/C++-Compiler, Version 12.00.8804, fuer x86
Copyright (C) Microsoft Corp 1984-1998. Alle Rechte vorbehalten.

* MS .NET SDK 1.1 Compiler (German version; non-optimizing VC7.1 compiler):

Microsoft (R) 32-Bit C/C++-Standardcompiler Version 13.10.3077 f?r 80x86
Copyright (C) Microsoft Corporation 1984-2002. Alle Rechte vorbehalten.

* MS VC7.1 (aka .NET 2003, US version; optimizing VC7.1 compiler)

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86
Copyright (C) Microsoft Corporation 1984-2002. All rights reserved.


[It looks as if optimizing vs. non-optimizing is not something that
you can detect by looking at the version number.]

Could you please provide me with more version numbers and logo
printouts ?

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 01 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From hyeshik at gmail.com  Wed Dec  1 16:43:12 2004
From: hyeshik at gmail.com (Hye-Shik Chang)
Date: Wed Dec  1 16:43:15 2004
Subject: [Python-Dev] MS VC compiler versions
In-Reply-To: <41ADDED2.30707@egenix.com>
References: <41ADDED2.30707@egenix.com>
Message-ID: <4f0b69dc0412010743352b1a8e@mail.gmail.com>

On Wed, 01 Dec 2004 16:10:10 +0100, M.-A. Lemburg <mal@egenix.com> wrote:
> Could you please provide me with more version numbers and logo
> printouts ?
> 

* MS Windows XP DDK (International version, optimizing VC 7.0):

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.00.9176 for 80x86
Copyright (C) Microsoft Corporation 1984-2001. All rights reserved.

* MS VS6 SP5 (International version, optimizing VC 6.0):

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8168 for 80x86
Copyright (C) Microsoft Corp 1984-1998. All rights reserved.

Hye-Shik
From theller at python.net  Wed Dec  1 19:14:51 2004
From: theller at python.net (Thomas Heller)
Date: Wed Dec  1 19:14:03 2004
Subject: [Python-Dev] Trouble installing 2.4
In-Reply-To: <000401c4d708$ba149dc0$6402a8c0@arkdesktop> (Andrew Koenig's
	message of "Tue, 30 Nov 2004 13:16:34 -0500")
References: <000401c4d708$ba149dc0$6402a8c0@arkdesktop>
Message-ID: <mzwxrilg.fsf@python.net>

"Andrew Koenig" <ark-mlist@att.net> writes:

> Follow-up:  When I install Python as Administrator, all is well.  In that
> case (but not when installing it as me), it asks whether I want to install
> it for all users or for myself only.  I then install pywin32 and it works.
>
> So it may be that a caveat is in order to people who do not install 2.4 as
> Administrator.

As Martin guessed, a distutils bug, triggered with non-admin Python
installs, and when the add-on package uses the pre-install-script or
post-install-script option.

Please submit a report to SF.

Thanks,

Thomas

From trentm at ActiveState.com  Wed Dec  1 19:54:31 2004
From: trentm at ActiveState.com (Trent Mick)
Date: Wed Dec  1 19:55:50 2004
Subject: [Python-Dev] MS VC compiler versions
In-Reply-To: <41ADDED2.30707@egenix.com>
References: <41ADDED2.30707@egenix.com>
Message-ID: <41AE1367.5000805@activestate.com>

M.-A. Lemburg wrote:
> Preparing for the distutils patch to allow building extensions
> using the .NET SDK compilers, I am compiling a list of version
> numbers and MS compiler logo outputs in order to use these to
> identify the correct compiler to use for the extensions.
> 
> These are the compilers I have found so far:
> 
> * MS VC6 (German version; optimizing VC6 compiler):
> 
> Optimierender Microsoft (R) 32-Bit C/C++-Compiler, Version 12.00.8804, 
> fuer x86
> Copyright (C) Microsoft Corp 1984-1998. Alle Rechte vorbehalten.

* MS VC6 (US version; optimizing VC6 compiler):

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8804 for 80x86
Copyright (C) Microsoft Corp 1984-1998. All rights reserved.



Trent

-- 
Trent Mick
trentm@activestate.com
From aahz at pythoncraft.com  Wed Dec  1 20:25:57 2004
From: aahz at pythoncraft.com (Aahz)
Date: Wed Dec  1 20:25:59 2004
Subject: [Python-Dev] Roster Deadline
In-Reply-To: <41AD5822.2000501@ieee.org>
References: <41AD5822.2000501@ieee.org>
Message-ID: <20041201192557.GA24980@panix.com>

On Tue, Nov 30, 2004, Tim Hochberg wrote:
>
> Hi Larry,
> 
> FYI: I asked EB about the roster deadline and she says that she doesn't 
> know when it is either. Checking on the Lei Out web page didn't help 
> much either.
> 
> So, you are no wiser now than at the start of this message.

We're even less wise now given that you probably didn't intend this for
python-dev.  ;-)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

WiFi is the SCSI of the 21st Century -- there are fundamental technical
reasons for sacrificing a goat.  (with no apologies to John Woods)
From steven.bethard at gmail.com  Wed Dec  1 22:03:50 2004
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed Dec  1 22:03:54 2004
Subject: [Python-Dev] adding key argument to min and max
Message-ID: <d11dcfba0412011303463121d3@mail.gmail.com>

This is my first post to Python dev, so I figured I should introduce myself.

My name's Steven Bethard and I'm a computer science Ph.D. student at
the University of Colorado at Boulder working primarily in the areas
of natural language processing and machine learning.  During my
undergrad at the University of Arizona, I worked as a teaching
assistant teaching Java for 2 1/2 years, though now that I'm at CU
Boulder I pretty much only work in Python.  I started getting active
on the Python list about 6 months ago, and I've been watching
python-dev for the last few months.

On to the real question...

I posted a few notes about this on the python-list and didn't hear
much of a response, so I thought that maybe python-dev is the more
appropriate place (since it involves a change to some of the builtin
functions).

For Python 2.5, I'd like to add a keyword argument 'key' to min and
max like we have now for list.sort and sorted.  I've needed this a
couple of times now, specifically when I have something like a dict of
word counts, and I want the most frequent word, I'd like to do
something like:

>>> d = dict(aaa=3000, bbb=2000, ccc=1000)
>>> max(d, key=d.__getitem__)
'aaa'

I've implemented a patch that provides this functionality, but there
are a few concerns about how it works.  Here's some examples of what
it does now:

>>> d = dict(aaa=3000, bbb=2000, ccc=1000)
>>> max(d)
'ccc'
>>> max(d, key=d.__getitem__)
'aaa'
>>> max(d, d.__getitem__)
{'aaa': 3000, 'bbb': 2000, 'ccc': 1000}

>>> max(('aaa', 3000), ('bbb', 2000), ('ccc', 1000))
('ccc', 1000)
>>> max(('aaa', 3000), ('bbb', 2000), ('ccc', 1000), key=operator.itemgetter(1))
('aaa', 3000)
>>> max(('aaa', 3000), ('bbb', 2000), ('ccc', 1000), operator.itemgetter(1))
('ccc', 1000)

Note the difference between the 2nd and 3rd use of max in each
example.  For backwards compatibility reasons, the 'key' argument
cannot be specified as a positional argument or it will look like max
is being used in the max(a, b, c, ...) form.  This means that a 'key'
argument can *only* be specified as a keyword parameter, thus giving
us the asymmetry we see in these examples.

My real question then is, is this asymmetry a problem?  Is it okay to
have a parameter that is *only* accessable as a keyword parameter?

Thanks,

Steve
-- 
You can wordify anything if you just verb it.
        --- Bucky Katt, Get Fuzzy
From bac at OCF.Berkeley.EDU  Wed Dec  1 22:42:02 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed Dec  1 22:42:20 2004
Subject: [Python-Dev] TRUNK UNFROZEN; release24-maint branch has been cut
In-Reply-To: <200412010023.17728.anthony@interlink.com.au>
References: <200412010023.17728.anthony@interlink.com.au>
Message-ID: <41AE3AAA.7010402@ocf.berkeley.edu>

Anthony Baxter wrote:
> I've cut the release24-maint branch, and updated the Include/patchlevel.h
> on trunk and branch (trunk is now 2.5a0, branch is 2.4+)
> 
> The trunk and the branch are now both unfrozen and suitable for checkins.
> The feature freeze on the trunk is lifted. Remember - if you're checking 
> bugfixes into the trunk, either backport them to the branch, or else mark 
> the commit message with 'bugfix candidate' or 'backport candidate' or the
> like.
> 
> Next up will be a 2.3.5 release. I'm going to be travelling for a large chunk
> of December (at very short notice) so it's likely that this will happen at the
> start of January.

OK, I will send out an email to python-list and python-announce mentioning this 
to the community and that if they have fixes they need to go into 2.3.5 they 
need to get it in ASAP so there is enough time to consider them (with no 
guarantee they will get in) at the end of the week if no one objects.

-Brett
From pje at telecommunity.com  Wed Dec  1 23:10:07 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Dec  1 23:08:35 2004
Subject: [Python-Dev] adding key argument to min and max
In-Reply-To: <d11dcfba0412011303463121d3@mail.gmail.com>
Message-ID: <5.1.1.6.0.20041201170900.02532c40@mail.telecommunity.com>

At 02:03 PM 12/1/04 -0700, Steven Bethard wrote:
>Is it okay to
>have a parameter that is *only* accessable as a keyword parameter?

Yes.

From python at rcn.com  Thu Dec  2 00:23:29 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Dec  2 00:26:09 2004
Subject: [Python-Dev] adding key argument to min and max
In-Reply-To: <d11dcfba0412011303463121d3@mail.gmail.com>
Message-ID: <001e01c4d7fc$c49efa20$e841fea9@oemcomputer>

[Steven Bethard]
> For Python 2.5, I'd like to add a keyword argument 'key' to min and
> max like we have now for list.sort and sorted. 
 . . .
> This means that a 'key'
> argument can *only* be specified as a keyword parameter, thus giving
> us the asymmetry we see in these examples.

FWIW, in Py2.5 I plan on adding a key= argument to heapq.nsmallest() and
heapq.nlargest().  There is no "assymmetry" issue with those functions,
so it can be implemented cleanly.  And, since min/max are essentially
the same nsmallest/nlargest with n==1, your use case is covered and
there is no need to mess with the min/max builtins.



> I've needed this a couple of times now, specifically when 
> I have something like a dict of word counts, and I want the most
frequent word

For Py2.4, you can cover your use cases readily adding the recipe for
mostcommon() to a module of favorite utilities:

   http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/347615

Alternatively, the recipe for a bag class is a more flexible and still
reasonably efficient:

    http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/259174



Raymond Hettinger

From steven.bethard at gmail.com  Thu Dec  2 02:02:33 2004
From: steven.bethard at gmail.com (Steven Bethard)
Date: Thu Dec  2 02:02:36 2004
Subject: [Python-Dev] adding key argument to min and max
In-Reply-To: <001e01c4d7fc$c49efa20$e841fea9@oemcomputer>
References: <d11dcfba0412011303463121d3@mail.gmail.com>
	<001e01c4d7fc$c49efa20$e841fea9@oemcomputer>
Message-ID: <d11dcfba041201170275239b1b@mail.gmail.com>

Raymond Hettinger <python@rcn.com> wrote:
> [Steven Bethard]
> > For Python 2.5, I'd like to add a keyword argument 'key' to min and
> > max like we have now for list.sort and sorted. 
>  . . .
> > This means that a 'key'
> > argument can *only* be specified as a keyword parameter, thus giving
> > us the asymmetry we see in these examples.
> 
> FWIW, in Py2.5 I plan on adding a key= argument to heapq.nsmallest() and
> heapq.nlargest().  There is no "assymmetry" issue with those functions,
> so it can be implemented cleanly.  And, since min/max are essentially
> the same nsmallest/nlargest with n==1, your use case is covered and
> there is no need to mess with the min/max builtins.

I don't want to put words into your mouth, so is this a vote against a
key= argument for min and max?

If nsmallest/nlargest get key= arguments, this would definitely cover
the same cases.  If a key= argument gets vetoed for min and max, I'd
at least like to add a bit of documentation pointing users of min/max
to nsmallest/nlargest if they want a key= argument...

Steve
-- 
You can wordify anything if you just verb it.
        --- Bucky Katt, Get Fuzzy
From python at rcn.com  Thu Dec  2 02:32:10 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Dec  2 02:34:51 2004
Subject: [Python-Dev] adding key argument to min and max
In-Reply-To: <d11dcfba041201170275239b1b@mail.gmail.com>
Message-ID: <002301c4d80e$beb08e00$e841fea9@oemcomputer>

> I don't want to put words into your mouth, so is this a vote against a
> key= argument for min and max?

Right.  I don't think there is any need.

 
> If nsmallest/nlargest get key= arguments, this would definitely cover
> the same cases. 

Right.


> If a key= argument gets vetoed for min and max, I'd
> at least like to add a bit of documentation pointing users of min/max
> to nsmallest/nlargest if they want a key= argument...

Sounds reasonable.



Raymond


P.S.  In case you're interested, here is the patch:


Index: heapq.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/heapq.py,v
retrieving revision 1.27
diff -u -r1.27 heapq.py
--- heapq.py	29 Nov 2004 05:54:47 -0000	1.27
+++ heapq.py	2 Dec 2004 01:32:44 -0000
@@ -307,6 +307,31 @@
 except ImportError:
     pass
 
+# Extend the implementations of nsmallest and nlargest to use a key=
argument
+_nsmallest = nsmallest
+def nsmallest(n, iterable, key=None):
+    """Find the n smallest elements in a dataset.
+
+    Equivalent to:  sorted(iterable, key=key)[:n]
+    """
+    if key is None:
+        return _nsmallest(n, iterable)
+    it = ((key(r), i, r) for i, r in enumerate(iterable))   # decorate
+    result = _nsmallest(n, it)
+    return [r for k, i, r in result]                        #
undecorate
+
+_nlargest = nlargest
+def nlargest(n, iterable, key=None):
+    """Find the n largest elements in a dataset.
+
+    Equivalent to:  sorted(iterable, key=key, reverse=True)[:n]
+    """
+    if key is None:
+        return _nlargest(n, iterable)
+    it = ((key(r), i, r) for i, r in enumerate(iterable))   # decorate
+    result = _nlargest(n, it)
+    return [r for k, i, r in result]                        #
undecorate
+
 if __name__ == "__main__":
     # Simple sanity test
     heap = []
Index: test/test_heapq.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test_heapq.py,v
retrieving revision 1.16
diff -u -r1.16 test_heapq.py
--- test/test_heapq.py	29 Nov 2004 05:54:48 -0000	1.16
+++ test/test_heapq.py	2 Dec 2004 01:32:44 -0000
@@ -105,13 +105,19 @@
 
     def test_nsmallest(self):
         data = [random.randrange(2000) for i in range(1000)]
+        f = lambda x:  x * 547 % 2000
         for n in (0, 1, 2, 10, 100, 400, 999, 1000, 1100):
             self.assertEqual(nsmallest(n, data), sorted(data)[:n])
+            self.assertEqual(nsmallest(n, data, key=f),
+                             sorted(data, key=f)[:n])
 
     def test_largest(self):
         data = [random.randrange(2000) for i in range(1000)]
+        f = lambda x:  x * 547 % 2000
         for n in (0, 1, 2, 10, 100, 400, 999, 1000, 1100):
             self.assertEqual(nlargest(n, data), sorted(data,
reverse=True)[:n])
+            self.assertEqual(nlargest(n, data, key=f),
+                             sorted(data, key=f, reverse=True)[:n])
 
 
 
#=======================================================================
=======

From gvanrossum at gmail.com  Thu Dec  2 02:58:48 2004
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu Dec  2 02:58:51 2004
Subject: [Python-Dev] adding key argument to min and max
In-Reply-To: <002301c4d80e$beb08e00$e841fea9@oemcomputer>
References: <d11dcfba041201170275239b1b@mail.gmail.com>
	<002301c4d80e$beb08e00$e841fea9@oemcomputer>
Message-ID: <ca471dc204120117582c69d6ac@mail.gmail.com>

> > I don't want to put words into your mouth, so is this a vote against a
> > key= argument for min and max?
> 
> Right.  I don't think there is any need.

Hm, min and max are probably needed 2-3 orders of magnitude more
frequently than nsmallest/nlargest. So I think it's reasonable to add
the key= argument to min and max as well. (We didn't leave it out of
sorted() because you can already do it with list.sort().)

> def test_largest(self):

shouldn't that be test_nlargest?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
From python at rcn.com  Thu Dec  2 03:04:57 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Dec  2 03:07:36 2004
Subject: [Python-Dev] adding key argument to min and max
In-Reply-To: <d11dcfba0412011303463121d3@mail.gmail.com>
Message-ID: <002e01c4d813$529cc300$e841fea9@oemcomputer>



> -----Original Message-----
> From: python-dev-bounces+python=rcn.com@python.org [mailto:python-dev-
> bounces+python=rcn.com@python.org] On Behalf Of Steven Bethard
> Sent: Wednesday, December 01, 2004 4:04 PM
> To: python-dev@python.org
> Subject: [Python-Dev] adding key argument to min and max
> 
> This is my first post to Python dev, so I figured I should introduce
> myself.
> 
> My name's Steven Bethard and I'm a computer science Ph.D. student at
> the University of Colorado at Boulder working primarily in the areas
> of natural language processing and machine learning.  During my
> undergrad at the University of Arizona, I worked as a teaching
> assistant teaching Java for 2 1/2 years, though now that I'm at CU
> Boulder I pretty much only work in Python.  I started getting active
> on the Python list about 6 months ago, and I've been watching
> python-dev for the last few months.
> 
> For Python 2.5, I'd like to add a keyword argument 'key' to min and
> max like we have now for list.sort and sorted.
 . . .
> I've implemented a patch that provides this functionality, but there
> are a few concerns about how it works.

Guido says yes.  So, load the patch to SF and assign to me for review,
testing, and documentation.


Raymond

From kbk at shore.net  Thu Dec  2 04:15:53 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Thu Dec  2 04:15:57 2004
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <873byp1jbq.fsf@hydra.localdomain>

Patch / Bug Summary
___________________

Patches :  258 open ( +4) /  2701 closed ( +1) /  2959 total ( +5)
Bugs    :  812 open (+28) /  4642 closed (+13) /  5454 total (+41)
RFE     :  160 open ( +4) /   136 closed ( +1) /   296 total ( +5)

New / Reopened Patches
______________________

#1074261 gzip dies on gz files with many appended headers  (2004-11-27)
       http://python.org/sf/1074381  opened by  Mark Eichin

Flush stdout/stderr if closed (fix for bug 1074011)  (2004-11-29)
       http://python.org/sf/1075147  opened by  Ben Hutchings

gcc compiler on Windows  (2004-11-30)
       http://python.org/sf/1075887  opened by  Michiel de Hoon

AUTH PLAIN in smtplib  (2004-11-30)
       http://python.org/sf/1075928  opened by  James Lan

readline does not need termcap  (2004-12-01)
       http://python.org/sf/1076826  opened by  Michal &#268;iha&#345;

Patches Closed
______________

bug#1021756: more informative error message  (2004-11-23)
       http://python.org/sf/1071739  closed by  effbot

New / Reopened Bugs
___________________

Bugs in _csv module - lineterminator  (2004-11-24)
       http://python.org/sf/1072404  opened by  Chris Withers

mkarg undocumented  (2004-11-24)
       http://python.org/sf/1072410  opened by  Gunter Ohrner

email as_string() omits trailing newline  (2004-11-24)
CLOSED http://python.org/sf/1072623  opened by  Tessa Lau

dyld: ./python.exe multiple definitions of symbol _BC  (2004-11-24)
       http://python.org/sf/1072642  opened by  Marius

thisid not intialized in pindent.py script  (2004-11-24)
       http://python.org/sf/1072853  opened by  Niraj Bajpai

^Z doesn't exit interpreter - 2.4c1 & Win2K  (2004-11-26)
       http://python.org/sf/1073736  opened by  Kent Johnson

subprocess.py doc bug in 2.4c1  (2004-11-26)
CLOSED http://python.org/sf/1073790  opened by  Dan Christensen

2 XML parsing errors  (2004-11-26)
CLOSED http://python.org/sf/1073864  opened by  Peer Janssen

write failure ignored in Py_Finalize()  (2004-11-27)
       http://python.org/sf/1074011  opened by  Matthias Klose

current directory in sys.path handles symlinks badly  (2004-11-26)
       http://python.org/sf/1074015  opened by  Eric M. Hopper

xml.dom.minidom produces errors with certain unicode chars  (2004-11-27)
       http://python.org/sf/1074200  opened by  Peer Janssen

gzip dies on gz files with many appended headers  (2004-11-27)
       http://python.org/sf/1074261  opened by  Mark Eichin

input from numeric pad always dropped when numlock off  (2004-11-27)
       http://python.org/sf/1074333  opened by  Rick Graves

FeedParser problem on end boundaries w/o newline  (2004-11-24)
       http://python.org/sf/1072623  reopened by  tlau

Irregular behavior of datetime.__str__()  (2004-11-27)
       http://python.org/sf/1074462  opened by  Wai Yip Tung

Errors and omissions in logging module documentation  (2004-11-28)
       http://python.org/sf/1074693  opened by  Joachim Boomberschloss

Windows 2.4c1 installer default location issues  (2004-11-28)
       http://python.org/sf/1074873  opened by  dmerrill

exceeding obscure weakproxy bug  (2004-11-29)
       http://python.org/sf/1075356  opened by  Michael Hudson

urllib2.HTTPBasicAuthHandler problem with [HOST]:[PORT]  (2004-11-29)
       http://python.org/sf/1075427  opened by  O-Zone

PyGILState_Ensure() deadlocks (ver 2.4c1)  (2004-11-29)
       http://python.org/sf/1075703  opened by  Andi Vajda

Build Bug on Solaris.  (2004-11-30)
CLOSED http://python.org/sf/1075934  opened by  Jeremy Whiting

Memory fault pyexpat.so on SGI  (2004-11-30)
       http://python.org/sf/1075984  opened by  Maik Hertha

Memory fault pyexpat.so on SGI  (2004-11-30)
       http://python.org/sf/1075990  opened by  Maik Hertha

HTMLParser can't handle page with javascript  (2004-11-30)
       http://python.org/sf/1076070  opened by  Jeremy Hylton

distutils.core.setup() with unicode arguments broken  (2004-11-30)
       http://python.org/sf/1076233  opened by  Walter D?rwald

Whats New for 2.4 "SafeTemplate" patch.  (2004-11-30)
CLOSED http://python.org/sf/1076365  opened by  Sean Reifschneider

test_shutil fails on x86-64 // Suse 9.1  (2004-11-30)
       http://python.org/sf/1076467  opened by  Ross G Baker Jr

Another message that croaks email.FeedParser  (2004-11-30)
       http://python.org/sf/1076485  opened by  Skip Montanaro

Sate/Save typo in Mac/scripts/BuildApplication.py  (2004-11-30)
       http://python.org/sf/1076490  opened by  Neil Mayhew

BuildApplication includes many unneeded modules  (2004-11-30)
       http://python.org/sf/1076492  opened by  Neil Mayhew

python24.msi  install error  (2004-12-01)
       http://python.org/sf/1076500  opened by  guan zi jing

shutil.move clobbers read-only files.  (2004-12-01)
       http://python.org/sf/1076515  opened by  Jeremy Fincher

test test_codecs failed  (2004-12-01)
       http://python.org/sf/1076790  opened by  Michal &#268;iha&#345;

test test_re produced unexpected output  (2004-12-01)
       http://python.org/sf/1076791  opened by  Michal &#268;iha&#345;

test test_unicodedata failed   (2004-12-01)
       http://python.org/sf/1076795  opened by  Michal &#268;iha&#345;

test test_shutil failed  (2004-12-01)
CLOSED http://python.org/sf/1076811  opened by  Michal &#268;iha&#345;

test_shelve failures  (2004-12-01)
       http://python.org/sf/1076819  opened by  Michal &#268;iha&#345;

Bad IDLE shortcut by 2.4 installer on XP  (2004-12-01)
       http://python.org/sf/1076861  opened by  Jean M. Brouwers

Tutorial corrections  (2004-12-01)
       http://python.org/sf/1076955  opened by  Fred L. Drake, Jr.

Incorrect behaviour of StreamReader.readline leads to crash  (2004-12-01)
       http://python.org/sf/1076985  opened by  Sebastian Hartte

Error building _bsddb extension  (2004-12-01)
       http://python.org/sf/1077040  opened by  Oleg Broytmann

Problem testing python 2.4  (2004-12-01)
       http://python.org/sf/1077103  opened by  Pierre

Negative numbers to os.read() cause segfault  (2004-12-01)
       http://python.org/sf/1077106  opened by  Jp Calderone

Bugs Closed
___________

coercing decimal to int doesn't work between -1 and 1  (2004-11-23)
       http://python.org/sf/1071588  closed by  rhettinger

OS X (Panther) Framework Install causes version mismatch  (2004-11-18)
       http://python.org/sf/1069198  closed by  bcannon

urllib2 authentication redirection error(?)  (2004-11-22)
       http://python.org/sf/1071147  closed by  facundobatista

shelve update fails  on &quot;large&quot; entry  (2002-02-27)
       http://python.org/sf/523425  closed by  facundobatista

new int overflow handling needs docs  (2001-08-23)
       http://python.org/sf/454446  closed by  facundobatista

subprocess.py doc bug in 2.4c1  (2004-11-26)
       http://python.org/sf/1073790  closed by  astrand

2 XML parsing errors  (2004-11-26)
       http://python.org/sf/1073864  closed by  peerjanssen

gzip module cannot write large files  (2003-02-08)
       http://python.org/sf/683061  closed by  jlgijsbers

FeedParser problem on end boundaries w/o newline  (2004-11-24)
       http://python.org/sf/1072623  closed by  bwarsaw

FeedParser problem on end boundaries w/o newline  (2004-11-24)
       http://python.org/sf/1072623  closed by  bwarsaw

Build Bug on Solaris.  (2004-11-30)
       http://python.org/sf/1075934  closed by  whitingjr

Whats New for 2.4 "SafeTemplate" patch.  (2004-11-30)
       http://python.org/sf/1076365  closed by  akuchling

'Demo/embed/' , It crash  (2003-12-04)
       http://python.org/sf/853862  closed by  guanzijing

test test_shutil failed  (2004-12-01)
       http://python.org/sf/1076811  closed by  nijel

New / Reopened RFE
__________________

Extension to optparse: options with facultative value  (2004-11-25)
       http://python.org/sf/1073198  opened by  pollastri

Extension to optparse: options with facultative value  (2004-11-25)
CLOSED http://python.org/sf/1073305  opened by  pollastri

ignore element format character for PyArg_ParseTuple  (2004-11-30)
       http://python.org/sf/1075902  opened by  Sean Proctor

Python Interpreter embedded in small memory system  (2004-12-01)
       http://python.org/sf/1076478  opened by  stoneabcwujx

RFE Closed
__________

Extension to optparse: options with facultative value  (2004-11-25)
       http://python.org/sf/1073305  closed by  nnorwitz


From phd at phd.pp.ru  Wed Dec  1 21:47:21 2004
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Thu Dec  2 11:46:39 2004
Subject: [Python-Dev] Python 2.4 README
Message-ID: <20041201204721.GA6993@phd.pp.ru>

Hello. Python 2.4 README still thinks it is 2.3.

   README lines 230-231:

"A number of features are not supported in Python 2.3 anymore. Some
support code is still present, but will be removed in Python 2.4."

   "Will be"? Isn't it 2.4 already? :)

Line 322: "Tcl to support it. To compile Python2.3 with Tkinter, you will"

   Python2.4, I suppose?

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd@phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.
From bjoti at home.se  Wed Dec  1 23:19:22 2004
From: bjoti at home.se (Bjorn Tillenius)
Date: Thu Dec  2 11:46:40 2004
Subject: [Python-Dev] Unicode in doctests
Message-ID: <20041201221922.GA26263@home.se>

There are some issues regarding the use of unicode in doctests. Consider
the following three tests.

    >>> foo = u'f??'
    >>> foo
    u'f\xf6\xf6'

    >>> foo
    u'f\u00f6\u00f6'

    >>> foo
    u'f??'

To me, those are identical. At least string comparison shows that
u'f\xf6\xf6' == u'f\u00f6\u00f6' == u'f??'. Yet, only the first one of
the tests passes, the other two fail. And that's if the tests are within
a doc string, where I can specify the encoding used. If DocFileSuite is
being used, there's no way of specify the encoding, thus all tests will
fail.

Is it supposed to be like this, or have I missed something? If I could
specify the encoding for DocFileSuite to use, I would at least be
partially happy.

Regards,

Bjorn
From niemeyer at conectiva.com  Thu Dec  2 17:31:06 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Thu Dec  2 17:31:13 2004
Subject: [Python-Dev] SRE bug and notifications
Message-ID: <20041202163106.GA16046@burma.localdomain>

I'm really ashamed. The SRE bug #1072259, reported in 2004-11-24,
and fixed a few minutes ago, got into 2.4 final. The only reason
it wasn't fixed on time for 2.4 is because I wasn't aware about
it. :-(

Is there any way to get notified about certain kinds of bugs
in SourceForge? Or, is there any way to follow all new bugs
posted? Or even, what's the easiest way to avoid that problem
by being notified of bugs as soon as possible?

Thank you, and I'm sorry.

-- 
Gustavo Niemeyer
http://niemeyer.net
From amk at amk.ca  Thu Dec  2 17:46:07 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Thu Dec  2 17:46:12 2004
Subject: [Python-Dev] SRE bug and notifications
In-Reply-To: <20041202163106.GA16046@burma.localdomain>
References: <20041202163106.GA16046@burma.localdomain>
Message-ID: <20041202164607.GA8027@rogue.amk.ca>

On Thu, Dec 02, 2004 at 02:31:06PM -0200, Gustavo Niemeyer wrote:
> in SourceForge? Or, is there any way to follow all new bugs
> posted? Or even, what's the easiest way to avoid that problem
> by being notified of bugs as soon as possible?

Kurt's weekly bug summaries list all new bugs.  For a sample, see
http://mail.python.org/pipermail/python-list/2004-December/252968.html ; they get 
posted to python-dev, too.

Perhaps the summaries should include an "unassigned bugs" list to nag
us to look at bugs and assign them to the right person.

--amk

From niemeyer at conectiva.com  Thu Dec  2 17:48:54 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Thu Dec  2 17:49:00 2004
Subject: [Python-Dev] Re: SRE bug and notifications
In-Reply-To: <20041202163106.GA16046@burma.localdomain>
References: <20041202163106.GA16046@burma.localdomain>
Message-ID: <20041202164854.GA16821@burma.localdomain>

> Is there any way to get notified about certain kinds of bugs
> in SourceForge? Or, is there any way to follow all new bugs
> posted? Or even, what's the easiest way to avoid that problem
> by being notified of bugs as soon as possible?

After some googling I've found python-bugs-list. Mentioning it
in http://www.python.org/community/lists.html might be a good
idea.

Additional suggestions still accepted.

Thanks!

-- 
Gustavo Niemeyer
http://niemeyer.net
From tim.peters at gmail.com  Thu Dec  2 18:02:51 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Thu Dec  2 18:02:53 2004
Subject: [Python-Dev] SRE bug and notifications
In-Reply-To: <20041202163106.GA16046@burma.localdomain>
References: <20041202163106.GA16046@burma.localdomain>
Message-ID: <1f7befae041202090237b3d159@mail.gmail.com>

FYI, I just changed SF so that bugs with Category "Regular
Expressions" are auto-assigned to Gustavo (they were being
auto-assigned to Fredrik, which doesn't appear to do much good
anymore).

[Gustavo Niemeyer]
> I'm really ashamed. The SRE bug #1072259, reported in
> 2004-11-24, and fixed a few minutes ago, got into 2.4 final. The
> only reason it wasn't fixed on time for 2.4 is because I wasn't
> aware about it. :-(

Unfortunately, that particular bug was in a wrong Category, so the new
auto-assign wouldn't have helped here anyway.
From python at rcn.com  Fri Dec  3 00:17:25 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Dec  3 00:20:05 2004
Subject: [Python-Dev] [Python-checkins] python/dist/src/Doc/lib
	liblogging.tex, 1.33, 1.34
Message-ID: <006301c4d8c5$15972380$e841fea9@oemcomputer>

Reminder:  The head is now for Py2.5.
Please also do a checkin for release24-maint.

Raymond

From noamr at myrealbox.com  Fri Dec  3 01:55:28 2004
From: noamr at myrealbox.com (Noam Raphael)
Date: Fri Dec  3 01:55:16 2004
Subject: [Python-Dev] An import hook which does nothing
Message-ID: <41AFB980.3050705@myrealbox.com>

Hello,

I'm currently writing an import hook which will cache information on the 
local disk, instead of retrieving it every time from the slow NFS where 
I work.
To make sure that I understand how import hooks work, and as a starting 
point, I wrote a dummy import hook, which is supposed to behave like the 
python's built-in import mechanism.
I post it here, for two reasons:
1. So that developers who understand the import mechanism better than I 
do may review it and find things which I didn't do exactly right.
2. Because I think other people might find it helpful when writing new 
import hooks, as a starting point as well as for better understanding -- 
there's nothing like a working example to help you settle up on what 
does which where. (Although perhaps a few more comments, in addition to 
those which I wrote, might help...)

(Note: I wrote the "DEBUG" parts in order to make sure that my importer 
works, because if it fails things might be done by the built-in importer 
and I won't notice. These parts can be removed, of course.)

Do you think that it might be useful? Maybe something like that can go 
into the "examples" section of the imp module?

Thanks,
Noam Raphael
-------------- next part --------------
A non-text attachment was scrubbed...
Name: importer.py
Type: text/x-python
Size: 1910 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20041203/378063b9/importer.py
From sxanth at ceid.upatras.gr  Fri Dec  3 10:54:25 2004
From: sxanth at ceid.upatras.gr (Stelios Xanthakis)
Date: Fri Dec  3 10:54:31 2004
Subject: [Python-Dev] PEP: __source__ proposal
Message-ID: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr>


Hi all.

Now that 2.4 is out and everything maybe it's
about time to start discussing the "use the
__source__ Luke" feature which IMO will really
boost python into a new domain of exciting
possibilities.

I've prepared a pre-PEP which is not very good
but it is a base.

In short, the feature is good and it enables
editing of python code at runtime instead of
the runfile-exit-edit-run-exit-edit-run cycle.

We have the following possibilities as to whether
__source__ data is marshalled and the feature is
always enabled.

[1] Command line switch and not marshalled
[2] Always on and not marshalled
[3] Always on and marshalled

There is also [4] which doesn't make much sense.

If I was BDFL I'd go for [1] so whoever wants it
can enable it and whoever doesn't can't complain,
and they'll all leave me alone.
Phillip J. Eby expressed some concerns that the
modules that depend on __source__ will eventually
take over and it will become a standard.

Anyway, the PEP is attached.
You can mail me with votes on the feature and if you
want on your preferred option from 1,2,3.
If I get votes I'll post the results later.

If this is accepted I'll try to come up with a good
patch vs 2.4


Thanks,

St.


-------------------ATTACHED PYTHON ENHANCEMENT PROPOSAL---
PEP: XXX
Title: The __source__ attribute
Version: $Revision: 1.10 $
Last-Modified: $Date: 2003/09/22 04:51:49 $
Author: Stelios Xanthakis
Status: Draft
Type: Standards Track
Content-Type: text/plain
Created: 19-Nov-2004
Python-Version: 2.4.1
Post-History:


Abstract

     This PEP suggests the implementation of __source__ attribute for
     functions and classes.  The attribute is a read-only string which
     is generated by the parser and is a copy of the original source
     code of the function/class (including comments, indentation and
     whitespace).


Motivation

     It is generally a tempting idea to use python as an interface to
     a program.  The developers can implement all the functionality
     and instead of designing a user interface, provide a python
     interpreter to their users.  Take for example one of the existing
     web browsers: they have everything that would be needed to write
     a script which downloads pages automatically or premutates the
     letters of web pages before they are displayed, but it is not
     possible for the user to do these things because the interface
     of these applications is static.

     A much more powerful approach would be an interface which is
     dynamically constructed by the user to meet the user's needs.
     The most common development cycle of python programs is:
     write .py file - execute .py file - exit - enhance .py file -
     execute .py file - etc.  With the implementation of the __source__
     attribute though the development/modification of python code
     can happen at run-time.  Functions and classes can be defined,
     modified or enhanced while the python shell is running and
     all the changes can be saved by saving the __source__ attribute
     of globals before termination.  Moreover, in such a system
     it is possible to modify the "code modification routines" and
     eventually we have a self-modifying interface.  Using a
     program also means improving its usability.

     The current solution of using 'inspect' to get the source
     code of functions is not adequate because it doesn't work
     for code defined with "exec" and it doesn't have the source
     of functions/classes defined in the interactive mode.  Generally,
     a "file" is something too abstract.  What is more real is the
     data received by the python parser and that is what is stored
     in __source__.


Specification

     The __source__ attribute is a read-only attribute of functions
     and classes.  Its type is string or None.  In the case of None
     it means that the source was not available.

     The indentation of the code block is the original identation
     obeying nested definitions.  For example:

         >>> class A:
         ...     def foo (self):
         ...         print """Santa-Clauss
         ... is coming to town"""
         >>> def spam ():
         ...     def closure ():
         ...         pass
         ...     return closure
         >>> print A.foo.__source__
             def foo (self):
                 print """Santa-Clauss
         is coming to town"""
         >>> print spam().__source__
             def closure ():
                 pass

     The attribute is not marshaled and therefore not stored in
     ".pyc" files.  As a consequence, functions and classes of
     imported modules have __source__==None.

     We propose that the generation of __source__ will be
     controlled by a command line option.  In the case this
     feature is not activated by the command line option, the
     attribute is absent.


Rationale

     Generally, "import" refers to modules that either have a file in
     a standard location or they are distributed in ".pyc" form only.
     Therefore in the case of modules, getting the source with
     "inspect" is adequate.  Moreover, it does not make sense saving
     __source__ in ".pyc" because the point would be to save
     modifications in the original ".py" file (if available).

     On the issue of the command-line option controlling the generation
     of __source__, please refer to the section about the overhead
     of this feature.  The rationale is that those applications that
     do not wish to use this feature can avoid it (cgi scripts in
     python benchmarked against another language).


Overhead

     The python's parser is not exactly well-suited for such a feature.
     Execution of python code goes through the stages of lexical
     analysis, tokenization, generation of AST and execution of
     bytecode.  In order to implement __source__, the tokenizer has
     to be modified to store the lines of the current translation
     unit.  Those lines are then attached the root node of the
     AST.  While the AST is compiled we have to keep a reference
     of the current node in order to be able to find the next node
     after the node for which we wish to generate __source__, get
     the first and the last line of our block and then refer to
     the root node to extract these lines and make a string.  All
     these actions add a minor overhead to some heavily optimized
     parts of python.  However, once compilation to bytecode is
     done, this feature no longer affects the performance of the
     execution of the bytecode.

     There is also the issue of the memory spent to store __source__.
     In our opinion, this is worth the tradeoff for those who
     are willing to take advantage of it.


Implementation

     There is a sample implementation at [2] which consists of a
     patch against python 2.3.4.  The patch has to be improved
     to avoid generating __source__ for the case we are importing
     modules for the first time (not from .pyc).  In the sample
     implementation there is also included a sample shell that
     takes advantage of __source__ and demonstrates some aspects
     that motivated us towards patching python and submitting this
     PEP.


References

     [1] PEP 1, PEP Purpose and Guidelines, Warsaw, Hylton
         http://www.python.org/peps/pep-0001.html

     [2] Sample implementation
         http://students.ceid.upatras.gr/~sxanth/ISYSTEM/python-PIESS.tar.gz


Copyright

     This document has been placed in the public domain.



Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
End:
From hpk at trillke.net  Fri Dec  3 13:36:16 2004
From: hpk at trillke.net (holger krekel)
Date: Fri Dec  3 13:36:20 2004
Subject: [Python-Dev] PEP: __source__ proposal
In-Reply-To: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr>
References: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr>
Message-ID: <20041203123616.GB4708@solar.trillke.net>

Hi Stelios, 

[Stelios Xanthakis Fri, Dec 03, 2004 at 11:54:25AM +0200]
> Abstract
> 
>     This PEP suggests the implementation of __source__ attribute for
>     functions and classes.  The attribute is a read-only string which
>     is generated by the parser and is a copy of the original source
>     code of the function/class (including comments, indentation and
>     whitespace).

I've had similar ideas in the past as we are doing dynamic
code generation in PyPy, as well as in other projects.  After
some discussion with Armin i think there is another
possibility for storing "source" or any other such meta information
with code/module objects: make __file__ and co_filename
instances of a subclass of 'str' providing an extra attribute.
For a simple example, they could have a 'source' attribute, which 
could be tried first by appropriate inspect functions and traceback 
related functionality.  

We are about to test out this approach with the py lib 
(http://codespeak.net/py) and want to have it work for 
for Python 2.2, 2.3. and 2.4.  I suspect there may be some 
issues lurking (also in your proposed PEP) especially with 
respect to encodings.  Also we have some use cases where we 
want to retrieve source code from non-local locations and
want to integrate this seemlessly with the introspection 
facilities of Python which obviously is an important part 
of the equation.  I can report back if there is interest. 

cheers, 

    holger
From fredrik at pythonware.com  Fri Dec  3 14:38:04 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri Dec  3 14:38:13 2004
Subject: [Python-Dev] Re: Unicode in doctests
References: <20041201221922.GA26263@home.se>
Message-ID: <copq7t$4d4$1@sea.gmane.org>

Bjorn Tillenius wrote:
> There are some issues regarding the use of unicode in doctests. Consider
> the following three tests.
>
>    >>> foo = u'föö'
>    >>> foo
>    u'f\xf6\xf6'
>
>    >>> foo
>    u'f\u00f6\u00f6'
>
>    >>> foo
>    u'föö'
>
> To me, those are identical.

really?  if a function is expected to print "öl", is it alright to print
"\u00f6l" instead?  wouldn't you complain if your newspaper used
Unicode escapes in headlines instead of Swedish characters?

> Is it supposed to be like this, or have I missed something? If I could
> specify the encoding for DocFileSuite to use, I would at least be
> partially happy.

repr() always generates the same output, no matter what encoding
you use.  just use repr, and you're done.

</F> 



From mal at egenix.com  Fri Dec  3 15:59:43 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri Dec  3 15:59:48 2004
Subject: [Python-Dev] Removing --with-wctype-functions support
Message-ID: <41B07F5F.9040207@egenix.com>

I would like to remove the support for using libc wctype functions
(e.g. towupper(), towlower(), etc.) from the code base.

The reason is that compiling Python using this switch not only
breaks the test suite, it also causes the functions .lower() and
.upper() to become locale aware and creates wrong results as
a result (which are hard to track down if you don't know whether
Python was compiled with the wctype switch or not).

The argument for having this switch at the time was to reduce
the interpreter size. This was true for Python 1.6 since the
Unicode type database was conditionally removed. Starting with
Python 2.0 a different approach was taken which resulted in
having to keep the type database in one piece - even with the
switch enabled.

As a result, performance became the only argument. However,
because the libc functions are locale aware, the added overhead
causes the libc functions not to perform better than the
builtin Python versions.

In the end, we loose complexity and reduce the Python configuration
space by one dimension.

Here's the bug that revealed the problem:

https://sourceforge.net/tracker/index.php?func=detail&aid=1076790&group_id=5470&atid=105470

Question is: Should this be done in 2.4.1 or will it have to
wait until 2.5 ?

We might want to remove the support for 2.4.1 and simply ignore
the compile time switch (or print a warning).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 03 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mal at egenix.com  Fri Dec  3 16:08:54 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri Dec  3 16:08:58 2004
Subject: [Python-Dev] MS VC compiler versions
In-Reply-To: <41AE1367.5000805@activestate.com>
References: <41ADDED2.30707@egenix.com> <41AE1367.5000805@activestate.com>
Message-ID: <41B08186.5070600@egenix.com>

Here's an update of the list I currently have:

* MS VS6 SP5 (International version, optimizing VC6 compiler with SP5):

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8168 for 80x86
Copyright (C) Microsoft Corp 1984-1998. All rights reserved.

* MS VC6 (German version; optimizing VC6 compiler with SP6):

Optimierender Microsoft (R) 32-Bit C/C++-Compiler, Version 12.00.8804, fuer x86
Copyright (C) Microsoft Corp 1984-1998. Alle Rechte vorbehalten.

* MS VC6 (US version; optimizing VC6 compiler with SP6):

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8804 for 80x86
Copyright (C) Microsoft Corp 1984-1998. All rights reserved.

* MS .NET SDK 1.1 Compiler (German version; non-optimizing VC7.1 compiler):

Microsoft (R) 32-Bit C/C++-Standardcompiler Version 13.10.3077 f?r 80x86
Copyright (C) Microsoft Corporation 1984-2002. Alle Rechte vorbehalten.

* MS VC7.1 (aka .NET 2003, German version; optimizing VC7.1 compiler)

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86
Copyright (C) Microsoft Corporation 1984-2002. All rights reserved.

* MS Windows XP DDK (International version, optimizing VC 7.0):

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.00.9176 for 80x86
Copyright (C) Microsoft Corporation 1984-2001. All rights reserved.


Does someone have the details for the MS Toolkit compiler ?

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 03 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From Scott.Daniels at Acm.Org  Fri Dec  3 17:07:25 2004
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Fri Dec  3 17:06:06 2004
Subject: [Python-Dev] Re: MS VC compiler versions
In-Reply-To: <41B08186.5070600@egenix.com>
References: <41ADDED2.30707@egenix.com> <41AE1367.5000805@activestate.com>
	<41B08186.5070600@egenix.com>
Message-ID: <coq2t8$ak$1@sea.gmane.org>

M.-A. Lemburg wrote:
> Here's an update of the list I currently have:
> 
> * MS VS6 SP5 (International version, optimizing VC6 compiler with SP5):
> 
> Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8168 for 80x86
> Copyright (C) Microsoft Corp 1984-1998. All rights reserved.
> 
> * MS VC6 (German version; optimizing VC6 compiler with SP6):
> 
> Optimierender Microsoft (R) 32-Bit C/C++-Compiler, Version 12.00.8804, 
> fuer x86
> Copyright (C) Microsoft Corp 1984-1998. Alle Rechte vorbehalten.
> 
> * MS VC6 (US version; optimizing VC6 compiler with SP6):
> 
> Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8804 for 80x86
> Copyright (C) Microsoft Corp 1984-1998. All rights reserved.
> 
> * MS .NET SDK 1.1 Compiler (German version; non-optimizing VC7.1 compiler):
> 
> Microsoft (R) 32-Bit C/C++-Standardcompiler Version 13.10.3077 f?r 80x86
> Copyright (C) Microsoft Corporation 1984-2002. Alle Rechte vorbehalten.
> 
> * MS VC7.1 (aka .NET 2003, German version; optimizing VC7.1 compiler)
> 
> Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86
> Copyright (C) Microsoft Corporation 1984-2002. All rights reserved.
> 
> * MS Windows XP DDK (International version, optimizing VC 7.0):
> 
> Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.00.9176 for 80x86
> Copyright (C) Microsoft Corporation 1984-2001. All rights reserved.
> 
> 
> Does someone have the details for the MS Toolkit compiler ?
> 
> Thanks,
\Program Files\Microsoft Visual C++ Toolkit 2003\bin\cl.exe:

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86
Copyright (C) Microsoft Corporation 1984-2002. All rights reserved.

-- 
-- Scott David Daniels
Scott.Daniels@Acm.Org

From bjoti at yahoo.se  Fri Dec  3 18:02:52 2004
From: bjoti at yahoo.se (Bjorn Tillenius)
Date: Fri Dec  3 18:20:43 2004
Subject: [Python-Dev] Re: Unicode in doctests
References: <20041201221922.GA26263@home.se> <copq7t$4d4$1@sea.gmane.org>
Message-ID: <loom.20041203T173955-552@post.gmane.org>

Fredrik Lundh <fredrik <at> pythonware.com> writes:
> Bjorn Tillenius wrote:
> > There are some issues regarding the use of unicode in doctests. Consider
> > the following three tests.
> >
> >    >>> foo = u'f??'
> >    >>> foo
> >    u'f\xf6\xf6'
> >
> >    >>> foo
> >    u'f\u00f6\u00f6'
> >
> >    >>> foo
> >    u'f??'
> >
> > To me, those are identical.
> 
> really?  if a function is expected to print "?l", is it alright to print
> "\u00f6l" instead?  wouldn't you complain if your newspaper used
> Unicode escapes in headlines instead of Swedish characters?

No, I wouldn't like the newspaper to use Unicode escapes. For the same
reason, I don't want my documentation to contain Unicode escapes. That's
why I would like the latter test to pass.

But I understand, it tries to match the output of repr(foo), I guess I can
live with that. I can always do:

    >>> foo == u'f??'
    True

On the other hand, since there already are some flags to modify the matching
algorithm, one could argue for adding another flag... or at least provide
the possibility for the user to alter the matching himself. Although it's
not that important for me.

> > Is it supposed to be like this, or have I missed something? If I could
> > specify the encoding for DocFileSuite to use, I would at least be
> > partially happy.
> 
> repr() always generates the same output, no matter what encoding
> you use.  just use repr, and you're done.

What is important for me, though, is to be able to specify an encoding to
DocFileSuite. As you said, one doesn't want to read Unicode escapes. At the
moment none of the tests I've given as example will pass in a DocFileSuite
(given that the text file is encoded using UTF-8). I do find it a bit
strange that I can't just copy a doctest within a docstring to a text file.
I have to Unicode escape all non-ASCII characters, which produces ugly
documentation.

Regards,

Bjorn

From kbk at shore.net  Fri Dec  3 18:40:42 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Fri Dec  3 18:42:14 2004
Subject: [Python-Dev] SRE bug and notifications
In-Reply-To: <20041202164607.GA8027@rogue.amk.ca> (A. M. Kuchling's message
	of "Thu, 2 Dec 2004 11:46:07 -0500")
References: <20041202163106.GA16046@burma.localdomain>
	<20041202164607.GA8027@rogue.amk.ca>
Message-ID: <87653jxoth.fsf@hydra.localdomain>

"A.M. Kuchling" <amk@amk.ca> writes:

> Perhaps the summaries should include an "unassigned bugs" list to nag
> us to look at bugs and assign them to the right person.

The bug report is derived from the bug and patch email lists, so that 
information isn't available without scraping it off SF.

However, the SF trackers can be set to query for Unassigned / Open.
It looks like about 2/3 of the open bugs and 3/4 of the open patches
are unassigned.  I often unassign IDLE bugs that I'm not planning to
fix right now, hoping that may encourage someone to step forward by
removing the illusion that something is happening.  Conversely, if I
make progress on an unassigned bug/patch, I'll assign it to myself as
an indication that it's being worked on, avoiding duplication of
effort.

I track IDLE bugs via the IDLE category.  That's very useful, maybe we
need more categories?

-- 
KBK
From sxanth at ceid.upatras.gr  Fri Dec  3 22:59:30 2004
From: sxanth at ceid.upatras.gr (Stelios Xanthakis)
Date: Fri Dec  3 22:59:40 2004
Subject: [Python-Dev] PEP: __source__ proposal
In-Reply-To: <20041203123616.GB4708@solar.trillke.net>
References: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr>
	<20041203123616.GB4708@solar.trillke.net>
Message-ID: <Pine.GSO.4.61.0412031554001.5550@zenon.ceid.upatras.gr>



On Fri, 3 Dec 2004, holger krekel wrote:

> ...
> there is another possibility for storing "source" or any other such meta 
> information with code/module objects: make __file__ and co_filename
> instances of a subclass of 'str' providing an extra attribute.
> For a simple example, they could have a 'source' attribute, which
> could be tried first by appropriate inspect functions and traceback
> related functionality.

Attaching such info on 'code objects' is indeed a more general
case. But, OTOH, AFAIK, a class is not a code object.  At least
by what I was able to figure out from python sources.

It seems reasonable to make 'source' a dynamic object
which will get its info from file/line if available.
Now the thing is that if we had __source__ from the start,
'inspect' would have been much different.  So the fact that
we have some functionality with inspect does not mean that
it's good enough. Probably inspect will be rewritten/improved
if __source__ is implemented.

> We are about to test out this approach with the py lib
> (http://codespeak.net/py) and want to have it work for
> for Python 2.2, 2.3. and 2.4.

Do you plan hacking python ?
It appears that tok_nextc() is the best place to
catch all the source passed to the interpreter.

A patch would be interesting.


Stelios
From martin at v.loewis.de  Sat Dec  4 00:26:33 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Dec  4 00:26:32 2004
Subject: [Python-Dev] PEP: __source__ proposal
In-Reply-To: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr>
References: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr>
Message-ID: <41B0F629.4040100@v.loewis.de>

Stelios Xanthakis wrote:
> Now that 2.4 is out and everything maybe it's
> about time to start discussing the "use the
> __source__ Luke" feature which IMO will really
> boost python into a new domain of exciting
> possibilities.

I'm opposed to this idea. It creates overhead in
the size of .pyc files, for no additional value
that couldn't be obtained otherwise.

As the rationale, the PEP lists:

1.
>     It is generally a tempting idea to use python as an interface to
>     a program.

I cannot see how this rationale is related to the PEP. You
can use Python as interface to a program with or without
__source__.

2.
>     The developers can implement all the functionality
>     and instead of designing a user interface, provide a python
>     interpreter to their users.

This does not require __source, either.

3.
>     A much more powerful approach would be an interface which is
>     dynamically constructed by the user to meet the user's needs.

Dynamic code generation doesn't require __source__, either.

4.
>     The most common development cycle of python programs is:
>     write .py file - execute .py file - exit - enhance .py file -
>     execute .py file - etc.  With the implementation of the __source__
>     attribute though the development/modification of python code
>     can happen at run-time.

This works just fine as well at the moment; see IDLE for an
example.

>     Functions and classes can be defined,
>     modified or enhanced while the python shell is running and
>     all the changes can be saved by saving the __source__ attribute
>     of globals before termination.

Currently, you can define classes dynamically, and you can also save
the source code of the class to a file in case you need it later.

 >     Moreover, in such a system
>     it is possible to modify the "code modification routines" and
>     eventually we have a self-modifying interface.  Using a
>     program also means improving its usability.

Self-modifying source code is currently also possible. Just read
the old source code from a .py file, modify it, and recompile it.

>     The current solution of using 'inspect' to get the source
>     code of functions is not adequate because it doesn't work
>     for code defined with "exec" and it doesn't have the source
>     of functions/classes defined in the interactive mode.

I fail to see why it isn't adequate. Anybody who wants to modify
source code that was originally passed to exec just needs to
preserve a copy of the source code, separately.

 >     Generally,
>     a "file" is something too abstract.  What is more real is the
>     data received by the python parser and that is what is stored
>     in __source__.

Not at all. A file is precisely the level of granularity that is
burnt into the Python language. A module is *always* a file, executed
from top to bottom. It is not possible to recreate the source code
of a module if you have only the source code of all functions, and
all classes.

Regards,
Martin
From edcjones at erols.com  Fri Dec  3 03:24:45 2004
From: edcjones at erols.com (Edward C. Jones)
Date: Sat Dec  4 03:23:03 2004
Subject: [Python-Dev] test_shutils.py fails for 2.4 install
Message-ID: <41AFCE6D.10300@erols.com>

I have a PC with an AMD cpu and Mandrake 10.1. While installing Python 
2.4 "make test" failed in "test_shutils.py". Here is the output of 
"./python ./Lib/test/test_shutil.py":

test_dont_copy_file_onto_link_to_itself (__main__.TestShutil) ... ok
test_dont_move_dir_in_itself (__main__.TestShutil) ... ok
test_on_error (__main__.TestShutil) ... FAIL
test_rmtree_dont_delete_file (__main__.TestShutil) ... ok
test_rmtree_errors (__main__.TestShutil) ... ok

======================================================================
FAIL: test_on_error (__main__.TestShutil)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./Lib/test/test_shutil.py", line 34, in test_on_error
    self.assertEqual(self.errorState, 2)
AssertionError: 0 != 2

----------------------------------------------------------------------
Ran 5 tests in 0.005s

FAILED (failures=1)
Traceback (most recent call last):
  File "./Lib/test/test_shutil.py", line 106, in ?
    test_main()
  File "./Lib/test/test_shutil.py", line 103, in test_main
    test_support.run_unittest(TestShutil)
  File "/usr/local/src/Python-2.4/Lib/test/test_support.py", line 290, 
in run_u
    run_suite(suite, testclass)
  File "/usr/local/src/Python-2.4/Lib/test/test_support.py", line 275, 
in run_s
    raise TestFailed(err)
test.test_support.TestFailed: Traceback (most recent call last):
  File "./Lib/test/test_shutil.py", line 34, in test_on_error
    self.assertEqual(self.errorState, 2)
AssertionError: 0 != 2

From michael.walter at gmail.com  Sat Dec  4 06:19:20 2004
From: michael.walter at gmail.com (Michael Walter)
Date: Sat Dec  4 06:19:23 2004
Subject: [Python-Dev] PEP: __source__ proposal
In-Reply-To: <877e9a17041203211849479ab5@mail.gmail.com>
References: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr>
	<877e9a17041203211849479ab5@mail.gmail.com>
Message-ID: <877e9a170412032119589f324e@mail.gmail.com>

Hi!

I think you were the omitting the more interesting closure examples
(namely with free variables inside the closure's source):

def foo(): pass
def bar(x):
  def fiep(): return x()
  return fiep

what's bar(foo).__source__?

Generally, I'm opposed to the feature -- I don't think it provides a
real advantage giving it's limitations (doesn't work for import'ed
modules, classes + methods != module, etc.).

Cheers,
Michael




On Fri, 3 Dec 2004 11:54:25 +0200 (EET), Stelios Xanthakis
<sxanth@ceid.upatras.gr> wrote:
>
> Hi all.
>
> Now that 2.4 is out and everything maybe it's
> about time to start discussing the "use the
> __source__ Luke" feature which IMO will really
> boost python into a new domain of exciting
> possibilities.
>
> I've prepared a pre-PEP which is not very good
> but it is a base.
>
> In short, the feature is good and it enables
> editing of python code at runtime instead of
> the runfile-exit-edit-run-exit-edit-run cycle.
>
> We have the following possibilities as to whether
> __source__ data is marshalled and the feature is
> always enabled.
>
> [1] Command line switch and not marshalled
> [2] Always on and not marshalled
> [3] Always on and marshalled
>
> There is also [4] which doesn't make much sense.
>
> If I was BDFL I'd go for [1] so whoever wants it
> can enable it and whoever doesn't can't complain,
> and they'll all leave me alone.
> Phillip J. Eby expressed some concerns that the
> modules that depend on __source__ will eventually
> take over and it will become a standard.
>
> Anyway, the PEP is attached.
> You can mail me with votes on the feature and if you
> want on your preferred option from 1,2,3.
> If I get votes I'll post the results later.
>
> If this is accepted I'll try to come up with a good
> patch vs 2.4
>
> Thanks,
>
> St.
>
> -------------------ATTACHED PYTHON ENHANCEMENT PROPOSAL---
> PEP: XXX
> Title: The __source__ attribute
> Version: $Revision: 1.10 $
> Last-Modified: $Date: 2003/09/22 04:51:49 $
> Author: Stelios Xanthakis
> Status: Draft
> Type: Standards Track
> Content-Type: text/plain
> Created: 19-Nov-2004
> Python-Version: 2.4.1
> Post-History:
>
> Abstract
>
>      This PEP suggests the implementation of __source__ attribute for
>      functions and classes.  The attribute is a read-only string which
>      is generated by the parser and is a copy of the original source
>      code of the function/class (including comments, indentation and
>      whitespace).
>
> Motivation
>
>      It is generally a tempting idea to use python as an interface to
>      a program.  The developers can implement all the functionality
>      and instead of designing a user interface, provide a python
>      interpreter to their users.  Take for example one of the existing
>      web browsers: they have everything that would be needed to write
>      a script which downloads pages automatically or premutates the
>      letters of web pages before they are displayed, but it is not
>      possible for the user to do these things because the interface
>      of these applications is static.
>
>      A much more powerful approach would be an interface which is
>      dynamically constructed by the user to meet the user's needs.
>      The most common development cycle of python programs is:
>      write .py file - execute .py file - exit - enhance .py file -
>      execute .py file - etc.  With the implementation of the __source__
>      attribute though the development/modification of python code
>      can happen at run-time.  Functions and classes can be defined,
>      modified or enhanced while the python shell is running and
>      all the changes can be saved by saving the __source__ attribute
>      of globals before termination.  Moreover, in such a system
>      it is possible to modify the "code modification routines" and
>      eventually we have a self-modifying interface.  Using a
>      program also means improving its usability.
>
>      The current solution of using 'inspect' to get the source
>      code of functions is not adequate because it doesn't work
>      for code defined with "exec" and it doesn't have the source
>      of functions/classes defined in the interactive mode.  Generally,
>      a "file" is something too abstract.  What is more real is the
>      data received by the python parser and that is what is stored
>      in __source__.
>
> Specification
>
>      The __source__ attribute is a read-only attribute of functions
>      and classes.  Its type is string or None.  In the case of None
>      it means that the source was not available.
>
>      The indentation of the code block is the original identation
>      obeying nested definitions.  For example:
>
>          >>> class A:
>          ...     def foo (self):
>          ...         print """Santa-Clauss
>          ... is coming to town"""
>          >>> def spam ():
>          ...     def closure ():
>          ...         pass
>          ...     return closure
>          >>> print A.foo.__source__
>              def foo (self):
>                  print """Santa-Clauss
>          is coming to town"""
>          >>> print spam().__source__
>              def closure ():
>                  pass
>
>      The attribute is not marshaled and therefore not stored in
>      ".pyc" files.  As a consequence, functions and classes of
>      imported modules have __source__==None.
>
>      We propose that the generation of __source__ will be
>      controlled by a command line option.  In the case this
>      feature is not activated by the command line option, the
>      attribute is absent.
>
> Rationale
>
>      Generally, "import" refers to modules that either have a file in
>      a standard location or they are distributed in ".pyc" form only.
>      Therefore in the case of modules, getting the source with
>      "inspect" is adequate.  Moreover, it does not make sense saving
>      __source__ in ".pyc" because the point would be to save
>      modifications in the original ".py" file (if available).
>
>      On the issue of the command-line option controlling the generation
>      of __source__, please refer to the section about the overhead
>      of this feature.  The rationale is that those applications that
>      do not wish to use this feature can avoid it (cgi scripts in
>      python benchmarked against another language).
>
> Overhead
>
>      The python's parser is not exactly well-suited for such a feature.
>      Execution of python code goes through the stages of lexical
>      analysis, tokenization, generation of AST and execution of
>      bytecode.  In order to implement __source__, the tokenizer has
>      to be modified to store the lines of the current translation
>      unit.  Those lines are then attached the root node of the
>      AST.  While the AST is compiled we have to keep a reference
>      of the current node in order to be able to find the next node
>      after the node for which we wish to generate __source__, get
>      the first and the last line of our block and then refer to
>      the root node to extract these lines and make a string.  All
>      these actions add a minor overhead to some heavily optimized
>      parts of python.  However, once compilation to bytecode is
>      done, this feature no longer affects the performance of the
>      execution of the bytecode.
>
>      There is also the issue of the memory spent to store __source__.
>      In our opinion, this is worth the tradeoff for those who
>      are willing to take advantage of it.
>
> Implementation
>
>      There is a sample implementation at [2] which consists of a
>      patch against python 2.3.4.  The patch has to be improved
>      to avoid generating __source__ for the case we are importing
>      modules for the first time (not from .pyc).  In the sample
>      implementation there is also included a sample shell that
>      takes advantage of __source__ and demonstrates some aspects
>      that motivated us towards patching python and submitting this
>      PEP.
>
> References
>
>      [1] PEP 1, PEP Purpose and Guidelines, Warsaw, Hylton
>          http://www.python.org/peps/pep-0001.html
>
>      [2] Sample implementation
>          http://students.ceid.upatras.gr/~sxanth/ISYSTEM/python-PIESS.tar.gz
>
> Copyright
>
>      This document has been placed in the public domain.
>
> Local Variables:
> mode: indented-text
> indent-tabs-mode: nil
> sentence-end-double-space: t
> fill-column: 70
> End:
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/michael.walter%40gmail.com
>
From hpk at trillke.net  Sat Dec  4 09:59:38 2004
From: hpk at trillke.net (holger krekel)
Date: Sat Dec  4 09:59:41 2004
Subject: [Python-Dev] PEP: __source__ proposal
In-Reply-To: <Pine.GSO.4.61.0412031554001.5550@zenon.ceid.upatras.gr>
References: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr>
	<20041203123616.GB4708@solar.trillke.net>
	<Pine.GSO.4.61.0412031554001.5550@zenon.ceid.upatras.gr>
Message-ID: <20041204085937.GC4708@solar.trillke.net>

[Stelios Xanthakis Fri, Dec 03, 2004 at 11:59:30PM +0200]
> On Fri, 3 Dec 2004, holger krekel wrote:
> >We are about to test out this approach with the py lib
> >(http://codespeak.net/py) and want to have it work for
> >for Python 2.2, 2.3. and 2.4.
> 
> Do you plan hacking python ?
> It appears that tok_nextc() is the best place to
> catch all the source passed to the interpreter.

Well, as we want to have the library work on past python 
versions modifying CPython 2.5 does not make much sense. 

It's more about (like Martin pointed out) organizing 
dynamic code generation so that Python's introspect 
and traceback logic works as much as possible - with 
tiny runtime "monkey" patches if needed. 

Now Martin also correctly pointed out that you can store 
source code before/after you pass it to compile/parse.  
We are doing this already with an external dictionary.  
This has multithreading issues, though.  So we think that 
hooking onto code's objects co_filename or a module's __file__ 
might be an interesting idea. 

cheers, 

    holger
From sxanth at ceid.upatras.gr  Sat Dec  4 10:02:26 2004
From: sxanth at ceid.upatras.gr (Stelios Xanthakis)
Date: Sat Dec  4 10:02:30 2004
Subject: [Python-Dev] PEP: __source__ proposal
In-Reply-To: <41B0F629.4040100@v.loewis.de>
References: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr>
	<41B0F629.4040100@v.loewis.de>
Message-ID: <Pine.GSO.4.61.0412041011200.6772@zenon.ceid.upatras.gr>


> I'm opposed to this idea. It creates overhead in
> the size of .pyc files,

No it doesn't.

> for no additional value that couldn't be obtained otherwise.

Martin:

I know it is possible to do all this with existing
python facilities.  I did write such a dynamic code
framework in python.  Specifically I used a function
'deyndef(code)' which was exactly like 'def' but also
stored the source string in a dictionary.

The key point is that I think think should be the
job of the parser and the functionality provided
at the interactive prompt w/o the user having to
write 'dyndef' or store the code of exec's or request
from himself to use specific commands to create
functions. It should be transparent built into python.

> A file is precisely the level of granularity that is
> burnt into the Python language. A module is *always* a file, executed
> from top to bottom. It is not possible to recreate the source code
> of a module if you have only the source code of all functions, and
> all classes.

That's exactly the rationale for NOT combining __source__
with import. It's in the PEP.

It appears that there are the 'module people' who
find this feature irrelevant. Indeed. If we are interested
in distributing modules and increasing the number of
people who use python programs,then  __source__ is redundant.
OTOH, programming python is easy and fun and I think
the proposed feature will make it even more fun and it
aims in increasing the number of people who program
python for their every day tasks. It'd be interesting to
hear if the developers of IDLE/ipython/etc could use this.


Oh well. I guess I'm ahead of my time again:)

St.

From mwh at python.net  Sat Dec  4 12:57:34 2004
From: mwh at python.net (Michael Hudson)
Date: Sat Dec  4 12:57:35 2004
Subject: [Python-Dev] test_shutils.py fails for 2.4 install
In-Reply-To: <41AFCE6D.10300@erols.com> (Edward C. Jones's message of "Thu,
	02 Dec 2004 21:24:45 -0500")
References: <41AFCE6D.10300@erols.com>
Message-ID: <2m7jnys2c1.fsf@starship.python.net>

"Edward C. Jones" <edcjones@erols.com> writes:

> I have a PC with an AMD cpu and Mandrake 10.1. While installing Python
> 2.4 "make test" failed in "test_shutils.py". Here is the output of
> "./python ./Lib/test/test_shutil.py":

Are you running 'make test' as root?  Don't do that (although possibly
the test suite should guard against it).

Also, did you search the bug tracker?  Please do do that.

Cheers,
mwh

-- 
  <glyph> CDATA is not an integration strategy.
                                                -- from Twisted.Quotes
From martin at v.loewis.de  Sat Dec  4 15:10:06 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Dec  4 15:10:03 2004
Subject: [Python-Dev] PEP: __source__ proposal
In-Reply-To: <Pine.GSO.4.61.0412041011200.6772@zenon.ceid.upatras.gr>
References: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr>
	<41B0F629.4040100@v.loewis.de>
	<Pine.GSO.4.61.0412041011200.6772@zenon.ceid.upatras.gr>
Message-ID: <41B1C53E.9050204@v.loewis.de>

Stelios Xanthakis wrote:
> The key point is that I think think should be the
> job of the parser and the functionality provided
> at the interactive prompt w/o the user having to
> write 'dyndef' or store the code of exec's or request
> from himself to use specific commands to create
> functions. It should be transparent built into python.

For the case of the interactive prompt, if preserving
the source code is somehow desirable (which I doubt),
it should be the job of the development environment
to offer saving interactively-defined methods.

Such IDE support is necessary even if __source__ was
available, since users probably would not want to write

open("demo.py").write(myfunc.__source__ + "\n" + myclass.__source)

> OTOH, programming python is easy and fun and I think
> the proposed feature will make it even more fun and it
> aims in increasing the number of people who program
> python for their every day tasks. It'd be interesting to
> hear if the developers of IDLE/ipython/etc could use this.

I think it is the other way 'round. If this is *only* for
interactive mode, than you should *first* change the interactive
environments. If you then find you absolutely need this feature
to implement the IDEs correctly, I'd like to hear the (new)
rationale for the change.

Regards,
Martin
From ncoghlan at iinet.net.au  Sat Dec  4 15:29:27 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sat Dec  4 15:29:34 2004
Subject: [Python-Dev] PEP: __source__ proposal
In-Reply-To: <Pine.GSO.4.61.0412041011200.6772@zenon.ceid.upatras.gr>
References: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr>	<41B0F629.4040100@v.loewis.de>
	<Pine.GSO.4.61.0412041011200.6772@zenon.ceid.upatras.gr>
Message-ID: <41B1C9C7.1020604@iinet.net.au>

[Resend, since a minor brain explosion caused me to send this to c.l.p instead 
of python-dev]

Stelios Xanthakis wrote:
> It appears that there are the 'module people' who
> find this feature irrelevant. Indeed. If we are interested
> in distributing modules and increasing the number of
> people who use python programs,then  __source__ is redundant.
> OTOH, programming python is easy and fun and I think
> the proposed feature will make it even more fun and it
> aims in increasing the number of people who program
> python for their every day tasks. It'd be interesting to
> hear if the developers of IDLE/ipython/etc could use this.

The feedback here (and the initial response on py-dev a while back) suggests to
me that you should look at making this a feature of the interactive mode.
Something that affects both Python's main interactive shell, plus the relevant
class in the standard library (CommandInterpreter or whatever it is called).

A late-night-train-of-thought example of what might be handy is below - keep in
mind that I haven't looked at what enhanced Python shells like IPython can do,
so it may be there are tools out there that do something like this already. It
would be handy to have a standard library module that supported "on-the-fly"
editing, though (this capability would then be available to anyone embedding
Python as a scripting engine).

Cheers,
Nick.
==============================
>>>import source
>>>class bob:
...  def mary():
...    pass
...  def tim():
...    print 'Tim'
...
>>>print bob.__source__
class bob:
   def mary():
     pass
   def tim():
     print 'Tim'

>>>print bob.mary.__source__
def mary():
   pass

>>> source.edit(bob.mary)
bob.mary(1)>def mary(text): # [1]
bob.mary(2)>  print "Mary:", text
bob.mary(3)>\save
>>> source.edit(bob.tim)
bob.tim(1)>\help
Commands: \help \cancel \save \deleteline
bob.tim(2)>\cancel
>>>print bob.__source__
"class bob:
   def mary(text):
     print "Mary:", text
   def tim():
     print 'Tim'
"
>>> bob().mary("Hi!")
Mary: Hi!

The basic ideas of the above:

"import source" triggers the storage of the __source__ attributes (e.g. via
installation of appropriate hooks in the class and function definition process)

The 'edit' function is then able to take advantage of the stored source code to
present each line of the original source for modification (e.g. to fix a minor
bug in one function of a class definition). When the 'edit' is complete, it can
be saved or cancelled.

1. The feature mentioned in the last paragraph is hard to show in the expected
output :)

-- 
Nick Coghlan   |   ncoghlan@email.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.skystorm.net

From tim.peters at gmail.com  Sat Dec  4 17:23:26 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Dec  4 17:23:30 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib whrandom.py,
	1.21, NONE
In-Reply-To: <E1CaXV7-0004ui-Sy@sc8-pr-cvs1.sourceforge.net>
References: <E1CaXV7-0004ui-Sy@sc8-pr-cvs1.sourceforge.net>
Message-ID: <1f7befae04120408235ffc9b8a@mail.gmail.com>

[rhettinger@users.sourceforge.net]
> Removed Files:
>        whrandom.py
> Log Message:
> Remove the deprecated whrandom module.

Woo hoo!  It's about friggin' time <wink>.
From edcjones at erols.com  Fri Dec  3 18:53:03 2004
From: edcjones at erols.com (Edward C. Jones)
Date: Sat Dec  4 18:51:20 2004
Subject: [Python-Dev] RE: test_shutils.py fails for 2.4 install
Message-ID: <41B0A7FF.9040305@erols.com>

The relevant bug appears to be 1076467 at SourceForge.
From raymond.hettinger at verizon.net  Sun Dec  5 03:20:09 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sun Dec  5 03:22:54 2004
Subject: [Python-Dev] Py2.5:  Deprecated Cookie classes and interface
Message-ID: <000901c4da70$f2201e40$ab2fc797@oemcomputer>

Any objections to removing Cookie.Cookie, Cookie.SmartCookie, and
Cookie.SerialCookie?
 
 
Raymond 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20041204/e3c8679b/attachment.html
From raymond.hettinger at verizon.net  Sun Dec  5 03:30:03 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sun Dec  5 03:32:47 2004
Subject: [Python-Dev] Deprecated xmllib module
Message-ID: <001801c4da72$543801a0$ab2fc797@oemcomputer>

Hmph.  The xmllib module has been deprecated since Py2.0 but is not
listed in PEP 4.

Question:  Do we have to keep it for another two years because of that
omission?

It seems somewhat silly to keep an obsolete, supplanted module that
doesn?t full support XML 1.0.


Raymond

From fredrik at pythonware.com  Sun Dec  5 08:40:47 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun Dec  5 08:38:34 2004
Subject: [Python-Dev] Re: Deprecated xmllib module
References: <001801c4da72$543801a0$ab2fc797@oemcomputer>
Message-ID: <coudtk$ef$1@sea.gmane.org>

Raymond Hettinger wrote:

> Hmph.  The xmllib module has been deprecated since Py2.0 but is not
> listed in PEP 4.
>
> Question:  Do we have to keep it for another two years because of that
> omission?
>
> It seems somewhat silly to keep an obsolete, supplanted module that
> doesn't full support XML 1.0.

it's better to be somewhat silly than to be arrogant and stupid.

if python-dev cares about existing users, xmllib should stay in there until 3.0.

</F> 



From bac at OCF.Berkeley.EDU  Sun Dec  5 08:52:14 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Dec  5 08:52:21 2004
Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling?
Message-ID: <41B2BE2E.3000907@ocf.berkeley.edu>

I noticed that Makefile.pre.in uses the value from the environment variable 
LDFLAGS but not CPPFLAGS.  Any reason for this?  ``./configure -h`` lists both 
(and some other environment variables which are not really used either) so it 
seems a little misleading to have those listed but to not utilize them.

The reason I ask is I plan on having setup.py add the directories specified in 
both of these environment variables for compiling the extension modules.  It 
would be nice to be able to use the same values as used by the Makefile to 
build the core, but I can if I must just get the values directly from the 
environment variables themselves.

This should allow for the removal of the direct support for Fink and 
DarwinPorts.  It should also remove any hand-editing needed to get setup.py to 
pick up any non-standard library and header locations.

-Brett
From martin at v.loewis.de  Sun Dec  5 09:36:00 2004
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Dec  5 09:35:59 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <001801c4da72$543801a0$ab2fc797@oemcomputer>
References: <001801c4da72$543801a0$ab2fc797@oemcomputer>
Message-ID: <41B2C870.5090609@v.loewis.de>

Raymond Hettinger wrote:
> Hmph.  The xmllib module has been deprecated since Py2.0 but is not
> listed in PEP 4.
> 
> Question:  Do we have to keep it for another two years because of that
> omission?
> 
> It seems somewhat silly to keep an obsolete, supplanted module that
> doesn?t full support XML 1.0.

I mostly agree with Fredrik. What good does removal of xmllib do?
It's not that it is causing any maintenance burden, so we could just
leave it in. Whether this means that we keep xmllib until P3k, I
don't know.

As for PEP 4: I don't know whether it needs to be listed there. It
appears that the PEP is largely unmaintained (I, personally, do not
really maintain it). So one option would be to just stop using PEP 4
for recording deprecations, since we now have the warnings module.
If we want to keep PEP 4, we need to follow the procedures it
requires (or modify them if we don't like them).

Regards,
Martin
From martin at v.loewis.de  Sun Dec  5 09:40:49 2004
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun Dec  5 09:40:45 2004
Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling?
In-Reply-To: <41B2BE2E.3000907@ocf.berkeley.edu>
References: <41B2BE2E.3000907@ocf.berkeley.edu>
Message-ID: <41B2C991.4090502@v.loewis.de>

Brett C. wrote:
> I noticed that Makefile.pre.in uses the value from the environment 
> variable LDFLAGS but not CPPFLAGS.  Any reason for this? 

How did you notice that? For LDFLAGS, Makefile.pre.in has

LDFLAGS=        @LDFLAGS@

This does *not* mean that the value from the environment is used.
Instead, it means that configure computes the value of LDFLAGS
when it generates Makefile.in. For CPPFLAGS, configure has nothing
to compute, so Makefile.pre.in just has the static value for
CPPFLAGS.

Regards,
Martin
From python at rcn.com  Sun Dec  5 10:27:15 2004
From: python at rcn.com (Raymond Hettinger)
Date: Sun Dec  5 10:30:03 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <41B2C870.5090609@v.loewis.de>
Message-ID: <000201c4daac$9bd871e0$3b01a044@oemcomputer>

> As for PEP 4: I don't know whether it needs to be listed there. It
> appears that the PEP is largely unmaintained (I, personally, do not
> really maintain it). So one option would be to just stop using PEP 4
> for recording deprecations, since we now have the warnings module.

+1



Raymond

From mwh at python.net  Sun Dec  5 11:40:08 2004
From: mwh at python.net (Michael Hudson)
Date: Sun Dec  5 11:40:10 2004
Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling?
In-Reply-To: <41B2BE2E.3000907@ocf.berkeley.edu> (Brett C.'s message of
	"Sat, 04 Dec 2004 23:52:14 -0800")
References: <41B2BE2E.3000907@ocf.berkeley.edu>
Message-ID: <2my8gdqb93.fsf@starship.python.net>

"Brett C." <bac@OCF.Berkeley.EDU> writes:

> I noticed that Makefile.pre.in uses the value from the environment
> variable LDFLAGS but not CPPFLAGS.  Any reason for this?
> ./configure -h`` lists both (and some other environment variables
> which are not really used either) so it seems a little misleading to
> have those listed but to not utilize them.

The whole story of how the compiling/linker flags get set up is a bit
of a mess, AFAIK.  I don't have the energy or the desire to try to
make what we want to happen precise (the hard bit) and then make that
happen (probably rather easier).

> This should allow for the removal of the direct support for Fink and
> DarwinPorts.

+lots.  I don't trust fink, but need it for latex...

Cheers,
mwh

-- 
  After a heavy night I travelled on, my face toward home - the comma
  being by no means guaranteed.           -- paraphrased from cam.misc
From python at rcn.com  Sun Dec  5 11:40:16 2004
From: python at rcn.com (Raymond Hettinger)
Date: Sun Dec  5 11:43:06 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <41B2C870.5090609@v.loewis.de>
Message-ID: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>

> > It seems somewhat silly to keep an obsolete, supplanted module that
> > doesn?t full support XML 1.0.
> 
> I mostly agree with Fredrik. What good does removal of xmllib do?


A few thoughts:

* Deprecated modules just get moved to the lib-old directory.  If
someone has ancient code relying on the module, it is a somewhat trivial
maintenance step to add lib-old to their PYTHONPATH.  IOW, I fail to see
the harm.

* For this particular module, xmllib, about six years will have elapsed
between its original deprecation in Py2.0 and us taking it out in a
Py2.5 release.  

* The number one current python complaint is the state of the standard
library:  http://www.internetnews.com/dev-news/article.php/3441931 .
Some of this may just be the fruits of AMK's highly publicized journal
entry; however, I think the concerns about library quality will persist.

* The average quality of the library improves as we take out junk (the
tzparse module for example) and put in high quality modules like
logging, csv, decimal, etc.

* After working through the tutorial, another huge task for aspiring
intermediate Python programmer is to learn about the standard library.
That task is made much difficult by the presence of obsolete,
incomplete, and/or supplanted modules.

For example, the high quality, actively maintained email package
supplants several older, incomplete, or non-actively maintained modules.
The library would be cleaner, more presentable, and easier to use if the
old ones were to fall by the wayside.

* Ideally, the library will develop in a way that doesn't have a
cluttered concept space.  This will help get it back to the point where
you can hold it in your head all at once and still be able to think
about the problem domain.  Keeping cruft impedes that goal.


my two cents,



Raymond

From martin at v.loewis.de  Sun Dec  5 12:12:12 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Dec  5 12:12:08 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>
References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>
Message-ID: <41B2ED0C.20509@v.loewis.de>

Raymond Hettinger wrote:
> * Deprecated modules just get moved to the lib-old directory.  If
> someone has ancient code relying on the module, it is a somewhat trivial
> maintenance step to add lib-old to their PYTHONPATH.  IOW, I fail to see
> the harm.

I have never considered this as an official policy. For example, when
deprecated C modules are removed, they are never moved to lib-old.

> * For this particular module, xmllib, about six years will have elapsed
> between its original deprecation in Py2.0 and us taking it out in a
> Py2.5 release.  

Correct. For regex, much more time has passed (it is deprecated since
1.5 or something).

> * The number one current python complaint is the state of the standard
> library:  http://www.internetnews.com/dev-news/article.php/3441931 .
> Some of this may just be the fruits of AMK's highly publicized journal
> entry; however, I think the concerns about library quality will persist.

I agree. Removing old modules might change this, indeed: the complaint
about unstability and incompatibility might then become the number one
complaint :-)

> * The average quality of the library improves as we take out junk (the
> tzparse module for example) and put in high quality modules like
> logging, csv, decimal, etc.

I am not convinced that all these more recent modules are really higher
quality than the modules that have been added ten years ago. Some are
(especially those which has seen extensive testing before being
integrated), some are not (especially those added in an ad-hoc manner).

Regards,
Martin
From anthony at interlink.com.au  Sun Dec  5 16:42:56 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sun Dec  5 16:43:28 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>
References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>
Message-ID: <200412060243.01129.anthony@interlink.com.au>

On Sunday 05 December 2004 21:40, Raymond Hettinger wrote:
> * The number one current python complaint is the state of the standard
> library:  http://www.internetnews.com/dev-news/article.php/3441931 .
> Some of this may just be the fruits of AMK's highly publicized journal
> entry; however, I think the concerns about library quality will persist.
>
> * The average quality of the library improves as we take out junk (the
> tzparse module for example) and put in high quality modules like
> logging, csv, decimal, etc.

We can't win - if we remove it, we'll get massive numbers of complaints
bitching about us breaking backwards compatibility. I think the solution
is to make sure the library docs say, very clearly, in big words at the
top of the relevant page "DON'T DO NEW CODE WITH THIS MODULE".
Perhaps Fred can make latex2html output a <blink> tag around it <wink>.
The library docs for, e.g. xmllib already say deprecated at the top - maybe
it should be larger?

Should the global module index should grow a "(deprecated)" line
next door to the module name of the modules that are known to be
old and kinda awful (such as xmllib)? Or make a seperate category -
normal modules, and modules-that-are-only-kept-for-backwards compat?
I suspect that rfc822 will end up in that latter category, rather than ever
being removed (at least, until 3.0). I went through just my own code, and
found an amazing amount of use of that module.  Python has a reputation 
for stability and not forcing people to rewrite their code - I think that this
is a good thing. 

IOW, I think breaking code is far worse than some folks whining 
that some of the stdlib is old and crufty. 

Alternately, we could lock the folks complaining about the stdlib's state
in a room with the folks who complain that every new thing has "wrecked
the language that they knew and loved" and let them fight it out.

> * After working through the tutorial, another huge task for aspiring
> intermediate Python programmer is to learn about the standard library.
> That task is made much difficult by the presence of obsolete,
> incomplete, and/or supplanted modules.

Surely this one is something that can be fixed in the documentation
by marking obsolete modules as such, or making the marking much
clearer?

Anthony
From barry at python.org  Sun Dec  5 17:35:13 2004
From: barry at python.org (Barry Warsaw)
Date: Sun Dec  5 17:35:17 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <41B2C870.5090609@v.loewis.de>
References: <001801c4da72$543801a0$ab2fc797@oemcomputer>
	<41B2C870.5090609@v.loewis.de>
Message-ID: <1102264513.29521.63.camel@presto.wooz.org>

On Sun, 2004-12-05 at 03:36, "Martin v. L?wis" wrote:

> I mostly agree with Fredrik. What good does removal of xmllib do?
> It's not that it is causing any maintenance burden, so we could just
> leave it in. Whether this means that we keep xmllib until P3k, I
> don't know.
> 
> As for PEP 4: I don't know whether it needs to be listed there. It
> appears that the PEP is largely unmaintained (I, personally, do not
> really maintain it). So one option would be to just stop using PEP 4
> for recording deprecations, since we now have the warnings module.
> If we want to keep PEP 4, we need to follow the procedures it
> requires (or modify them if we don't like them).

I agree.  We don't need to use PEP 4 for listing module deprecation
schedules.  It would be better if we included that information in the
DeprecationWarning.  Probably nobody reads PEP 4 when they get a
DeprecationWarning, but everyone reads the message that's printed, so if
it said something like "This module is deprecated and slated for removal
in Python 2.6", I think we'd be providing better information to our
users.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20041205/094182eb/attachment.pgp
From barry at python.org  Sun Dec  5 17:50:22 2004
From: barry at python.org (Barry Warsaw)
Date: Sun Dec  5 17:50:25 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>
References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>
Message-ID: <1102265422.29524.74.camel@presto.wooz.org>

On Sun, 2004-12-05 at 05:40, Raymond Hettinger wrote:

> * The number one current python complaint is the state of the standard
> library:  http://www.internetnews.com/dev-news/article.php/3441931 .
> Some of this may just be the fruits of AMK's highly publicized journal
> entry; however, I think the concerns about library quality will persist.

I don't know if I'd use one quote from this article to infer that
standard library issues are the #1 complaint.  But besides that, there's
no question IMO that the standard library should be the place where
future Python development efforts are focused (not exclusively, but
predominately).

> * The average quality of the library improves as we take out junk (the
> tzparse module for example) and put in high quality modules like
> logging, csv, decimal, etc.
> 
> * After working through the tutorial, another huge task for aspiring
> intermediate Python programmer is to learn about the standard library.
> That task is made much difficult by the presence of obsolete,
> incomplete, and/or supplanted modules.

You don't have to remove modules to make this happen.  Few developers
find modules by doing an 'ls' on Lib -- they look to the library
reference docs, books, and existing code.  So for example, the re-org I
did on the string module docs for 2.4 should help, because now you can't
look up a string function such as atoi() without seeing that it's
deprecated.   A developer writing new code hopefully wouldn't even waste
time learning about those functions, and instead hop right to the
preferred string methods.  Yet, there's no push in the community to
actually remove those deprecated functions from string.py.

> For example, the high quality, actively maintained email package
> supplants several older, incomplete, or non-actively maintained modules.
> The library would be cleaner, more presentable, and easier to use if the
> old ones were to fall by the wayside.

Yes, but there's still /a lot/ of work to do to update the existing
standard library to use the email package.  We debated this during 2.4
and decided we can't even add deprecation warnings to things like
rfc822.py until we do.

> * Ideally, the library will develop in a way that doesn't have a
> cluttered concept space.  This will help get it back to the point where
> you can hold it in your head all at once and still be able to think
> about the problem domain.  Keeping cruft impedes that goal.

Agreed, but I think it's mostly going to be an organic process over
time.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20041205/71ac1d23/attachment.pgp
From carribeiro at gmail.com  Sun Dec  5 18:42:45 2004
From: carribeiro at gmail.com (Carlos Ribeiro)
Date: Sun Dec  5 18:42:48 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>
References: <41B2C870.5090609@v.loewis.de>
	<000701c4dab6$cf5e8e00$3b01a044@oemcomputer>
Message-ID: <864d370904120509421ec3dd50@mail.gmail.com>

On Sun, 5 Dec 2004 05:40:16 -0500, Raymond Hettinger <python@rcn.com> wrote:
> * Deprecated modules just get moved to the lib-old directory.  If
> someone has ancient code relying on the module, it is a somewhat trivial
> maintenance step to add lib-old to their PYTHONPATH.  IOW, I fail to see
> the harm.

In principle, that's a good idea, but I don't know if it's really
practical. I fear that the distribution would end up simply adding
lib-old to the PYTHONPATH anyway :-) But some variations may be worth
considering:

1) Deprecated modules would first generate a warning (as it is today).
On future versions, instead of being removed, the level of the warning
would be raised, causing a fatal exception (an ImportError?), unless
explicitly configured otherwise (using a config file or some similar
mechanism). I believe that the current warnings module already
provides a good basis for this implementation.

(IOW - the module would still be there, and could be activated, but
it's "off" by default. Turning it on again has to be easy, though. And
the message on "ImportError" have to meaningful for non-technical
users, allowing for easier support in these situations)

2) Split the documentation: the "old-and-deprecated" modules would be
listed separated from the active & recommended ones, as a different
section, or even as a separate book. That would be a clear sign to new
users to keep in mind while they read the documentation, perhaps more
effective than writing the deprecation warning on the top of the page.


My half-a-cent.

-- 
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: carribeiro@gmail.com
mail: carribeiro@yahoo.com
From martin at v.loewis.de  Sun Dec  5 18:54:42 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Dec  5 18:54:39 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <200412060243.01129.anthony@interlink.com.au>
References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>
	<200412060243.01129.anthony@interlink.com.au>
Message-ID: <41B34B62.7040201@v.loewis.de>

Anthony Baxter wrote:
> The library docs for, e.g. xmllib already say deprecated at the top - maybe
> it should be larger?

I think it would be better to *remove* the xmllib documentation.
Existing code which needs the module will continue to work even
without documentation, and new code is unlikely to be written for
a module that has no documentation, and where importing the module
gives a DeprecationWarning.

> Alternately, we could lock the folks complaining about the stdlib's state
> in a room with the folks who complain that every new thing has "wrecked
> the language that they knew and loved" and let them fight it out.

That sounds like a fair and democratic way of solving the issue.

Regards,
Martin
From allison at sumeru.stanford.EDU  Sun Dec  5 19:30:28 2004
From: allison at sumeru.stanford.EDU (Dennis Allison)
Date: Sun Dec  5 19:30:35 2004
Subject: [Python-Dev] Python 2.4 and Zope 2.7.X
In-Reply-To: <1102265422.29524.74.camel@presto.wooz.org>
Message-ID: <Pine.LNX.4.10.10412051027310.4966-100000@sumeru.stanford.EDU>


A report on the zope@zope.org list suggests that Python 2.4 is not fully
compatible with Zope 2.7.3.  Has any tested against Zope? 

From lists at andreas-jung.com  Sun Dec  5 21:06:26 2004
From: lists at andreas-jung.com (Andreas Jung)
Date: Sun Dec  5 21:06:33 2004
Subject: [Python-Dev] Python 2.4 and Zope 2.7.X
In-Reply-To: <Pine.LNX.4.10.10412051027310.4966-100000@sumeru.stanford.EDU>
References: <Pine.LNX.4.10.10412051027310.4966-100000@sumeru.stanford.EDU>
Message-ID: <17891C1A2E0C2C4C4580A33B@[192.168.0.102]>



--On Sonntag, 5. Dezember 2004 10:30 Uhr -0800 Dennis Allison 
<allison@sumeru.stanford.EDU> wrote:

>
> A report on the zope@zope.org list suggests that Python 2.4 is not fully
> compatible with Zope 2.7.3.  Has any tested against Zope?
>

To which report do you refer? The only thing I mentioned is that there has 
not been
any audit for Python 2.4 + Zope 2.7 as with Python 2.3  + Zope 2.6/2.7.

-aj
From rwgk at cci.lbl.gov  Mon Dec  6 00:37:12 2004
From: rwgk at cci.lbl.gov (Ralf W. Grosse-Kunstleve)
Date: Mon Dec  6 00:37:19 2004
Subject: [Python-Dev] proposal+patch: sys.gettickeraccumulation()
Message-ID: <200412052337.iB5NbC5Y318887@boa.lbl.gov>

Brett C. wrote:
> Ralf W. Grosse-Kunstleve wrote:
> >  Brett C. wrote:
> >  
> > >I have been mulling over this proposal and I think I am finally
> > >settling on +0 as long as it can be shown that it does not hurt
> > >performance at all (yes, it shouldn't but this is the eval loop we are
> > >talking about and that thing is a touchy beast).
> >  
> >  
> >  Is there a standard benchmark I could use to demonstrate the
> >  impact of the two new additions on the runtime?
> >  Thanks!
> 
> =)  Parrotbench and PyBench would be enough for me to sign off on any 
> performance hit.  There is also pystone.

I have done my homework now, based on parrotbench. Timings are below.
The most important conclusion from my viewpoint is that my original
goal can be achieved with a small patch and with a runtime penalty that
is in the noise. However, mysterious runtime behavior of
Objects/longobject.c lead me to change my original patch, and to the
unanswered question "could small rearrangements in longobject.c
potentially boost performance by 45% on some platforms?"

Please don't be put off by the size of this message. My patch
is essentially just one line in ceval.c (_Py_TickerAccumulation++)!

To restate my original goal:

  I am looking for a simple way to answer the question: How much of a
  speedup can I expect if I reimplement a piece of Python code in C or
  C++?

Note that I am not asking how much time is spent in Python including
all the extensions that come with it, but how much time is spent in the
Python bytecode interpreter loop. To clarify, look at the timings for
two tests in parrotbench:

   b1      5.8287 time        30049474 ticks        0.1940 us/tick
   b2      1.5944 time          482584 ticks        3.3039 us/tick

Each of the tests was run with range(10) in b.py to increase accuracy.
The first column shows time.time() differences, the second the number
of bytecodes processed in ceval.c, and the last column shows
micro-seconds per tick. Why is the us/tick value for b2 17 times larger
than that for b1?

The answer is that b2 makes heavy use of long integers, and that a lot
of time is spent in Objects/longobject.c doing long-integer arithmetic.
Compared to b1, relatively little time is spent interpreting bytecodes.

Back to the original question: how much sense does it make to
reimplement b1 or b2 in C? Simply looking at the us/tick values is
telling me that I can expect much more of a performance boost by
reimplementing b1 rather than b2. This is because b2 spends a lot of
time in C already. A similar situation arises if extensions like
Numeric are used. Computationally intensive operations like matrix
multiplication and matrix inversion are already done at C speed.

The us/tick let us quickly estimate how disappointed we would be after
moving Python code to C or C++. We get this estimate without having to
do detailed source code analysis, and before we have wasted our time
doing the recoding (we have been through this a couple of times ...).

The timings below show that the patched Python which counts the number
of interpreted bytecodes is in the worst case 0.6% slower than the
original Python, but some tests even run faster at face value:

$gcc296_compiled/original/python -O b_timings.py | grep reporter
   b0     16.8831 time
   b1      5.9553 time
   b2      1.5914 time
   b3     10.8149 time
   b4      5.2528 time
   b5     11.6437 time
   b6     11.0248 time
   b7     27.6960 time
  all     90.9276 time
$gcc296_compiled/work/python -O b_timings.py | grep reporter
   b0     16.9193 time        50979585 ticks        0.3319 us/tick
   b1      5.8287 time        30049474 ticks        0.1940 us/tick
   b2      1.5944 time          482584 ticks        3.3039 us/tick
   b3     10.7931 time        43577404 ticks        0.2477 us/tick
   b4      5.2865 time        18463044 ticks        0.2863 us/tick
   b5     11.6086 time        17065750 ticks        0.6802 us/tick
   b6     10.0992 time        60000464 ticks        0.1683 us/tick
   b7     27.6830 time         1928644 ticks       14.3536 us/tick
  all     89.8760 time       222548399 ticks        0.4038 us/tick

The b7 test is the same as b2 but with xrange(10000) instead of
xrange(1000), and with all print statements removed.

Ratios (rounded to 3 decimals):
    16.9193/16.8831=1.002
     5.8287/5.9553 =0.979
     1.5944/1.5914 =1.002
    10.7931/10.8149=0.998
     5.2865/5.2528 =1.006
    11.6086/11.6437=0.997
    10.0992/11.0248=0.916
    27.6830/27.6960=1.000

Therefore I'd argue that the runtime penalty for the one additional
long long increment in ceval.c (_Py_TickerAccumulation++) is in the
noise.

The timings were collected on a 2.8GHz Dual Xeon, Redhat WS 3.
Python was compiled under Red Hat 7.3 with gcc 2.96. See below why.

My latest patch can be found at the end of this posting.
It can be applied to the Python-2.4 distribution like this:

    tar zxf Python-2.4.tgz
    patch --strip=1 -d Python-2.4 < patch_file

------------------------------------------------------------------------------
------------------------------------------------------------------------------
------------------------------------------------------------------------------

Now to the mysterious runtime behavior of Objects/longobject.c

In the original Python 2.4 sources _Py_Ticker is decremented in
two places:

In the interpreter loop in Python/ceval.c:

                if (--_Py_Ticker < 0) {

In Objects/longobject.c:

#define SIGCHECK(PyTryBlock) \
        if (--_Py_Ticker < 0) { \
                _Py_Ticker = _Py_CheckInterval; \
                if (PyErr_CheckSignals()) { PyTryBlock; } \
        }

This macro is used in four places.
My initial patch was to insert updates of a new, global
_Py_TickerAccumulation variable in these two places:

Python/ceval.c:

                                    a try: finally: block uninterruptable. */
                                 goto fast_next_opcode;
                         }
+			_Py_TickerAccumulation += _Py_CheckInterval - _Py_Ticker;
 			_Py_Ticker = _Py_CheckInterval;
 			tstate->tick_counter++;

Objects/longobject.c:

 #define SIGCHECK(PyTryBlock) \
 	if (--_Py_Ticker < 0) { \
+		_Py_TickerAccumulation += _Py_CheckInterval - _Py_Ticker; \
 		_Py_Ticker = _Py_CheckInterval; \
 		if (PyErr_CheckSignals()) { PyTryBlock; } \
 	}

This lead to the timings below. All timings were collected on the
same machine, but the original and the patched Pythons were each
compiled with two different compilers, gcc 2.96 and gcc 3.2.3. The
important timings to look at are those for b2 and b7 again.

$gcc296_compiled/original/python -O b_timings.py | grep reporter
   b0     16.8831 time
   b1      5.9553 time
   b2      1.5914 time
   b3     10.8149 time
   b4      5.2528 time
   b5     11.6437 time
   b6     11.0248 time
   b7     27.6960 time
  all     90.9276 time
$gcc323_compiled/original/python -O b_timings.py | grep reporter
   b0     18.6390 time
   b1      6.4681 time
   b2      2.2588 time
   b3     11.0215 time
   b4      5.6490 time
   b5     12.3022 time
   b6     10.3815 time
   b7     40.2735 time
  all    107.0636 time

This shows that the gcc 2.96 optimizer is generally a little bit
better than the gcc 3.2.3 optimizer, but not by very much. Except
for the b2 and b7 tests. E.g. 40.2735/27.6960 = 1.454!

Now with the patches in both ceval.c and longobject.c:

$gcc296_compiled/ticker/python -O b_timings.py | grep reporter
   b0     17.1190 time        60792625 ticks        0.2816 us/tick
   b1      6.1171 time        30049474 ticks        0.2036 us/tick
   b2      1.8764 time          705754 ticks        2.6587 us/tick
   b3     10.7070 time        43577404 ticks        0.2457 us/tick
   b4      5.2677 time        22944694 ticks        0.2296 us/tick
   b5     11.7448 time        17433190 ticks        0.6737 us/tick
   b6     10.9603 time        60000504 ticks        0.1827 us/tick
   b7     33.2320 time         2837124 ticks       11.7133 us/tick
  all     97.0893 time       238342235 ticks        0.4074 us/tick
$gcc323_compiled/ticker/python -O b_timings.py | grep reporter
   b0     18.3115 time        60792625 ticks        0.3012 us/tick
   b1      6.1193 time        30049474 ticks        0.2036 us/tick
   b2      2.2522 time          705754 ticks        3.1912 us/tick
   b3     11.2202 time        43577404 ticks        0.2575 us/tick
   b4      5.4596 time        22944694 ticks        0.2379 us/tick
   b5     11.8197 time        17433190 ticks        0.6780 us/tick
   b6     10.7407 time        60000504 ticks        0.1790 us/tick
   b7     40.2397 time         2837124 ticks       14.1833 us/tick
  all    106.2283 time       238342235 ticks        0.4457 us/tick

The gcc 3.2.3 timings for b7 are hardly affected by my patch in
longobject.c, but the gcc 2.96 timing shoots up from 27.6960 to
33.2320. Still not as bad as the gcc 3.2.3 timing, but why?

Interestingly, if I declare _Py_TickerAccumulation as long instead
of PY_LONG_LONG, the gcc 2.96 timing is also hardly affected by
my patch in longobject.c.

Even more interestingly, if I change longobject.c like this...

#define SIGCHECK(PyTryBlock) \
        { \
                if (PyErr_CheckSignals()) { PyTryBlock; } \
        }

the runtime is also hardly affected (times not shown in this posting)
even though PyErr_CheckSignals() is executed 100 times more often.

Curious to learn how other compilers deal with longobject.c
I collected timings with this compiler:
Compaq C V6.3-028 on Compaq Tru64 UNIX V5.1 (Rev. 732)

$compaq_compiled/original/python -O b_timings.py | grep reporter 
   b1     19.4492 time
   b2      4.9395 time
   b3     23.6787 time
   b5     28.2568 time
   b6     21.6230 time
   b7     86.5283 time
  all    192.0498 time
$compaq_compiled/ticker/python -O b_timings.py | grep reporter 
   b1     19.4775 time        30049474 ticks        0.6482 us/tick
   b2      4.9971 time          704784 ticks        7.0902 us/tick
   b3     24.2559 time        43577404 ticks        0.5566 us/tick
   b5     26.3320 time        17420746 ticks        1.5115 us/tick
   b6     21.7227 time        60000464 ticks        0.3620 us/tick
   b7     86.9395 time         2836154 ticks       30.6540 us/tick
  all    190.8936 time       165555937 ticks        1.1530 us/tick

The b0 and b4 timings are missing because of exceptions that I didn't
investigate. As with the gcc 3.2.3 compilation, my longobject.c
patch hardly makes a difference.

What does all this mean? Does the gcc 2.96 surprise tell us that small
rearrangements in longobject.c could potentially boost performance by
45% on some platforms?

Since I am not interested in optimizing longobject.c this is where I
stopped. Eventually I decided that the problematic patch in
longobject.c is not helping me with my original goal as stated near the
beginning of this posting. I am only interested in counting the
iterations of the interpreter loop. However, the _Py_Ticker decrements
in longobject.c are not inside the interpreter loop, but in C loops!
This means _Py_Ticker is useless for my purposes. Therefore I decoupled
_Py_Ticker and _Py_TickerAccumulation.

BTW: I think strictly speaking this documentation is incorrect because
of the _Py_Ticker manipulations in longobject.c:

    sys.setcheckinterval.__doc__:
    Tell the Python interpreter to check for asynchronous events every
    n instructions.

I feel very dizzy now after dealing with all the timings and the nasty
details. A lot of fallout caused by the simple idea to add one innocent
line to ceval.c. I hope it makes sense to somebody!

Cheers,
        Ralf

------------------------------------------------------------------------------
------------------------------------------------------------------------------
------------------------------------------------------------------------------

diff -r -u original/Include/ceval.h work/Include/ceval.h
--- original/Include/ceval.h	2004-10-10 19:40:35.000000000 -0700
+++ work/Include/ceval.h	2004-11-30 23:27:11.000000000 -0800
@@ -68,6 +68,11 @@
 
 /* this used to be handled on a per-thread basis - now just two globals */
 PyAPI_DATA(volatile int) _Py_Ticker;
+#ifndef HAVE_LONG_LONG
+PyAPI_DATA(volatile long) _Py_TickerAccumulation;
+#else
+PyAPI_DATA(volatile PY_LONG_LONG) _Py_TickerAccumulation;
+#endif
 PyAPI_DATA(int) _Py_CheckInterval;
 
 /* Interface for threads.
diff -r -u original/PC/os2emx/python24.def work/PC/os2emx/python24.def
--- original/PC/os2emx/python24.def	2004-10-10 19:40:50.000000000 -0700
+++ work/PC/os2emx/python24.def	2004-11-30 23:27:11.000000000 -0800
@@ -743,6 +743,7 @@
   "_Py_CheckRecursionLimit"
   "_Py_CheckInterval"
   "_Py_Ticker"
+  "_Py_TickerAccumulation"
 
 ; From python24_s.lib(compile)
   "PyCode_New"
diff -r -u original/Python/ceval.c work/Python/ceval.c
--- original/Python/ceval.c	2004-11-23 10:06:08.000000000 -0800
+++ work/Python/ceval.c	2004-12-03 19:39:36.000000000 -0800
@@ -373,6 +373,7 @@
 	pendinglast = j;
 
 	_Py_Ticker = 0;
+	_Py_TickerAccumulation = 0;
 	things_to_do = 1; /* Signal main loop */
 	busy = 0;
 	/* XXX End critical section */
@@ -476,6 +477,11 @@
    per thread, now just a pair o' globals */
 int _Py_CheckInterval = 100;
 volatile int _Py_Ticker = 100;
+#ifndef HAVE_LONG_LONG
+volatile long _Py_TickerAccumulation = 0;
+#else
+volatile PY_LONG_LONG _Py_TickerAccumulation = 0;
+#endif
 
 PyObject *
 PyEval_EvalCode(PyCodeObject *co, PyObject *globals, PyObject *locals)
@@ -770,6 +776,7 @@
 		   async I/O handler); see Py_AddPendingCall() and
 		   Py_MakePendingCalls() above. */
 
+                _Py_TickerAccumulation++;
 		if (--_Py_Ticker < 0) {
                         if (*next_instr == SETUP_FINALLY) {
                                 /* Make the last opcode before
diff -r -u original/Python/sysmodule.c work/Python/sysmodule.c
--- original/Python/sysmodule.c	2004-08-12 11:19:17.000000000 -0700
+++ work/Python/sysmodule.c	2004-12-03 19:36:52.000000000 -0800
@@ -442,6 +442,20 @@
 "getcheckinterval() -> current check interval; see setcheckinterval()."
 );
 
+static PyObject *
+sys_gettickeraccumulation(PyObject *self, PyObject *args)
+{
+#ifndef HAVE_LONG_LONG
+	return PyInt_FromLong(_Py_TickerAccumulation);
+#else
+	return PyLong_FromLongLong(_Py_TickerAccumulation);
+#endif
+}
+
+PyDoc_STRVAR(gettickeraccumulation_doc,
+"gettickeraccumulation() -> current count of bytecodes processed by the interpreter."
+);
+
 #ifdef WITH_TSC
 static PyObject *
 sys_settscdump(PyObject *self, PyObject *args)
@@ -763,6 +777,8 @@
 	 setcheckinterval_doc}, 
 	{"getcheckinterval",	sys_getcheckinterval, METH_NOARGS,
 	 getcheckinterval_doc}, 
+	{"gettickeraccumulation", sys_gettickeraccumulation, METH_NOARGS,
+	 gettickeraccumulation_doc}, 
 #ifdef HAVE_DLOPEN
 	{"setdlopenflags", sys_setdlopenflags, METH_VARARGS, 
 	 setdlopenflags_doc},
From bac at OCF.Berkeley.EDU  Mon Dec  6 00:49:56 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Dec  6 00:51:03 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <41B34B62.7040201@v.loewis.de>
References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>	<200412060243.01129.anthony@interlink.com.au>
	<41B34B62.7040201@v.loewis.de>
Message-ID: <41B39EA4.9030105@ocf.berkeley.edu>

Martin v. L?wis wrote:
> Anthony Baxter wrote:
> 
>> The library docs for, e.g. xmllib already say deprecated at the top - 
>> maybe
>> it should be larger?
> 
> 
> I think it would be better to *remove* the xmllib documentation.
> Existing code which needs the module will continue to work even
> without documentation, and new code is unlikely to be written for
> a module that has no documentation, and where importing the module
> gives a DeprecationWarning.
> 

I like this solution.  And if anyone cares about having good docs still for a 
deprecated module they can just read the docstrings.  We don't get new people 
using it but others don't have to change their code.

And as for the mention of dropping PEP 4, I agree with the running consensus 
that it isn't needed thanks to the warning module.  Do we need to have a more 
formal vote to drop PEP 4, or should one the PEP maintainers just delete it?

-Brett
From bac at OCF.Berkeley.EDU  Mon Dec  6 00:55:54 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Dec  6 00:57:05 2004
Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling?
In-Reply-To: <41B2C991.4090502@v.loewis.de>
References: <41B2BE2E.3000907@ocf.berkeley.edu> <41B2C991.4090502@v.loewis.de>
Message-ID: <41B3A00A.30500@ocf.berkeley.edu>

Martin v. L?wis wrote:
> Brett C. wrote:
> 
>> I noticed that Makefile.pre.in uses the value from the environment 
>> variable LDFLAGS but not CPPFLAGS.  Any reason for this? 
> 
> 
> How did you notice that? For LDFLAGS, Makefile.pre.in has
> 
> LDFLAGS=        @LDFLAGS@
> 
> This does *not* mean that the value from the environment is used.
> Instead, it means that configure computes the value of LDFLAGS
> when it generates Makefile.in. For CPPFLAGS, configure has nothing
> to compute, so Makefile.pre.in just has the static value for
> CPPFLAGS.
> 

I realize that much.  But if you look in configure.in it seems to use the 
previous value of LDFLAGS every time it is redefined as a base and thus gets 
its initial value from the environment variable the first time it is tweaked.

Not a big deal, though.  I will just use the environment variables in setup.py .

-Brett
From python at rcn.com  Mon Dec  6 01:05:56 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Dec  6 01:09:05 2004
Subject: [Python-Dev] proposal+patch: sys.gettickeraccumulation()
In-Reply-To: <200412052337.iB5NbC5Y318887@boa.lbl.gov>
Message-ID: <000601c4db27$6b1bad80$e841fea9@oemcomputer>

> To restate my original goal:
> 
>   I am looking for a simple way to answer the question: How much of a
>   speedup can I expect if I reimplement a piece of Python code in C or
>   C++?
 . . .
> Ratios (rounded to 3 decimals):
>     16.9193/16.8831=1.002
>      5.8287/5.9553 =0.979
>      1.5944/1.5914 =1.002
>     10.7931/10.8149=0.998
>      5.2865/5.2528 =1.006
>     11.6086/11.6437=0.997
>     10.0992/11.0248=0.916
>     27.6830/27.6960=1.000
> 
> Therefore I'd argue that the runtime penalty for the one additional
> long long increment in ceval.c (_Py_TickerAccumulation++) is in the
> noise.

The measurements are too imprecise to draw any worthwhile conclusions.
Try running:
    python timeit.py -r9 "pass"

That ought to give more stable measurements.

The proposed "analysis tool" has no benefit to a majority of Python
users.  Even a 1% hit is not worth it.



> I am only interested in counting the
> iterations of the interpreter loop. However, the _Py_Ticker decrements
> in longobject.c are not inside the interpreter loop, but in C loops!
> This means _Py_Ticker is useless for my purposes. Therefore I
decoupled
> _Py_Ticker and _Py_TickerAccumulation.

Why add this to everyone's build?  Just put it in when doing your own
analysis.

The eval loop already pays a penalty for Py2.4's extra function tracing
code.  And ceval.c has been cluttered with #ifdefs for hardware
timestamps.  And there have been other additions for signal handling and
whatnot.  This is enough.



>. A lot of fallout caused by the simple idea to add one innocent
> line to ceval.c. 

I do not find it to be innocent.  A great of work was expended over the
years just trying to eliminate a small step or two from the eval-loop.
Those efforts should not be discarded lightly.

-1 on adding it directly.

-0 on adding it as a #ifdeffed compile option (with the default being to
exclude it).



Raymond Hettinger

From python at rcn.com  Mon Dec  6 01:33:17 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Dec  6 01:36:01 2004
Subject: [Python-Dev] pystone rant [was] proposal+patch:
	sys.gettickeraccumulation()
In-Reply-To: <200412052337.iB5NbC5Y318887@boa.lbl.gov>
Message-ID: <000f01c4db2b$2e119f40$e841fea9@oemcomputer>

(Old message)
> > =)  Parrotbench and PyBench would be enough for me to sign off on
any
> > performance hit.  There is also pystone.

pystone is good a predicting the relative performance of python apps on
difference hardware/software environments.

As a tool for evaluating proposed language changes, it completely sucks
and should *never* be used for anything other than an extra
informational datapoint.

The foremost issue is that it exercises only a tiny portion of the
language.  

Its design makes it worse than totally useless for evaluating eval-loop
overhead.  It runs a loop twice, once with benchmarking code and once
without.  Only the difference is reported.  In theory, that means that
all eval-loop speedups and slowdowns are netted out of the result.  In
practice, the reported result may indicate exactly the opposite of
reality because the empty loop has vastly different cache effects than
the benchmarked loop.

For useful timings, run timeit on the specific feature in question.
Then check for overall impact using pybench, parrotbench, and
test_decimal.


Raymond


From barry at python.org  Mon Dec  6 02:10:28 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Dec  6 02:10:31 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <41B39EA4.9030105@ocf.berkeley.edu>
References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>
	<200412060243.01129.anthony@interlink.com.au>
	<41B34B62.7040201@v.loewis.de>  <41B39EA4.9030105@ocf.berkeley.edu>
Message-ID: <1102295428.8361.8.camel@presto.wooz.org>

On Sun, 2004-12-05 at 18:49, Brett C. wrote:

> And as for the mention of dropping PEP 4, I agree with the running consensus 
> that it isn't needed thanks to the warning module.  Do we need to have a more 
> formal vote to drop PEP 4, or should one the PEP maintainers just delete it?

There's really no such thing as "dropping" a PEP, but in any event, we
should still keep PEP 4 to document the procedure for deprecating
modules.  It just doesn't need to list any modules (i.e. remove the
sections labeled Obsolete modules, Deprecated modules, and Undeprecated
modules).

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20041205/c1ad06b4/attachment.pgp
From anthony at interlink.com.au  Mon Dec  6 02:19:31 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Dec  6 02:41:22 2004
Subject: [Python-Dev] pystone rant [was] proposal+patch:
	sys.gettickeraccumulation()
In-Reply-To: <000f01c4db2b$2e119f40$e841fea9@oemcomputer>
References: <000f01c4db2b$2e119f40$e841fea9@oemcomputer>
Message-ID: <200412061219.32190.anthony@interlink.com.au>

On Monday 06 December 2004 11:33, Raymond Hettinger wrote:
> pystone is good a predicting the relative performance of python apps on
> difference hardware/software environments.

This comes up so often that I'm almost tempted to add a disclaimer
to the pystone output. I can't count the number of times people have
tried to claim that pystone showed that a change was good/neutral.

From ncoghlan at iinet.net.au  Mon Dec  6 09:53:42 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Mon Dec  6 09:53:49 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <41B34B62.7040201@v.loewis.de>
References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>	<200412060243.01129.anthony@interlink.com.au>
	<41B34B62.7040201@v.loewis.de>
Message-ID: <41B41E16.90100@iinet.net.au>

Martin v. L?wis wrote:
> Anthony Baxter wrote:
> 
>> The library docs for, e.g. xmllib already say deprecated at the top - 
>> maybe
>> it should be larger?
> 
> 
> I think it would be better to *remove* the xmllib documentation.
> Existing code which needs the module will continue to work even
> without documentation, and new code is unlikely to be written for
> a module that has no documentation, and where importing the module
> gives a DeprecationWarning.

If we went down this path, I would suggest we include a list of deprecated 
modules in the docs to forestall "where is the documentation for these modules?" 
questions. Such a list would also make life easier for anyone stuck with 
maintaining legacy Python code.

E.g.

X.X Deprecated modules

   The following modules exist in the standard library solely for backwards 
compatibility with older versions of Python. They are no longer supported, and 
should not be used for new code. Anyone maintaining code using these modules 
should refer to the documentation for the Python version listed in the table below.

   Module Name  | Last documented in:
   -------------------------------------
    lib1        |  Python 2.2
    lib2        |  Python 2.3

etc. . .

I'd also suggest including a link to this list at the bottom of the global 
module index.

>> Alternately, we could lock the folks complaining about the stdlib's state
>> in a room with the folks who complain that every new thing has "wrecked
>> the language that they knew and loved" and let them fight it out.
> 
> That sounds like a fair and democratic way of solving the issue.

If we sold tickets, it could even be a nice little earner for the PSF ;)

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan@email.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.skystorm.net
From mwh at python.net  Mon Dec  6 13:12:48 2004
From: mwh at python.net (Michael Hudson)
Date: Mon Dec  6 13:12:50 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <41B34B62.7040201@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Sun,
	05 Dec 2004 18:54:42 +0100")
References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>
	<200412060243.01129.anthony@interlink.com.au>
	<41B34B62.7040201@v.loewis.de>
Message-ID: <2mhdmzr5fj.fsf@starship.python.net>

"Martin v. L?wis" <martin@v.loewis.de> writes:

> Anthony Baxter wrote:
>> The library docs for, e.g. xmllib already say deprecated at the top - maybe
>> it should be larger?
>
> I think it would be better to *remove* the xmllib documentation.
> Existing code which needs the module will continue to work even
> without documentation, and new code is unlikely to be written for
> a module that has no documentation, and where importing the module
> gives a DeprecationWarning.

+1, at least for *hiding* the documentation of xmllib + similar
modules.  I'm not sure wholesale removal is really a good idea.

It's like the new "Non-essential Built-in Functions" section of the
lib ref.

Cheers,
mwh

-- 
  <Acapnotic> jemfinch: What's to parse?  A numeric code, perhaps a
              chicken, and some arguments
                                                -- from Twisted.Quotes
From carribeiro at gmail.com  Mon Dec  6 13:23:09 2004
From: carribeiro at gmail.com (Carlos Ribeiro)
Date: Mon Dec  6 13:23:11 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <2mhdmzr5fj.fsf@starship.python.net>
References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>
	<200412060243.01129.anthony@interlink.com.au>
	<41B34B62.7040201@v.loewis.de> <2mhdmzr5fj.fsf@starship.python.net>
Message-ID: <864d3709041206042352920501@mail.gmail.com>

On Mon, 06 Dec 2004 12:12:48 +0000, Michael Hudson <mwh@python.net> wrote:
> "Martin v. L?wis" <martin@v.loewis.de> writes:
> 
> > Anthony Baxter wrote:
> >> The library docs for, e.g. xmllib already say deprecated at the top - maybe
> >> it should be larger?
> >
> > I think it would be better to *remove* the xmllib documentation.
> > Existing code which needs the module will continue to work even
> > without documentation, and new code is unlikely to be written for
> > a module that has no documentation, and where importing the module
> > gives a DeprecationWarning.
> 
> +1, at least for *hiding* the documentation of xmllib + similar
> modules.  I'm not sure wholesale removal is really a good idea.

1) Move the deprecated documentation into a separate book. 

2) Don't include the "Deprecated Modules Reference" in the standard
distribution, but make it available as a separate download.


-- 
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: carribeiro@gmail.com
mail: carribeiro@yahoo.com
From ncoghlan at iinet.net.au  Mon Dec  6 13:35:11 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Mon Dec  6 13:35:18 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <864d3709041206042352920501@mail.gmail.com>
References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>	<200412060243.01129.anthony@interlink.com.au>	<41B34B62.7040201@v.loewis.de>
	<2mhdmzr5fj.fsf@starship.python.net>
	<864d3709041206042352920501@mail.gmail.com>
Message-ID: <41B451FF.1000909@iinet.net.au>

Carlos Ribeiro wrote:
> On Mon, 06 Dec 2004 12:12:48 +0000, Michael Hudson <mwh@python.net> wrote:
>>+1, at least for *hiding* the documentation of xmllib + similar
>>modules.  I'm not sure wholesale removal is really a good idea.
> 
> 
> 1) Move the deprecated documentation into a separate book. 
> 
> 2) Don't include the "Deprecated Modules Reference" in the standard
> distribution, but make it available as a separate download.

What about the idea of pointing to the last version of the docs that contained 
the relevant documentation?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan@email.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.skystorm.net
From python at rcn.com  Mon Dec  6 13:44:53 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Dec  6 13:47:37 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <864d3709041206042352920501@mail.gmail.com>
Message-ID: <001101c4db91$627e4600$e841fea9@oemcomputer>

> > > I think it would be better to *remove* the xmllib documentation.
> > > Existing code which needs the module will continue to work even
> > > without documentation, and new code is unlikely to be written for
> > > a module that has no documentation, and where importing the module
> > > gives a DeprecationWarning.
> >
> > +1, at least for *hiding* the documentation of xmllib + similar
> > modules.  I'm not sure wholesale removal is really a good idea.

> 1) Move the deprecated documentation into a separate book.
> 
> 2) Don't include the "Deprecated Modules Reference" in the standard
> distribution, but make it available as a separate download.

-1

We are trying to remove clutter, not keep it in perpetuity.  Leaving the
docs means continuing to have to test it, field bug reports, be aware of
its existence, etc.

Do not keep this on a respirator.  Let it die in peace.



Raymond

From barry at python.org  Mon Dec  6 13:55:35 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Dec  6 13:55:37 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <001101c4db91$627e4600$e841fea9@oemcomputer>
References: <001101c4db91$627e4600$e841fea9@oemcomputer>
Message-ID: <1102337735.8362.36.camel@presto.wooz.org>

On Mon, 2004-12-06 at 07:44, Raymond Hettinger wrote:

> We are trying to remove clutter, not keep it in perpetuity.  Leaving the
> docs means continuing to have to test it, field bug reports, be aware of
> its existence, etc.
> 
> Do not keep this on a respirator.  Let it die in peace.

Old documentation lives on in http://www.python.org/doc/versions.html
which seems sufficient for looking up deprecated modules.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20041206/5e733f08/attachment.pgp
From bac at OCF.Berkeley.EDU  Mon Dec  6 21:32:02 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Dec  6 21:32:24 2004
Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling?
In-Reply-To: <41B2C991.4090502@v.loewis.de>
References: <41B2BE2E.3000907@ocf.berkeley.edu> <41B2C991.4090502@v.loewis.de>
Message-ID: <41B4C1C2.7060207@ocf.berkeley.edu>

Martin v. L?wis wrote:
> Brett C. wrote:
> 
>> I noticed that Makefile.pre.in uses the value from the environment 
>> variable LDFLAGS but not CPPFLAGS.  Any reason for this? 
> 
> 
> How did you notice that? For LDFLAGS, Makefile.pre.in has
> 
> LDFLAGS=        @LDFLAGS@
> 
> This does *not* mean that the value from the environment is used.
> Instead, it means that configure computes the value of LDFLAGS
> when it generates Makefile.in. For CPPFLAGS, configure has nothing
> to compute, so Makefile.pre.in just has the static value for
> CPPFLAGS.
> 

I am not so sure that is true.  Checking configure.in, there is no mention of 
CPPFLAGS anywhere.  And yet if I modify the definition of CPPFLAGS in 
Makefile.pre.in to ``-I. -I./Include @CPPFLAGS@`` it ends up containing the 
value I have for the environment variable at the end of it.  I think the '@@' 
syntax uses a value from configure.in if it is defined else it defaults to the 
value the shell has.

-Brett
From bac at OCF.Berkeley.EDU  Mon Dec  6 07:36:14 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Dec  6 21:37:41 2004
Subject: [Python-Dev] Ptyon 2.3.5 probably coming in January;
	get your bugs/patches reported!
Message-ID: <41B3FDDE.3040302@ocf.berkeley.edu>

Anthony Baxter, our ever-diligent release manager, mentioned this past week 
that Python 2.3.5 will most likely come to fruition some time in January (this 
is not guaranteed date).  This means that in order to have enough time to 
proper evaluate new patches and bugs they must be reported **now**!  A one 
month lead time is necessary to properly look at, test, and commit patches, let 
alone coming up with solutions to any reported bugs.

Please realize, though, that reporting a bug or submitting a patch now does not 
guarantee that it will committed in time!  The free time of the development 
team is limited.  If you want to help a bug or patch along to make it easier to 
be evaluated and thus raise its chances of being dealt with please see the 
"Helping Out" section of the 'Intro to Development' essay at 
http://www.python.org/dev/dev_intro.html .

As always, both bugs and patches should be reported to Python's SourceForge 
tracker at http://sourceforge.net/bugs/?group_id=5470 and 
http://sourceforge.net/patch/?group_id=5470, respectively.

-Brett Cannon
From martin at v.loewis.de  Mon Dec  6 22:28:39 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Dec  6 22:28:33 2004
Subject: [Python-Dev] Rewriting PEP4
In-Reply-To: <41B39EA4.9030105@ocf.berkeley.edu>
References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>	<200412060243.01129.anthony@interlink.com.au>
	<41B34B62.7040201@v.loewis.de> <41B39EA4.9030105@ocf.berkeley.edu>
Message-ID: <41B4CF07.4050108@v.loewis.de>

Brett C. wrote:
> And as for the mention of dropping PEP 4, I agree with the running 
> consensus that it isn't needed thanks to the warning module.  Do we need 
> to have a more formal vote to drop PEP 4, or should one the PEP 
> maintainers just delete it?

I would do what Barry suggests: rephrase the module to document the
deprecation/removal procedure. This, of course, needs
consensus/pronouncement, too, but I think I would put the following
aspects into it:

Requirements
============
Removal of module needs proper advance warning for users; the
module should be removed only if
- continued usage poses a security threat to the application
   using it, or
- there are no reported usages of the module anymore for quite
   some time
- the module is unmaintained (i.e. there are serious outstanding
   unfixed bugs for it), and there are alternatives which are
   maintained.
There must be a way for users to stop removal of the module,
e.g. by volunteering to maintain an unmaintained module.

Deprecation
===========
If a module is candidate for removal, it must be deprecated first.
In order to deprecate the module
1. a warning must be added to the module, indicating the expected
    version when the module is removed, and a reference to the PEP
2. a comment must be added just above the warning, giving a
    the date and version of deprecation, and a
    rationale for removal (e.g.: no known users, is security risk)
3. the module documentation must be removed from the Python
    documentation (alternatively: moved into a "deprecated" section)

Undeprecation
=============
A module can be undeprecated if the deprecation reasons turns out
to be invalid, e.g. if users of the module appear when the module
was believed to have no users, or if a maintainer shows up for a
previously unmaintained module. This fact must be recorded in the
module; the original state (documentation, no warning) be restored.

Removal
=======
If the module has been deprecated for atleast a year and atleast
a version, it can be removed. Removal should move it to old-libs
for pure Python modules; a removal procedure for pure C modules
has not been defined yet.

If the module was deprecated because it was unmaintained, the module
should only be removed if there have been no recent reports about
usage of the module.

Bug reports against deprecated modules
======================================
If a bug is reported against a deprecated module, the bug report
can be closed without further consideration, pointing to the
deprecation status. If a patch is submitted against the module,
it should be considered whether the patch could undeprecate
the module; if not, it can be rejected with the same rationale.

If this policy can be agreed, I will replace PEP4 with it.

Modules that have currently deprecation messages in them often
fail to identify the Python version in which removal will occur.
For modules that have been deprecated since 2.1, I would suggest
to remove them for 2.5, with the option of bringing them back
in 2.5.1 if people complain.

Regards,
Martin
From martin at v.loewis.de  Mon Dec  6 22:31:30 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Dec  6 22:31:23 2004
Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling?
In-Reply-To: <41B3A00A.30500@ocf.berkeley.edu>
References: <41B2BE2E.3000907@ocf.berkeley.edu> <41B2C991.4090502@v.loewis.de>
	<41B3A00A.30500@ocf.berkeley.edu>
Message-ID: <41B4CFB2.6080907@v.loewis.de>

Brett C. wrote:
> I realize that much.  But if you look in configure.in it seems to use 
> the previous value of LDFLAGS every time it is redefined as a base and 
> thus gets its initial value from the environment variable the first time 
> it is tweaked.

Yes, it would be possible to do the same thing for CPPFLAGS. However,
you really need to combine the values. If you assume you don't know
anything about the current value of CPPFLAGS, this might get tricky -
you might cause build failures if you honor CPPFLAGS too much.

> Not a big deal, though.  I will just use the environment variables in 
> setup.py .

You shouldn't do this - you do need to consider the values in the
Makefile as well. If you think you need both, you should modify
configure.in appropriately.

Regards,
Martin
From martin at v.loewis.de  Mon Dec  6 22:39:30 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Dec  6 22:39:25 2004
Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling?
In-Reply-To: <41B4C1C2.7060207@ocf.berkeley.edu>
References: <41B2BE2E.3000907@ocf.berkeley.edu> <41B2C991.4090502@v.loewis.de>
	<41B4C1C2.7060207@ocf.berkeley.edu>
Message-ID: <41B4D192.1040307@v.loewis.de>

Brett C. wrote:
>> How did you notice that? For LDFLAGS, Makefile.pre.in has
>>
>> LDFLAGS=        @LDFLAGS@
>>
>> This does *not* mean that the value from the environment is used.
>> Instead, it means that configure computes the value of LDFLAGS
>> when it generates Makefile.in. For CPPFLAGS, configure has nothing
>> to compute, so Makefile.pre.in just has the static value for
>> CPPFLAGS.
>>
> 
> I am not so sure that is true.  Checking configure.in, there is no 
> mention of CPPFLAGS anywhere.

Right. That's what I meant when I said "has nothing to compute", so
it does not even mention CPPFLAGS.

> And yet if I modify the definition of 
> CPPFLAGS in Makefile.pre.in to ``-I. -I./Include @CPPFLAGS@`` it ends up 
> containing the value I have for the environment variable at the end of 
> it.  I think the '@@' syntax uses a value from configure.in if it is 
> defined else it defaults to the value the shell has.

Indeed, that seems to be the case. However, absence of @CPPFLAGS@
means that Makefile.pre will just use the static value from
Makefile.pre.in.

Whether or not adding @CPPFLAGS@ to the end is the right thing,
I don't know.

Regards,
Martin
From barry at python.org  Mon Dec  6 22:54:52 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Dec  6 22:55:04 2004
Subject: [Python-Dev] Rewriting PEP4
In-Reply-To: <41B4CF07.4050108@v.loewis.de>
References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>
	<200412060243.01129.anthony@interlink.com.au>
	<41B34B62.7040201@v.loewis.de> <41B39EA4.9030105@ocf.berkeley.edu>
	<41B4CF07.4050108@v.loewis.de>
Message-ID: <1102370092.9565.30.camel@geddy.wooz.org>

On Mon, 2004-12-06 at 16:28, "Martin v. L?wis" wrote:

Martin, +1 on everything you wrote, with one minor quibble.

> Removal
> =======
> If the module has been deprecated for atleast a year and atleast
> a version, it can be removed. Removal should move it to old-libs
> for pure Python modules; a removal procedure for pure C modules
> has not been defined yet.

I wonder if the one year/one version rule is too short.  Given that new
versions come about every 18 months, I'd probably give it a 2 year/one
version limit.

> Modules that have currently deprecation messages in them often
> fail to identify the Python version in which removal will occur.
> For modules that have been deprecated since 2.1, I would suggest
> to remove them for 2.5, with the option of bringing them back
> in 2.5.1 if people complain

+1 here too.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20041206/54057bcd/attachment.pgp
From eirik.mikkelsen at unix.net  Mon Dec  6 23:58:40 2004
From: eirik.mikkelsen at unix.net (Eirik Mikkelsen)
Date: Mon Dec  6 23:58:42 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <41B34B62.7040201@v.loewis.de>
References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>
	<200412060243.01129.anthony@interlink.com.au>
	<41B34B62.7040201@v.loewis.de>
Message-ID: <1102373920.29451.4.camel@localhost>

On Sun, 2004-12-05 at 18:54 +0100, "Martin v. L?wis" wrote:

> I think it would be better to *remove* the xmllib documentation.
> Existing code which needs the module will continue to work even
> without documentation, and new code is unlikely to be written for
> a module that has no documentation, and where importing the module
> gives a DeprecationWarning.

Not sure I'm a registered voter, but I'll chip in with a
+1

If someone needs the documentation for xmllib it will still be available
in the older versions.

Eirik.

From janssen at parc.com  Tue Dec  7 01:25:07 2004
From: janssen at parc.com (Bill Janssen)
Date: Tue Dec  7 01:25:23 2004
Subject: [Python-Dev] Deprecated xmllib module 
In-Reply-To: Your message of "Sun, 05 Dec 2004 02:40:16 PST."
	<000701c4dab6$cf5e8e00$3b01a044@oemcomputer> 
Message-ID: <04Dec6.162511pst."58617"@synergy1.parc.xerox.com>

> * The average quality of the library improves as we take out junk (the
> tzparse module for example) and put in high quality modules like
> logging, csv, decimal, etc.

Yes and no.  The added modules have to be relevant to what users want
to do.  While (relatively) minor stuff like csv and decimal are good
ideas, of course, logging is kind of an "insider's" module.

What many more users want, however, are things like an XML parser, or
a CSS parser, or a usable HTTP server, or...  The fact that XML 1.1
can't be parsed with already-written Python is a *big* strike against.

So removing highly demanded functionality like XML parsing, and adding
specialist modules such as csv, do not overall help what users see as
the quality of the standard library.

Bill
From t-meyer at ihug.co.nz  Tue Dec  7 01:35:16 2004
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Tue Dec  7 01:36:03 2004
Subject: [Python-Dev] Deprecated xmllib module 
In-Reply-To: <ECBA357DDED63B4995F5C1F5CBE5B1E8019B0136@its-xchg4.massey.ac.nz>
Message-ID: <ECBA357DDED63B4995F5C1F5CBE5B1E86C3854@its-xchg4.massey.ac.nz>

>> * The average quality of the library improves as we take 
>> out junk (the tzparse module for example) and put in high
>> quality modules like logging, csv, decimal, etc.
> 
> Yes and no.  The added modules have to be relevant to what 
> users want to do.  While (relatively) minor stuff like csv 
> and decimal are good ideas, of course, logging is kind of an 
> "insider's" module.

What do you mean by "insiders"?  The logging module is great (ok, it could
be simpler to use in some cases) for any Python programmer.

> What many more users want, however, are things like an XML 
> parser, or a CSS parser, or a usable HTTP server, or...

Statements like this are pretty common, but there's no evidence (that I've
ever seen pointed to) that someone has *measured* how many people want
modules for X.  People who work with HTML at lot are probably keen on those
things you list, yes.  OTOH, other people (e.g. me) have no use for any of
those, but use CSV and logging daily.  Others want something completely
different.

There's quite a difference between quality and relevance.  It's certainly
worthwhile to ensure that all the standard library modules are as of high a
quality as possible (e.g. email > rfc822).  You'll never be able to get
everyone to agree on the same set of modules that are relevant.

If there are that many people that want (e.g.) a CSS parser, wouldn't there
be a 3rd party one that everyone is using that could be proposed for
addition into the standard library?

=Tony.Meyer

From python at rcn.com  Tue Dec  7 01:43:07 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Dec  7 01:46:02 2004
Subject: [Python-Dev] Rewriting PEP4
In-Reply-To: <41B4CF07.4050108@v.loewis.de>
Message-ID: <002401c4dbf5$b847cc00$f301a044@oemcomputer>

> I would do what Barry suggests: rephrase the module to document the
> deprecation/removal procedure. This, of course, needs
> consensus/pronouncement, too, but I think I would put the following
> aspects into it:
> 
> Requirements
> ============
> Removal of module needs proper advance warning for users; the
> module should be removed only if
> - continued usage poses a security threat to the application
>    using it, or
> - there are no reported usages of the module anymore for quite
>    some time
> - the module is unmaintained (i.e. there are serious outstanding
>    unfixed bugs for it), and there are alternatives which are
>    maintained.
> There must be a way for users to stop removal of the module,
> e.g. by volunteering to maintain an unmaintained module.

The effect of coding this into the PEP is to make deprecation
unnecessarily involved.  Also, the list of reasons is insufficiently
comprehensive.  In the past we've gotten along fine with discussing a
situation on python-dev and either reaching a consensus or having Guido
give an okay.  



> 2. a comment must be added just above the warning, giving a
>     the date and version of deprecation, and a
>     rationale for removal (e.g.: no known users, is security risk)

We should only talk in terms of versions.  Dates are irrelevant and
confusing.  In the last version of the PEP, it was far from clear the
significane of deprecating or removing on date xx-xx-xx.  All that
matters is the version the warning first appears and the version where
the module is removed.  


> Modules that have currently deprecation messages in them often
> fail to identify the Python version in which removal will occur.
> For modules that have been deprecated since 2.1, I would suggest
> to remove them for 2.5, with the option of bringing them back
> in 2.5.1 if people complain.

I do not think the version number messages are useful.  When someone
imports a file and gets a warning, they know a deprecation is pending
and to stop writing code with that module.  Further, I want to avoid the
previous PEP 4 situation where one nit or another wasn't followed to the
letter so we had to keep the module around for another two versions.  It
can and should be as simple as having us agree to deprecation, inserting
a warning, and then removing it two versions later.

Also, we need to shy away from the idea of having 2.5.1 with different
capabilities than 2.5.  This sort of thing is a portability disaster.
Anthony, please speak up.


General comments:

I had thought the purpose of rewording the PEP was just to take out
language naming specific modules and to add language on using the
warning module.  Instead, it looks like an attempt to make it much more
difficult and slow to deprecate a module.  

If so, I would like to understand what situation is motivating it.  Is
there some module that had been deprecated but should not have been?
Why the change in procedure?

It doesn't affect me personally (I haven't proposed any modules for
deprecation and do not have any in mind); however, I think the
discussion should take place in the broader context of what we intend to
do with the standard library -- do we want regex still there in 2010.

Also, as proposed, the PEP lists many forces against deprecation and
none of the forces for it.   If that is the reality, then we could just
never deprecate anything because someone, somewhere is bound to be
irritated by it.  That would certainly simplify the discussion.

Going back to specifics, it may be worthwhile to think through the
reason we kept the xmllib code but not whrandom.  Both were documented,
non-buggy, tested, marked as deprecated for a long time, and it was
within the realm of possibility that some code still used them.

Also, the PEP should discuss the use of the lib-old directory and it
should include Barry's link to old documentation.


Raymond

From bac at OCF.Berkeley.EDU  Tue Dec  7 01:52:08 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Dec  7 01:52:15 2004
Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling?
In-Reply-To: <41B4D192.1040307@v.loewis.de>
References: <41B2BE2E.3000907@ocf.berkeley.edu> <41B2C991.4090502@v.loewis.de>
	<41B4C1C2.7060207@ocf.berkeley.edu> <41B4D192.1040307@v.loewis.de>
Message-ID: <41B4FEB8.90906@ocf.berkeley.edu>

Martin v. L?wis wrote:
> Brett C. wrote:
> 
>>> How did you notice that? For LDFLAGS, Makefile.pre.in has
>>>
>>> LDFLAGS=        @LDFLAGS@
>>>
>>> This does *not* mean that the value from the environment is used.
>>> Instead, it means that configure computes the value of LDFLAGS
>>> when it generates Makefile.in. For CPPFLAGS, configure has nothing
>>> to compute, so Makefile.pre.in just has the static value for
>>> CPPFLAGS.
>>>
>>
>> I am not so sure that is true.  Checking configure.in, there is no 
>> mention of CPPFLAGS anywhere.
> 
> 
> Right. That's what I meant when I said "has nothing to compute", so
> it does not even mention CPPFLAGS.
> 
>> And yet if I modify the definition of CPPFLAGS in Makefile.pre.in to 
>> ``-I. -I./Include @CPPFLAGS@`` it ends up containing the value I have 
>> for the environment variable at the end of it.  I think the '@@' 
>> syntax uses a value from configure.in if it is defined else it 
>> defaults to the value the shell has.
> 
> 
> Indeed, that seems to be the case. However, absence of @CPPFLAGS@
> means that Makefile.pre will just use the static value from
> Makefile.pre.in.
> 

That's basically the functionality I need, so I am going with it.

> Whether or not adding @CPPFLAGS@ to the end is the right thing,
> I don't know.
> 

Well, we will soon find out.  =)  My checkin made this change and everything 
seems fine.  If it doesn't work out I will have to create another environment 
variable to store the value.

Michael's desire of getting the Fink and DarwinPorts special casing in setup.py 
is now solved; setup.py now uses the directories specified in LDFLAGS and 
CPPFLAGS for compiling the extension modules.  I didn't bother with CFLAGS or 
CC since the former is mostly handled by BASECFLAGS it seems and the latter is 
specified by arguments to configure.

-Brett
From python at rcn.com  Tue Dec  7 01:53:05 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Dec  7 01:55:58 2004
Subject: [Python-Dev] Rewriting PEP4
In-Reply-To: <002401c4dbf5$b847cc00$f301a044@oemcomputer>
Message-ID: <002801c4dbf7$1ce6a860$f301a044@oemcomputer>


One other thought:  In deciding how long to leave module in, we should
consider that Python books are updated infrequently, if at all.  It
would be a bummer if code in them stopped working as advertised.



Raymond

From anthony at interlink.com.au  Tue Dec  7 02:31:13 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Dec  7 02:31:29 2004
Subject: [Python-Dev] Rewriting PEP4
In-Reply-To: <002401c4dbf5$b847cc00$f301a044@oemcomputer>
References: <002401c4dbf5$b847cc00$f301a044@oemcomputer>
Message-ID: <200412071231.13878.anthony@interlink.com.au>

On Tuesday 07 December 2004 11:43, Raymond Hettinger wrote:
> > Modules that have currently deprecation messages in them often
> > fail to identify the Python version in which removal will occur.
> > For modules that have been deprecated since 2.1, I would suggest
> > to remove them for 2.5, with the option of bringing them back
> > in 2.5.1 if people complain.
>
> [...]
>
> Also, we need to shy away from the idea of having 2.5.1 with different
> capabilities than 2.5.  This sort of thing is a portability disaster.
> Anthony, please speak up.

Sorry - been a bit busy the last few days. Raymond is right here - there's
no way we can or should do this. A bugfix release contains _bugfixes_.
Making new modules available in a bugfix is a sucky sucky approach.
Worse - once we remove a module, it should _stay_ removed. Otherwise,
we're going to end up with crap like:

     if sys.version < (2,5) or sys.version >= (2,6):
          import froggie
     else:
          # froggie was removed in 2.5 and reinstated in 2.6
          from compat import froggie

and people will be shipping their own versions of the code to get around
our misfeature.

-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From janssen at parc.com  Tue Dec  7 02:49:40 2004
From: janssen at parc.com (Bill Janssen)
Date: Tue Dec  7 02:50:15 2004
Subject: [Python-Dev] Deprecated xmllib module 
In-Reply-To: Your message of "Mon, 06 Dec 2004 16:35:16 PST."
	<ECBA357DDED63B4995F5C1F5CBE5B1E86C3854@its-xchg4.massey.ac.nz> 
Message-ID: <04Dec6.174948pst."58617"@synergy1.parc.xerox.com>

> Statements like this are pretty common, but there's no evidence (that I've
> ever seen pointed to) that someone has *measured* how many people want
> modules for X.

I almost didn't send this in, because I figured someone would have to
argue with it.

> If there are that many people that want (e.g.) a CSS parser, wouldn't there
> be a 3rd party one that everyone is using that could be proposed for
> addition into the standard library?

As far as I can tell, there are no CSS or XML 1.1 parsers for Python, period.

Bill
From t-meyer at ihug.co.nz  Tue Dec  7 03:04:49 2004
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Tue Dec  7 03:05:23 2004
Subject: [Python-Dev] Deprecated xmllib module 
In-Reply-To: <ECBA357DDED63B4995F5C1F5CBE5B1E8019B0164@its-xchg4.massey.ac.nz>
Message-ID: <ECBA357DDED63B4995F5C1F5CBE5B1E86C3858@its-xchg4.massey.ac.nz>

> As far as I can tell, there are no CSS or XML 1.1 parsers for 
> Python, period.

This belongs on c.l.p, I suppose, but the first page of google results
includes:

<http://www.python.org/pypi?:action=display&name=TG%20CSS%20Tools&version=1.
0a1>
<http://cthedot.de/cssutils/>

=Tony.Meyer

From sjoerd at acm.org  Mon Dec  6 21:47:37 2004
From: sjoerd at acm.org (Sjoerd Mullender)
Date: Tue Dec  7 03:19:07 2004
Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling?
In-Reply-To: <41B4C1C2.7060207@ocf.berkeley.edu>
References: <41B2BE2E.3000907@ocf.berkeley.edu> <41B2C991.4090502@v.loewis.de>
	<41B4C1C2.7060207@ocf.berkeley.edu>
Message-ID: <41B4C569.8060608@acm.org>

Brett C. wrote:
> Martin v. L?wis wrote:
> 
>> Brett C. wrote:
>>
>>> I noticed that Makefile.pre.in uses the value from the environment 
>>> variable LDFLAGS but not CPPFLAGS.  Any reason for this? 
>>
>>
>>
>> How did you notice that? For LDFLAGS, Makefile.pre.in has
>>
>> LDFLAGS=        @LDFLAGS@
>>
>> This does *not* mean that the value from the environment is used.
>> Instead, it means that configure computes the value of LDFLAGS
>> when it generates Makefile.in. For CPPFLAGS, configure has nothing
>> to compute, so Makefile.pre.in just has the static value for
>> CPPFLAGS.
>>
> 
> I am not so sure that is true.  Checking configure.in, there is no 
> mention of CPPFLAGS anywhere.  And yet if I modify the definition of 
> CPPFLAGS in Makefile.pre.in to ``-I. -I./Include @CPPFLAGS@`` it ends up 
> containing the value I have for the environment variable at the end of 
> it.  I think the '@@' syntax uses a value from configure.in if it is 
> defined else it defaults to the value the shell has.

It's autoconf that deals with these flags.  See the output of "configure 
--help".


-- 
Sjoerd Mullender <sjoerd@acm.org>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 374 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20041206/bfe3b473/signature.pgp
From kbk at shore.net  Tue Dec  7 05:33:45 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Tue Dec  7 05:35:36 2004
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <200412070433.iB74XjGf019407@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  259 open ( +1) /  2705 closed ( +4) /  2964 total ( +5)
Bugs    :  800 open (-12) /  4662 closed (+20) /  5462 total ( +8)
RFE     :  160 open ( +0) /   137 closed ( +1) /   297 total ( +1)

New / Reopened Patches
______________________

add key  (2004-12-01)
CLOSED http://python.org/sf/1077353  opened by  Steven Bethard

Simple webbrowser fix for netscape -remote  (2004-12-02)
       http://python.org/sf/1077979  opened by  Josh Cherry

readline does not need termcap  (2004-12-01)
       http://python.org/sf/1076826  reopened by  nijel

Make cgi.py use logging module  (2004-12-05)
       http://python.org/sf/1079729  opened by  Josh Hoyt

Make cgi.py use email instead of rfc822 or mimetools  (2004-12-05)
       http://python.org/sf/1079734  opened by  Josh Hoyt

list sort is not "in place"  (2004-12-06)
       http://python.org/sf/1080078  reopened by  rhettinger

list sort is not "in place"  (2004-12-06)
CLOSED http://python.org/sf/1080078  opened by  Heikki Orsila

Patches Closed
______________

add key  (2004-12-02)
       http://python.org/sf/1077353  closed by  rhettinger

readline does not need termcap  (2004-12-01)
       http://python.org/sf/1076826  closed by  loewis

raise error for common mistake with subprocess  (2004-11-23)
       http://python.org/sf/1071755  closed by  astrand

list sort is not "in place"  (2004-12-06)
       http://python.org/sf/1080078  closed by  rhettinger

list sort is not "in place"  (2004-12-06)
       http://python.org/sf/1080078  closed by  tim_one

AUTH PLAIN in smtplib  (2004-11-30)
       http://python.org/sf/1075928  closed by  jlgijsbers

New / Reopened Bugs
___________________

problem with make test on OS/X  (2004-12-02)
CLOSED http://python.org/sf/1077302  opened by  Ian Holsman

threads: segfault or Py_FatalError at exit  (2004-11-08)
       http://python.org/sf/1061968  reopened by  mhammond

assert fail to threw exception when run python with '-O'  (2004-12-02)
CLOSED http://python.org/sf/1077862  opened by  tj128

Python2.4: building '_socket' extension fails with `INET_ADD  (2004-12-03)
       http://python.org/sf/1078245  opened by  Michael Str?der

Docs for run() methods are misleading  (2004-12-04)
CLOSED http://python.org/sf/1078905  opened by  Kent Johnson

Email.Header encodes non-ASCII content incorrectly  (2004-12-04)
       http://python.org/sf/1078919  opened by  Tessa Lau

Incorrect error message (somewhat)  (2004-12-04)
       http://python.org/sf/1079011  opened by  Gerrit Holl

datetime changes missing from "Porting from 2.3 to 2.4"  (2004-12-04)
       http://python.org/sf/1079134  opened by  Sadruddin Rejeb

python-2.4.msi install error  (2004-12-05)
       http://python.org/sf/1079545  opened by  maharal

IDLE and old color themes  (2004-12-06)
       http://python.org/sf/1080387  opened by  projecktzero

Bugs Closed
___________

threads: segfault or Py_FatalError at exit  (2004-11-07)
       http://python.org/sf/1061968  closed by  arigo

Tutorial corrections  (2004-12-01)
       http://python.org/sf/1076955  closed by  rhettinger

test_shutil fails on x86-64 // Suse 9.1  (2004-12-01)
       http://python.org/sf/1076467  closed by  jlgijsbers

test test_unicodedata failed   (2004-12-01)
       http://python.org/sf/1076795  closed by  lemburg

Problem testing python 2.4  (2004-12-01)
       http://python.org/sf/1077103  closed by  lemburg

re module segfaulting in large regular expression  (2004-11-24)
       http://python.org/sf/1072259  closed by  niemeyer

Memory fault pyexpat.so on SGI  (2004-11-30)
       http://python.org/sf/1075990  closed by  akuchling

assert fail to threw exception when run python with '-O'  (2004-12-02)
       http://python.org/sf/1077862  closed by  tim_one

Errors and omissions in logging module documentation  (2004-11-28)
       http://python.org/sf/1074693  closed by  vsajip

xml.dom.minidom produces errors with certain unicode chars  (2004-11-27)
       http://python.org/sf/1074200  closed by  effbot

test test_re produced unexpected output  (2004-12-01)
       http://python.org/sf/1076791  closed by  nijel

^Z doesn't exit interpreter - 2.4c1 & Win2K  (2004-11-26)
       http://python.org/sf/1073736  closed by  facundobatista

test_macostools failure on OS X  (2004-12-01)
       http://python.org/sf/1077302  closed by  bcannon

PyGILState_Ensure() deadlocks (ver 2.4)  (2004-11-29)
       http://python.org/sf/1075703  closed by  andivajda

calendar.weekheader() undocumented  (2004-05-04)
       http://python.org/sf/947894  closed by  doerwalter

Docs for unittest run() methods are misleading  (2004-12-04)
       http://python.org/sf/1078905  closed by  rhettinger

Another message that croaks email.FeedParser  (2004-11-30)
       http://python.org/sf/1076485  closed by  bwarsaw

Windows msi doesn't install site-packages directory  (2004-11-23)
       http://python.org/sf/1071594  closed by  loewis

win32con missing codes VK_VOLUME_MUTE, VK_BROWSER_BACK, ...  (2004-11-16)
       http://python.org/sf/1067153  closed by  loewis

test_sutil fails on cygwin  (2004-11-23)
       http://python.org/sf/1071513  closed by  jlgijsbers

CFLAGS, CPPFLAGS, LDFLAGS are NOT used when building modules  (2004-05-17)
       http://python.org/sf/955197  closed by  bcannon

RFE Closed
__________

Python Interpreter embedded in small memory system  (2004-11-30)
       http://python.org/sf/1076478  closed by  rhettinger

From martin at v.loewis.de  Tue Dec  7 08:08:46 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Dec  7 08:08:39 2004
Subject: [Python-Dev] Rewriting PEP4
In-Reply-To: <002401c4dbf5$b847cc00$f301a044@oemcomputer>
References: <002401c4dbf5$b847cc00$f301a044@oemcomputer>
Message-ID: <41B556FE.20504@v.loewis.de>

Raymond Hettinger wrote:
> The effect of coding this into the PEP is to make deprecation
> unnecessarily involved.  

Can you please explain why you consider this involvement
unnecessary? I think it is important that we do not prematurely
remove (or even deprecate) modules that are still being actively
used.

 > Also, the list of reasons is insufficiently
> comprehensive.  In the past we've gotten along fine with discussing a
> situation on python-dev and either reaching a consensus or having Guido
> give an okay.  

I don't think we "got along", but rather "got away". Discussion on
python-dev does not cause sufficient user involvement, IMO. And indeed,
some people (even on python-dev) think that we *never* should deprecate
any modules, as it breaks existing code. They might not be as vocal as
supporters of early removal of "ugly" code, but I think their opinion
is just as valuable.

> We should only talk in terms of versions.  Dates are irrelevant and
> confusing.

Why is a date confusing? I think everybody on Earth understands the
notion of a date, there is nothing confusing in this notion.

Why are they irrelevant? If we release Python 2.5 next month (and
stop developing 2.4), are we entitled to remove all features that
were scheduled for removal? I think not: users would have not gotten
sufficient advance warning. Users need *time* to react on proposed
feature changes.

 > In the last version of the PEP, it was far from clear the
> significane of deprecating or removing on date xx-xx-xx.  All that
> matters is the version the warning first appears and the version where
> the module is removed.  

No. It also matters how much time users have had to react.

> I do not think the version number messages are useful.  When someone
> imports a file and gets a warning, they know a deprecation is pending
> and to stop writing code with that module.

Yes, but what about code that currently uses the module? People will
not start rewriting (perhaps they aren't the authors in the first
place) just because they see a deprecation warning. Instead, they will
silence the warning, and go on. If we plan to remove the feature at
some point, we need to give these users an indication that they need
to act beyond silencing the warning. Such actions could be
- have the author of the software update it appropriately
- look for a replacement software
- ask for an extension of the grace period on python-dev

Currently, users do not really have these options, since they do not
have an indication that action on their side is really needed if they
want their software continue to work with future Python versions.

> Further, I want to avoid the
> previous PEP 4 situation where one nit or another wasn't followed to the
> letter so we had to keep the module around for another two versions.

Why do you want to avoid that situation? What is the problem with
waiting for two more versions? No harm is done in doing so.

 > It
> can and should be as simple as having us agree to deprecation, inserting
> a warning, and then removing it two versions later.

I strongly disagree with that position.

> I had thought the purpose of rewording the PEP was just to take out
> language naming specific modules and to add language on using the
> warning module.  Instead, it looks like an attempt to make it much more
> difficult and slow to deprecate a module.  

I want the PEP to be followed. For that, it needs to be precise, and
the procedures need to be agreed upon (or, if we cannot agree, it
needs to be Pronounced).

> If so, I would like to understand what situation is motivating it.  Is
> there some module that had been deprecated but should not have been?

Deprecation is fine for all the modules so far. Generating the warning
is a bit of a problem, removal would be a serious problem, for the
following modules:
- regex
- xmllib
- whrandom
since I expect that these modules are still widely used, in existing
code, by users who are not the authors of that code. This is a problem,
since these users cannot do anything about the usage of the deprecated
code. They just find that the software that ran happily with the
previous Python version first shouts at them in the next Python version,
then totally breaks with the subsequent version. They will hate Python
for that.

> Why the change in procedure?

I think we agree that the previously-documented procedure needs to
be changed - it was not followed.

> It doesn't affect me personally (I haven't proposed any modules for
> deprecation and do not have any in mind); however, I think the
> discussion should take place in the broader context of what we intend to
> do with the standard library -- do we want regex still there in 2010.

As to the latter question: Definitely. We don't want regex to be used
for new code, and we want developers to fully understand that, but we
also do want code that currently works correctly with regex continue
to do so in 2010. The only exception would be that 2010 no code is left
in active use that uses regex, which I doubt.

> Also, as proposed, the PEP lists many forces against deprecation and
> none of the forces for it.

That is not true. Security flaws are giving priority. They can only
be undeprecated if the reason for deprecation goes away, which means
that somebody fixes the security flaw.

> If that is the reality, then we could just
> never deprecate anything because someone, somewhere is bound to be
> irritated by it.  That would certainly simplify the discussion.

Well, we do want to move developers to the new direction. However,
many people use Python code who are *not* developers. We need to
take their concerns into account as well.

> Going back to specifics, it may be worthwhile to think through the
> reason we kept the xmllib code but not whrandom.  Both were documented,
> non-buggy, tested, marked as deprecated for a long time, and it was
> within the realm of possibility that some code still used them.

Indeed. If it is in the realm of possibility that code still uses a
module, and that this would not raise security concerns, the module
should not be removed.

> Also, the PEP should discuss the use of the lib-old directory and it
> should include Barry's link to old documentation.

I'm happy to adjust details of the procedures - but it seems we disagree
on the principles.

Regards,
Martin
From martin at v.loewis.de  Tue Dec  7 08:12:21 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Dec  7 08:12:14 2004
Subject: [Python-Dev] Rewriting PEP4
In-Reply-To: <002801c4dbf7$1ce6a860$f301a044@oemcomputer>
References: <002801c4dbf7$1ce6a860$f301a044@oemcomputer>
Message-ID: <41B557D5.2080700@v.loewis.de>

Raymond Hettinger wrote:
> One other thought:  In deciding how long to leave module in, we should
> consider that Python books are updated infrequently, if at all.  It
> would be a bummer if code in them stopped working as advertised.

Correct. Thanks for reminding me - that is indeed a reasoning that
is typically brought up by book authors, so I should have brought
it up myself :-)

Of course, developers would first see the deprecation warning,
which would tell them that their books are outdated. However, if
the code is removed, they get an import error, which leaves them
guessing what the problem might have been.

Regards,
Martin
From python at rcn.com  Tue Dec  7 08:50:55 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Dec  7 08:53:39 2004
Subject: [Python-Dev] Rewriting PEP4
In-Reply-To: <41B556FE.20504@v.loewis.de>
Message-ID: <000701c4dc31$7bf6da20$e841fea9@oemcomputer>

> I'm happy to adjust details of the procedures - but it seems we
disagree
> on the principles.

Not really.  I have no objection to making module deprecation/removal
rare or even not doing it at all.  Personally, I've never asked for a
module deprecation and don't expect to.  

I also agree that more public participation is a wise step.

I would just like to see a clean, actionable PEP.  To me, the draft text
outlined a timid, faltering process that would be hard to follow, prone
to reversal, and accomplish little.

I especially find burdensome the obligation to undo a deprecation the
moment some random user sends a grumpy email.

Instead, there should be a clear decision to deprecate or not.  If that
entails a comp.lang.python.announce notice and comment period, so be it.
Once a decision is made, document it, add a warning, and wait.

Once a couple versions pass, some useful action needs to take place.  If
the code is left in-place and the doc index is just re-arranged, then we
would have been better off not doing anything at all.  

Ideally, the PEP should also provide some informational value.  It
should list Barry's link as a reference for old docs.  It should
describe the appropriate use of lib-old (like whrandom) vs. renaming a
module (like stringold) vs. leaving it in-place (like xmllib) vs.
removing the module 

The questions of dates was somewhat minor.  I was unclear on the
implication of, for example, "remove on 15Jan2005".  Since Py2.5 won't
go out for at least a year, does that mean that the work can't be done
now while I have time instead of later when I don't.  The only time the
removal becomes visible is when a new version goes out the door.
Further, if a version is going to have something removed, we should do
it at the outset rather than just before a release goes out the door (to
allow for early surfacing of any issues).



> > Further, I want to avoid the
> > previous PEP 4 situation where one nit or another wasn't followed to
the
> > letter so we had to keep the module around for another two versions.
> 
> Why do you want to avoid that situation? What is the problem with
> waiting for two more versions? No harm is done in doing so.

If we really don't care whether it gets done, then we shouldn't bother
in the first place.



Raymond

From python at rcn.com  Tue Dec  7 09:12:20 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Dec  7 09:15:03 2004
Subject: [Python-Dev] Rewriting PEP4
In-Reply-To: <000701c4dc31$7bf6da20$e841fea9@oemcomputer>
Message-ID: <000801c4dc34$7949b420$e841fea9@oemcomputer>

There is one other issue that should get addressed, modules in limbo.

I'm concerned about the old email modules.  They haven't been deprecated
and may stay around for a good while.  However, some SF requests have
been passed over on the basis that the modules are supplanted and will
be phased out.  I don't think it is good to leave active modules as
orphans.


Raymond

From martin at v.loewis.de  Tue Dec  7 09:16:05 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Dec  7 09:15:58 2004
Subject: [Python-Dev] Rewriting PEP4
In-Reply-To: <000701c4dc31$7bf6da20$e841fea9@oemcomputer>
References: <000701c4dc31$7bf6da20$e841fea9@oemcomputer>
Message-ID: <41B566C5.2040102@v.loewis.de>

Raymond Hettinger wrote:
> Instead, there should be a clear decision to deprecate or not.  If that
> entails a comp.lang.python.announce notice and comment period, so be it.
> Once a decision is made, document it, add a warning, and wait.

Ok, that might be reasonable.

> Once a couple versions pass, some useful action needs to take place.  If
> the code is left in-place and the doc index is just re-arranged, then we
> would have been better off not doing anything at all.  

I disagree. The primary purpose (move developers to the alternative
approach) is achieved as soon as the warning is added, and the
documentation is deleted. Whether the module is actually deleted is
of minor importance: it costs nothing to continue including it; disk
space is cheap.

> The questions of dates was somewhat minor.  I was unclear on the
> implication of, for example, "remove on 15Jan2005".  Since Py2.5 won't
> go out for at least a year, does that mean that the work can't be done
> now while I have time instead of later when I don't.  The only time the
> removal becomes visible is when a new version goes out the door.

You could remove it now, but if we release Py2.5 in two months, you
would have to put it back in. So if you don't have time then, maybe
somebody else will. If nobody has time to remove the module, the next
release will ship with it, again - no big issue.

> Further, if a version is going to have something removed, we should do
> it at the outset rather than just before a release goes out the door (to
> allow for early surfacing of any issues).

That is true.

> If we really don't care whether it gets done, then we shouldn't bother
> in the first place.

What do you mean by "bother"? Not put the deprecation warning in? We
do want users to move to the new approach of doing something. However,
two version are just not enough - it may take 10 or 20 years to remove
a widely used feature (e.g. the string module). That it will take so
long does not mean we should not start the process *now* - if we start
the process in five years, it will *still* take 10 or 20 years. Just
be patient.

Regards,
Martin
From python at rcn.com  Tue Dec  7 09:35:57 2004
From: python at rcn.com (Raymond Hettinger)
Date: Tue Dec  7 09:38:42 2004
Subject: [Python-Dev] Rewriting PEP4
In-Reply-To: <41B566C5.2040102@v.loewis.de>
Message-ID: <000e01c4dc37$c673b400$e841fea9@oemcomputer>

> > Instead, there should be a clear decision to deprecate or not.  If
that
> > entails a comp.lang.python.announce notice and comment period, so be
it.
> > Once a decision is made, document it, add a warning, and wait.
> 
> Ok, that might be reasonable.

Please word the PEP accordingly.


> > Once a couple versions pass, some useful action needs to take place.
If
> > the code is left in-place and the doc index is just re-arranged,
then we
> > would have been better off not doing anything at all.
> 
> I disagree. The primary purpose (move developers to the alternative
> approach) is achieved as soon as the warning is added, and the
> documentation is deleted. Whether the module is actually deleted is
> of minor importance: it costs nothing to continue including it; disk
> space is cheap.

That is reasonable.  Please make that goal/assumption explicit in the
PEP.



> > The questions of dates was somewhat minor.  I was unclear on the
> > implication of, for example, "remove on 15Jan2005".  Since Py2.5
won't
> > go out for at least a year, does that mean that the work can't be
done
> > now while I have time instead of later when I don't.  The only time
the
> > removal becomes visible is when a new version goes out the door.
> 
> You could remove it now, but if we release Py2.5 in two months, you
> would have to put it back in. So if you don't have time then, maybe
> somebody else will. If nobody has time to remove the module, the next
> release will ship with it, again - no big issue.

Okay.  Again, please say that in the PEP.


> > Further, if a version is going to have something removed, we should
do
> > it at the outset rather than just before a release goes out the door
(to
> > allow for early surfacing of any issues).
> 
> That is true.

That should also be recommended in the PEP.  



> > If we really don't care whether it gets done, then we shouldn't
bother
> > in the first place.
> 
> What do you mean by "bother"? Not put the deprecation warning in? We
> do want users to move to the new approach of doing something. However,
> two version are just not enough - it may take 10 or 20 years to remove
> a widely used feature (e.g. the string module). That it will take so
> long does not mean we should not start the process *now* - if we start
> the process in five years, it will *still* take 10 or 20 years. Just
> be patient.

I see.  That also may useful to include in the motivation section of the
PEP.

With these adjustments, I think the PEP will be fine.   Also, be sure
get rid of the mid-version undo (see Anthony's note) and to address the
situation with Python books.

I look forward to a revised draft.


Raymond

From ncoghlan at iinet.net.au  Tue Dec  7 09:39:06 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Tue Dec  7 09:39:12 2004
Subject: [Python-Dev] Re: long number multiplication
In-Reply-To: <t1wfz2jfn9p.fsf@kermit.wreck.org>
References: <Pine.LNX.4.44.0412050900300.12327-100000@localhost>
	<t1wfz2jfn9p.fsf@kermit.wreck.org>
Message-ID: <41B56C2A.3030609@iinet.net.au>

Christopher A. Craig wrote:
>>i needed to implement this myself and was thinking of storing the digits 
>>of an integer in a list.
> 
> 
> That's sort of what Python does except the "digits" are 15 bits,
> not base 10.  Doing it in base 10 would be a huge pain because of the
> problems with base 10->base 2 conversion.
>

Indeed. Python's Decimal data type stores the digits in a list for ease of power 
of 10 multiplication & rounding (the precision rules means a *lot* of that goes 
on). However, it converts that list of digits to a long integer in order to do 
addition, multiplication or division.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan@email.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.skystorm.net
From ncoghlan at iinet.net.au  Tue Dec  7 10:46:41 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Tue Dec  7 10:46:49 2004
Subject: [Python-Dev] Rewriting PEP4
In-Reply-To: <000e01c4dc37$c673b400$e841fea9@oemcomputer>
References: <000e01c4dc37$c673b400$e841fea9@oemcomputer>
Message-ID: <41B57C01.9050404@iinet.net.au>

One other question occurred to me for "deprecate X in favour of Y" situations - 
should the deprecation warning added to X point developers towards Y?

Also, should the PEP include example wordings for deprecation warnings, rather 
than being completely freeform? Martin listed some information that should be 
included, but it seems an example or two would make it easy to get right.

E.g.:

Unmaintained, with a maintained alternative:

"Due to lack of maintenance, this module has been deprecated in favour of module 
Y and will be removed in Python 2.6 (see PEP 4 for information on the 
deprecation process)"

Security problems, no alternative:

"Due to security concerns, this module has been deprecated and will be removed 
in Python 2.6 (see PEP 4 for information on the deprecation process)"

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan@email.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.skystorm.net
From ncoghlan at iinet.net.au  Tue Dec  7 10:51:01 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Tue Dec  7 10:51:10 2004
Subject: [Python-Dev] Re: long number multiplication
In-Reply-To: <41B56C2A.3030609@iinet.net.au>
References: <Pine.LNX.4.44.0412050900300.12327-100000@localhost>	<t1wfz2jfn9p.fsf@kermit.wreck.org>
	<41B56C2A.3030609@iinet.net.au>
Message-ID: <41B57D05.1090509@iinet.net.au>

Gah, sorry folks. I really ought to pay more attention to that send line.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan@email.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.skystorm.net
From barry at python.org  Tue Dec  7 15:06:10 2004
From: barry at python.org (Barry Warsaw)
Date: Tue Dec  7 15:06:15 2004
Subject: [Python-Dev] Rewriting PEP4
In-Reply-To: <000801c4dc34$7949b420$e841fea9@oemcomputer>
References: <000801c4dc34$7949b420$e841fea9@oemcomputer>
Message-ID: <1102428369.25008.31.camel@geddy.wooz.org>

On Tue, 2004-12-07 at 03:12, Raymond Hettinger wrote:

> I'm concerned about the old email modules.  They haven't been deprecated
> and may stay around for a good while.  However, some SF requests have
> been passed over on the basis that the modules are supplanted and will
> be phased out.  I don't think it is good to leave active modules as
> orphans.

I would like to add deprecation warnings to some of the old
email-related modules for Python 2.5, but I doubt we can remove any of
them until Python 3.0 (or 2014, whichever comes first <wink>).

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20041207/2226a04a/attachment.pgp
From fdrake at acm.org  Tue Dec  7 15:47:37 2004
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue Dec  7 15:48:02 2004
Subject: [Python-Dev] Rewriting PEP4
In-Reply-To: <41B57C01.9050404@iinet.net.au>
References: <000e01c4dc37$c673b400$e841fea9@oemcomputer>
	<41B57C01.9050404@iinet.net.au>
Message-ID: <200412070947.38173.fdrake@acm.org>

On Tuesday 07 December 2004 04:46 am, Nick Coghlan wrote:
 > One other question occurred to me for "deprecate X in favour of Y"
 > situations - should the deprecation warning added to X point developers
 > towards Y?

Not sure about the warning, but the documentation certainly should.  When the 
mapping from the old module to the new is not obvious and trivial, sufficient 
information should be included in the documentation for the old module to 
make it fairly easy to perform the conversion.  (Placing this in the docs for 
the old module ensures that it goes away when the documentation for the old 
module goes away, and the docs for the new module need never be loaded with 
information about the old module.)

 > Also, should the PEP include example wordings for deprecation warnings,
 > rather than being completely freeform? Martin listed some information that
 > should be included, but it seems an example or two would make it easy to
 > get right.

Examples would be good, though specific wording should not be proscribed.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>

From mwh at python.net  Tue Dec  7 15:54:50 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Dec  7 15:54:53 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test
 test_shutil.py, 1.10, 1.11
In-Reply-To: <E1CbPoH-0005LW-Qo@sc8-pr-cvs1.sourceforge.net>
	(jlgijsbers@users.sourceforge.net's
	message of "Mon, 06 Dec 2004 12:50:17 -0800")
References: <E1CbPoH-0005LW-Qo@sc8-pr-cvs1.sourceforge.net>
Message-ID: <2m4qiyqhtx.fsf@starship.python.net>

jlgijsbers@users.sourceforge.net writes:

> Update of /cvsroot/python/python/dist/src/Lib/test
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv20531
>
> Modified Files:
> 	test_shutil.py 
> Log Message:
> SF bug #1076467: don't run test_on_error as root, as the permission
> errors don't get provoked that way. Also add a bunch of cross-references
> to bugs.
>
>
> Index: test_shutil.py
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/Lib/test/test_shutil.py,v
> retrieving revision 1.10
> retrieving revision 1.11
> diff -u -d -r1.10 -r1.11
> --- test_shutil.py	23 Nov 2004 09:27:27 -0000	1.10
> +++ test_shutil.py	6 Dec 2004 20:50:15 -0000	1.11
> @@ -16,7 +16,10 @@
>          filename = tempfile.mktemp()
>          self.assertRaises(OSError, shutil.rmtree, filename)
>  
> -    if hasattr(os, 'chmod') and sys.platform[:6] != 'cygwin':
> +    # See bug #1071513 for why we don't run this on cygwin
> +    # and bug #1076467 for why we don't run this as root.
> +    if (hasattr(os, 'chmod') and sys.platform[:6] != 'cygwin'
> +        and os.getenv('USER') != 'root'):

Is that really the best way to check for root?  It may be, I guess,
but I'd have expected os.geteuid() to be more reliable...

Or is it os.getuid()?  I always get confused by these functions...

Cheers,
mwh

-- 
  <glyph> CDATA is not an integration strategy.
                                                -- from Twisted.Quotes
From lists at andreas-jung.com  Tue Dec  7 18:24:31 2004
From: lists at andreas-jung.com (Andreas Jung)
Date: Tue Dec  7 18:24:37 2004
Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL
Message-ID: <9DFB75B2DC037BC6AAD05F9E@[192.168.0.102]>

While using Zope 2.7 with Python 2.4 we discovered some strange behaviour 
of the security machinery.
I could track this down to some Zope code in cAccessControl.c where an 
Unauthorized exception is
raised because of a call to PyInt_FromLong(1) which returns NULL. What 
could be the reason that
such a "stupid" call return NULL in a reproducable way?

-aj
From mwh at python.net  Tue Dec  7 18:28:45 2004
From: mwh at python.net (Michael Hudson)
Date: Tue Dec  7 18:28:47 2004
Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL
In-Reply-To: <9DFB75B2DC037BC6AAD05F9E@[192.168.0.102]> (Andreas Jung's
	message of "Tue, 07 Dec 2004 18:24:31 +0100")
References: <9DFB75B2DC037BC6AAD05F9E@[192.168.0.102]>
Message-ID: <2mzn0qow4y.fsf@starship.python.net>

Andreas Jung <lists@andreas-jung.com> writes:

> While using Zope 2.7 with Python 2.4 we discovered some strange
> behaviour of the security machinery.
> I could track this down to some Zope code in cAccessControl.c where an
> Unauthorized exception is
> raised because of a call to PyInt_FromLong(1) which returns NULL. What
> could be the reason that
> such a "stupid" call return NULL in a reproducable way?

A memory scribble somewhere else?  Possibly a DECREF too many somewhere?

Is an exception set?  Have you tried a debug build?  Etc.

Cheers,
mwh

-- 
  All obscurity will buy you is time enough to contract venereal
  diseases.                                  -- Tim Peters, python-dev
From jim at zope.com  Tue Dec  7 18:30:19 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Dec  7 18:30:24 2004
Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL
In-Reply-To: <9DFB75B2DC037BC6AAD05F9E@[192.168.0.102]>
References: <9DFB75B2DC037BC6AAD05F9E@[192.168.0.102]>
Message-ID: <41B5E8AB.8060405@zope.com>

Andreas Jung wrote:
> While using Zope 2.7 with Python 2.4 we discovered some strange 
> behaviour of the security machinery.
> I could track this down to some Zope code in cAccessControl.c where an 
> Unauthorized exception is
> raised because of a call to PyInt_FromLong(1) which returns NULL. What 
> could be the reason that
> such a "stupid" call return NULL in a reproducable way?

Ugh. Part of the problem is that all of those calls are unchecked,
Dang us.  If they were checked, then, who knows, we might have
gotten informative exceptions.

I'd say the first step should be to add checks.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From tim.peters at gmail.com  Tue Dec  7 18:58:12 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Dec  7 18:59:03 2004
Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL
In-Reply-To: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102>
References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102>
Message-ID: <1f7befae041207095831b00ae0@mail.gmail.com>

[Andreas Jung]
> While using Zope 2.7

Do you mean 2.7, or do you mean 2.7.3, or ...?

> with Python 2.4 we discovered some strange behaviour
> of the security machinery.
> I could track this down to some Zope code in cAccessControl.c
> where an Unauthorized exception is raised because of a call to
> PyInt_FromLong(1) which returns NULL. What could be the
> reason that such a "stupid" call return NULL in a reproducable
> way?

Any C function that returns a Python object can return NULL if
malloc() says there's not enough memory to create a new object. 
PyInt_FromLong() actually allocates about 1KB at a time, and will
return NULL if malloc() can't find that much.

OTOH, 1KB isn't big, and PyInt_FromLong(1) specifically should be
returning a shared reference to a pre-existing PyIntObject (a number
of small integer objects are constructed at Python initialization
time, and PyInt_FromLong() returns references to them instead of
allocating new memory).

So PyInt_FromLong(1) should have taken this path:

		v = small_ints[ival + NSMALLNEGINTS];

But if a wild store had stuffed NULL into 1's slot in the small_ints
vector, the *next* line would have blown up with a NULL-pointer
dereference before PyInt_FromLong() could have returned:

		Py_INCREF(v);
		return (PyObject *) v;

So, in all, it appears impossible for PyInt_FromLong(1) to return NULL.

If it's reproducible, run it under a debugger and step into the errant
PyInt_FromLong(1) to see what's happening?  Could be a compiler
optimization bug (while rare, they have happened in Python).
From tim.peters at gmail.com  Tue Dec  7 19:02:51 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Dec  7 19:03:52 2004
Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL
In-Reply-To: <41B5E8AB.8060405@zope.com>
References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102>
	<41B5E8AB.8060405@zope.com>
Message-ID: <1f7befae04120710026748686c@mail.gmail.com>

[Jim Fulton]
> Ugh. Part of the problem is that all of those calls are unchecked,
> Dang us.  If they were checked, then, who knows, we might have
> gotten informative exceptions.

They certainly "should" be checked, but as a pragmatic matter
PyInt_FromLong(1) can't fail -- Python allocates an int object for 1
(and for about 100 other popular little integers) when it starts up,
and PyInt_FromLong() just returns a new reference to these
pre-existing objects whenever possible.  So, wrt:

> I'd say the first step should be to add checks

that's probably not going to help.  I'd make it the fourth thing <wink>.
From jim at zope.com  Tue Dec  7 19:06:54 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Dec  7 19:07:00 2004
Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL
In-Reply-To: <1f7befae04120710026748686c@mail.gmail.com>
References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102>	
	<41B5E8AB.8060405@zope.com>
	<1f7befae04120710026748686c@mail.gmail.com>
Message-ID: <41B5F13E.5080901@zope.com>

Tim Peters wrote:
> [Jim Fulton]
> 
>>Ugh. Part of the problem is that all of those calls are unchecked,
>>Dang us.  If they were checked, then, who knows, we might have
>>gotten informative exceptions.
> 
> 
> They certainly "should" be checked, but as a pragmatic matter
> PyInt_FromLong(1) can't fail -- Python allocates an int object for 1
> (and for about 100 other popular little integers) when it starts up,
> and PyInt_FromLong() just returns a new reference to these
> pre-existing objects whenever possible.

I know. I'm sure that's why we don't bother.  But, obviously,
it can fail.

Jim


-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From tim.peters at gmail.com  Tue Dec  7 19:17:19 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Dec  7 19:18:30 2004
Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL
In-Reply-To: <41B5F13E.5080901@zope.com>
References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102>
	<41B5E8AB.8060405@zope.com>
	<1f7befae04120710026748686c@mail.gmail.com>
	<41B5F13E.5080901@zope.com>
Message-ID: <1f7befae04120710175ae5882@mail.gmail.com>

[Tim]
>> ... but as a pragmatic matter PyInt_FromLong(1) can't fail --

[Jim]
> I know. I'm sure that's why we don't bother.  But, obviously,
> it can fail.

I disagree -- it's not obvious at all.  Looking at the code, it's far
more likely that Andreas misunderstood the cause of the failure than
that PyInt_FromLong(1) actually returned NULL.  If it did return NULL,
then it's got to be something as rare as bad code generation (for
reasons I explained earlier), or a non-standard compilation that
fiddled the NSMALLPOSINTS and/or NSMALLNEGINTS #defines to insane
values.

This is the entire expected path in PyInt_FromLong(1):

PyObject *
PyInt_FromLong(long ival)
{
	register PyIntObject *v;
#if NSMALLNEGINTS + NSMALLPOSINTS > 0
	if (-NSMALLNEGINTS <= ival && ival < NSMALLPOSINTS) {
		v = small_ints[ival + NSMALLNEGINTS];
		Py_INCREF(v);
#ifdef COUNT_ALLOCS
		if (ival >= 0)
			quick_int_allocs++;
		else
			quick_neg_int_allocs++;
#endif
		return (PyObject *) v;
	}
#endif

It's not possible for that to return NULL -- even if small_ints[] got
corrupted, so that v == NULL, the Py_INCREF(v) would have blown up
before the function could have returned.
From amk at amk.ca  Tue Dec  7 19:30:20 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Tue Dec  7 19:30:41 2004
Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL
In-Reply-To: <1f7befae04120710026748686c@mail.gmail.com>
References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102>
	<41B5E8AB.8060405@zope.com>
	<1f7befae04120710026748686c@mail.gmail.com>
Message-ID: <20041207183020.GA22351@rogue.amk.ca>

On Tue, Dec 07, 2004 at 01:02:51PM -0500, Tim Peters wrote:
> pre-existing objects whenever possible.  So, wrt:
> > I'd say the first step should be to add checks
> that's probably not going to help.  I'd make it the fourth thing <wink>.

Is it possible that some other Python API call is raising an
exception, but a NULL return isn't being checked for, and
PyInt_FromLong() happens to be the first bit of code that does 'if
(PyErr_Occurred())'?  Though from a quick glance at PyInt_FromLong()
and the macros it uses, I don't see any references to PyErr_Occurred()...

--amk

From lists at andreas-jung.com  Tue Dec  7 19:34:32 2004
From: lists at andreas-jung.com (Andreas Jung)
Date: Tue Dec  7 19:34:37 2004
Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL
In-Reply-To: <1f7befae04120710175ae5882@mail.gmail.com>
References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102>
	<41B5E8AB.8060405@zope.com>	<1f7befae04120710026748686c@mail.gmail.com>
	<41B5F13E.5080901@zope.com> <1f7befae04120710175ae5882@mail.gmail.com>
Message-ID: <47517AA25D790FF4C7D2A450@[192.168.0.102]>

Sorry, false alarm :-(  There assignment of the NULL occurs in the 
if-clause of the
corresponding code (I have overseen the ASSIGN call):

                    if (! PyInt_Check(p))
                      {
                        if (PyDict_Check(p))
                          {
                            if (PyString_Check(name) || 
PyUnicode_Check(name))
                              {
                                ASSIGN(p, PyObject_GetItem(p, name));
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

                                if (p == NULL) {
                                    puts("PyObject returned NULL");
                                  PyErr_Clear();
                                }
                              }
                            else
                              p = PyInt_FromLong((long)1);

...doing some further investigations on that.

-aj






From tim.peters at gmail.com  Tue Dec  7 19:40:01 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Dec  7 19:40:16 2004
Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL
In-Reply-To: <20041207183020.GA22351@rogue.amk.ca>
References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102>
	<41B5E8AB.8060405@zope.com>
	<1f7befae04120710026748686c@mail.gmail.com>
	<20041207183020.GA22351@rogue.amk.ca>
Message-ID: <1f7befae0412071040346940c8@mail.gmail.com>

[A.M. Kuchling]
> Is it possible that some other Python API call is raising an
> exception, but a NULL return isn't being checked for, and
> PyInt_FromLong() happens to be the first bit of code that does 'if
> (PyErr_Occurred())'?  Though from a quick glance at
> PyInt_FromLong() and the macros it uses, I don't see any
> references to PyErr_Occurred()...

A long stare wouldn't uncover any either <wink>.  This isn't it.  That
1 is passed as an argument is important too, which cuts out all but
the simplest path thru this code.

The (current Zope 2.7 branch) source for cAccessControl.c contains six
calls to PyInt_FromLong().  Five of them pass the literal 1.  The
other passes a variable with value -1, 0, or 1.  So regardless of
which of these Andreas is talking about, it's going thru
PyInt_FromLong's fast path.
From tim.peters at gmail.com  Tue Dec  7 20:10:20 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Dec  7 20:10:54 2004
Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL
In-Reply-To: <47517AA25D790FF4C7D2A450@192.168.0.102>
References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102>
	<41B5E8AB.8060405@zope.com>
	<1f7befae04120710026748686c@mail.gmail.com>
	<41B5F13E.5080901@zope.com> <1f7befae04120710175ae5882@mail.gmail.com>
	<47517AA25D790FF4C7D2A450@192.168.0.102>
Message-ID: <1f7befae0412071110707ff1db@mail.gmail.com>

[Andreas Jung]
> Sorry, false alarm :-(  There assignment of the NULL occurs in the
> if-clause of the corresponding code (I have overseen the ASSIGN
> call):

Thanks for the followup!

>                    if (! PyInt_Check(p))
>                      {
>                        if (PyDict_Check(p))
>                          {
>                            if (PyString_Check(name) ||
> PyUnicode_Check(name))
>                              {
>                                ASSIGN(p, PyObject_GetItem(p, name));
>                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
>                                if (p == NULL) {
>                                    puts("PyObject returned NULL");
>                                  PyErr_Clear();
>                                }
>                              }
>                            else
>                              p = PyInt_FromLong((long)1);
>
> ...doing some further investigations on that.

I note that all of this is nested inside another "if (p) {...}" block.
 That implies the "p = PyInt_FromLong((long)1);" line is at least a
memory leak:  it overwrites p without decref'ing p first.
From jim at zope.com  Tue Dec  7 20:19:43 2004
From: jim at zope.com (Jim Fulton)
Date: Tue Dec  7 20:19:46 2004
Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL
In-Reply-To: <1f7befae0412071110707ff1db@mail.gmail.com>
References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102>	
	<41B5E8AB.8060405@zope.com>	
	<1f7befae04120710026748686c@mail.gmail.com>	
	<41B5F13E.5080901@zope.com>
	<1f7befae04120710175ae5882@mail.gmail.com>	
	<47517AA25D790FF4C7D2A450@192.168.0.102>
	<1f7befae0412071110707ff1db@mail.gmail.com>
Message-ID: <41B6024F.4070201@zope.com>

Tim Peters wrote:
> [Andreas Jung]
> 
>>Sorry, false alarm :-(  There assignment of the NULL occurs in the
>>if-clause of the corresponding code (I have overseen the ASSIGN
>>call):
> 
> 
> Thanks for the followup!
> 
> 
>>                   if (! PyInt_Check(p))
>>                     {
>>                       if (PyDict_Check(p))
>>                         {
>>                           if (PyString_Check(name) ||
>>PyUnicode_Check(name))
>>                             {
>>                               ASSIGN(p, PyObject_GetItem(p, name));
>>                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>
>>                               if (p == NULL) {
>>                                   puts("PyObject returned NULL");
>>                                 PyErr_Clear();
>>                               }
>>                             }
>>                           else
>>                             p = PyInt_FromLong((long)1);
>>
>>...doing some further investigations on that.
> 
> 
> I note that all of this is nested inside another "if (p) {...}" block.
>  That implies the "p = PyInt_FromLong((long)1);" line is at least a
> memory leak:  it overwrites p without decref'ing p first.

The ASSIGN macro DECREFs it's first argument if it is non-NULL.

It loosly models a Python assignment, assuming that it owns the
reference to the second argument.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org
From tim.peters at gmail.com  Tue Dec  7 21:25:06 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Dec  7 21:25:19 2004
Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL
In-Reply-To: <41B6024F.4070201@zope.com>
References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102>
	<41B5E8AB.8060405@zope.com>
	<1f7befae04120710026748686c@mail.gmail.com>
	<41B5F13E.5080901@zope.com> <1f7befae04120710175ae5882@mail.gmail.com>
	<47517AA25D790FF4C7D2A450@192.168.0.102>
	<1f7befae0412071110707ff1db@mail.gmail.com>
	<41B6024F.4070201@zope.com>
Message-ID: <1f7befae04120712255df4861e@mail.gmail.com>

>>>                   if (! PyInt_Check(p))
>>>                     {
>>>                       if (PyDict_Check(p))
>>>                         {
>>>                           if (PyString_Check(name) || PyUnicode_Check(name))
>>>                             {
>>>                               ASSIGN(p, PyObject_GetItem(p, name));
>>>                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>
>>>                               if (p == NULL) {
>>>                                   puts("PyObject returned NULL");
>>>                                 PyErr_Clear();
>>>                               }
>>>                             }
>>>                           else
>>>                             p = PyInt_FromLong((long)1);
>>>

[Tim]
>> I note that all of this is nested inside another "if (p) {...}" block.
>> That implies the "p = PyInt_FromLong((long)1);" line is at least a
>> memory leak:  it overwrites p without decref'ing p first.

[Jim]
> The ASSIGN macro DECREFs it's first argument if it is non-NULL.
> 
> It loosly models a Python assignment, assuming that it owns the
> reference to the second argument.

ASSIGN isn't executed on the path in question.  I really can't follow
nesting with that indentation style.  Reformatting in Python style
makes it obvious to me:

if (p) {
    if (! PyInt_Check(p)) {
        if (PyDict_Check(p)) {
            if (PyString_Check(name) || PyUnicode_Check(name)) {
                ASSIGN(p, PyObject_GetItem(p, name));
                if (p == NULL)
                    PyErr_Clear();
            }
            else
                p = PyInt_FromLong(1);
        }
        else {
            ASSIGN(p, callfunction2(p, name, value));
            if (p == NULL)
                goto err;
        }
    }
}

"p = PyInt_FromLong(1)" is in an ``else`` block.  The only ASSIGN
before it is in that ``else`` block's disjoint ``if`` block.
From jlg at dds.nl  Tue Dec  7 23:52:45 2004
From: jlg at dds.nl (Johannes Gijsbers)
Date: Tue Dec  7 23:49:55 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test
	test_shutil.py, 1.10, 1.11
In-Reply-To: <2m4qiyqhtx.fsf@starship.python.net>
References: <E1CbPoH-0005LW-Qo@sc8-pr-cvs1.sourceforge.net>
	<2m4qiyqhtx.fsf@starship.python.net>
Message-ID: <20041207225245.GB9734@authsmtp.dds.nl>

On Tue, Dec 07, 2004 at 02:54:50PM +0000, Michael Hudson wrote:
> Is that really the best way to check for root?  It may be, I guess,
> but I'd have expected os.geteuid() to be more reliable...

Ah yes, I suppose it is. I'll change it tomorrow night if no Unix-guru
steps up with an alternative solution.
 
> Or is it os.getuid()?  I always get confused by these functions...

Based on my reading of the difference between geteuid and getuid, I'd go
for geteuid.

Cheers,

Johannes
From tjreedy at udel.edu  Wed Dec  8 02:41:45 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed Dec  8 02:41:41 2004
Subject: [Python-Dev] Re: Rewriting PEP4
References: <002401c4dbf5$b847cc00$f301a044@oemcomputer>
	<41B556FE.20504@v.loewis.de>
Message-ID: <cp5m4c$fkv$1@sea.gmane.org>


""Martin v. Löwis"" <martin@v.loewis.de> wrote in message 
news:41B556FE.20504@v.loewis.de...

As a (currently) casual user of Python, this is my view of the standard 
library dilemma and its solution:

1. It is too small: I may someday want to use a module not yet added.

2. It is too big: I cannot keep everything in mind at once and cannot 
remember, without referring back to the lib manual or pydev, which modules 
are currently or slated to become deprecated.

3. I do not wish to have other people's code broken,or books made obsolete, 
without sufficient reason (ie, the code is actively dangerous versus merely 
broken or superceded).

My preferred solution is to apply the 'out of sight, out of mind' 
principle.

1. Move obsolete modules to a separate directory (lib_old sounds fine) and 
put that directory in pythonpath.

When I ran Python (1.3) from a 20 meg disk, I would have preferred complete 
removal, but with 60+ gigs, the small extra space is no longer an issue. 
If  someone needs the space or wants to guarantee non-use of old modules, 
deleting lib_old is easy enough.

2. Remove the docs for obsolete modules from the main part of the current 
lib reference.  Sub-choices for legacy chapters are either complete removal 
or segregation into a separate document or apppendix to current one.

Terry J. Reedy



From gvanrossum at gmail.com  Wed Dec  8 23:18:48 2004
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Wed Dec  8 23:18:50 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
Message-ID: <ca471dc204120814186f5dc423@mail.gmail.com>

I was pleasantly surprised to find a pointer to this article in a news
digest that the ACM emails me regularly (ACM TechNews).

http://gcn.com/vol1_no1/daily-updates/28026-1.html

One thing that bugs me: the article says 3 or 4 times that Python is
slow, each time with a refutation ("but it's so flexible", "but it's
fast enough") but still, they sure seem to harp on the point. This is
a PR issue that Python needs to fight -- any ideas?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
From pje at telecommunity.com  Wed Dec  8 23:39:31 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Dec  8 23:39:34 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com>
Message-ID: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>

At 02:18 PM 12/8/04 -0800, Guido van Rossum wrote:
>I was pleasantly surprised to find a pointer to this article in a news
>digest that the ACM emails me regularly (ACM TechNews).
>
>http://gcn.com/vol1_no1/daily-updates/28026-1.html
>
>One thing that bugs me: the article says 3 or 4 times that Python is
>slow, each time with a refutation ("but it's so flexible", "but it's
>fast enough") but still, they sure seem to harp on the point. This is
>a PR issue that Python needs to fight -- any ideas?

The only thing that will fix the PR issue is to have a Python compiler 
distributed as part of the language.  It doesn't matter if it doesn't 
support the full generality of Python, or even if it doesn't speed many 
operations up much.  The only real requirements are that it can be used to 
produce "native" executables, and that it be an official part of the 
language, not a separately-distributed tool like Psyco or Pyrex.  Then, it 
will perhaps be a sufficient "security blanket" to stop people FUDding 
about it.

I imagine you could speed up the Python interpreter until it's faster than 
half the Java JIT's out there, and people will still ask, "But can I 
compile to an .exe?"  On the other hand, if you add a compiler, we'll see 
articles like the above talking about how Python can now be compiled and so 
therefore it's suitable for all kinds of things it wasn't before.  :)

Of course, it would be *really* useful if the compiler were coupled with 
optional type declarations for Python-the-language, because then we could 
ultimately dispense with the current syntax warts of Pyrex (and the 
runtime/distribution warts of ctypes) for interfacing with C.  But I don't 
think that having the compiler actually be useful is a prerequisite for 
solving the PR issue.  :)

From pythondev at bitfurnace.com  Wed Dec  8 23:59:05 2004
From: pythondev at bitfurnace.com (damien morton)
Date: Wed Dec  8 23:59:11 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
Message-ID: <41B78739.1080200@bitfurnace.com>

Guido van Rossum wrote:
> I was pleasantly surprised to find a pointer to this article in a news
> digest that the ACM emails me regularly (ACM TechNews).
> 
> http://gcn.com/vol1_no1/daily-updates/28026-1.html
> 
> One thing that bugs me: the article says 3 or 4 times that Python is
> slow, each time with a refutation ("but it's so flexible", "but it's
> fast enough") but still, they sure seem to harp on the point. This is
> a PR issue that Python needs to fight -- any ideas?

In public documents - documentation, release notes, etc etc.:

Remove all references to the performance of Python. From now on, Python 
has always been extremely fast, has no need to get any faster, and any 
speed increases are of so little importance that they barely rate a 
mention (no percentage increases mentioned in release notes).

If you must mention performance, do it in terms of the use of C modules 
such as numarray, and for benchmarks that other languages cant compete 
in. Write a benchmark which uses common C extensions, that Java cant 
compete with.

Replace all references to a 'Python interpreter' with 'Python Virtual 
Machine'

Ensure that the terms 'Python compiler' and 'compiled python' are 
liberally sprinkled around. Start emphasising the compilation step from 
python source to python bytecodes instead of making it transparent.



From sxanth at ceid.upatras.gr  Thu Dec  9 00:25:20 2004
From: sxanth at ceid.upatras.gr (Stelios Xanthakis)
Date: Thu Dec  9 00:25:24 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
Message-ID: <Pine.GSO.4.61.0412090113520.15784@zenon.ceid.upatras.gr>



> The only thing that will fix the PR issue is to have a Python compiler
> distributed as part of the language.  It doesn't matter if it doesn't
> support the full generality of Python, or even if it doesn't speed many
> operations up much.  The only real requirements are that it can be used 
> to produce "native" executables

I don't hink it's a matter of native executables.
Hopefully all the real algorithms are already in native
executables as builtins (dictionary lookup, etc).

And generally, one can wrap all the tough code in C
and take advantage of the flexibility of python at a
higher level.  pygame is the best example of this!
Basic stuff from complexity theory: the real complex
parts are few and isolated in C.  For the other we can
take advantage of a higher level dynamic language.


Stelios
From nas at arctrix.com  Thu Dec  9 00:29:09 2004
From: nas at arctrix.com (Neil Schemenauer)
Date: Thu Dec  9 00:29:13 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
Message-ID: <20041208232909.GA6354@mems-exchange.org>

On Wed, Dec 08, 2004 at 02:18:48PM -0800, Guido van Rossum wrote:
> This is a PR issue that Python needs to fight -- any ideas?

I'm not good at PR so I will continue to try to make it faster.  In
my copious free time I plan to:

    * finish the AST compiler (no performance benefit but makes
      other things easier)

    * optimize module namespaces as per the PEPs (whose numbers I
      cannot recall)

    * optimize function/method invocation (we must be able to do
      something better here, they are currently very expensive).
      Fredrik has mentioned "call-site caching".

    * write a new, register based VM

From hpk at trillke.net  Thu Dec  9 01:39:38 2004
From: hpk at trillke.net (holger krekel)
Date: Thu Dec  9 01:39:40 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
Message-ID: <20041209003938.GE4708@solar.trillke.net>

[Guido van Rossum Wed, Dec 08, 2004 at 02:18:48PM -0800]
> I was pleasantly surprised to find a pointer to this article in a news
> digest that the ACM emails me regularly (ACM TechNews).
> 
> http://gcn.com/vol1_no1/daily-updates/28026-1.html
> 
> One thing that bugs me: the article says 3 or 4 times that Python is
> slow, each time with a refutation ("but it's so flexible", "but it's
> fast enough") but still, they sure seem to harp on the point. This is
> a PR issue that Python needs to fight -- any ideas?

What about doing a survey on c.l.py asking questions like:

    what do you find runs slowly in Python 
    that should run faster? 

    Could you help with some concrete - 
    preferably real life examples with speed problems?". 

If python-dev takes the time to seriously debate (and debunk :-) 
upcoming code and suggestions then such a process could 
be very useful for all sides and also for PR purposes.  

IMO the biggest PR problem is that python programmers themselves [*]
tend to say that Python is slow and it's interesting to find
out why they think so, document and discuss the "answers" if any. 

I am not saying that such questioning/discussion doesn't already 
happen sometimes. But doing a survey in a more systematic way
might let us find out how pervasive the perception of "Python
is too slow" really is.  Maybe it turns out that not many people 
have concrete problems to offer?  Anyway, this would probably also 
be interesting for the various alternative Python implementations 
currently in the works.  

just an idea, 

    holger


[*] For example, something i stumbled upon today: 

    http://unununium.org/articles/languages

    where it states (without providing any details!): 

    But what about that fast system? Python isn't a slow
    language; it just has a slow implementation. There are many
    projects underway to correct this situation: Psyco, PyPy,
    Starkiller, IronPython, and Parrotcode are among them. It's
    likely these projects will be nearing completion when the time
    comes for Unununium to look at optimizations.
From pje at telecommunity.com  Thu Dec  9 02:30:02 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Dec  9 02:30:11 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <Pine.GSO.4.61.0412090113520.15784@zenon.ceid.upatras.gr>
References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com>

At 01:25 AM 12/9/04 +0200, Stelios Xanthakis wrote:


>>The only thing that will fix the PR issue is to have a Python compiler
>>distributed as part of the language.  It doesn't matter if it doesn't
>>support the full generality of Python, or even if it doesn't speed many
>>operations up much.  The only real requirements are that it can be used 
>>to produce "native" executables
>
>I don't hink it's a matter of native executables.

As I explained later in that message, "native" simply means, "has an .exe 
extension on Windows".

For PR purposes, it would suffice to bundle py2exe with Python 2.5 and say 
that Python "now includes a compiler to produce executable files".  This 
will then be picked up and misinterpreted by the trade press in exactly the 
same way that the article Guido cited picked up and misinterpreted what was 
said about 2.4.

If you read the article carefully, you will notice that the author 
translated "we rewrote a few modules in C" into "we made Python faster by 
switching to C".  If you ask anybody what language is faster, language X or 
C, most everybody will answer "C", regardless of what X is (unless it's 
assembly, of course).

All of the discussion about *actually* improving Python's performance is 
moot for PR purposes.  Public perception is not swayed by mere facts (as 
one might cynically observe of the U.S. political system).  If the goal is 
to achieve a PR win, the important thing is to pick a meme that's capable 
of succeeding, and stay "on message" with it.  The *only* meme that's 
organically capable of trumping "Python is slow because it's interpreted" 
is "Python is compiled now".

Me, I don't really care one way or the other.  I used to sell software I 
wrote in TRS-80 Basic, so Python's performance is fine for me, and I'm 
certainly not a compiler bigot.  I'm just responding to Guido's inquiry 
about what might work to increase Python's *perceived* speed in popular 
consciousness, not its actual speed.

From andrew at indranet.co.nz  Thu Dec  9 03:32:10 2004
From: andrew at indranet.co.nz (Andrew McGregor)
Date: Thu Dec  9 03:32:28 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
Message-ID: <86F33E75-498A-11D9-BB31-000D93C603C0@indranet.co.nz>

Well, for a lot of applications for Python, the performance that really  
counts is time from no code but a pile of data to code and processed  
data.  Python shines at that because nearly always the time to write  
the code dominates, so it doesn't matter what the run time is.

I wrote a little tool to translate a bunch of free data into scenery  
for the X-Plane flight simulator.  If I'd tried to do it in C, I'd  
still be debugging it, whereas I released it and the scenery I was  
building six months ago.  The run time of the C version would be, I  
speculate, about 5 times faster than the Python (given that psyco  
speeds it up about 8 times, and I would not have been so fancy with the  
algorithms in C).  But a 5x improvement on a 24 hour runtime is not 6  
months of improvement.

The other thing we can do is finish the portable backend for psyco and  
make it a standard module.  Then Python won't be slow, it will be  
compiled, and py2exe will be able to make a single-file executable.

Andrew

On 9/12/2004, at 11:18 AM, Guido van Rossum wrote:

> I was pleasantly surprised to find a pointer to this article in a news
> digest that the ACM emails me regularly (ACM TechNews).
>
> http://gcn.com/vol1_no1/daily-updates/28026-1.html
>
> One thing that bugs me: the article says 3 or 4 times that Python is
> slow, each time with a refutation ("but it's so flexible", "but it's
> fast enough") but still, they sure seem to harp on the point. This is
> a PR issue that Python needs to fight -- any ideas?
>
> --  
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:  
> http://mail.python.org/mailman/options/python-dev/ 
> andrew%40indranet.co.nz
>
>

From ilya at bluefir.net  Thu Dec  9 04:57:25 2004
From: ilya at bluefir.net (Ilya Sandler)
Date: Thu Dec  9 04:55:08 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
Message-ID: <Pine.LNX.4.58.0412081904450.1012@bagira>



On Wed, 8 Dec 2004, Guido van Rossum wrote:

> One thing that bugs me: the article says 3 or 4 times that Python is
> slow, each time with a refutation ("but it's so flexible", "but it's
> fast enough") but still, they sure seem to harp on the point. This is
> a PR issue that Python needs to fight -- any ideas?
>

On PR side of the issue one idea would be to avoid using the words
interpreted/intepreter less and use the
words compiler/byte-compilation/virtual machine instead...

On non-PR side of the issue (I do think that python slowness is a real
problem for at least some people/domains): would
bundling of Psyco with Python be a good idea?

Ilya

From python at rcn.com  Thu Dec  9 04:55:59 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Dec  9 05:00:16 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com>
Message-ID: <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer>

> One thing that bugs me: the article says 3 or 4 times that Python is
> slow, each time with a refutation ("but it's so flexible", "but it's
> fast enough") but still, they sure seem to harp on the point. This is
> a PR issue that Python needs to fight -- any ideas?

* Ask a prominent PSF member, ESR, to stop bagging on that point in his
book (python is much faster since he wrote it and Moore's law has made
it less of an issue over time).  Py1.5.2 running on a 133Mhz 386 is not
the same beast as well designed Py2.4 code running on a 3Ghz Pentium IV
(also I would argue that many apps are computationally bound anyway).


* Have python.org prominently feature an article of Python's use in
high-performance environments.  IIRC, somebody wrote a realtime voice
over internet system and found that with good design, there was no speed
issue.  Also, the cellphone apps may make a good example.


* Another prominent developer and PSF member wrote a much publicized,
unfavorable log entry basically saying that he was unimpressed with
Py2.4.  While that log entry was subsequently revised, it did serve as
anti-advocacy.  Also, my request was denied for better presentation of
performance related benchmarks in the widely read What's New document.
Right now, it features somewhat unimpressive and misleading pystone
results.  
Were we to show comparative pybench results 2.0 to 2.1 to 2.2 to 2.3 to
2.4, it would become clear that some of the performance concerns are out
of date.
Parrotbench and test_decimal also reveal some modest gains over 2.3.


* A ZDNet reporter had setup a phone interview with me but did not
follow through.  I had planned to nip the issue in the bud by focusing
on the Py2.4's vastly improved scalability:

- The trend toward use of iterators, generators, generator expressions,
and itertools scale-up well because of their superb memory performance
and ability to save state.  Apps using generators and genexps save all
the time that was formerly lost to accessing instance variables.  And
apps keying off of itertools can sometimes completely avoid interpreter
overhead.  IOW, Py2.4 provides a strong toolset for writing clean,
high-performance apps with a minimum of development time.

- Behind the scenes, almost every important building block has either
improved algorithms, memory optimizations, speed optimizations, and/or
has been rewritten in C (while still preserving their pure Python
equivalents).  The bytecodes get optimized for speed and readability,
and the eval-loop itself is tighter.  Lists initialize, append, pop, and
extend much faster and are more conservative with memory.  List comps
are up to twice as fast.  Dictionary iteration is speedier.  Sets,
bisect, and heaps now have C implementations like other basic building
blocks.  All of the fifo queues in the library now use an O(1) algorithm
instead of O(n) -- this improves scalability for everything from
threading and queues to mutexs and asynchat.


* Any PR effort should also emphasize that no usability trade-offs were
made along the way.  A number of features make Py2.4 easier to use than
1.5.6:  list comps, genexps, generators, sets, nested scopes, int/long
unification, decorators, etc.  Some might argue that it takes a modicum
effort to learn listcomps, generators, and decorators, but they cannot
make any bones about the cleanliness, simplicity, beauty, and speed of
the resulting code.  Python will always lead in maintainability and
development time.


* Python's website has traditionally been self-centered, leaving others
to have to make the case for their own products.  Perhaps, it is time to
change that.  Those who really care about speed cannot make a balanced
decision about Python without considering psyco, pyrex, numpy, and the
like as part of the total development environment.


* There is another angle that is difficult to present but a reality
never-the-less.  It is easy to write toy C / C++ code that is blazingly
fast.  However, when you add error checking, fault tolerance,
interaction with real world, etc, the code gets slower.  Also because
that code is harder to write, it is somewhat common to use crummy
algorithms (linear searches for example).  I really liked Tim's comment
that code using Python's dictionaries runs 10,000 times faster than
anything written in any other language.  That seems to get to the point
across well enough.


* A bit of good news is that the upcoming Python cookbook will make a
strong case for Py2.4 being a tool of choice for those who care speed,
space, maintainability, and development time.  It is a nice, highly
visible bit of advocacy.


* One last thought.  Articles will always pick on something.  It's not
really a terrible thing to have someone say the superb productivity
gains come at the price of running slower than C.  I would much rather
hear that than have people bag on the docs or standard library or launch
into a diatribe @decocrator destroying the beauty of the language.



Raymond

From mdehoon at ims.u-tokyo.ac.jp  Thu Dec  9 05:47:01 2004
From: mdehoon at ims.u-tokyo.ac.jp (Michiel Jan Laurens de Hoon)
Date: Thu Dec  9 05:43:46 2004
Subject: [Python-Dev] PyOS_InputHook enhancement proposal
Message-ID: <41B7D8C5.9000008@ims.u-tokyo.ac.jp>

PyOS_InputHook is a pointer to a function that is called periodically (ten
times per second) while Python is idle, for example, when waiting for a
user command. Python C extension modules can set this pointer to a hook
function defined in the extension module. For example, _tkinter.c makes
use of PyOS_InputHook to get messages delivered to its widgets.

A problem arises when two or more extension modules want to set
PyOS_InputHook. For example, the scientific plotting package pygist needs
PyOS_InputHook to get messages delivered to its graphics windows, and may
therefore conflict with the Python GUI IDLE, which uses Tkinter. Chaining
won't work, as it will fail when an extension module wants to remove its
hook function.

My suggestion is therefore to replace PyOS_InputHook by two functions
PyOS_AddInputHook and PyOS_RemoveInputHook, and let Python keep track of
which hooks are installed. This way, an extension module can add a hook
function without having to worry about other extension modules trying
to use the same hook.

Any comments? Would I need to submit a PEP for this proposal?

--Michiel, U Tokyo.


-- 
Michiel de Hoon, Assistant Professor
University of Tokyo, Institute of Medical Science
Human Genome Center
4-6-1 Shirokane-dai, Minato-ku
Tokyo 108-8639
Japan
http://bonsai.ims.u-tokyo.ac.jp/~mdehoon

From python at rcn.com  Thu Dec  9 05:43:49 2004
From: python at rcn.com (Raymond Hettinger)
Date: Thu Dec  9 05:48:03 2004
Subject: [Python-Dev] The other Py2.4 issue
Message-ID: <001401c4dda9$ad605680$e841fea9@oemcomputer>

Acceptance for Py2.4 partially hinges on how quickly third party apps
have their binaries updated.

I wonder if there is anything we can do to help.


Raymond



-----Original Message-----
From: image-sig-bounces@python.org [mailto:image-sig-bounces@python.org]
On Behalf Of Tuure Laurinolli
Sent: Tuesday, December 07, 2004 2:51 AM
To: image-sig@python.org
Subject: [Image-SIG] Re: Python 2.4

Spencer Ogden wrote:
> I was wondering if there was a time frame for a windows binary for Py 
> 2.4. Is it possible to compile the source against 2.4 myself?

There didn't seem to be any problems compiling it against 2.4. I managed

to get some version up and running, but problems with the required 
shared libraries(I seem to have many, incompatible versions of jpeg 
around...) killed my enthusiasm and I returned to 2.3.

_______________________________________________
Image-SIG maillist  -  Image-SIG@python.org
http://mail.python.org/mailman/listinfo/image-sig

From phd at mail2.phd.pp.ru  Thu Dec  9 11:17:02 2004
From: phd at mail2.phd.pp.ru (Oleg Broytmann)
Date: Thu Dec  9 11:17:05 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com>
References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com>
Message-ID: <20041209101702.GA12506@phd.pp.ru>

On Wed, Dec 08, 2004 at 08:30:02PM -0500, Phillip J. Eby wrote:
> As I explained later in that message, "native" simply means, "has an .exe 
> extension on Windows".

   And very soon that strategy will backfire - people will start PRing
"but those .exe's are nothing more than a python interpreter in
disguise" which in my opnion is even worse.

> All of the discussion about *actually* improving Python's performance is 
> moot for PR purposes.

   Hence we must stop spending our very valuable time thinking about PR
and return to actually improving Python (not only performance).

> If the goal is 
> to achieve a PR win, the important thing is to pick a meme that's capable 
> of succeeding, and stay "on message" with it.

   Translating to a plain langauge: "PSF should spend money spreading
around a counter-PR". I am afraid PSF doesn't have enough money, and
even if it has - should we really run down that path?

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd@phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.
From phd at mail2.phd.pp.ru  Thu Dec  9 11:22:50 2004
From: phd at mail2.phd.pp.ru (Oleg Broytmann)
Date: Thu Dec  9 11:22:52 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<000b01c4dda2$fe7a87e0$e841fea9@oemcomputer>
Message-ID: <20041209102250.GB12506@phd.pp.ru>

On Wed, Dec 08, 2004 at 10:55:59PM -0500, Raymond Hettinger wrote:
> * Have python.org prominently feature an article of Python's use in
> high-performance environments.  IIRC, somebody wrote a realtime voice
> over internet system and found that with good design, there was no speed
> issue.  Also, the cellphone apps may make a good example.

   +Games.

> * Python's website has traditionally been self-centered, leaving others
> to have to make the case for their own products.  Perhaps, it is time to
> change that.  Those who really care about speed cannot make a balanced
> decision about Python without considering psyco, pyrex, numpy, and the
> like as part of the total development environment.

   That's overreaction, I think. People always say this and that about
python - python is slow, and python is not like java, and python does
not have static type checks... And what? Should the webmasters adapt the
site to every complain?

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd@phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.
From aahz at pythoncraft.com  Thu Dec  9 14:15:55 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Dec  9 14:15:58 2004
Subject: [Python-Dev] PyOS_InputHook enhancement proposal
In-Reply-To: <41B7D8C5.9000008@ims.u-tokyo.ac.jp>
References: <41B7D8C5.9000008@ims.u-tokyo.ac.jp>
Message-ID: <20041209131555.GB10465@panix.com>

On Thu, Dec 09, 2004, Michiel Jan Laurens de Hoon wrote:
>
> My suggestion is therefore to replace PyOS_InputHook by two functions
> PyOS_AddInputHook and PyOS_RemoveInputHook, and let Python keep track of
> which hooks are installed. This way, an extension module can add a hook
> function without having to worry about other extension modules trying
> to use the same hook.
> 
> Any comments? Would I need to submit a PEP for this proposal?

Because this is only for the C API, your best bet is to write a patch
and submit it to SF.  If people whine or it gets rejected, then write a
PEP.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing."  --Alan Perlis
From stephen.kennedy at havok.com  Thu Dec  9 14:20:38 2004
From: stephen.kennedy at havok.com (Stephen Kennedy)
Date: Thu Dec  9 14:32:15 2004
Subject: [Python-Dev] How to install tile (or any other tcl module)
Message-ID: <loom.20041209T141319-804@post.gmane.org>


I've been trying to get Tile to work with python.
It can make your tkinter apps look like
http://tktable.sourceforge.net/tile/screenshots/demo-alt-unix.png
See http://tktable.sourceforge.net/tile/

Under linux I built tile from source, installed and it just works.

import Tkinter
root = Tkinter.Tk()
root.tk.call('package', 'require', 'tile')
root.tk.call('namespace', 'import', '-force', 'ttk::*')
root.tk.call('tile::setTheme', 'alt')
### Widgets are now pretty!

Under win32, I installed the binary package into python/tcl
(i.e. python/tcl/tile0.5) with all the other tcl packages, but tcl
can't find it. Any ideas?

Traceback (most recent call last):
  File "Script1.py", line 5, in ?
    root.tk.call('package', 'require', 'tile')
_tkinter.TclError: can't find package tile

Stephen.

From anthony at interlink.com.au  Thu Dec  9 14:36:16 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu Dec  9 14:36:33 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer>
References: <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer>
Message-ID: <200412100036.17253.anthony@interlink.com.au>

On Thursday 09 December 2004 14:55, Raymond Hettinger wrote:
> * Have python.org prominently feature an article of Python's use in
> high-performance environments.  IIRC, somebody wrote a realtime voice
> over internet system and found that with good design, there was no speed
> issue.  Also, the cellphone apps may make a good example.

That was me. I gave a keynote (45 minute) version of the talk last week
at OSDC, and I believe it was videoed and will be available eventually. This
is good propaganda.

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From Scott.Daniels at Acm.Org  Thu Dec  9 15:07:41 2004
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Thu Dec  9 15:06:19 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <20041209102250.GB12506@phd.pp.ru>
References: <ca471dc204120814186f5dc423@mail.gmail.com>	<000b01c4dda2$fe7a87e0$e841fea9@oemcomputer>
	<20041209102250.GB12506@phd.pp.ru>
Message-ID: <cp9m4m$isn$1@sea.gmane.org>

Oleg Broytmann wrote:
> Raymond Hettinger wrote:
>>* Python's website has traditionally been self-centered, leaving others
>>to have to make the case for their own products.  Perhaps, it is time to
>>change that.  Those who really care about speed cannot make a balanced
>>decision about Python without considering psyco, pyrex, numpy, and the
>>like as part of the total development environment.
> 
>    That's overreaction, I think. People always say this and that about
> python - python is slow, and python is not like java, and python does
> not have static type checks... And what? Should the webmasters adapt the
> site to every complain?
> 

Perhaps a link on the main page to a "for even more speed" page where we
can combine advice on how to improve application performance (measure,
abstract, choose good data structures) with references to these other
projects for particular applications.  This is the place to say things
like "operating on every character of a string is seldom efficient."

-- 
-- Scott David Daniels
Scott.Daniels@Acm.Org

From phd at mail2.phd.pp.ru  Thu Dec  9 15:11:04 2004
From: phd at mail2.phd.pp.ru (Oleg Broytmann)
Date: Thu Dec  9 15:11:07 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <cp9m4m$isn$1@sea.gmane.org>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<000b01c4dda2$fe7a87e0$e841fea9@oemcomputer>
	<20041209102250.GB12506@phd.pp.ru> <cp9m4m$isn$1@sea.gmane.org>
Message-ID: <20041209141104.GA18348@phd.pp.ru>

On Thu, Dec 09, 2004 at 06:07:41AM -0800, Scott David Daniels wrote:
> Oleg Broytmann wrote:
> >   That's overreaction, I think.
> 
> Perhaps a link on the main page

   Why on the main page? There are Topics Guide at
http://python.org/topics/ that describes the ways Python can be used in
some popular areas. Let's add another topic, "Making things fast". Let's
even make it the first topic, though I personnaly dont see a need for
this.

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd@phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.
From anthony at interlink.com.au  Thu Dec  9 15:17:50 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu Dec  9 15:18:49 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <20041209141104.GA18348@phd.pp.ru>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<cp9m4m$isn$1@sea.gmane.org> <20041209141104.GA18348@phd.pp.ru>
Message-ID: <200412100117.50910.anthony@interlink.com.au>

FWIW, I was planning on writing a tutorial (working title:
"Making Python Code Not Suck") for some conference
or another...  talked to a bunch of people last week at
OSDC, and it seems like something people are interested
in. Got a bunch of stuff already down from various notes
I've written in the past when helping coworkers and the like.

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.
From aahz at pythoncraft.com  Thu Dec  9 15:55:40 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Dec  9 15:55:54 2004
Subject: [Python-Dev] How to install tile (or any other tcl module)
In-Reply-To: <loom.20041209T141319-804@post.gmane.org>
References: <loom.20041209T141319-804@post.gmane.org>
Message-ID: <20041209145540.GB13555@panix.com>

On Thu, Dec 09, 2004, Stephen Kennedy wrote:
>
> I've been trying to get Tile to work with python.
> It can make your tkinter apps look like
> http://tktable.sourceforge.net/tile/screenshots/demo-alt-unix.png
> See http://tktable.sourceforge.net/tile/

Sorry, this is not a good place to get Python support; python-dev is for
people actively working on Python language development.  Please use
comp.lang.python or one of the other resources.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing."  --Alan Perlis
From pje at telecommunity.com  Thu Dec  9 16:30:10 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Dec  9 16:30:50 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <20041209101702.GA12506@phd.pp.ru>
References: <5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com>
	<5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20041209102402.042c8580@mail.telecommunity.com>

At 01:17 PM 12/9/04 +0300, Oleg Broytmann wrote:
>On Wed, Dec 08, 2004 at 08:30:02PM -0500, Phillip J. Eby wrote:
> > As I explained later in that message, "native" simply means, "has an .exe
> > extension on Windows".
>
>    And very soon that strategy will backfire - people will start PRing
>"but those .exe's are nothing more than a python interpreter in
>disguise" which in my opnion is even worse.

ISTR that for a long time, Visual Basic actually did the same thing.  A few 
magazines mentioned the fact, but nobody really cared.  However, if this is 
really a concern, bundle Pyrex as well.  Both Pyrex and py2exe are 
distutils-based, so at that point you have a complete solution, including 
the "C" meme as well as the ".exe" meme.


> > If the goal is
> > to achieve a PR win, the important thing is to pick a meme that's capable
> > of succeeding, and stay "on message" with it.
>
>    Translating to a plain langauge: "PSF should spend money spreading
>around a counter-PR". I am afraid PSF doesn't have enough money, and
>even if it has - should we really run down that path?

I'm not suggesting spending any money.  Heck, I'm not suggesting *doing* 
anything.  I just answered Guido's question about PR.  Whether anybody 
wants to *do* anything about it is an entirely separate issue from 
discussing *what* would need to be done, if something was going to be done.

From gmccaughan at synaptics-uk.com  Thu Dec  9 17:39:21 2004
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Thu Dec  9 17:39:53 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
Message-ID: <200412091639.21596.gmccaughan@synaptics-uk.com>

On Wednesday 2004-12-08 22:39, Phillip J. Eby wrote:

[Guido:]
>> One thing that bugs me: the article says 3 or 4 times that Python is
>> slow, each time with a refutation ("but it's so flexible", "but it's
>> fast enough") but still, they sure seem to harp on the point. This is
>> a PR issue that Python needs to fight -- any ideas?

[Philip:]
> The only thing that will fix the PR issue is to have a Python compiler 
> distributed as part of the language.  It doesn't matter if it doesn't 
> support the full generality of Python, or even if it doesn't speed many 
> operations up much.  The only real requirements are that it can be used to 
> produce "native" executables, and that it be an official part of the 
> language, not a separately-distributed tool like Psyco or Pyrex.  Then, it 
> will perhaps be a sufficient "security blanket" to stop people FUDding 
> about it.

Unfortunately, this may not be enough. Consider, by way of counterexample,
Common Lisp, which
  - is compiled to native code
  - has optional type declarations
  - actually *does* run fast when compiled
  - has had all these properties for years and years
but is still almost universally decried as "slow" by people who have
never actually used it. It's true that it doesn't (as part of the
standard, and in the free implementations I know of) have the ability
to generate standalone executables with filenames ending in ".exe";
perhaps that's the key thing.

The other thing that might work is to change the name of the language
to "C" plus optional punctuation.

-- 
g

From janssen at parc.com  Thu Dec  9 17:56:09 2004
From: janssen at parc.com (Bill Janssen)
Date: Thu Dec  9 17:56:57 2004
Subject: [Python-Dev] 2.4 news reaches interesting places 
In-Reply-To: Your message of "Thu, 09 Dec 2004 08:39:21 PST."
	<200412091639.21596.gmccaughan@synaptics-uk.com> 
Message-ID: <04Dec9.085613pst."58617"@synergy1.parc.xerox.com>

> The other thing that might work is to change the name of the language
> to "C" plus optional punctuation.

You mean "C@@" (pronounced C-pie-pie)?

Bill
From fredrik at pythonware.com  Thu Dec  9 18:57:10 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu Dec  9 18:54:56 2004
Subject: [Python-Dev] Re: How to install tile (or any other tcl module)
References: <loom.20041209T141319-804@post.gmane.org>
	<20041209145540.GB13555@panix.com>
Message-ID: <cpa3hb$29m$1@sea.gmane.org>

>> I've been trying to get Tile to work with python.
>> It can make your tkinter apps look like
>> http://tktable.sourceforge.net/tile/screenshots/demo-alt-unix.png
>> See http://tktable.sourceforge.net/tile/
>
> Sorry, this is not a good place to get Python support; python-dev is for
> people actively working on Python language development.  Please use
> comp.lang.python or one of the other resources.

in this case,

    http://mail.python.org/mailman/listinfo/tkinter-discuss

is the right forum

(also see: http://tkinter.unpythonic.net/wiki/TkinterDiscuss )

</F> 



From fumanchu at amor.org  Thu Dec  9 19:21:00 2004
From: fumanchu at amor.org (Robert Brewer)
Date: Thu Dec  9 19:23:07 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
Message-ID: <3A81C87DC164034AA4E2DDFE11D258E324538E@exchange.hqamor.amorhq.net>

Anthony Baxter wrote:
> FWIW, I was planning on writing a tutorial (working title:
> "Making Python Code Not Suck") for some conference
> or another...

Perhaps, given your high profile in the Python developer community, you
might reconsider the title? Little details like that are what PR is made
of. Imagine Bush's next Executive Order being titled "Making American
[Business|Military|People] Not Suck"... ;)


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org
From mwh at python.net  Thu Dec  9 19:28:01 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Dec  9 19:28:03 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <3A81C87DC164034AA4E2DDFE11D258E324538E@exchange.hqamor.amorhq.net>
	(Robert Brewer's message of "Thu, 9 Dec 2004 10:21:00 -0800")
References: <3A81C87DC164034AA4E2DDFE11D258E324538E@exchange.hqamor.amorhq.net>
Message-ID: <2mk6rrpbri.fsf@starship.python.net>

"Robert Brewer" <fumanchu@amor.org> writes:

> Anthony Baxter wrote:
>> FWIW, I was planning on writing a tutorial (working title:
>> "Making Python Code Not Suck") for some conference
>> or another...
>
> Perhaps, given your high profile in the Python developer community, you
> might reconsider the title? Little details like that are what PR is made
> of. Imagine Bush's next Executive Order being titled "Making American
> [Business|Military|People] Not Suck"... ;)

Anthony's Australian, people expect this sort of thing from him :)

Cheers,
mwh

-- 
  Also, remember to put the galaxy back when you've finished, or an
  angry mob of astronomers will come round and kneecap you with a
  small telescope for littering. 
       -- Simon Tatham, ucam.chat, from Owen Dunn's review of the year
From pje at telecommunity.com  Thu Dec  9 19:31:15 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Dec  9 19:32:05 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <200412091639.21596.gmccaughan@synaptics-uk.com>
References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20041209131843.038f3530@mail.telecommunity.com>

At 04:39 PM 12/9/04 +0000, Gareth McCaughan wrote:
>On Wednesday 2004-12-08 22:39, Phillip J. Eby wrote:
> > The only thing that will fix the PR issue is to have a Python compiler
> > distributed as part of the language.  It doesn't matter if it doesn't
> > support the full generality of Python, or even if it doesn't speed many
> > operations up much.  The only real requirements are that it can be used to
> > produce "native" executables, and that it be an official part of the
> > language, not a separately-distributed tool like Psyco or Pyrex.  Then, it
> > will perhaps be a sufficient "security blanket" to stop people FUDding
> > about it.
>
>Unfortunately, this may not be enough. Consider, by way of counterexample,
>Common Lisp, which
>   - is compiled to native code
>   - has optional type declarations
>   - actually *does* run fast when compiled
>   - has had all these properties for years and years
>but is still almost universally decried as "slow" by people who have
>never actually used it. It's true that it doesn't (as part of the
>standard, and in the free implementations I know of) have the ability
>to generate standalone executables with filenames ending in ".exe";
>perhaps that's the key thing.

Among the great unwashed masses of Windows programmers, it probably 
is.  Look at the marketing for virtually any off-beat commercial language 
for Windows (that's still around to look at), and you will usually find a 
prominent bullet point about how it makes .exe's.  This is an incredibly 
powerful meme, though I don't entirely understand why.

I do agree that it still may not be enough; I'm just saying that it's not a 
coincidence that so many development tools for non-C languages on Windows 
all ended up deciding to feature that bullet point.  My theory is that it's 
because it's a FAQ, and it's often found on the official FAQ's of open 
source languages as well.

Granted, the scope of the question is much broader than just speed, since 
it also encompasses application distribution issues, and the social status 
of the developer.  One of the subtexts of the question is, "Can I make a 
*real* program with this thing?", or put more honestly, "Will other 
programmers laugh at me when they see my program isn't an .exe?"


>The other thing that might work is to change the name of the language
>to "C" plus optional punctuation.

Well, we could always just call it CPython, and bill it as a faster C 
implementation of that popular dynamic language for Java, Jython.  That 
way, we can also leverage the meme that C is faster than Java.  :)

Isn't it funny, by the way, that people don't go around talking about how 
slow Jython is?  At least, I haven't seen it to the extent that I've seen 
with CPython.

From foom at fuhm.net  Thu Dec  9 20:16:18 2004
From: foom at fuhm.net (James Y Knight)
Date: Thu Dec  9 20:16:24 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <5.1.1.6.0.20041209131843.038f3530@mail.telecommunity.com>
References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<5.1.1.6.0.20041209131843.038f3530@mail.telecommunity.com>
Message-ID: <CDB95F16-4A16-11D9-88DA-000A95A50FB2@fuhm.net>

On Dec 9, 2004, at 1:31 PM, Phillip J. Eby wrote:
> Isn't it funny, by the way, that people don't go around talking about 
> how slow Jython is?  At least, I haven't seen it to the extent that 
> I've seen with CPython.

People talk about how slow CPython is, is because they are writing code 
targeted for it that traditionally would have been written in C. Python 
*is* slower than C. You can try to deny it, but it is simple fact. 
However, python is a really nice language to program in. So people use 
it anyways. It is "fast enough" for many things.

Nobody talks about how slow Jython is, because nobody(1) is writing 
entire programs in it. Jython is used as a way to script Java programs, 
and as such, it is pretty much completely unimportant how fast it runs.

Java has also traditionally got a lot of bad press for its lack of 
speed, and it was usually of the "not fast enough" variety, which is 
much worse than what's said of Python. However, a huge amount of 
progress has been made and Java really is quite speedy these days. It 
was not PR that made it faster, but hard work on the part of Sun.

There is currently a lot of work going on in the python community to 
make Python faster, which indicates both that Python is too slow for 
some people, and that it will be faster in the future. This is a good 
thing, and not something that should be shoved under the rug to try to 
pretend python is ultra mega super fast already.

James

1) Of course someone probably is, but you get the idea.

From tdelaney at avaya.com  Thu Dec  9 20:25:54 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Thu Dec  9 20:27:14 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE4A915A@au3010avexu1.global.avaya.com>

Michael Hudson wrote:

> Anthony's Australian, people expect this sort of thing from him :)

As another Australian, I think that "Making Python Not Suck" implies
that if you don't do extra things, Python Sucks.

This is not a good thing IMO.

"Making Python Suck Less" would be even worse.

How about "Python - You Can Have Your Cake And Eat It Too".

Tim Delaney
From jhylton at gmail.com  Thu Dec  9 20:56:41 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Thu Dec  9 20:56:45 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <CDB95F16-4A16-11D9-88DA-000A95A50FB2@fuhm.net>
References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<5.1.1.6.0.20041209131843.038f3530@mail.telecommunity.com>
	<CDB95F16-4A16-11D9-88DA-000A95A50FB2@fuhm.net>
Message-ID: <e8bf7a5304120911564cef0c53@mail.gmail.com>

On Thu, 9 Dec 2004 14:16:18 -0500, James Y Knight <foom@fuhm.net> wrote:
> On Dec 9, 2004, at 1:31 PM, Phillip J. Eby wrote:
> > Isn't it funny, by the way, that people don't go around talking about
> > how slow Jython is?  At least, I haven't seen it to the extent that
> > I've seen with CPython.
> 
> People talk about how slow CPython is, is because they are writing code
> targeted for it that traditionally would have been written in C. Python
> *is* slower than C. You can try to deny it, but it is simple fact.
> However, python is a really nice language to program in. So people use
> it anyways. It is "fast enough" for many things.

I think you are exactly right.  It's easy to get the impression that
Python is slow when many programs written in Python *are* slowly than
the same program written in C, C++, or Java.  If you look are other
"dynamic" languages, you'll find that IronPython is often faster than
CPython and that Smalltalk and Common Lisp are often substantially
faster than Python.  Python isn't remarkably slow for a scripting
language, but it is not fast.

> There is currently a lot of work going on in the python community to
> make Python faster, which indicates both that Python is too slow for
> some people, and that it will be faster in the future. This is a good
> thing, and not something that should be shoved under the rug to try to
> pretend python is ultra mega super fast already.

I agree, although it's not clear to me how much faster it will be in
the future.  Making a *fast* Python based on our own virtual execution
environment (as opposed to piggybacking a JVM or CLR) is a big
project.  It's not clear than anyone has enough resources to make
credible effort, so we're left to piecemeal improvements.  The
improvement I see from 2.2 to 2.4 on my box is about 55% (measured
using pystone) over almost three years.

I think the right thing to do with PR is frame the question
differently.  We need to say that people use Python because it lets
them right clear, simple code or that it is easy to maintain or that
dynamic languages are excellent for prototyping -- whatever the
message is -- and simply avoid talking about speed.  There are a lot
of issues that affect the selection of a programming language, and
speed is not always the most important one.  It is easy for a
journalist to write about and it is easy to report (possibly
meaningless) measurements of speed.

Jeremy
From s.percivall at chello.se  Thu Dec  9 21:10:02 2004
From: s.percivall at chello.se (Simon Percivall)
Date: Thu Dec  9 21:10:07 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <cp9m4m$isn$1@sea.gmane.org>
References: <ca471dc204120814186f5dc423@mail.gmail.com>	<000b01c4dda2$fe7a87e0$e841fea9@oemcomputer>
	<20041209102250.GB12506@phd.pp.ru> <cp9m4m$isn$1@sea.gmane.org>
Message-ID: <4FA61149-4A1E-11D9-9ADB-0003934AD54A@chello.se>

On 2004-12-09, at 15.07, Scott David Daniels wrote:
> Oleg Broytmann wrote:
>> Raymond Hettinger wrote:
>>> * Python's website has traditionally been self-centered, leaving 
>>> others
>>> to have to make the case for their own products.  Perhaps, it is 
>>> time to
>>> change that.  Those who really care about speed cannot make a 
>>> balanced
>>> decision about Python without considering psyco, pyrex, numpy, and 
>>> the
>>> like as part of the total development environment.
>>    That's overreaction, I think. People always say this and that about
>> python - python is slow, and python is not like java, and python does
>> not have static type checks... And what? Should the webmasters adapt 
>> the
>> site to every complain?
>
> Perhaps a link on the main page to a "for even more speed" page where 
> we
> can combine advice on how to improve application performance (measure,
> abstract, choose good data structures) with references to these other
> projects for particular applications.  This is the place to say things
> like "operating on every character of a string is seldom efficient."

+1

Many examples of common idioms and tasks and how to do them right in 
Python
would also be useful for such a page.

//Simon

From allison at sumeru.stanford.EDU  Thu Dec  9 21:18:07 2004
From: allison at sumeru.stanford.EDU (Dennis Allison)
Date: Thu Dec  9 21:18:59 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE4A915A@au3010avexu1.global.avaya.com>
Message-ID: <Pine.LNX.4.10.10412091151530.11555-100000@sumeru.stanford.EDU>


The goal here is to make Python better known and to counter some of the
prevalent myths.  One way to accomplish this goal is to publish literate
technical articles with real content including performance measurements
and pointers to the code.  Perhaps Guido could be a real-life N. Bourbaki
and co-publish with developers.

A significant issue is: where to publish.  Certainly publishing in the
technical society journals (ACM & IEEE CS) would make sense.  Software
Practice and Experience would also be good.  Likewise DDJ and Game
Developer Magazine. But I suspect that much of our target audience will be
missed if we limit ourselves to these journals.  We also need to get
articles placed in the mass market computer mags.  And we need to build an
editorial relationship with the mass market journals so that when X says 
they think "Python is slow" there's some they know they can call for a
truth-squad quote.

And, parenthetically, I continue to be amazed at the number of projects
that use Python, but do it in stealth-mode and view it as their
silver-bullet and competative edge.  I wish more people would publish
their experience.





From glyph at divmod.com  Thu Dec  9 22:11:33 2004
From: glyph at divmod.com (Glyph Lefkowitz)
Date: Thu Dec  9 22:10:35 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
Message-ID: <1102626693.16291.57.camel@localhost>

On Wed, 2004-12-08 at 17:39 -0500, Phillip J. Eby wrote:

> The only thing that will fix the PR issue is to have a Python compiler 
> distributed as part of the language.  It doesn't matter if it doesn't 
> support the full generality of Python, or even if it doesn't speed many 
> operations up much.  The only real requirements are that it can be used to 
> produce "native" executables, and that it be an official part of the 
> language, not a separately-distributed tool like Psyco or Pyrex.  Then, it 
> will perhaps be a sufficient "security blanket" to stop people FUDding 
> about it.

I am aware that Pyrex is still in flux, so perhaps it is too soon to
propose this even for 2.5, but I think it's worth bringing up anyway: I
would like Pyrex to be distributed with the Python core.  I agree that
it should be modified to produce full .exe files and not just .dlls on
Windows, but many potential users *are* seriously concerned about
efficiency and not just simplifying distribution.

The main benefit would be imparting a Pythonesque flavor to optimized
code.  The thing which experienced Python-ers say is, technically,
correct: "If you need speed, rewrite the hot-spots in C.".  To someone
already steeped in the Python way, this means, "write your application
in Python, do your architecture in Python, profile it, and write the 300
lines or so of your 25000 line application that are *really*
speed-critical in a faster language".  There are a few, very very
specialized areas where this approach is not the best one, but
unfortunately C programmers do not hear this when you say it.

What they hear is, "If speed is important to your application, write it
in C, and only write the parts where you need flexibility in Python.
They will all be deadly slow.  If you notice that they are slow, you are
going to have to rewrite them in C anyway."  To wit, Python is only
interesting to the people who are going to "script" your application.

Although writing code in Pyrex ends up being a technically similar
solution, politically it is radically different.  If the Python-expert's
answer were instead, "write the slow parts in Pyrex", then the C
programmer's plan changes - instead of writing in C, because they know
they are going to need to rewrite all the slow parts anyway, they
realize that there is an explicitly Python way to rewrite the slow
parts, that may leverage SOME of their existing C knowledge.  They learn
Python and appreciate Python secure in the knowledge that this effort
will not be wasted.

Finally, there are real benefits to Pyrex when making the transition
between levels in your optimized code.  All the common areas for
mistakes when interfacing with Python - missing refcounts when getting,
changing, or deleting attributes, calling Python callbacks, and using
basic Python data structures - are all taken care of so that the dive
into C is far less abrupt.  The people this PR is geared to are those
who would write in C because of efficiency concerns, so they would
likely be looking at Pyrex before choosing to start a large project, and
the simple code examples would be both familiar enough (hey, that's a 32
bit integer!  I remember that!) but also different enough (wow, you can
just call a script with "foo()"?) that they would be impressed.


From barry at python.org  Thu Dec  9 22:20:32 2004
From: barry at python.org (Barry Warsaw)
Date: Thu Dec  9 22:20:37 2004
Subject: Freezing Python (was Re: [Python-Dev] 2.4 news reaches interesting
	places)
In-Reply-To: <5.1.1.6.0.20041209102402.042c8580@mail.telecommunity.com>
References: <5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com>
	<5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com>
	<5.1.1.6.0.20041209102402.042c8580@mail.telecommunity.com>
Message-ID: <1102627232.16867.47.camel@geddy.wooz.org>

On Thu, 2004-12-09 at 10:30, Phillip J. Eby wrote:

> ISTR that for a long time, Visual Basic actually did the same thing.  A few 
> magazines mentioned the fact, but nobody really cared.  However, if this is 
> really a concern, bundle Pyrex as well.  Both Pyrex and py2exe are 
> distutils-based, so at that point you have a complete solution, including 
> the "C" meme as well as the ".exe" meme.

As an aside, I wonder if there's interest in collaborating on freeze
tools among the various existing solutions here.  I've recently needed
to look at this and I chose cx_Freeze, primarily because it supported
the platforms I needed.  It's a very good tool.

I've had some conversations with Anthony Tuininga on the cx_Freeze
mailing list and there seems to be some interest in perhaps putting
together a SIG or something.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20041209/ce332aec/attachment.pgp
From bob at redivi.com  Thu Dec  9 22:30:30 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Dec  9 22:31:08 2004
Subject: Freezing Python (was Re: [Python-Dev] 2.4 news reaches
	interesting places)
In-Reply-To: <1102627232.16867.47.camel@geddy.wooz.org>
References: <5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com>
	<5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com>
	<5.1.1.6.0.20041209102402.042c8580@mail.telecommunity.com>
	<1102627232.16867.47.camel@geddy.wooz.org>
Message-ID: <8CDE78F4-4A29-11D9-8CCD-000A9567635C@redivi.com>


On Dec 9, 2004, at 4:20 PM, Barry Warsaw wrote:

> On Thu, 2004-12-09 at 10:30, Phillip J. Eby wrote:
>
>> ISTR that for a long time, Visual Basic actually did the same thing.  
>> A few
>> magazines mentioned the fact, but nobody really cared.  However, if 
>> this is
>> really a concern, bundle Pyrex as well.  Both Pyrex and py2exe are
>> distutils-based, so at that point you have a complete solution, 
>> including
>> the "C" meme as well as the ".exe" meme.
>
> As an aside, I wonder if there's interest in collaborating on freeze
> tools among the various existing solutions here.  I've recently needed
> to look at this and I chose cx_Freeze, primarily because it supported
> the platforms I needed.  It's a very good tool.
>
> I've had some conversations with Anthony Tuininga on the cx_Freeze
> mailing list and there seems to be some interest in perhaps putting
> together a SIG or something.

That's certainly worth doing.  Currently we have, in general use, at 
least cx_Freeze, py2exe, and py2app.

I have been abstracting away a lot of the dependency finding issues in 
py2app into a cross-platform module called modulegraph 
<http://svn.red-bean.com/bob/py2app/trunk/src/modulegraph/>.  
Modulegraph supercedes the standard library modulefinder.  Because it 
uses a graph data structure, it gives you the flexibility say "pydoc 
doesn't *really* depend on Tkinter".  It also has functionality that 
lets you specify implicit dependencies (via PyImport_.. or such), so 
that it plays nicely enough with C extensions.

This code is basically at the point where py2exe and cx_Freeze could 
import it and use it instead of modulefinder (with a small amount of 
hacking, the API is different).

-bob

From aahz at pythoncraft.com  Thu Dec  9 22:35:56 2004
From: aahz at pythoncraft.com (Aahz)
Date: Thu Dec  9 22:35:58 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <Pine.LNX.4.10.10412091151530.11555-100000@sumeru.stanford.EDU>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE4A915A@au3010avexu1.global.avaya.com>
	<Pine.LNX.4.10.10412091151530.11555-100000@sumeru.stanford.EDU>
Message-ID: <20041209213556.GB4013@panix.com>

On Thu, Dec 09, 2004, Dennis Allison wrote:
>
> And, parenthetically, I continue to be amazed at the number of projects
> that use Python, but do it in stealth-mode and view it as their
> silver-bullet and competative edge.  I wish more people would publish
> their experience.

http://www.paulgraham.com/avg.html
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing."  --Alan Perlis
From pje at telecommunity.com  Thu Dec  9 22:37:59 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Dec  9 22:38:58 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <1102626693.16291.57.camel@localhost>
References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20041209162547.06531700@mail.telecommunity.com>

At 04:11 PM 12/9/04 -0500, Glyph Lefkowitz wrote:
>On Wed, 2004-12-08 at 17:39 -0500, Phillip J. Eby wrote:
>
> > The only thing that will fix the PR issue is to have a Python compiler
> > distributed as part of the language.  It doesn't matter if it doesn't
> > support the full generality of Python, or even if it doesn't speed many
> > operations up much.  The only real requirements are that it can be used to
> > produce "native" executables, and that it be an official part of the
> > language, not a separately-distributed tool like Psyco or Pyrex.  Then, it
> > will perhaps be a sufficient "security blanket" to stop people FUDding
> > about it.
>
>I am aware that Pyrex is still in flux, so perhaps it is too soon to
>propose this even for 2.5, but I think it's worth bringing up anyway: I
>would like Pyrex to be distributed with the Python core.  I agree that
>it should be modified to produce full .exe files and not just .dlls on
>Windows, but many potential users *are* seriously concerned about
>efficiency and not just simplifying distribution.

+1 on all the stuff you said, with one minor exception.  Pyrex-the-language 
is often unpythonically ugly and verbose at present.  If Python had an 
official syntax for optional static type declaration, Pyrex's syntax could 
be aligned with that, and that would at least eliminate most of the inline 
'cdef' ugliness, leaving only C type declarations and Python property 
declarations as the main syntax issues to be resolved.  (Maybe by using 
something like the 'ctypes' API, and having the compiler recognize that 
API, such that C is directly callable from Python anyway, so compiling or 
interpreting Python makes no difference to ability to access C...  but I 
digress.)

Of course, this would take some effort from the core developers, especially 
Guido, to consider the various syntax needs and formulate official 
solutions.  But if it were done, the Python-vs.-Pyrex distinction could 
fade away altogether, replaced with the meme, "just add type declarations 
to slow parts, and tell Python you want the module compiled to C."

IOW, if Pyrex is merely the name of a compiler, not a separate language, 
then our master plan for world domination is complete.  :)

From michael.walter at gmail.com  Thu Dec  9 22:43:19 2004
From: michael.walter at gmail.com (Michael Walter)
Date: Thu Dec  9 22:43:22 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <5.1.1.6.0.20041209162547.06531700@mail.telecommunity.com>
References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<1102626693.16291.57.camel@localhost>
	<5.1.1.6.0.20041209162547.06531700@mail.telecommunity.com>
Message-ID: <877e9a17041209134356f9290e@mail.gmail.com>

If I parse you correctly, this would be great.

- Michael


On Thu, 09 Dec 2004 16:37:59 -0500, Phillip J. Eby
<pje@telecommunity.com> wrote:
> At 04:11 PM 12/9/04 -0500, Glyph Lefkowitz wrote:
> 
> 
> >On Wed, 2004-12-08 at 17:39 -0500, Phillip J. Eby wrote:
> >
> > > The only thing that will fix the PR issue is to have a Python compiler
> > > distributed as part of the language.  It doesn't matter if it doesn't
> > > support the full generality of Python, or even if it doesn't speed many
> > > operations up much.  The only real requirements are that it can be used to
> > > produce "native" executables, and that it be an official part of the
> > > language, not a separately-distributed tool like Psyco or Pyrex.  Then, it
> > > will perhaps be a sufficient "security blanket" to stop people FUDding
> > > about it.
> >
> >I am aware that Pyrex is still in flux, so perhaps it is too soon to
> >propose this even for 2.5, but I think it's worth bringing up anyway: I
> >would like Pyrex to be distributed with the Python core.  I agree that
> >it should be modified to produce full .exe files and not just .dlls on
> >Windows, but many potential users *are* seriously concerned about
> >efficiency and not just simplifying distribution.
> 
> +1 on all the stuff you said, with one minor exception.  Pyrex-the-language
> is often unpythonically ugly and verbose at present.  If Python had an
> official syntax for optional static type declaration, Pyrex's syntax could
> be aligned with that, and that would at least eliminate most of the inline
> 'cdef' ugliness, leaving only C type declarations and Python property
> declarations as the main syntax issues to be resolved.  (Maybe by using
> something like the 'ctypes' API, and having the compiler recognize that
> API, such that C is directly callable from Python anyway, so compiling or
> interpreting Python makes no difference to ability to access C...  but I
> digress.)
> 
> Of course, this would take some effort from the core developers, especially
> Guido, to consider the various syntax needs and formulate official
> solutions.  But if it were done, the Python-vs.-Pyrex distinction could
> fade away altogether, replaced with the meme, "just add type declarations
> to slow parts, and tell Python you want the module compiled to C."
> 
> IOW, if Pyrex is merely the name of a compiler, not a separate language,
> then our master plan for world domination is complete.  :)
> 
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/michael.walter%40gmail.com
>
From glyph at divmod.com  Thu Dec  9 23:11:45 2004
From: glyph at divmod.com (Glyph Lefkowitz)
Date: Thu Dec  9 23:10:47 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <5.1.1.6.0.20041209162547.06531700@mail.telecommunity.com>
References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<5.1.1.6.0.20041209162547.06531700@mail.telecommunity.com>
Message-ID: <1102630305.16291.62.camel@localhost>

On Thu, 2004-12-09 at 16:37 -0500, Phillip J. Eby wrote:

> +1 on all the stuff you said, with one minor exception.  Pyrex-the-language 
> is often unpythonically ugly and verbose at present.

Personally I have no problem with Pyrex's current tradeoffs, but given
the amount of effort that this is going to inevitably involve, I'm sure
that at least some of the changes you suggested would be necessary.
So... +0 on your suggestions, I suppose.


From faassen at infrae.com  Fri Dec 10 01:09:05 2004
From: faassen at infrae.com (Martijn Faassen)
Date: Fri Dec 10 01:09:08 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
Message-ID: <41B8E921.2090903@infrae.com>

Guido van Rossum wrote:
[snip]
> One thing that bugs me: the article says 3 or 4 times that Python is
> slow, each time with a refutation ("but it's so flexible", "but it's
> fast enough") but still, they sure seem to harp on the point. This is
> a PR issue that Python needs to fight -- any ideas?

One thing you could do is announce loudly that the European Union is 
actually funding a international consortium of scientists, industry and 
software engineers to speed up Python (among other things :) -- PyPy.

I'm sure that a press release about this in the right channels could 
make quite a bit of positive noise. "Python, backed by the EU". Of 
course one worry is that the 'slow bureaucracy' meme would rub off on it.

Of course any press release would need to be discussed with the PyPy people.

Regards,

Martijn
From bob at redivi.com  Fri Dec 10 02:17:21 2004
From: bob at redivi.com (Bob Ippolito)
Date: Fri Dec 10 02:17:58 2004
Subject: [Python-Dev] Proposed Mac OS X change for 2.3.5 and 2.4.1 -
	MACOSX_DEPLOYMENT_TARGET vs. Python, round two, fight!
Message-ID: <3DBFC242-4A49-11D9-8CCD-000A9567635C@redivi.com>

I've included a lot of background information here, if you just want to 
know the details
of the proposed change, skip to the bottom.

As some of you may know, Python 2.4's configure script and distutils 
has some
tricky behavior with regard to the ``MACOSX_DEPLOYMENT_TARGET`` 
environment
variable on Mac OS X 10.3 and later.  Unless otherwise specified, 
assume that I
am talking about a build environment of Mac OS X 10.3 or later.  This 
behavior
is as follows:

If ``MACOSX_DEPLOYMENT_TARGET`` is not set during configure:

1.  distutils builds modules with
     ``-F/Python/Installation/Location -framework Python`` as per usual 
2.3.x
     behavior if it is also not set
2.  If ``MACOSX_DEPLOYMENT_TARGET`` is set during a later run of 
distutils, then
     distutils complains that "10.3" mismatches the configure time 
setting of ""

This Python framework has the following misfeatures:

1.  All your extensions are dependent on the installation location of 
this
     particular Python
2.  This installation of Python 2.4 *may break the building of 
extensions* for
     any previous version of Python that uses the same distutils 
behavior.  It
     will certainly break them if they are installed to the same 
framework
     directory, but if the version of Python 2.3 is early enough, such 
as the
     stock 2.3.0, it will break that too.  Also, any future version of 
Python
     installed to the same frameworks directory *will break the building 
of
     extensions* for this Python installation!
3.  The Python framework will not be compatible with versions of Mac OS 
X
     earlier than 10.3 anyway due to changes in libSystem!

This is stupid, and it should NOT be default behavior!

If ``MACOSX_DEPLOYMENT_TARGET`` is set to "10.3" or higher during 
configure:

1.  distutils builds modules with ``-undefined dynamic_lookup``
2.  If ``MACOSX_DEPLOYMENT_TARGET`` is set to something other than 
"10.3", or
     unset, it will complain that the current setting mismatches the 
configure
     setting of "10.3"

The features:

1.  All your extensions are independent of the Python installation 
location,
     and are thus compatible with any other Python with the same major 
version.
2.  This installation of Python 2.4 will still, unavoidably, break the 
building
     of extensions for any previous version of Python using the bad 
distutils
     behavior mentioned above.  This installation is not susceptible to
     breakage, however.

The misfeatures:

1.  You need to have ``MACOSX_DEPLOYMENT_TARGET`` set during configure, 
most
     people don't know to do this.
2.  You also need to have ``MACOSX_DEPLOYMENT_TARGET`` set when using
     distutils.  Even less people know to do this.

The solution to this is to have the following behavior instead:

1.  If no ``MACOSX_DEPLOYMENT_TARGET`` is set during configure, and the 
build
     machine is Mac OS X 10.3 or later, then set it to "10.3".
2.  If no ``MACOSX_DEPLOYMENT_TARGET`` is set during distutils, but it 
WAS
     set during configure, then set it to the configure time value.
3.  Otherwise, if it is set to a LOWER value, then fail.  Checking for 
an
     exact match is bad, because the user or extension author should be
     free to build a particular extension using 10.4 specific features
     against a Python that is 10.3+ compatible.

For a build machine running Mac OS X 10.2 or earlier, it should ignore
all of this behavior as per usual.

Unless anyone has a reasonable objection to this proposed solution, then
we should make sure it goes into Python 2.4.1 and Python 2.3.5.

These threads might also be of interest:
http://mail.python.org/pipermail/pythonmac-sig/2004-November/012192.html
http://mail.python.org/pipermail/pythonmac-sig/2004-December/012237.html
http://mail.python.org/pipermail/pythonmac-sig/2004-December/012275.html

-bob

From skip at pobox.com  Thu Dec  9 13:12:31 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Dec 10 03:29:04 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<000b01c4dda2$fe7a87e0$e841fea9@oemcomputer>
Message-ID: <16824.16687.343664.151558@montanaro.dyndns.org>


    Raymond> * Any PR effort should also emphasize that no usability
    Raymond>   trade-offs were made along the way.  A number of features
    Raymond>   make Py2.4 easier to use than 1.5.6: list comps, genexps,
    Raymond>   generators, sets, nested scopes, int/long unification,
    Raymond>   decorators, etc. 

Not to mention which, such beasts aren't commonly available for C.  What
about C++?

I found it interesting that a guy at work wrote a string caching class for
our C++ programmers to use.  He told me he got the idea from Python's int
caching.

Skip
From anthony at interlink.com.au  Fri Dec 10 03:31:20 2004
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Dec 10 03:31:37 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE4A915A@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE4A915A@au3010avexu1.global.avaya.com>
Message-ID: <200412101331.21291.anthony@interlink.com.au>

On Friday 10 December 2004 06:25, Delaney, Timothy C (Timothy) wrote:
> Michael Hudson wrote:
> > Anthony's Australian, people expect this sort of thing from him :)
>
> As another Australian, I think that "Making Python Not Suck" implies
> that if you don't do extra things, Python Sucks.
>
> This is not a good thing IMO.
>
> "Making Python Suck Less" would be even worse.

Don't worry - it will have a nice friendly title by the time I present it.
I'm currently aiming for OSCON, which isn't until August. It's going
to be aimed at people who _know_ Python, but aren't necessarily 
experts, and focus not just on performance, but maintainable and 
sane code (for instance, not many people seem to realise that 
__del__ methods are almost always not the right answer to a problem).

> How about "Python - You Can Have Your Cake And Eat It Too".

Nah - I have a bunch of ideas kicking around in my head, something
will pop out. I doubt I'll ever be able to top the title of my pycon talk,
though... 

Anthony
From kdart at kdart.com  Fri Dec 10 04:41:19 2004
From: kdart at kdart.com (Keith Dart)
Date: Fri Dec 10 04:41:30 2004
Subject: [Python-Dev] PyOS_InputHook enhancement proposal
In-Reply-To: <20041209131555.GB10465@panix.com>
References: <41B7D8C5.9000008@ims.u-tokyo.ac.jp>
	<20041209131555.GB10465@panix.com>
Message-ID: <41B91ADF.6040800@kdart.com>

Aahz wrote:
> On Thu, Dec 09, 2004, Michiel Jan Laurens de Hoon wrote:
> 
>>My suggestion is therefore to replace PyOS_InputHook by two functions
>>PyOS_AddInputHook and PyOS_RemoveInputHook, and let Python keep track of
>>which hooks are installed. This way, an extension module can add a hook
>>function without having to worry about other extension modules trying
>>to use the same hook.
>>
>>Any comments? Would I need to submit a PEP for this proposal?
> 
> 
> Because this is only for the C API, your best bet is to write a patch
> and submit it to SF.  If people whine or it gets rejected, then write a
> PEP.

I did modify the readline module that hooks this and can call back to a 
Python function. There are also methods for installing and removing the 
Python function. I did this for a different reason. I need Python signal 
handlers to run, and they don't run when the execution path is currently 
in some C code (such as readline). The hook forces the interpreter to 
run, and check for signals as a side effect. Using this I can be sitting 
in raw_input(), or interactive mode, and signal handlers are still run.

If you want it, let me know.

Actually, the modded readline module is part of pyNMS, on sourceforge.



-- 
                            \/ \/
                            (O O)
-- --------------------oOOo~(_)~oOOo----------------------------------------
Keith Dart <kdart@kdart.com>
vcard: <http://www.kdart.com/~kdart/kdart.vcf>
public key: ID: F3D288E4       URL: <http://www.kdart.com/~kdart/public.key>
============================================================================
From arigo at tunes.org  Fri Dec 10 10:41:28 2004
From: arigo at tunes.org (Armin Rigo)
Date: Fri Dec 10 10:50:49 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <86F33E75-498A-11D9-BB31-000D93C603C0@indranet.co.nz>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<86F33E75-498A-11D9-BB31-000D93C603C0@indranet.co.nz>
Message-ID: <20041210094128.GA15865@vicky.ecs.soton.ac.uk>

Hi Andrew,

On Thu, Dec 09, 2004 at 03:32:10PM +1300, Andrew McGregor wrote:
> The other thing we can do is finish the portable backend for psyco and  
> make it a standard module.  Then Python won't be slow, it will be  
> compiled, and py2exe will be able to make a single-file executable.

You probably mean that Psyco can dynamically compile Python bytecodes even if
they have been hidden into an .exe file by py2exe.  At first reading, it
appears that you are propagating the misconception "compiler == speed == .exe"
i.e. Psyco could "compile your program into an .exe file".  As far as I see,
this is definitely not possible.

Of course, for PR purposes the difference is small -- an .exe file that runs
at compiled speed, or at least machine-level VM speed -- but advertising
should work around and not build on top of existing confusion IMHO.


A bientot,

Armin
From arigo at tunes.org  Fri Dec 10 11:06:59 2004
From: arigo at tunes.org (Armin Rigo)
Date: Fri Dec 10 11:16:15 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <001401c4dda9$ad605680$e841fea9@oemcomputer>
References: <001401c4dda9$ad605680$e841fea9@oemcomputer>
Message-ID: <20041210100659.GB15865@vicky.ecs.soton.ac.uk>

Hi,

On Wed, Dec 08, 2004 at 11:43:49PM -0500, Raymond Hettinger wrote:
> Acceptance for Py2.4 partially hinges on how quickly third party apps
> have their binaries updated.
> 
> I wonder if there is anything we can do to help.

For people like myself, Linux programmers not developing on Windows every day,
there is precious little information available about how to compile our
extension modules for the new Windows distribution.  I was actually very
disappointed to have to google my way around until I found a page that
explained to me that to use Microsoft's free compilers you need to manually
patch the distutils included with 2.4.  (I know this is being worked on right
now but I'd have expected it to be present in the 2.4 release.)  (The only
page I could find still refers to a pre-final 2.4 so their distutils patch
doesn't apply without hand-editing, though that should be fixed by now.)

Moreover, the standard documentation's "Extending and Embedding" has
apparently not been updated at all since 2.3.  That's quite unexpected too...

In other words, if you want 3rd parties to compile Windows binaries for 2.4,
tell them how.

(I even hoped that just doing "python setup.py build" would fail but refer me
to a web page that explains me which free compilers I should get and install
from Microsoft.)


A bientot,

Armin.
From skip at pobox.com  Fri Dec 10 11:49:30 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Dec 10 11:49:47 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <20041210094128.GA15865@vicky.ecs.soton.ac.uk>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<86F33E75-498A-11D9-BB31-000D93C603C0@indranet.co.nz>
	<20041210094128.GA15865@vicky.ecs.soton.ac.uk>
Message-ID: <16825.32570.898575.571826@montanaro.dyndns.org>


    >> The other thing we can do is finish the portable backend for psyco
    >> and make it a standard module.  Then Python won't be slow, it will be
    >> compiled, and py2exe will be able to make a single-file executable.

    Armin> You probably mean that Psyco can dynamically compile Python
    Armin> bytecodes even if they have been hidden into an .exe file by
    Armin> py2exe.

I didn't read it that way.  My impression was that py2exe be modified to
include and enable psyco if it's available when building an .exe.  You
would, in theory, get a single file distribution as well as dynamic
compilation.

Skip
From theller at python.net  Fri Dec 10 12:12:36 2004
From: theller at python.net (Thomas Heller)
Date: Fri Dec 10 12:11:49 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <16825.32570.898575.571826@montanaro.dyndns.org> (Skip
	Montanaro's message of "Fri, 10 Dec 2004 04:49:30 -0600")
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<86F33E75-498A-11D9-BB31-000D93C603C0@indranet.co.nz>
	<20041210094128.GA15865@vicky.ecs.soton.ac.uk>
	<16825.32570.898575.571826@montanaro.dyndns.org>
Message-ID: <y8g676fv.fsf@python.net>

Skip Montanaro <skip@pobox.com> writes:

>     >> The other thing we can do is finish the portable backend for psyco
>     >> and make it a standard module.  Then Python won't be slow, it will be
>     >> compiled, and py2exe will be able to make a single-file executable.
>
>     Armin> You probably mean that Psyco can dynamically compile Python
>     Armin> bytecodes even if they have been hidden into an .exe file by
>     Armin> py2exe.
>
> I didn't read it that way.  My impression was that py2exe be modified to
> include and enable psyco if it's available when building an .exe.  You
> would, in theory, get a single file distribution as well as dynamic
> compilation.

I haven't tried it, but using psyco in a script and building an exe
from it with py2exe should work right out of the box - but maybe this
isn't what you had in mind?

And single-file exes with py2exe are on my plan for a medium time range,
although currently I'm not really sure which way to go.

Thomas

From mwh at python.net  Fri Dec 10 12:56:38 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Dec 10 12:56:39 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <e8bf7a5304120911564cef0c53@mail.gmail.com> (Jeremy Hylton's
	message of "Thu, 9 Dec 2004 14:56:41 -0500")
References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<5.1.1.6.0.20041209131843.038f3530@mail.telecommunity.com>
	<CDB95F16-4A16-11D9-88DA-000A95A50FB2@fuhm.net>
	<e8bf7a5304120911564cef0c53@mail.gmail.com>
Message-ID: <2mfz2epds9.fsf@starship.python.net>

Jeremy Hylton <jhylton@gmail.com> writes:

> I agree, although it's not clear to me how much faster it will be in
> the future.  Making a *fast* Python based on our own virtual execution
> environment (as opposed to piggybacking a JVM or CLR) is a big
> project.  It's not clear than anyone has enough resources to make
> credible effort

I think the EU does!

Cheers,
mwh

-- 
  NUTRIMAT:  That drink was individually tailored to meet your
             personal requirements for nutrition and pleasure.
    ARTHUR:  Ah.  So I'm a masochist on a diet am I?
                    -- The Hitch-Hikers Guide to the Galaxy, Episode 9
From mwh at python.net  Fri Dec 10 13:02:59 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Dec 10 13:03:01 2004
Subject: [Python-Dev] PyOS_InputHook enhancement proposal
In-Reply-To: <41B91ADF.6040800@kdart.com> (Keith Dart's message of "Thu, 09
	Dec 2004 19:41:19 -0800")
References: <41B7D8C5.9000008@ims.u-tokyo.ac.jp>
	<20041209131555.GB10465@panix.com> <41B91ADF.6040800@kdart.com>
Message-ID: <2mbrd2pdho.fsf@starship.python.net>

Keith Dart <kdart@kdart.com> writes:

> I did modify the readline module that hooks this and can call back
> to a Python function. There are also methods for installing and
> removing the Python function. I did this for a different reason. I
> need Python signal handlers to run, and they don't run when the
> execution path is currently in some C code (such as readline). The
> hook forces the interpreter to run, and check for signals as a side
> effect. Using this I can be sitting in raw_input(), or interactive
> mode, and signal handlers are still run.

Have you seen what happened to the signal handling code in readline in
Python 2.4?  I don't think your modifications will be needed anymore,
but you should check.

Cheers,
mwh

-- 
  I don't remember any dirty green trousers.
                                             -- Ian Jackson, ucam.chat
From p.f.moore at gmail.com  Fri Dec 10 13:06:01 2004
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri Dec 10 13:06:04 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <20041210100659.GB15865@vicky.ecs.soton.ac.uk>
References: <001401c4dda9$ad605680$e841fea9@oemcomputer>
	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>
Message-ID: <79990c6b041210040671584abc@mail.gmail.com>

On Fri, 10 Dec 2004 10:06:59 +0000, Armin Rigo <arigo@tunes.org> wrote:
> For people like myself, Linux programmers not developing on Windows every day,
> there is precious little information available about how to compile our
> extension modules for the new Windows distribution.  I was actually very
> disappointed to have to google my way around until I found a page that
> explained to me that to use Microsoft's free compilers you need to manually
> patch the distutils included with 2.4.  (I know this is being worked on right
> now but I'd have expected it to be present in the 2.4 release.)  (The only
> page I could find still refers to a pre-final 2.4 so their distutils patch
> doesn't apply without hand-editing, though that should be fixed by now.)
[...]
> In other words, if you want 3rd parties to compile Windows binaries for 2.4,
> tell them how.

I think that the details about the Microsoft free compilers is a bit
misleading. Sure, it's great that it exists, but at the present time,
it is not the best free option (too many downloads required, too many
rough edges).

For most C extensions, the best free option is mingw. This is fully
supported by distutils, and has been for a while. It is documented in
the manuals, but basically all you need is to do the following:

1. Obtain a reasonably recent mingw (3.3.3 definitely works, you need
msvcr71 support).
2. Install it, and make sure it is in your PATH.
3. Build libpython24.a as documented in the Python manuals (recent
versions of mingw can read MS import libraries, so this may no longer
be needed, but I haven't tested this yet).
4. python setup.py build --compiler=mingw32.

The only difficulties are:

1. Obtaining support libraries like jpeg, sdl, etc. It *may* be a
nuisance getting mingw-compatible build, but as I say above, MS-format
libraries may now be fine, and there are good sources of
mingw-compatible libraries like gnuwin32.sourceforge.net.
2. Microsoft-specific code. (As I'm replying to Armin, maybe the fact
that inline assembler formats are different is relevant :-) :-))

To be honest, I'd only ever use the MS free compilers if I had a
critical need to build an extension which had some highly MS-specific
code or build processes. And even then, I'd probably give up and wait
for someone with MSVC7 to produce a binary...

If anyone has any particular modules (of their own, or third party)
they have problems with, I'd be happy to have a go at building, and
report back. Maybe a page on the Python Wiki about building modules
using mingw would be worth adding. Hmm, might do that tonight...

Hope this helps,
Paul.
From skip at pobox.com  Fri Dec 10 16:10:41 2004
From: skip at pobox.com (Skip Montanaro)
Date: Fri Dec 10 16:10:52 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <y8g676fv.fsf@python.net>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<86F33E75-498A-11D9-BB31-000D93C603C0@indranet.co.nz>
	<20041210094128.GA15865@vicky.ecs.soton.ac.uk>
	<16825.32570.898575.571826@montanaro.dyndns.org>
	<y8g676fv.fsf@python.net>
Message-ID: <16825.48241.997055.384959@montanaro.dyndns.org>


    Thomas> I haven't tried it, but using psyco in a script and building an
    Thomas> exe from it with py2exe should work right out of the box - but
    Thomas> maybe this isn't what you had in mind?

I was thinking of implicitly mixing in psyco, even if the script didn't use
it.  Maybe I have too much faith in psyco. ;-)

Skip
From pje at telecommunity.com  Fri Dec 10 16:17:42 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Dec 10 16:19:19 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <20041210100659.GB15865@vicky.ecs.soton.ac.uk>
References: <001401c4dda9$ad605680$e841fea9@oemcomputer>
	<001401c4dda9$ad605680$e841fea9@oemcomputer>
Message-ID: <5.1.1.6.0.20041210101609.020ed6e0@mail.telecommunity.com>

At 10:06 AM 12/10/04 +0000, Armin Rigo wrote:
>Hi,
>
>On Wed, Dec 08, 2004 at 11:43:49PM -0500, Raymond Hettinger wrote:
> > Acceptance for Py2.4 partially hinges on how quickly third party apps
> > have their binaries updated.
> >
> > I wonder if there is anything we can do to help.
>
>For people like myself, Linux programmers not developing on Windows every day,
>there is precious little information available about how to compile our
>extension modules for the new Windows distribution.

I personally find MinGW (by way of Cygwin) to be the easiest way to do 
it.  I posted here previously with the procedure and even a script to 
prepare the necessary libpython24.a file.

From arigo at tunes.org  Fri Dec 10 18:07:50 2004
From: arigo at tunes.org (Armin Rigo)
Date: Fri Dec 10 18:16:55 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <16825.32570.898575.571826@montanaro.dyndns.org>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<86F33E75-498A-11D9-BB31-000D93C603C0@indranet.co.nz>
	<20041210094128.GA15865@vicky.ecs.soton.ac.uk>
	<16825.32570.898575.571826@montanaro.dyndns.org>
Message-ID: <20041210170750.GA12032@vicky.ecs.soton.ac.uk>

Hi Skip,

On Fri, Dec 10, 2004 at 04:49:30AM -0600, Skip Montanaro wrote:
> 
>     >> The other thing we can do is finish the portable backend for psyco
>     >> and make it a standard module.  Then Python won't be slow, it will be
>     >> compiled, and py2exe will be able to make a single-file executable.
> 
>     Armin> You probably mean that Psyco can dynamically compile Python
>     Armin> bytecodes even if they have been hidden into an .exe file by
>     Armin> py2exe.
> 
> I didn't read it that way.  My impression was that py2exe be modified to
> include and enable psyco if it's available when building an .exe.  You
> would, in theory, get a single file distribution as well as dynamic
> compilation.

Yes, I agree with this.  What I meant is that when I first read the original
paragraph (the 1st one quoted above), I thought it meant that in the future
py2exe and Psyco could be combined in such a way that we'd essentially have a
compiler from Python producing a "classical" compiled binary.  A lot of people
could read it that way.

The question is if we should advertise a Psyco+py2exe combination using a
similar wording, such that it superficially sounds like we are doing a
"classical" compilation from .py to .exe, whereas it actually means that we
are hiding Psyco with the bytecodes in a .exe.  After all, from a user's (or
journalist's) point of view the result is similar, performancewise.

Whether Psyco is reliable enough for this is yet another issue...  I'd
classify Psyco as "mostly reliable" only...


A bientot,

Armin.
From arigo at tunes.org  Fri Dec 10 18:19:21 2004
From: arigo at tunes.org (Armin Rigo)
Date: Fri Dec 10 18:28:22 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <79990c6b041210040671584abc@mail.gmail.com>
References: <001401c4dda9$ad605680$e841fea9@oemcomputer>
	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>
	<79990c6b041210040671584abc@mail.gmail.com>
Message-ID: <20041210171921.GB12032@vicky.ecs.soton.ac.uk>

Hi,

On Fri, Dec 10, 2004 at 12:06:01PM +0000, Paul Moore wrote:
> For most C extensions, the best free option is mingw.

Sorry, I was not aware that mingw supports the new VC7.1-type of runtime that 
is needed for the extension module to load with the official Python 2.4 
distribution.

Note that compiling with Microsoft's free compiler (after the huge downloads
and the manual patching) worked just fine, so I think it is worth a note
somewhere.

Another note: can you report on whether building libpython24.a can be skipped
for mingw?  I'm thinking about the specific situation where we want on-site
compilation of extension modules with a minimal number of things to install
first.  E.g. if we need to compile libpython24.a it means we need to fetch the
Python sources themselves first.  (A use case is for prorgams that dynamically
produce and compile extension modules, as "weave" or the PyPy test suite do.)


A bientot,

Armin.
From pje at telecommunity.com  Fri Dec 10 19:05:17 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Dec 10 19:07:03 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <20041210171921.GB12032@vicky.ecs.soton.ac.uk>
References: <79990c6b041210040671584abc@mail.gmail.com>
	<001401c4dda9$ad605680$e841fea9@oemcomputer>
	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>
	<79990c6b041210040671584abc@mail.gmail.com>
Message-ID: <5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com>

At 05:19 PM 12/10/04 +0000, Armin Rigo wrote:

>Another note: can you report on whether building libpython24.a can be skipped
>for mingw?  I'm thinking about the specific situation where we want on-site
>compilation of extension modules with a minimal number of things to install
>first.  E.g. if we need to compile libpython24.a it means we need to fetch the
>Python sources themselves first.

Actually, no, you don't need the sources.  See:

http://mail.python.org/pipermail/python-dev/2004-January/041676.html

for a script that builds libpython24.a from the python24.lib distributed 
with Python for Windows.

From bob at redivi.com  Fri Dec 10 19:12:04 2004
From: bob at redivi.com (Bob Ippolito)
Date: Fri Dec 10 19:12:42 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com>
References: <79990c6b041210040671584abc@mail.gmail.com>
	<001401c4dda9$ad605680$e841fea9@oemcomputer>
	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>
	<79990c6b041210040671584abc@mail.gmail.com>
	<5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com>
Message-ID: <FF3971F9-4AD6-11D9-80F2-000A9567635C@redivi.com>

On Dec 10, 2004, at 1:05 PM, Phillip J. Eby wrote:

> At 05:19 PM 12/10/04 +0000, Armin Rigo wrote:
>
>> Another note: can you report on whether building libpython24.a can be 
>> skipped
>> for mingw?  I'm thinking about the specific situation where we want 
>> on-site
>> compilation of extension modules with a minimal number of things to 
>> install
>> first.  E.g. if we need to compile libpython24.a it means we need to 
>> fetch the
>> Python sources themselves first.
>
> Actually, no, you don't need the sources.  See:
>
> http://mail.python.org/pipermail/python-dev/2004-January/041676.html
>
> for a script that builds libpython24.a from the python24.lib 
> distributed with Python for Windows.

Shouldn't this be distributed with binary distributions of Python, to 
save people the trouble?

Or, if it can be skipped, the procedure for doing a mingw build with 
the .lib should be documented and that'd be the end of it.

-bob

From pje at telecommunity.com  Fri Dec 10 19:19:10 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Dec 10 19:20:58 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <FF3971F9-4AD6-11D9-80F2-000A9567635C@redivi.com>
References: <5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com>
	<79990c6b041210040671584abc@mail.gmail.com>
	<001401c4dda9$ad605680$e841fea9@oemcomputer>
	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>
	<79990c6b041210040671584abc@mail.gmail.com>
	<5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20041210131425.030fd360@mail.telecommunity.com>

At 01:12 PM 12/10/04 -0500, Bob Ippolito wrote:
>On Dec 10, 2004, at 1:05 PM, Phillip J. Eby wrote:
>>At 05:19 PM 12/10/04 +0000, Armin Rigo wrote:
>>
>>>Another note: can you report on whether building libpython24.a can be 
>>>skipped
>>>for mingw?  I'm thinking about the specific situation where we want on-site
>>>compilation of extension modules with a minimal number of things to install
>>>first.  E.g. if we need to compile libpython24.a it means we need to 
>>>fetch the
>>>Python sources themselves first.
>>
>>Actually, no, you don't need the sources.  See:
>>
>>http://mail.python.org/pipermail/python-dev/2004-January/041676.html
>>
>>for a script that builds libpython24.a from the python24.lib distributed 
>>with Python for Windows.
>
>Shouldn't this be distributed with binary distributions of Python, to save 
>people the trouble?

The Python developers who produce the Windows binaries don't use 
mingw/cygwin, so this would put a maintenance burden on them.


>Or, if it can be skipped, the procedure for doing a mingw build with the 
>.lib should be documented and that'd be the end of it.

It's actually documented now, in the "Installing Python Modules" manual.  See:

http://docs.python.org/inst/tweak-flags.html#SECTION000622000000000000000

It just would need to have the libpython.a instructions removed in that case.

From amk at amk.ca  Fri Dec 10 20:01:55 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Fri Dec 10 20:02:27 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <20041209141104.GA18348@phd.pp.ru>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<000b01c4dda2$fe7a87e0$e841fea9@oemcomputer>
	<20041209102250.GB12506@phd.pp.ru> <cp9m4m$isn$1@sea.gmane.org>
	<20041209141104.GA18348@phd.pp.ru>
Message-ID: <20041210190155.GA11443@rogue.amk.ca>

On Thu, Dec 09, 2004 at 05:11:04PM +0300, Oleg Broytmann wrote:
> some popular areas. Let's add another topic, "Making things fast". Let's
> even make it the first topic, though I personnaly dont see a need for
> this.

The topic guides are migrating into the Wiki, and there's already a Wiki page
about this: http://www.python.org/moin/PythonSpeed

--amk
From phd at mail2.phd.pp.ru  Fri Dec 10 20:07:58 2004
From: phd at mail2.phd.pp.ru (Oleg Broytmann)
Date: Fri Dec 10 20:08:00 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <20041210190155.GA11443@rogue.amk.ca>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<000b01c4dda2$fe7a87e0$e841fea9@oemcomputer>
	<20041209102250.GB12506@phd.pp.ru> <cp9m4m$isn$1@sea.gmane.org>
	<20041209141104.GA18348@phd.pp.ru>
	<20041210190155.GA11443@rogue.amk.ca>
Message-ID: <20041210190758.GC16678@phd.pp.ru>

On Fri, Dec 10, 2004 at 02:01:55PM -0500, A.M. Kuchling wrote:
> On Thu, Dec 09, 2004 at 05:11:04PM +0300, Oleg Broytmann wrote:
> > some popular areas. Let's add another topic, "Making things fast". Let's
> > even make it the first topic, though I personnaly dont see a need for
> > this.
> 
> The topic guides are migrating into the Wiki, and there's already a Wiki page
> about this: http://www.python.org/moin/PythonSpeed

   Thank you. Actually, I've thought about wiki when I was writing that
email, but I was afarid that people who (in my very humble opinion) are
overreacting - will not tolerate putting such an important topic into a
wiki. My personnal opinion is that a good topic about the speed of
Python in the wiki is enough.

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd@phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.
From jhylton at gmail.com  Fri Dec 10 20:28:13 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Fri Dec 10 20:28:16 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <20041210190155.GA11443@rogue.amk.ca>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<000b01c4dda2$fe7a87e0$e841fea9@oemcomputer>
	<20041209102250.GB12506@phd.pp.ru> <cp9m4m$isn$1@sea.gmane.org>
	<20041209141104.GA18348@phd.pp.ru>
	<20041210190155.GA11443@rogue.amk.ca>
Message-ID: <e8bf7a530412101128326aea7b@mail.gmail.com>

On Fri, 10 Dec 2004 14:01:55 -0500, A.M. Kuchling <amk@amk.ca> wrote:
> On Thu, Dec 09, 2004 at 05:11:04PM +0300, Oleg Broytmann wrote:
> > some popular areas. Let's add another topic, "Making things fast". Let's
> > even make it the first topic, though I personnaly dont see a need for
> > this.
> 
> The topic guides are migrating into the Wiki, and there's already a Wiki page
> about this: http://www.python.org/moin/PythonSpeed

The Wiki entry seems to reinforce the impression that bugged Guido to
begin with.  It provides a bunch of "but ..." explanations about why
Python's speed isn't that important.  Python is slow, but "speed of
development is far more important."

If you want to avoid people saying "Python is slow, but ..." you need
to give them a different message.  "Writing code quickly is crucial in
today's marketplace."  (whatever, that's not much of a message.) 
David's dynamic languages whitepaper has more of message, for example.

Jeremy
From p.f.moore at gmail.com  Fri Dec 10 21:40:10 2004
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri Dec 10 21:40:13 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <20041210171921.GB12032@vicky.ecs.soton.ac.uk>
References: <001401c4dda9$ad605680$e841fea9@oemcomputer>
	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>
	<79990c6b041210040671584abc@mail.gmail.com>
	<20041210171921.GB12032@vicky.ecs.soton.ac.uk>
Message-ID: <79990c6b041210124039803de9@mail.gmail.com>

On Fri, 10 Dec 2004 17:19:21 +0000, Armin Rigo <arigo@tunes.org> wrote:
> Another note: can you report on whether building libpython24.a can be skipped
> for mingw?

A first attempt seems to almost work. There is a problem with
structures, however - I get an error about unresolved references to
_imp___Py_NoneStruct. My guess is that there's some issue with
resolving references to data (as opposed to functions) from DLLs. I'll
investigate further and then report back.

But this is specifically if you *don't* build libpytho24.a. If you
build that, there's no problem at all.

Paul.
From p.f.moore at gmail.com  Fri Dec 10 23:20:29 2004
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri Dec 10 23:20:32 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <79990c6b041210124039803de9@mail.gmail.com>
References: <001401c4dda9$ad605680$e841fea9@oemcomputer>
	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>
	<79990c6b041210040671584abc@mail.gmail.com>
	<20041210171921.GB12032@vicky.ecs.soton.ac.uk>
	<79990c6b041210124039803de9@mail.gmail.com>
Message-ID: <79990c6b0412101420522d59f3@mail.gmail.com>

On Fri, 10 Dec 2004 20:40:10 +0000, Paul Moore <p.f.moore@gmail.com> wrote:
> On Fri, 10 Dec 2004 17:19:21 +0000, Armin Rigo <arigo@tunes.org> wrote:
> > Another note: can you report on whether building libpython24.a can be skipped
> > for mingw?
> 
> A first attempt seems to almost work. There is a problem with
> structures, however - I get an error about unresolved references to
> _imp___Py_NoneStruct. My guess is that there's some issue with
> resolving references to data (as opposed to functions) from DLLs. I'll
> investigate further and then report back.

OK. After some more extensive investigation:

1. You cannot use python24.lib (ie, skip building libpython24.a). If
you do, data values exported from python24.dll (specifically,
Py_NoneStruct, which is quite important :-)) do not get exported
correctly. I don't know exactly why - I guess it's something to do
with how the MS LIB format works.

2. You can, however, link direct to python24.dll, which works fine.
Unfortunately, there are 2 problems with this - first, distutils
doesn't support it as things stand, and second, it isn't possible to
get the full path of the Python DLL without using Mark Hammond's
pywin32, or custom C code :-( (You need to get from sys.dllhandle to
the filename, and the API to do this isn't exposed).

3. In some cases, an extension built with mingw still contains
references to msvcrt (I think Martin has noted this). This seems *not*
to be an issue. What is happening, is that any symbols referenced from
user code are satisfied from msvcr71, and so match the Python DLL.
However, if the mingw startup code uses symbols which have not been
satisfied by the time all references from user code are resolved, then
these references are satisfied from msvcrt. But I can't see any way
that these references are going to do any harm, as they are startup
code, which by its nature will have been written to work cleanly in
this situation (linking like this is explicitly supported by mingw).
Martin, as the CRT compatibility expert, does this sound reasonable?
The mingw people certainly seem to think that the situation is OK...

You could probably incorporate Philip Eby's script, posted here a
while ago, which he mentioned earlier in this thread, into a setup.py
so that if libpython24.a didn't already exist, the setup script would
build it. This would give you a reasonably clean way of making sure it
was present. Philip's script doesn't need anything but mingw
installed, AIUI.

Longer term, I'll see if I can put together a distutils patch to make
it reference pythonXX.dll directly, so removing the need for
libpythonXX.a in the future. Obviously, this is only going to be
relevant for Python 2.5 and later, though.

Sorry for the long mail, but I thought it would be useful to get the
details in the archives...

Paul.
From python at rcn.com  Fri Dec 10 23:42:29 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Dec 10 23:45:36 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <e8bf7a530412101128326aea7b@mail.gmail.com>
Message-ID: <001d01c4df09$87e5caa0$e841fea9@oemcomputer>

> The Wiki entry seems to reinforce the impression that bugged Guido to
> begin with.  It provides a bunch of "but ..." explanations about why
> Python's speed isn't that important.  Python is slow, but "speed of
> development is far more important."

I felt the same way when reading it.  Also, it seemed to embody the
political outlook that optimization is sinful.  The document could be
much more positive, fact based, and informative.  Also, the wording
seems somewhat outdated.

A draft for a new entry is included below.  Review and feedback are
requested.



Raymond





Techniques for Improving Performance and Scalability
====================================================

Here are some coding guidelines for applications that demand peek
performance (in terms of memory utilization, speed, or scalability).


Use the best algorithms and fastest tools
-----------------------------------------

.	Membership testing with sets and dictionaries is much faster,
O(1), than searching sequences, O(n).  When testing "a in b", b should
be a set or dictionary instead of a list or tuple.

.	String concatenation is best done with ''.join(seq) which is an
O(n) process.  In contrast, using the '+' or '+=' operators can result
in an O(n**2) process because new strings may be built for each
intermediate step.  Py2.4 mitigates this issue somewhat; however,
''.join(seq) remains the best practice.

.	Many tools come in both list form and iterator form (map and
itertools.imap, list comprehension and generator expressions, dict.items
and dict.iteritems).  In general, the iterator forms are more memory
friendly and more scalable.  They are preferred whenever a real list is
not required.

.	Many core building blocks are coded in optimized C.
Applications that take advantage of them can make substantial
performance gains.  The building blocks include all of the builtin
datatypes (lists, tuples, sets, and dictionaries) and extension modules
like array, itertools, and collections.deque.

.	Likewise, the builtin functions run faster than hand-built
equivalents.  For example, map(operator.add, v1, v2) is faster than
map(lambda x,y: x+y, v1, v2).

.	List comprehensions run a bit faster than equivalent for-loops.

.	Lists perform well as fixed length arrays and as stacks.  For
queue applications using pop(0) or insert(0,v)), collections.deque()
offers superior O(1) performance because it avoids rebuilding a full
list for each insertion or deletion, an O(n) process.

.	Custom sort ordering is best performed with Py2.4's key= option
or with the traditional decorate-sort-undecorate technique.  Both
approaches call the key function just once per element.  In contrast,
sort's cmp= option is called many times per element during a sort.  For
example, sort(key=str.lower) is faster than sort(cmp=lambda a,b:
cmp(a.lower(), b.lower())).


Take advantage of interpreter optimizations
-------------------------------------------

.	In functions, local variables are accessed more quickly than
global variables, builtins, and attribute lookups.  So, it is sometimes
worth localizing variable access in inner-loops.  For example, the code
for random.shuffle() localizes access with the line, random=self.random.
That saves the shuffling loop from having to repeatedly lookup
self.random.  Outside of loops, the gain is minimal and rarely worth it.

.	The previous recommendation is a generalization of the rule to
factor constant expressions out of loops.  Also, constant folding should
be done manually.  Inside loops, write "x=3" instead of "x=1+2".

.	Function call overhead is large compared to other instructions.
Accordingly, it is sometimes worth in-lining code in the middle of a few
time-critical loops.

.	Starting with Py2.3, the interpreter optimizes "while 1" to just
a single jump.  In contrast "while True" takes several more steps.
While the latter is preferred for clarity, time critical code should use
the first form.

.	Multiple assignment is slower than individual assignment.  For
example "x,y=a,b" is slower than "x=a; y=b".  However, multiple
assignment is faster for variable swaps.  For example, "x,y=y,x" is
faster than "t=x; x=y; y=t".

.	Chained comparisons are faster than using the "and" operator.
Write "x < y < z" instead of "x < y and y < z".

.	A few fast approaches should be considered hacks and reserved
for only the most demanding applications.  For example, "x and True or
False" is faster than "bool(x)".


Take advantage of diagnostic tools
----------------------------------

.	The hotshot and profile modules help identify performance
bottlenecks.

.	The timeit module offers immediate performance comparisons
between alternative approaches to writing individual lines of code.


Performance can dictate overall strategy
----------------------------------------

.	SAX is typically faster and more memory efficient than DOM
approaches to XML.
.	Threading can improve the response time in applications that
would otherwise waste time waiting for I/O.
.	Select can help minimize the overhead for polling multiple
sockets.

From martin at v.loewis.de  Sat Dec 11 00:45:01 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Dec 11 00:45:06 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <20041210100659.GB15865@vicky.ecs.soton.ac.uk>
References: <001401c4dda9$ad605680$e841fea9@oemcomputer>
	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>
Message-ID: <41BA34FD.6050306@v.loewis.de>

Armin Rigo wrote:
> In other words, if you want 3rd parties to compile Windows binaries for 2.4,
> tell them how.

The straight-forward answer is: Get VC7.1 (aka VS.NET 2003), and invoke

python setup.py bdist_wininst

That's not too hard to do, I think.

Regards,
Martin
From martin at v.loewis.de  Sat Dec 11 00:52:52 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Dec 11 00:52:55 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <5.1.1.6.0.20041210131425.030fd360@mail.telecommunity.com>
References: <5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com>	<79990c6b041210040671584abc@mail.gmail.com>	<001401c4dda9$ad605680$e841fea9@oemcomputer>	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>	<79990c6b041210040671584abc@mail.gmail.com>	<5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com>
	<5.1.1.6.0.20041210131425.030fd360@mail.telecommunity.com>
Message-ID: <41BA36D4.2080903@v.loewis.de>

Phillip J. Eby wrote:
> The Python developers who produce the Windows binaries don't use 
> mingw/cygwin, so this would put a maintenance burden on them.

If this were completely automatic (e.g. part of msi.py), I'd happily
add all utilities needed to integrated this into the build process.
For this to happen:
1. Somebody else needs to do the actual work. I.e. write a patch
    to msi.py that invokes all the necessary magic, and test that
    patch.
2. The patch should avoid manual configuration of the "use cygwin"
    case. I.e. it should try to find the cygwin on the path, or in
    their standard location, or use any other mechanism available
    to locate cygwin. It should produce an error message if cygwin
    tools cannot be found, and then proceed without out creating
    libpython24.a

Regards,
Martin
From chui.tey at advdata.com.au  Fri Dec 10 12:53:16 2004
From: chui.tey at advdata.com.au (Chui G. Tey)
Date: Sat Dec 11 04:24:58 2004
Subject: [Python-Dev] Supporting Third Party Modules (was The other Py2.4
	issue)
Message-ID: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au>

>Raymond:
>
>Acceptance for Py2.4 partially hinges on how quickly third party apps
>have their binaries updated.
>
>I wonder if there is anything we can do to help.


One good way of helping out is to provide an dynamic loading function
that third party modules could access the basic python functions such as
PyArgParseTuple, PyString_AsString etc regardless of which python the
user is running. This would be similar to the COM approach. You can load
all the function pointers into a struct and then call them. 

Third party modules would link against this DLL independent of which
python is being used. 

Chui



From gvwilson at cs.toronto.edu  Tue Dec  7 21:32:58 2004
From: gvwilson at cs.toronto.edu (Greg Wilson)
Date: Sat Dec 11 04:25:52 2004
Subject: [Python-Dev] decorators article for DDJ?
Message-ID: <Pine.GSO.4.58.0412071532260.9235@dvp.cs>

Hi folks.  Is anyone interested in doing an article on Python decorators
for "Doctor Dobb's Journal"?  If so, please drop me a line...

Thanks,
Greg
From neal at metaslash.com  Thu Dec  9 03:33:50 2004
From: neal at metaslash.com (Neal Norwitz)
Date: Sat Dec 11 04:26:49 2004
Subject: [Python-Dev] removing aclocal.m4
Message-ID: <20041209023350.GB7217@janus.swcomplete.com>

Can we remove aclocal.m4?  The last log message states:

        fix for bug #811160 - autoconf vs. hp/ux system header files.
        also applied to release23-maint.

        Note that aclocal.m4 can go away when autoconf 2.58 is out.

And configure says:

        # Generated by GNU Autoconf 2.59 for python 2.5.

Neal
From ncoghlan at iinet.net.au  Sat Dec 11 05:41:24 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sat Dec 11 05:41:31 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <001d01c4df09$87e5caa0$e841fea9@oemcomputer>
References: <001d01c4df09$87e5caa0$e841fea9@oemcomputer>
Message-ID: <41BA7A74.1010302@iinet.net.au>

Raymond Hettinger wrote:
> guidelines for applications that demand peek performance (in terms of memory

Peak performance, perhaps? :) Anyway, it looks pretty good to me, but I have a 
few additional ideas.


Add a section of Caveats (we know they exist - might as well be upfront about it):

Caveats
--------
.  This document is based primarily on the CPython interpreter from 
www.python.org. However, most of the recommendations should apply to any Python 
interpreter that supports the mentioned feature.

.  For small data sets, some of the suggestions will perform more slowly than 
the approaches they are given as alternatives too. This is due to the fact that 
those suggestions introduce a little more fixed overhead in order to avoid 
overhead in processing each data item. The crossover point (where the more 
scaleable approach begins to show a net performance gain) is generally 
application specific. Use the diagnostic tools mentioned later to make an 
informed decision for your application.

> Use the best algorithms and fastest tools 
> -----------------------------------------
 > .	Membership testing with sets and dictionaries is much faster,
 > O(1), than searching sequences, O(n).  When testing "a in b", b should
 > be a set or dictionary instead of a list or tuple.

Should the bisect module be mentioned? If you have a sorted list already, using
bisect may be faster than constructing an intermediate set. Also, you can use 
bisect if an item's *position* in the list is important. list.index(x) uses a 
linear search (since it can't assume a sorted list), whereas bisect expects the 
list to be pre-sorted.

> intermediate step.  Py2.4 mitigates this issue somewhat; however, 
> ''.join(seq) remains the best practice.

I'd say 'The CPython 2.4 interpreter', rather than just Py2.4.

> .	List comprehensions run a bit faster than equivalent for-loops.

I'd move this to the next section (the algorithms are the same, but the
interpreter can speed up one more than the other)

> .	Custom sort ordering is best performed with Py2.4's key= option or with the
> traditional decorate-sort-undecorate technique.  Both approaches call the key
> function just once per element.  In contrast, sort's cmp= option is called
> many times per element during a sort.  For example, sort(key=str.lower) is
> faster than sort(cmp=lambda a,b: cmp(a.lower(), b.lower())).

If sorting is going to be performed repeatedly (e.g. maintaining a list in 
sorted order), it may be feasible to store the list of keys, rather than 
regenerating them every time. This also plays nicely with using the bisect 
module to update the list (as the list of keys is available to determine the 
correct index for insertion).

I can't find a 'sorted list' recipe in the Cookbook, so I might have to create one.

> Take advantage of interpreter optimizations 
> -------------------------------------------
[snip]

. Prefer iteration (using for loops, list comprehensions, or generator 
expressions) over iterables to explicit invocation of .next()

> Take advantage of diagnostic tools
> ----------------------------------
> 
> .	The hotshot and profile modules help identify performance bottlenecks.

Currently profile is a fair bit more useful than hotshot, mainly due to the fact 
that it isolates time spent in C functions from the Python code that calls those 
functions.

Perhaps another section for External Libraries? If you're doing serious number 
crunching in Python, using NumPy is practically a requirement.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan@email.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.skystorm.net
From mdehoon at ims.u-tokyo.ac.jp  Sat Dec 11 07:13:39 2004
From: mdehoon at ims.u-tokyo.ac.jp (Michiel Jan Laurens de Hoon)
Date: Sat Dec 11 07:10:24 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com>
References: <79990c6b041210040671584abc@mail.gmail.com>	<001401c4dda9$ad605680$e841fea9@oemcomputer>	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>	<79990c6b041210040671584abc@mail.gmail.com>
	<5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com>
Message-ID: <41BA9013.50101@ims.u-tokyo.ac.jp>

>> E.g. if we need to compile libpython24.a it means we need to 
>> fetch the Python sources themselves first.
> 
> Actually, no, you don't need the sources.  See:
> 
> http://mail.python.org/pipermail/python-dev/2004-January/041676.html
> 
> for a script that builds libpython24.a from the python24.lib distributed 
> with Python for Windows.
> 

Previously I built libpython24.a; it can be downloaded from my website. See 
http://bonsai.ims.u-tokyo.ac.jp/~mdehoon/software/python/cygwin.html

--Michiel.


-- 
Michiel de Hoon, Assistant Professor
University of Tokyo, Institute of Medical Science
Human Genome Center
4-6-1 Shirokane-dai, Minato-ku
Tokyo 108-8639
Japan
http://bonsai.ims.u-tokyo.ac.jp/~mdehoon
From martin at v.loewis.de  Sat Dec 11 09:39:28 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Dec 11 09:39:31 2004
Subject: [Python-Dev] Supporting Third Party Modules (was The other Py2.4
	issue)
In-Reply-To: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au>
References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au>
Message-ID: <41BAB240.6060101@v.loewis.de>

Chui G. Tey wrote:
> One good way of helping out is to provide an dynamic loading function
> that third party modules could access the basic python functions such as
> PyArgParseTuple, PyString_AsString etc regardless of which python the
> user is running. This would be similar to the COM approach. You can load
> all the function pointers into a struct and then call them. 
> 
> Third party modules would link against this DLL independent of which
> python is being used. 

I believe this is not implementable: How can the DLL know which Python
DLL to use?

Regards,
Martin
From martin at v.loewis.de  Sat Dec 11 09:53:29 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Dec 11 09:53:32 2004
Subject: [Python-Dev] removing aclocal.m4
In-Reply-To: <20041209023350.GB7217@janus.swcomplete.com>
References: <20041209023350.GB7217@janus.swcomplete.com>
Message-ID: <41BAB589.4030303@v.loewis.de>

Neal Norwitz wrote:
> Can we remove aclocal.m4?  The last log message states:
> 
>         fix for bug #811160 - autoconf vs. hp/ux system header files.
>         also applied to release23-maint.
> 
>         Note that aclocal.m4 can go away when autoconf 2.58 is out.

It appears to me that 2.59 indeed fixes the HP-UX problem; a diff
with and without aclocal.m4 shows chunks like

  cat confdefs.h >>conftest.$ac_ext
  cat >>conftest.$ac_ext <<_ACEOF
  /* end confdefs.h.  */
+/* Define $ac_func to an innocuous variant, in case <limits.h> declares 
$ac_func.
+   For example, HP-UX 11i <limits.h> declares gettimeofday.  */
+#define $ac_func innocuous_$ac_func
+
  /* System header to define __stub macros and hopefully few prototypes,
      which can conflict with char $ac_func (); below.
      Prefer <limits.h> to <assert.h> if __STDC__ is defined, since
-    <limits.h> exists even on freestanding compilers.  Under hpux,
-    including <limits.h> includes <sys/time.h> and causes problems
-    checking for functions defined therein.  */
-#if defined (__STDC__) && !defined (_HPUX_SOURCE)
+    <limits.h> exists even on freestanding compilers.  */
+
+#ifdef __STDC__
  # include <limits.h>
  #else
  # include <assert.h>
  #endif
+
+#undef $ac_func
+

So they manage to get limits.h define, say, innoucuous_clock(),
instead of clock(), whereas we currently avoid including limits.h
on HP-UX.

IOW, it seems like it should work, but somebody should test it
on HP-UX to be sure.

Regards,
Martin
From bob at redivi.com  Sat Dec 11 10:51:30 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sat Dec 11 10:52:07 2004
Subject: [Python-Dev] Supporting Third Party Modules (was The other Py2.4
	issue)
In-Reply-To: <41BAB240.6060101@v.loewis.de>
References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au>
	<41BAB240.6060101@v.loewis.de>
Message-ID: <3BBC89A1-4B5A-11D9-80F2-000A9567635C@redivi.com>

On Dec 11, 2004, at 3:39 AM, Martin v. L?wis wrote:

> Chui G. Tey wrote:
>> One good way of helping out is to provide an dynamic loading function
>> that third party modules could access the basic python functions such 
>> as
>> PyArgParseTuple, PyString_AsString etc regardless of which python the
>> user is running. This would be similar to the COM approach. You can 
>> load
>> all the function pointers into a struct and then call them. Third 
>> party modules would link against this DLL independent of which
>> python is being used.
>
> I believe this is not implementable: How can the DLL know which Python
> DLL to use?

Well for py2app on Mac OS X, I wrote an executable stub that chooses a 
Python runtime from an XML file, looks up and binds a few symbols from 
it dynamically, and then starts doing stuff.  Works for 2.3 (as long as 
it's not compiled debug or trace refs) and 2.4 (universally, because it 
has exported functions for incref and decref).

For extension modules, this makes much less sense, but I wanted to be 
able to bootstrap Python programs with a precompiled executable that I 
didn't have to maintain across several versions.

-bob

From fredrik at pythonware.com  Sat Dec 11 11:14:38 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sat Dec 11 11:12:23 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
References: <ca471dc204120814186f5dc423@mail.gmail.com>
Message-ID: <cpeh62$ldh$1@sea.gmane.org>

> One thing that bugs me: the article says 3 or 4 times that Python is
> slow, each time with a refutation ("but it's so flexible", "but it's
> fast enough") but still, they sure seem to harp on the point.

fwiw, IDG's Computer Sweden,  "sweden's leading IT-newspaper" has a
surprisingly big Python article in their most recent issue:

    PYTHON FEELS WELL
    Better performance biggest news in 2.4

and briefly interviews swedish zope-developer Johan Carlsson and Python-
Ware co-founder Håkan Karlsson.

</F>

PS. the heading is a play on the swedish phrase "må pyton", which means
"feel like crap"... 



From fredrik at pythonware.com  Sat Dec 11 15:06:05 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sat Dec 11 15:03:51 2004
Subject: [Python-Dev] Re: Supporting Third Party Modules (was The other
	Py2.4issue)
References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au>
	<41BAB240.6060101@v.loewis.de>
Message-ID: <cpeuo1$g58$1@sea.gmane.org>

Martin v. Löwis wrote:

>> Third party modules would link against this DLL independent of which
>> python is being used.
>
> I believe this is not implementable: How can the DLL know which Python
> DLL to use?

the Python interpreter could initialize this DLL, using some suitable mechanism.

alternatively, it could figure out what EXE you used to start the program in the
first place, and figure out what Python DLL that EXE uses.  my ExeMaker tool
can do that, for example:

    http://effbot.org/zone/exemaker.htm

(see the #!python.exe entry in the table)

</F> 



From faassen at infrae.com  Sat Dec 11 15:07:27 2004
From: faassen at infrae.com (Martijn Faassen)
Date: Sat Dec 11 15:07:30 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <cpeh62$ldh$1@sea.gmane.org>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<cpeh62$ldh$1@sea.gmane.org>
Message-ID: <41BAFF1F.3060200@infrae.com>

Fredrik Lundh wrote:
[snip]
> fwiw, IDG's Computer Sweden,  "sweden's leading IT-newspaper" has a
> surprisingly big Python article in their most recent issue:
> 
>     PYTHON FEELS WELL
>     Better performance biggest news in 2.4
[snip]

Perhaps the message getting out is actually that Python's performance is 
getting better, so nothing to worry about. The original article that 
Guido cited had a headline along the same lines: "Faster Python grabs 
programmers".

So perhaps there really is nothing to worry about in this department: 
the headlines are saying "Python now faster". That to me sounds like an 
excellent way to combat "Python is slow" memes. :)

Regards,

Martijn
From gvanrossum at gmail.com  Sat Dec 11 17:22:17 2004
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Sat Dec 11 17:22:20 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <cpeh62$ldh$1@sea.gmane.org>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<cpeh62$ldh$1@sea.gmane.org>
Message-ID: <ca471dc204121108225511d43@mail.gmail.com>

> fwiw, IDG's Computer Sweden,  "sweden's leading IT-newspaper" has a
> surprisingly big Python article in their most recent issue:
> 
>     PYTHON FEELS WELL
>     Better performance biggest news in 2.4
> 
> and briefly interviews swedish zope-developer Johan Carlsson and Python-
> Ware co-founder H?kan Karlsson.

Maybe we've done this to ourselves then. I'm sure the interviewer
didn't know what was the biggest news until after Johan and H?kan told
him. And this despite all the functional improvements that are much
more interesting for most Python programmers: generator expressions,
decorators, a ton of new modules...

Ah, but several of those modules are rewrites in C of modules that
were previously written in Python. Maybe we really *should* stop
emphasizing speed so much to ourselves, and focus more on fixing bugs
and adding features instead of looking for speedup opportunities.

I think Python's message ought to be more something like "programming
for the rest of us -- a child can learn Python, but Python's no toy."

BTW I strongly disagree that making easy .EXE binaries available will
address this issue; while not bundled, there are plenty of solutions
for maning .EXEs for those who need them, and this is not something
that typically worries managers. But the perception of Python as
"slow" does worry managers.

(Wish I could add more to this thread but my family is calling...)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
From phd at mail2.phd.pp.ru  Sat Dec 11 17:27:14 2004
From: phd at mail2.phd.pp.ru (Oleg Broytmann)
Date: Sat Dec 11 17:27:16 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <ca471dc204121108225511d43@mail.gmail.com>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<cpeh62$ldh$1@sea.gmane.org>
	<ca471dc204121108225511d43@mail.gmail.com>
Message-ID: <20041211162714.GA32590@phd.pp.ru>

On Sat, Dec 11, 2004 at 08:22:17AM -0800, Guido van Rossum wrote:
> But the perception of Python as
> "slow" does worry managers.

   Much more those managers are worried that Python isn't backed by a
large corporation. For Java there is Sun, for Visual Basic there is the
biggest and most powerful corporation. But who is for Python? Who is
responsible? Whom to blame?

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd@phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.
From arigo at tunes.org  Sat Dec 11 17:55:09 2004
From: arigo at tunes.org (Armin Rigo)
Date: Sat Dec 11 18:04:11 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <41BA34FD.6050306@v.loewis.de>
References: <001401c4dda9$ad605680$e841fea9@oemcomputer>
	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>
	<41BA34FD.6050306@v.loewis.de>
Message-ID: <20041211165508.GA10252@vicky.ecs.soton.ac.uk>

Hi Martin,

On Sat, Dec 11, 2004 at 12:45:01AM +0100, "Martin v. L?wis" wrote:
> The straight-forward answer is: Get VC7.1 (aka VS.NET 2003), and invoke
>
> python setup.py bdist_wininst
> 
> That's not too hard to do, I think.

Hum, this is getting into a Linux-vs-Windows argument.  I don't want to invest
time and money on Windows tools just to compile my extension module for
Windows users...


Armin
From arigo at tunes.org  Sat Dec 11 17:55:16 2004
From: arigo at tunes.org (Armin Rigo)
Date: Sat Dec 11 18:04:22 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <5.1.1.6.0.20041210131425.030fd360@mail.telecommunity.com>
References: <5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com>
	<79990c6b041210040671584abc@mail.gmail.com>
	<001401c4dda9$ad605680$e841fea9@oemcomputer>
	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>
	<79990c6b041210040671584abc@mail.gmail.com>
	<5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com>
	<5.1.1.6.0.20041210131425.030fd360@mail.telecommunity.com>
Message-ID: <20041211165516.GB10252@vicky.ecs.soton.ac.uk>

Hi,

On Fri, Dec 10, 2004 at 01:19:10PM -0500, Phillip J. Eby wrote:
> >>http://mail.python.org/pipermail/python-dev/2004-January/041676.html
> >
> >Shouldn't this be distributed with binary distributions of Python, to save 
> >people the trouble?
> 
> The Python developers who produce the Windows binaries don't use 
> mingw/cygwin, so this would put a maintenance burden on them.

This thread started as "how to convince people to build their extension
modules quickly for Python 2.4".  I was just suggesting that taking a bit of
that burden away from us non-Windows-users would be helpful :-)

> >Or, if it can be skipped, the procedure for doing a mingw build with the 
> >.lib should be documented and that'd be the end of it.
> 
> It's actually documented now, in the "Installing Python Modules" manual.  

Oh right...  Well, I didn't notice because, oddly, there is another set of
pages on the same subject (but with different details and up-to-date-ness)  
in "Extending and Embedding the Python Interpreter".

I guess that one of them should be removed and point to the other...


A bientot,

Armin.
From pje at telecommunity.com  Sat Dec 11 18:07:16 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat Dec 11 18:05:43 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <ca471dc204121108225511d43@mail.gmail.com>
References: <cpeh62$ldh$1@sea.gmane.org>
	<ca471dc204120814186f5dc423@mail.gmail.com>
	<cpeh62$ldh$1@sea.gmane.org>
Message-ID: <5.1.1.6.0.20041211120241.03c12b50@mail.telecommunity.com>

At 08:22 AM 12/11/04 -0800, Guido van Rossum wrote:
>BTW I strongly disagree that making easy .EXE binaries available will
>address this issue; while not bundled, there are plenty of solutions
>for maning .EXEs for those who need them, and this is not something
>that typically worries managers. But the perception of Python as
>"slow" does worry managers.

The relevant memes are that "compiled == fast", and that ".exe == compiled".

It's not about .exe as a distribution format, but rather that producing an 
.exe is supporting evidence of having a "real" compiler.

IOW, if there exists a compiler, but it doesn't produce .exe's, it won't be 
considered a "real" compiler by many programmers.

From fredrik at pythonware.com  Sat Dec 11 19:06:19 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sat Dec 11 19:04:06 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
References: <ca471dc204120814186f5dc423@mail.gmail.com><cpeh62$ldh$1@sea.gmane.org>
	<ca471dc204121108225511d43@mail.gmail.com>
Message-ID: <cpfcqh$ejc$1@sea.gmane.org>

Guido van Rossum wrote:

> > fwiw, IDG's Computer Sweden,  "sweden's leading IT-newspaper" has a
> > surprisingly big Python article in their most recent issue:
> >
> >     PYTHON FEELS WELL
> >     Better performance biggest news in 2.4
> >

(hmm.  I seem to have accidentally deleted a line here...)

> > and briefly interviews swedish zope-developer Johan Carlsson and Python-
> > Ware co-founder Håkan Karlsson.
>
> Maybe we've done this to ourselves then. I'm sure the interviewer
> didn't know what was the biggest news until after Johan and Håkan told
> him. And this despite all the functional improvements that are much
> more interesting for most Python programmers: generator expressions,
> decorators, a ton of new modules...

I don't know Johan, but I'm pretty sure Håkan didn't talk about performance;
he's more interested in *development* speed... (and I don't think he's been
following 2.4 that closely).

Looking again, the article says:

    "Among the news in 2.4 is better performance, especially for new
    functions that were added in 2.3."

and then mentions integer/long integer unification and the decimal type as notable
additions, and continues "and so it continues, with a long list of improved details".
The article also points to www.python.org/2.4, where the first link is:

    http://www.python.org/2.4/highlights.html

which says:

    Here are the (subjective) highlights of what's new in Python 2.4.

    * Faster

    A number of modules that were added in Python 2.3 (such as sets and heapq)
    have been recoded in C.   In addition, there's been a number of other speedups
    to the interpreter. (See section 8.1, Optimizations, of the "What's New" document
    for more [this lists a couple of thousand tweaks by Raymond, and Armin's string
    concatenation hack]).

    * New language features

    /.../ unification of integers and long integers /.../ decimal - a new numeric type
    that allows for the accurate representation of floating point numbers /.../

so I don't think you can blame Johan or Håkan...  the writer simply read the
python.org material, and picked a couple of things that he found interesting
(decorators and generator expressions may be a big thing for an experienced
pythoneer, but they are probably a bit too obscure for a general audience...)

> Ah, but several of those modules are rewrites in C of modules that
> were previously written in Python. Maybe we really *should* stop
> emphasizing speed so much to ourselves, and focus more on fixing bugs
> and adding features instead of looking for speedup opportunities.

yes please.

> (Wish I could add more to this thread but my family is calling...)

same here.  more later.

</F> 



From tismer at stackless.com  Sat Dec 11 19:57:55 2004
From: tismer at stackless.com (Christian Tismer)
Date: Sat Dec 11 19:57:57 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <20041211165508.GA10252@vicky.ecs.soton.ac.uk>
References: <001401c4dda9$ad605680$e841fea9@oemcomputer>	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>	<41BA34FD.6050306@v.loewis.de>
	<20041211165508.GA10252@vicky.ecs.soton.ac.uk>
Message-ID: <41BB4333.2070003@stackless.com>

Armin Rigo wrote:

> Hi Martin,
> 
> On Sat, Dec 11, 2004 at 12:45:01AM +0100, "Martin v. L?wis" wrote:
> 
>>The straight-forward answer is: Get VC7.1 (aka VS.NET 2003), and invoke
>>
>>python setup.py bdist_wininst
>>
>>That's not too hard to do, I think.
> 
> 
> Hum, this is getting into a Linux-vs-Windows argument.  I don't want to invest
> time and money on Windows tools just to compile my extension module for
> Windows users...

Maybe we can set this up as a service?
I have the compiler and could do something like
doing a checkout, build, and upload new binary
for a number of projects.
What is needed is a simple but secure notification
method. Probably I need one windows machine which is
always online.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/
From ncoghlan at iinet.net.au  Sun Dec 12 01:39:13 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sun Dec 12 01:39:17 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <000201c4df6b$5c9cef60$e841fea9@oemcomputer>
References: <000201c4df6b$5c9cef60$e841fea9@oemcomputer>
Message-ID: <41BB9331.2060408@iinet.net.au>

Raymond Hettinger wrote:
   <some sensible things>

Makes sense to me - I look forward to seeing the next version.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan@email.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.skystorm.net
From ncoghlan at iinet.net.au  Sun Dec 12 03:20:01 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sun Dec 12 03:20:07 2004
Subject: [Python-Dev] PEP 338: Executing modules inside packages with '-m'
Message-ID: <41BBAAD1.4090208@iinet.net.au>

Python 2.4's -m command line switch only works for modules directly on sys.path.
Trying to use it with modules inside packages will fail with a "Module not
found" error. This PEP aims to fix that for Python 2.5.

Previously, posting of a draft version of the PEP to python-dev and python-list
didn't actually generate any responses. I'm not sure if that's an indication
that people don't see the restriction to top-level modules as a problem (and
hence think the PEP is unecessary), or think the extension to handle packages is
obvious (and hence see no need to comment).

For c.l.p, it could just be a sign that Python 2.4 hasn't been out long enough 
for anyone to care what I'm yabbering on about :)

Anyway, all comments are appreciated (even a simple "Sounds good to me").

Cheers,
Nick.

***********************************************************************
PEP: 338
Title: Executing modules inside packages with '-m'
Version: $Revision: 1.2 $
Last-Modified: $Date: 2004/12/11 20:31:10 $
Author: Nick Coghlan <ncoghlan@email.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 16-Oct-2004
Python-Version: 2.5
Post-History: 8-Nov-2004


Abstract
========

This PEP defines semantics for executing modules inside packages as
scripts with the ``-m`` command line switch.

The proposed semantics are that the containing package be imported
prior to execution of the script.


Rationale
=========

Python 2.4 adds the command line switch ``-m`` to allow modules to be
located using the Python module namespace for execution as scripts.
The motivating examples were standard library modules such as ``pdb``
and ``profile``.

A number of users and developers have requested extension of the
feature to also support running modules located inside packages.  One
example provided is pychecker's ``pychecker.checker`` module.  This
capability was left out of the Python 2.4 implementation because the
appropriate semantics were not entirely clear.

The opinion on python-dev was that it was better to postpone the
extension to Python 2.5, and go through the PEP process to help make
sure we got it right.


Scope of this proposal
==========================

In Python 2.4, a module located using ``-m`` is executed just as if
its filename had been provided on the command line.  The goal of this
PEP is to get as close as possible to making that statement also hold
true for modules inside packages.

Prior discussions suggest it should be noted that this PEP is **not**
about any of the following:

- changing the idiom for making Python modules also useful as scripts
   (see PEP 299 [1]_).

- lifting the restriction of ``-m`` to modules of type PY_SOURCE or
   PY_COMPILED (i.e. ``.py``, ``.pyc``, ``.pyo``, ``.pyw``).

- addressing the problem of ``-m`` not understanding zip imports or
   Python's sys.metapath.

The issues listed above are considered orthogonal to the specific
feature addressed by this PEP.


Current Behaviour
=================

Before describing the new semantics, it's worth covering the existing
semantics for Python 2.4 (as they are currently defined only by the
source code).

When ``-m`` is used on the command line, it immediately terminates the
option list (like ``-c``).  The argument is interpreted as the name of
a top-level Python module (i.e. one which can be found on
``sys.path``).

If the module is found, and is of type ``PY_SOURCE`` or
``PY_COMPILED``, then the command line is effectively reinterpreted
from ``python <options> -m <module> <args>`` to ``python <options>
<filename> <args>``.  This includes setting ``sys.argv[0]`` correctly
(some scripts rely on this - Python's own ``regrtest.py`` is one
example).

If the module is not found, or is not of the correct type, an error
is printed.


Proposed Semantics
==================

The semantics proposed are fairly simple: if ``-m`` is used to execute
a module inside a package as a script, then the containing package is
imported before executing the module in accordance with the semantics
for a top-level module.

This is necessary due to the way Python's import machinery locates
modules inside packages.  A package may modify its own __path__
variable during initialisation.  In addition, paths may affected by
``*.pth`` files.  Accordingly, the only way for Python to reliably
locate the module is by importing the containing package and
inspecting its __path__ variable.

Note that the package is *not* imported into the ``__main__`` module's
namespace.  The effects of these semantics that will be visible to the
executed module are:

- the containing package will be in sys.modules

- any external effects of the package initialisation (e.g. installed
   import hooks, loggers, atexit handlers, etc.)


Reference Implementation
========================

A reference implementation is available on SourceForge [2]_.  In this
implementation, if the ``-m`` switch fails to locate the requested
module at the top level, it effectively reinterprets the command from
``python -m <script>`` to ``python -m execmodule <script>``.  (There
is one caveat: when reinterpreted in this way, ``sys.argv[0]`` may not
actually contain the filename of ``execmodule``.  This only affects
``execmodule`` itself, not the requested module).

``execmodule`` is a proposed standard library module that contains a
single function (also called ``execmodule``).  When invoked as a
script, this module finds and executes the module supplied as the
first argument.  It adjusts ``sys.argv`` by deleting ``sys.argv[0]``
and replacing the new ``sys.argv[0]`` with the module's filename
instead of its Python name.

The function ``execmodule`` is like ``execfile``, but uses the Python
module namespace to locate the script instead of the filesystem.  It
has an additional optional argument ``set_argv0`` which causes the
filename of the located module to be written to ``sys.argv[0]`` before
the module is executed.

A hybrid C/Python implementation is used as the Python module is much
more flexible and extensible than the equivalent C code would be.  It
also allows the ``execmodule`` function to be made available.  Scripts
which execute other scripts (e.g. ``profile``, ``pdb``) have the
option to use this function to provide ``-m`` style support for
identifying the script to be executed.

The Python code for ``execmodule`` has also been posted as a
cookbook recipe for Python 2.4 [3]_.


Open Issues
===========

- choosing a name for the standard library module containing
   ``execmodule``.  The reference implementation uses ``execmodule``.
   An alternative name proposed on python-dev is ``runpy``.


Alternatives
============

The main alternative implementation considered ignored packages'
__path__ variables, and looked only in the main package directory.  A
Python script with this behaviour can be found in the discussion of
the ``execmodule`` cookbook recipe [3]_.

This approach was not used as it does not meet the main goal of the
``-m`` switch -- to allow the full Python namespace to be used to
locate modules for execution.


References
==========

.. [1] Special __main__() function in modules
    (http://www.python.org/peps/pep-0299.html)

.. [2] Native ``-m`` execmodule support

(http://sourceforge.net/tracker/?func=detail&aid=1043356&group_id=5470&atid=305470 )

.. [3] execmodule Python Cookbook Recipe
    (http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/307772)


Copyright
=========

This document has been placed in the public domain.

..
    Local Variables:
    mode: indented-text
    indent-tabs-mode: nil
    sentence-end-double-space: t
    fill-column: 70
    End:


-- 
Nick Coghlan   |   ncoghlan@email.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.skystorm.net

From bac at OCF.Berkeley.EDU  Sun Dec 12 04:10:09 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Dec 12 04:10:16 2004
Subject: [Python-Dev] python-dev Summary for 2004-10-16 through 2004-10-31
	[draft]
Message-ID: <41BBB691.40604@ocf.berkeley.edu>

Winter Break is upon me which means I have time to catch up on the Summaries. 
I will definitely be caught up by the end of the month.

As for this summary, I will send this out around Wednesday.  As always 
corrections are appreciated.  If you feel one of the skipped threads deserves 
coverage please feel free to write up a summary and I will edit it and include it.

--------------------------------------

=====================
Summary Announcements
=====================
`Python 2.4`_ final is now out!

As I mentioned in the last summary my school schedule this past quarter has 
been insane.  But now I am on Winter Break and will hopefully be able to catch 
up on my Summary backlog.

.. _Python 2.4: http://www.python.org/2.4/

=========
Summaries
=========
--------------------------------------------------------
Specifying main functions and calling packages with '-m'
--------------------------------------------------------
In my opinion, the new '-m' command line option in Python 2.4 is really handy. 
  But wouldn't it be even handier if you could execute modules in a package? 
That exact question came up.  The reason this kind of thing didn't just go 
directly into 2.4 was that the semantics are not as obvious nor is it as 
simple.  A PEP is on the way, though.  Until then, you can use Nick Coghlan's 
recipe at http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/307772.

This also brought up the discussion of being able to specify a 'main' function 
to take the place of the good old ``if __name__ == "__main__"`` idiom.  Some 
liked the idea of allowing one to define a function named 'main', others 
'__main__'.  But the discussion never went any farther.  This will require a 
PEP to ever even be seriously considered.

Contributing threads:
   - `python-dev Summary for 2004-09-16 through 2004-09-30 [draft] <>`__
   - `Magic main functions <>`__
   - `Supporting packages on the command line <>`__

----------------------------
ConfigParser shootout begins
----------------------------
As mentioned in the `last summary`_, a desire for a replacement for 
ConfigParser has come into being.  It seems the ideas are being hashed out in 
the wiki at http://www.python.org/moin/ConfigParserShootout .

Contributing threads:
   - `ConfigParser shootout, preliminary entry <>`__
   - `What are the goals for ConfigParser 3000? <>`__

----------------------------------------------------
Making pymalloc friendlier to long running processes
----------------------------------------------------
Pymalloc, when a small chunk of memory is requested that is less than 256 
bytes, fetches the memory from a pool of previously used memory that is now 
available.  While this speeds up memory allocation since it does not directly 
involve calling the OS to free up the memory, it does cause issues for long 
running processes that have at some point requested a large portion of memory. 
  The example in the OP for this whole topic was an app that needs 1GB for 
about five minutes, but the amount of allocated memory stays at 1 GB since 
pymalloc does not free it to the OS.

This was brought up back in June and is summarized at 
http://www.python.org/dev/summary/2004-06-01_2004-06-15.html#id17 .

No code has been checked in at the moment to change the behavior partially 
thanks to the difficulty of the problem.

Contributing threads:
   - `Changing pymalloc behaviour for long running processes <>`__
   - `Re: Python-Dev Digest, Vol 15, Issue 46 <>`__

----------------------------
How to get a patch looked at
----------------------------
Often times people have a specific patch that they want reviewed/applied to 
solve a specific problem they are having.  Unfortunately the Python developers 
have limited time; we just can't get to every patch there in a timely fashion. 
  This can be a problem who *really* need to get a patch in.

Enter Martin v. L?wis and his deal to review a specific patch:

1. Review 10 other patches on SF
2. Send an email to python-dev listing the 10 patches that you reviewed and the 
one patch you want to be reviewed
3. Your specific patch gets reviewed by Martin

You can see the exact details of Martin's requirements at 
http://mail.python.org/pipermail/python-dev/2004-October/049495.html and can 
also read http://www.python.org/dev/dev_intro.html for ideas on what to do for 
reviewing.

Contributing threads:
   - `Patches <>`__

----------------------------------------------
Discussion of including pysqlite in the stdlib
----------------------------------------------
The idea of including pysqlite_ in the stdlib came up once again (not this is 
the wrapper for SQLite_ and not SQLite itself).  The arguments for including 
the module were the SQLite is most likely used more than Sleepycat and thus 
deserved a place in the stdlib if bsddb did.  It also goes along with Python's 
idea of "batteries included".

Arguments against started with the idea of sanctioning pysqlite over other 
SQLite wrappers did not seem proper.  People also thought that including bsddb 
might not be correct anymore and thus should not be used as a basis for 
including pysqlite.

This all then led into a discussion about package management and how to 
simplify extending what can be installed at compile-time.  The idea of pushing 
PyPI_ came up as well as revising `PEP 206`_.

.. _pysqlite: http://pysqlite.org/
.. _SQLite: http://www.sqlite.org/
.. _PyPI: http://www.python.org/pypi
.. _PEP 206: http://www.python.org/peps/pep-0206.html

Contributing threads:
   - `SQLite module for Python 2.5 <>`__

===============
Skipped Threads
===============
- Python 2.4b1 on win32 installer bug?
- test_subprocess 2.4b1 fails on FreeBSD 5.2
- adding Py{String|Unicode}_{Lower|Upper} and fixing CreateProcess in 
_subprocess.pyd and PyWin32
- IPv6 bug in 2.3.4??
- logging needs better documentation
- bsddb todo for someone - auto-upgrade db format
- Tests broken on Windows
- import w/options
- Adding a developer
       People are only added as a developer if they ask for it and have shown 
they are going to stick around
- audiotest.au and possible copyright issues?
- Re: [Python-checkins]	python/dist/src/Pythoncompile.c, 2.330, 2.331
- Bug [ 959379 ] Implicit close() should check for errors
- weakref callback vs gc vs threads
From bac at OCF.Berkeley.EDU  Sun Dec 12 04:17:28 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sun Dec 12 04:17:51 2004
Subject: [Python-Dev] PEP 338: Executing modules inside packages with '-m'
In-Reply-To: <41BBAAD1.4090208@iinet.net.au>
References: <41BBAAD1.4090208@iinet.net.au>
Message-ID: <41BBB848.4040408@ocf.berkeley.edu>

Nick Coghlan wrote:
> Python 2.4's -m command line switch only works for modules directly on 
> sys.path.
> Trying to use it with modules inside packages will fail with a "Module not
> found" error. This PEP aims to fix that for Python 2.5.
> 
> Previously, posting of a draft version of the PEP to python-dev and 
> python-list
> didn't actually generate any responses. I'm not sure if that's an 
> indication
> that people don't see the restriction to top-level modules as a problem 
> (and
> hence think the PEP is unecessary), or think the extension to handle 
> packages is
> obvious (and hence see no need to comment).
> 
> For c.l.p, it could just be a sign that Python 2.4 hasn't been out long 
> enough for anyone to care what I'm yabbering on about :)
> 
> Anyway, all comments are appreciated (even a simple "Sounds good to me").
> 

Sounds good to me. =)

+1

-Brett
From ncoghlan at iinet.net.au  Sun Dec 12 05:10:59 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sun Dec 12 05:11:04 2004
Subject: [Python-Dev] python-dev Summary for 2004-10-16 through 2004-10-31
	[draft]
In-Reply-To: <41BBB691.40604@ocf.berkeley.edu>
References: <41BBB691.40604@ocf.berkeley.edu>
Message-ID: <41BBC4D3.1050604@iinet.net.au>

Brett C. wrote:
> This also brought up the discussion of being able to specify a 'main' 
> function to take the place of the good old ``if __name__ == "__main__"`` 
> idiom.  Some liked the idea of allowing one to define a function named 
> 'main', others '__main__'.  But the discussion never went any farther.  
> This will require a PEP to ever even be seriously considered.

There's a PEP already - PEP 299.

The PEP actually describes a reasonable approach, since code using the current 
idiom is unlikely to define a __main__() function. However, it seems more like a 
Py3K idea, since if it's only in 2.5 and later, we'd see code like this to 
support earlier 2.x versions:

==========================
def __main__(*args):
   ...

if __name__ == "__main__":
   import sys as _sys
   if _sys.version_info < (2, 5):
     __main__(_sys.argv)
==========================

Or, instead (using only the current idiom):

==========================
def _main(*args):
   ...

if __name__ == "__main__":
   import sys as _sys
   _main(_sys.argv)
==========================

So, to my mind, the backwards compatibility issue completely defeats the PEP's 
goal of finding a cleaner idiom than the current one.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan@email.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.skystorm.net
From faassen at infrae.com  Sun Dec 12 13:15:29 2004
From: faassen at infrae.com (Martijn Faassen)
Date: Sun Dec 12 13:15:32 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <5.1.1.6.0.20041211120241.03c12b50@mail.telecommunity.com>
References: <cpeh62$ldh$1@sea.gmane.org>	<ca471dc204120814186f5dc423@mail.gmail.com>	<cpeh62$ldh$1@sea.gmane.org>
	<5.1.1.6.0.20041211120241.03c12b50@mail.telecommunity.com>
Message-ID: <41BC3661.4030008@infrae.com>

Phillip J. Eby wrote:
> At 08:22 AM 12/11/04 -0800, Guido van Rossum wrote:
> 
>> BTW I strongly disagree that making easy .EXE binaries available will
>> address this issue; while not bundled, there are plenty of solutions
>> for maning .EXEs for those who need them, and this is not something
>> that typically worries managers. But the perception of Python as
>> "slow" does worry managers.
> 
> The relevant memes are that "compiled == fast", and that ".exe == 
> compiled".
> 
> It's not about .exe as a distribution format, but rather that producing 
> an .exe is supporting evidence of having a "real" compiler.
> 
> IOW, if there exists a compiler, but it doesn't produce .exe's, it won't 
> be considered a "real" compiler by many programmers.

This is indeed an extremely common set of memes. It especially haunts 
people who have done some programming in the past but don't really have 
it as their main focus now. Many managers would be in this group.

It's a PC-platform thing mostly. In the early days of the PC, you had 
BASIC, and you had compiled languages. Interpreted BASIC was considered 
to be unprofessional and slow. Distributing your program as a .bas file 
was considered to be a sign of amateurism. Besides, interpreters on that 
hardware *were* sometimes unacceptably slow. If you got a compiled .exe 
it was a sign of performance and more professionalism. Microsoft was 
quite aware of this meme: they did the trick with Visual Basic 
(packaging their interpreter in the .exe and called it 'compiled'.

The "compiled == fast" meme is also very common among programmers 
themselves; I know I myself had to wrestle free of it (I didn't care 
about .exes by that time as I was on linux). That's probably why we have 
so many Python programmers saying, "well, yeah, Python is not as fast as 
compiled languages but it's fast enough"; that's the counter meme that 
replaced the "compilation is good" meme that was there in those people 
before. It's quite possible that some of these programmers influence 
managers.

Regards,

Martijn
From jjl at pobox.com  Sun Dec 12 16:09:57 2004
From: jjl at pobox.com (John J Lee)
Date: Sun Dec 12 16:13:03 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer>
References: <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer>
Message-ID: <Pine.LNX.4.58.0412121505030.6267@alice>

On Wed, 8 Dec 2004, Raymond Hettinger wrote:

> > One thing that bugs me: the article says 3 or 4 times that Python is
> > slow, each time with a refutation ("but it's so flexible", "but it's
> > fast enough") but still, they sure seem to harp on the point. This is
> > a PR issue that Python needs to fight -- any ideas?
[...]
> * Have python.org prominently feature an article of Python's use in
> high-performance environments.  IIRC, somebody wrote a realtime voice
> over internet system and found that with good design, there was no speed
> issue.  Also, the cellphone apps may make a good example.
[...]

IIRC, Alex Martelli mentioned a video application at last year's ACCU /
Python UK conference.  He said the system never went into production, but
it sounded like a good meme from the speed point of view: it triggered
surprised and gleeful noises from the mixed Python / C++ / Java audience
;-).


John
From jjl at pobox.com  Sun Dec 12 16:26:00 2004
From: jjl at pobox.com (John J Lee)
Date: Sun Dec 12 16:29:06 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
Message-ID: <Pine.LNX.4.58.0412121514220.6267@alice>

On Wed, 8 Dec 2004, Phillip J. Eby wrote:

> At 02:18 PM 12/8/04 -0800, Guido van Rossum wrote:
> >I was pleasantly surprised to find a pointer to this article in a news
> >digest that the ACM emails me regularly (ACM TechNews).
> >
> >http://gcn.com/vol1_no1/daily-updates/28026-1.html
> >
> >One thing that bugs me: the article says 3 or 4 times that Python is
> >slow, each time with a refutation ("but it's so flexible", "but it's
> >fast enough") but still, they sure seem to harp on the point. This is
> >a PR issue that Python needs to fight -- any ideas?
> 
> The only thing that will fix the PR issue is to have a Python compiler 
> distributed as part of the language.  It doesn't matter if it doesn't 

I suspect you're correct, but the suggestion elsewhere to bundle py2exe
seems likely to be counterproductive to me: merely emphasizing the
"interpreterness" of Python the moment the idea spreads that Python-built
.exe's are so big because they're just an interpreter plus a script.

I'm sure PyPy, if successful, will be a big win on both PR and technical
fronts.

On a seperate PR issue, I use the word 'script' above advisedly: At work,
I've noticed technical employees of clients who use Java often seem to
take some satisfaction in referring to our web applications (which of
course, consist of who knows how many packages, modules and classes) as
"CGI scripts".  We do use CGI, but the CGI scripts themselves are always
about five lines long and just contain boilerplate code and configuration
to kick off our framework.  You can see them imagining a great long script
named doeverything.cgi...


John
From fredrik at pythonware.com  Sun Dec 12 16:38:43 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun Dec 12 16:38:59 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com>
	<Pine.LNX.4.58.0412121514220.6267@alice>
Message-ID: <cphom3$aj2$1@sea.gmane.org>

John J Lee wrote:

> the idea spreads that Python-built .exe's are so big because they're just
> an interpreter plus a script.

that's why exemaker is so great, of course: all EXE files are 4 kilobytes.

as for the PYC files, hide them in a ZIP archive named DAT, HLP, OCX,
or even DLL, and you're done.

</F> 



From martin at v.loewis.de  Sun Dec 12 16:40:22 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Dec 12 16:40:26 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <20041211165508.GA10252@vicky.ecs.soton.ac.uk>
References: <001401c4dda9$ad605680$e841fea9@oemcomputer>
	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>
	<41BA34FD.6050306@v.loewis.de>
	<20041211165508.GA10252@vicky.ecs.soton.ac.uk>
Message-ID: <41BC6666.8000500@v.loewis.de>

Armin Rigo wrote:
> Hum, this is getting into a Linux-vs-Windows argument.  I don't want to invest
> time and money on Windows tools just to compile my extension module for
> Windows users...

If you don't have Windows at all, you cannot create Windows installers
for your users, anyway.

If you do have Windows, but you normally don't use it for software
development (so you have no copy of MSVC), you have to invest money,
indeed. You don't have to invest time - atleast not as much time
as you need to invest finding out how these alternative compilers work.

However, for free software, things work quite differently, anyway:
if you (as a package author) don't have the time and money to create
a Windows package, tell your users that you won't create one. Instead,
ask for a volunteer to create a package for you. If your package has
a large Windows user community, somebody will have the money, and they
will find the time if the build process is as simple as bdist_wininst.

If none of your users volunteers to do the build for you, I would stop
worrying about the Windows users.

Regards,
Martin
From martin at v.loewis.de  Sun Dec 12 16:45:02 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Dec 12 16:45:08 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <41BB4333.2070003@stackless.com>
References: <001401c4dda9$ad605680$e841fea9@oemcomputer>	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>	<41BA34FD.6050306@v.loewis.de>
	<20041211165508.GA10252@vicky.ecs.soton.ac.uk>
	<41BB4333.2070003@stackless.com>
Message-ID: <41BC677E.6060604@v.loewis.de>

Christian Tismer wrote:
> Maybe we can set this up as a service?
> I have the compiler and could do something like
> doing a checkout, build, and upload new binary
> for a number of projects.

I don't think this needs to be too automated.
Offering this service is a good thing, and if somebody
volunteers, I will applaud. However, I believe that
it will be used less often than this thread might
suggest: Many developers do have VS.NET available, or
know somebody in their user community who does (or
would know if they asked), so a "global" compilation
service would likely be an infrequently-used fallback.

Nevertheless: if somebody wants to offer this for,
say, 6 months, I'd be really curious what the feedback
will be. We could announce this on pydotorg and elsewhere
if that volunteers desires.

Regards,
Martin
From martin at v.loewis.de  Sun Dec 12 17:02:57 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Dec 12 17:02:59 2004
Subject: [Python-Dev] Supporting Third Party Modules (was The other Py2.4
	issue)
In-Reply-To: <3BBC89A1-4B5A-11D9-80F2-000A9567635C@redivi.com>
References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au>	<41BAB240.6060101@v.loewis.de>
	<3BBC89A1-4B5A-11D9-80F2-000A9567635C@redivi.com>
Message-ID: <41BC6BB1.9010401@v.loewis.de>

Bob Ippolito wrote:
>> I believe this is not implementable: How can the DLL know which Python
>> DLL to use?
> 
> 
> Well for py2app on Mac OS X, I wrote an executable stub that chooses a 
> Python runtime from an XML file, looks up and binds a few symbols from 
> it dynamically, and then starts doing stuff.

While that would work, I think this is inappropriate for this specific
issue: we want to write extension modules which are independent of
the Python version, and might even be used with multiple Python
installations on the same system. In that case, adding configuration
files won't work, as each usage of the extension might require a
different Python DLL.

Regards,
Martin
From erik at heneryd.com  Sun Dec 12 17:10:58 2004
From: erik at heneryd.com (Erik Heneryd)
Date: Sun Dec 12 17:11:04 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <cpfcqh$ejc$1@sea.gmane.org>
References: <ca471dc204120814186f5dc423@mail.gmail.com><cpeh62$ldh$1@sea.gmane.org>	<ca471dc204121108225511d43@mail.gmail.com>
	<cpfcqh$ejc$1@sea.gmane.org>
Message-ID: <41BC6D92.5050802@heneryd.com>

Fredrik Lundh wrote:
>>>fwiw, IDG's Computer Sweden,  "sweden's leading IT-newspaper" has a
>>>surprisingly big Python article in their most recent issue:
>>>
>>>    PYTHON FEELS WELL
>>>    Better performance biggest news in 2.4
>>>

>>>and briefly interviews swedish zope-developer Johan Carlsson and Python-
>>>Ware co-founder H?kan Karlsson.
>>

...

> 
> so I don't think you can blame Johan or H?kan...  the writer simply read the
> python.org material, and picked a couple of things that he found interesting
> (decorators and generator expressions may be a big thing for an experienced
> pythoneer, but they are probably a bit too obscure for a general audience...)

I'm a bit puzzled by the last paragraph, where Python is grouped 
together with PHP and Perl - names starting with p, being popular on 
Linux and not having big, commercial backers.  The article then 
concludes "Since Python is copyrighted, it's not truly open.  However, 
it can be freely used and redistributed, even commercially."

Huh?  Where did THAT come from?  You might argue the merits of Python 
being associated with Perl/PHP, but it's a fact that it is.  But when it 
is, it's seen as less free?


Erik
From martin at v.loewis.de  Sun Dec 12 17:19:24 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Dec 12 17:19:27 2004
Subject: [Python-Dev] Re: Supporting Third Party Modules (was The other
	Py2.4issue)
In-Reply-To: <cpeuo1$g58$1@sea.gmane.org>
References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au>	<41BAB240.6060101@v.loewis.de>
	<cpeuo1$g58$1@sea.gmane.org>
Message-ID: <41BC6F8C.2030301@v.loewis.de>

Fredrik Lundh wrote:
> the Python interpreter could initialize this DLL, using some suitable mechanism.

What suitable mechanism could that be? Suppose the Python script imports
foo.pyd, which in turn is linked with pythonany.dll, which in turn is
supposed to find the right Python DLL.

Now, python24.dll runs LoadLibary for foo.pyd, but has no control over
pythonany.dll. How could the suitable mechanism work?

> alternatively, it could figure out what EXE you used to start the program in the
> first place, and figure out what Python DLL that EXE uses.  my ExeMaker tool
> can do that, for example:
> 
>     http://effbot.org/zone/exemaker.htm
> 
> (see the #!python.exe entry in the table)

That requires a module handle for the EXE, right? How do you get that 
handle?

Out of curiosity: How come that the hModule is the start address of
the image? On what Windows versions does that work? Is that documented?

Regards,
Martin
From p.f.moore at gmail.com  Sun Dec 12 17:57:07 2004
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun Dec 12 17:57:10 2004
Subject: [Python-Dev] PEP 338: Executing modules inside packages with '-m'
In-Reply-To: <41BBB848.4040408@ocf.berkeley.edu>
References: <41BBAAD1.4090208@iinet.net.au> <41BBB848.4040408@ocf.berkeley.edu>
Message-ID: <79990c6b0412120857196eadb0@mail.gmail.com>

On Sat, 11 Dec 2004 19:17:28 -0800, Brett C. <bac@ocf.berkeley.edu> wrote:
> Nick Coghlan wrote:
> 
> > Anyway, all comments are appreciated (even a simple "Sounds good to me").
> >
> 
> Sounds good to me. =)

And me :-)

+1

Paul.
From carribeiro at gmail.com  Sun Dec 12 18:14:09 2004
From: carribeiro at gmail.com (Carlos Ribeiro)
Date: Sun Dec 12 18:14:13 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <41BC6D92.5050802@heneryd.com>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<cpeh62$ldh$1@sea.gmane.org>
	<ca471dc204121108225511d43@mail.gmail.com>
	<cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com>
Message-ID: <864d370904121209147b713f31@mail.gmail.com>

On Sun, 12 Dec 2004 17:10:58 +0100, Erik Heneryd <erik@heneryd.com> wrote:
> Fredrik Lundh wrote:
> >>>fwiw, IDG's Computer Sweden,  "sweden's leading IT-newspaper" has a
> >>>surprisingly big Python article in their most recent issue:
> >>>
> >>>    PYTHON FEELS WELL
> >>>    Better performance biggest news in 2.4
> >>>
> 
> >>>and briefly interviews swedish zope-developer Johan Carlsson and Python-
> >>>Ware co-founder H?kan Karlsson.
> >>
> 
> ...
> 
> >
> > so I don't think you can blame Johan or H?kan...  the writer simply read the
> > python.org material, and picked a couple of things that he found interesting
> > (decorators and generator expressions may be a big thing for an experienced
> > pythoneer, but they are probably a bit too obscure for a general audience...)
> 
> I'm a bit puzzled by the last paragraph, where Python is grouped
> together with PHP and Perl - names starting with p, being popular on
> Linux and not having big, commercial backers.  The article then
> concludes "Since Python is copyrighted, it's not truly open.  However,
> it can be freely used and redistributed, even commercially."
> 
> Huh?  Where did THAT come from?  You might argue the merits of Python
> being associated with Perl/PHP, but it's a fact that it is.  But when it
> is, it's seen as less free?

The author was probably referring to the old (and as AFAIK already
solved) CRNI copyright issue that ocurred into the 1.x to 2.x series
transition. It's amazing how old memes from Python keep being
remembered and repeated, even years after the fact. It also
illustrates something very important - the community is not doing a
good job at spreading the news; perhaps we talk too much between
ourselves, and too little with the outside market. IMHO the website is
a great part of this, its message being more important to "sell"
Python than the standard library or native .exes.

About the website, a note from my own experience: when I search for
documentation on Python, I'm usually directed to some of the mirror of
the main python.org site. To find it inside the main site, I have to
use "site:python.org", or even "site:docs.python.org". Usually Google
does a good job at ranking pages, and if it doesn't rank the main
Python website very highly, it's because they're not being referred
to. A campaign to ask people to put links back to the canonical
documentation at the Python website would be nice.

-- 
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: carribeiro@gmail.com
mail: carribeiro@yahoo.com
From p.f.moore at gmail.com  Sun Dec 12 18:26:49 2004
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun Dec 12 18:26:52 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <41BB4333.2070003@stackless.com>
References: <001401c4dda9$ad605680$e841fea9@oemcomputer>
	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>
	<41BA34FD.6050306@v.loewis.de>
	<20041211165508.GA10252@vicky.ecs.soton.ac.uk>
	<41BB4333.2070003@stackless.com>
Message-ID: <79990c6b0412120926647402e7@mail.gmail.com>

On Sat, 11 Dec 2004 19:57:55 +0100, Christian Tismer
<tismer@stackless.com> wrote:
> Armin Rigo wrote:

> > Hum, this is getting into a Linux-vs-Windows argument.  I don't want to invest
> > time and money on Windows tools just to compile my extension module for
> > Windows users...

First of all, I'm assuming this is a timing issue. If I understood
your initial posting, your concern was that people wanted Windows
build of your extension *now*, and it was going to take you time to
make it available.

That's a different issue from you not having the facilities to build
the Windows installers at all, where you rely on 3rd parties making
builds available.

As Martin points out, ultimately the provision of Windows binaries is
an issue for the extension project - is the demand high enough to
justify the effort, can you find tools and/or a helper, etc etc.

But the former issue (how quickly you can provide binaries, assuming
that you will do so ultimately) is more relevant for python-dev.
Specifically, because lack of binary extensions can be a barrier to
take-up of the new version. Certainly, in the past, you could pretty
much guarantee that there would be very few Windows users testing beta
releases, because pywin32 binaries weren't available. With 2.4, I have
at least one system I'd upgrade *now* but for the lack of a critical
extension in binary form (I haven't yet had the time to try to adapt
the build process to mingw for myself).

> Maybe we can set this up as a service?

That sounds like a good idea. What I'd suggest is needed is a website
where the following can take place:

1. People have a way of posting rquests for particular modules.
2. Installers can be uploaded to satisfy those requests.
3. There is somewhere to put step-by-step "build it yourself"
instructions, using free components, so that people *without* access
to VS.NET can make progress themselves. Obviously, if a particular
extension can't be built with free compilers, then binaries or access
to VS.NET are the only options.

The installers should be clearly noted as having no warranty or
support implied, to encourage people to offer binaries they have built
without feeling that they are taking on a support burden. Conversely,
as soon as "official" binaries are available from the extension
project, the binaries available here should be removed (and replaced
with a link to the official site) to reinforce the "unofficial" nature
of the binaries provided here.

The biggest potential issue with such a site is clearly validation.
I've no idea how to make something like this work without it being a
major virus risk. Which may, sadly, be enough to kill the idea :-(

Paul.
From carribeiro at gmail.com  Sun Dec 12 18:32:03 2004
From: carribeiro at gmail.com (Carlos Ribeiro)
Date: Sun Dec 12 18:32:05 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <864d370904121209147b713f31@mail.gmail.com>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<cpeh62$ldh$1@sea.gmane.org>
	<ca471dc204121108225511d43@mail.gmail.com>
	<cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com>
	<864d370904121209147b713f31@mail.gmail.com>
Message-ID: <864d3709041212093274becde2@mail.gmail.com>

Hello all,

Just to complement my previous remarks, I would like to point out how
do a competing language defines itself in its own website. The
perl.org website has a simple faq that is a good piece of marketing.
What follows are direct quotes, just to point out how ot handle the
market perception about their quality & speed.

-- "Perl takes the best features from other languages, such as C, awk,
sed, sh, and BASIC, among others."

(A claim can't possibly be any more generic than this. Strangely
enough, it mentions only older languages -- not Java, C++, or even
Python (!). This is possibly a sign of an old quote, but anyway: it
has a good marketing effect, specially for non-techies.)

-- "Perl can be embedded into web servers to speed up processing by as
much as 2000%."

(BTW, this quote is embarrassing misleading -- it probably means that
Perl is 20x slower when started on a request basis by the web server,
and that embedding it will accelerate response by a huge factor. I'm
sure non-techies will read it as "Perl is able to accelerate my server
20x!")


Of course, the point here is not Perl-bashing. The point here is that
we should be able to "sell" Python better than we do now, even without
the need to resort to such poor measures. I'm sure the Python
community does have good & creative people that can write a good
"selling" FAQ for Python, emphasizing the main points of the language.

For those who believe that a non-profit project should not do any
marketing, a reminder. If the perception about Python is one of a slow
language, it's much more difficult to find places where you can use
Python. In the long run, many of us may be forced to work with other
languages & tools, just because that's where the money is. I
personally take it a matter of personal interest, because I know how
hard it is to "sell" Python to companies here in Brazil.


-- 
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: carribeiro@gmail.com
mail: carribeiro@yahoo.com
From bob at redivi.com  Sun Dec 12 18:46:18 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sun Dec 12 18:46:26 2004
Subject: [Python-Dev] Supporting Third Party Modules (was The other Py2.4
	issue)
In-Reply-To: <41BC6BB1.9010401@v.loewis.de>
References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au>	<41BAB240.6060101@v.loewis.de>
	<3BBC89A1-4B5A-11D9-80F2-000A9567635C@redivi.com>
	<41BC6BB1.9010401@v.loewis.de>
Message-ID: <BA266FBA-4C65-11D9-BE18-000A9567635C@redivi.com>


On Dec 12, 2004, at 11:02 AM, Martin v. L?wis wrote:

> Bob Ippolito wrote:
>>> I believe this is not implementable: How can the DLL know which 
>>> Python
>>> DLL to use?
>> Well for py2app on Mac OS X, I wrote an executable stub that chooses 
>> a Python runtime from an XML file, looks up and binds a few symbols 
>> from it dynamically, and then starts doing stuff.
>
> While that would work, I think this is inappropriate for this specific
> issue: we want to write extension modules which are independent of
> the Python version, and might even be used with multiple Python
> installations on the same system. In that case, adding configuration
> files won't work, as each usage of the extension might require a
> different Python DLL.

Yes, of course, I was talking about the executable, not extensions.  On 
Mac OS X 10.3+, the linker flag -undefined dynamic_lookup allows 
extensions to link to no Python whatsoever.  The extensions will just 
find the symbols it needs from some other image already loaded into the 
process at runtime.  If it weren't for the "forced" ABI 
incompatibility, we'd already have extensions that work 
cross-Python-major-version (assuming they used a safe subset of 
functions and structures).

-bob
From carribeiro at gmail.com  Sun Dec 12 18:53:10 2004
From: carribeiro at gmail.com (Carlos Ribeiro)
Date: Sun Dec 12 18:53:12 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <41BC6666.8000500@v.loewis.de>
References: <001401c4dda9$ad605680$e841fea9@oemcomputer>
	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>
	<41BA34FD.6050306@v.loewis.de>
	<20041211165508.GA10252@vicky.ecs.soton.ac.uk>
	<41BC6666.8000500@v.loewis.de>
Message-ID: <864d37090412120953187d31c9@mail.gmail.com>

On Sun, 12 Dec 2004 16:40:22 +0100, Martin v. L?wis <martin@v.loewis.de> wrote:
> If none of your users volunteers to do the build for you, I would stop
> worrying about the Windows users.

Sorry, Martin. I understand your point, but I think you are not being
realistic. I for myself took the decision to use only free tools for
my own development, but I still have to suport my Windows customers. I
can't force them to change to Linux. I don't own a copy of MSVC. Also,
one of the reasons to choose a third part module is to save time. The
moment I am required to recompile everything myself I'm losing this
convenience. This of course impacts my ability to focus on my own
work, and so the story goes.

I'm not saying that module authors should work for free just to save
me some time & hassle. It's fair if an author decides to release a
Linux-only module. But again -- this is not realistic. The world has a
lot of Windows users, and I depend on them for my own income. If I
can't find a good set of Python tools for my projects, what should I
do? Picking another language is not a choice, mind you :-)

All in all, I sincerely hope that this discussion end up in a high
note. I'm not qualified to talk about the particulars of C compilers &
development environments, but as a Python user, I have hope that a
good solution will be found to make the process of building Python
extensions for Windows more convenient. The dream scenario is not to
require recompiling, at least inside the same major release (2.4 to
2.5, for example). That would be really great.

-- 
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: carribeiro@gmail.com
mail: carribeiro@yahoo.com
From fredrik at pythonware.com  Sun Dec 12 18:53:27 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun Dec 12 18:53:35 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
References: <e8bf7a530412101128326aea7b@mail.gmail.com>
	<001d01c4df09$87e5caa0$e841fea9@oemcomputer>
Message-ID: <cpi0io$s64$1@sea.gmane.org>

> . Multiple assignment is slower than individual assignment.  For
> example "x,y=a,b" is slower than "x=a; y=b".  However, multiple
> assignment is faster for variable swaps.  For example, "x,y=y,x" is
> faster than "t=x; x=y; y=t".

marginally faster in 2.4, a lot slower in earlier versions.  maybe you
should mark sections that rely on 2.4-specific optimizations, so that
people who use earlier versions don't end up slowing their programs
down...

</F> 



From martin at v.loewis.de  Sun Dec 12 19:06:44 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Dec 12 19:06:47 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <864d37090412120953187d31c9@mail.gmail.com>
References: <001401c4dda9$ad605680$e841fea9@oemcomputer>	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>	<41BA34FD.6050306@v.loewis.de>	<20041211165508.GA10252@vicky.ecs.soton.ac.uk>	<41BC6666.8000500@v.loewis.de>
	<864d37090412120953187d31c9@mail.gmail.com>
Message-ID: <41BC88B4.2010703@v.loewis.de>

Carlos Ribeiro wrote:
>>If none of your users volunteers to do the build for you, I would stop
>>worrying about the Windows users.
> 
> 
> Sorry, Martin. I understand your point, but I think you are not being
> realistic. I for myself took the decision to use only free tools for
> my own development, but I still have to suport my Windows customers. I
> can't force them to change to Linux. I don't own a copy of MSVC. Also,
> one of the reasons to choose a third part module is to save time. The
> moment I am required to recompile everything myself I'm losing this
> convenience. This of course impacts my ability to focus on my own
> work, and so the story goes.

I did not suggest that *all* your Windows users should recompile
your module - just a single one would be sufficient.

> I'm not saying that module authors should work for free just to save
> me some time & hassle. It's fair if an author decides to release a
> Linux-only module. But again -- this is not realistic. The world has a
> lot of Windows users, and I depend on them for my own income. If I
> can't find a good set of Python tools for my projects, what should I
> do? Picking another language is not a choice, mind you :-)

As I said: Find a volunteer that has the necessary build infrastructure,
and have that volunteer build the extension for you.

> The dream scenario is not to
> require recompiling, at least inside the same major release (2.4 to
> 2.5, for example). That would be really great.

That is guaranteed. Extensions built for 2.4 will certainly continue
to work in 2.4.1, and later 2.4.x. They will stop working with 2.5
(as they are linked with python24.dll).

Regards,
Martin
From martin at v.loewis.de  Sun Dec 12 19:09:05 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Dec 12 19:09:08 2004
Subject: [Python-Dev] Supporting Third Party Modules (was The other Py2.4
	issue)
In-Reply-To: <BA266FBA-4C65-11D9-BE18-000A9567635C@redivi.com>
References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au>	<41BAB240.6060101@v.loewis.de>	<3BBC89A1-4B5A-11D9-80F2-000A9567635C@redivi.com>	<41BC6BB1.9010401@v.loewis.de>
	<BA266FBA-4C65-11D9-BE18-000A9567635C@redivi.com>
Message-ID: <41BC8941.7090605@v.loewis.de>

Bob Ippolito wrote:
> Yes, of course, I was talking about the executable, not extensions.  On 
> Mac OS X 10.3+, the linker flag -undefined dynamic_lookup allows 
> extensions to link to no Python whatsoever.

It's the same on SysV ELF shared libraries, and in most other unices.

> The extensions will just 
> find the symbols it needs from some other image already loaded into the 
> process at runtime.  If it weren't for the "forced" ABI incompatibility, 
> we'd already have extensions that work cross-Python-major-version 
> (assuming they used a safe subset of functions and structures).

Are you talking about a forced ABI incompatibility, beyond the Windows
issue of linking with a specific pythonxy.dll?

On Unix, you certainly can have extensions across Python major versions.

Regards,
Martin
From nick at craig-wood.com  Sun Dec 12 19:15:37 2004
From: nick at craig-wood.com (Nick Craig-Wood)
Date: Sun Dec 12 19:15:41 2004
Subject: [Python-Dev] The other Py2.4 issue
In-Reply-To: <41BC6666.8000500@v.loewis.de>
References: <001401c4dda9$ad605680$e841fea9@oemcomputer>
	<20041210100659.GB15865@vicky.ecs.soton.ac.uk>
	<41BA34FD.6050306@v.loewis.de>
	<20041211165508.GA10252@vicky.ecs.soton.ac.uk>
	<41BC6666.8000500@v.loewis.de>
Message-ID: <20041212181537.GA22022@craig-wood.com>

On Sun, Dec 12, 2004 at 04:40:22PM +0100, "Martin v. L?wis" wrote:
> Armin Rigo wrote:
> >Hum, this is getting into a Linux-vs-Windows argument.  I don't
> >want to invest time and money on Windows tools just to compile my
> >extension module for Windows users...
> 
> If you don't have Windows at all, you cannot create Windows installers
> for your users, anyway.

Actually you are wrong there!  We cross compile our major application
(about 500k lines of C/C++) on linux/x86 for various architectures,
Windows/x86 being one of them.  We create the Windows installer using
the Nullsoft installer (which we run under Wine) and compile with a
gcc / mingw cross compiler.  This all works very well and means we can
build all our architectures every night on a single machine.

I'm following this thread with interest because we are considering
embedding Python into this project, and I'm wondering whether we can
cross compile python using mingw (almost certainly by the sound of
it), but probably harder would be to make python module build and
install system work cross-wise.

-- 
Nick Craig-Wood <nick@craig-wood.com> -- http://www.craig-wood.com/nick
From bob at redivi.com  Sun Dec 12 19:24:15 2004
From: bob at redivi.com (Bob Ippolito)
Date: Sun Dec 12 19:24:21 2004
Subject: [Python-Dev] Supporting Third Party Modules (was The other Py2.4
	issue)
In-Reply-To: <41BC8941.7090605@v.loewis.de>
References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au>	<41BAB240.6060101@v.loewis.de>	<3BBC89A1-4B5A-11D9-80F2-000A9567635C@redivi.com>	<41BC6BB1.9010401@v.loewis.de>
	<BA266FBA-4C65-11D9-BE18-000A9567635C@redivi.com>
	<41BC8941.7090605@v.loewis.de>
Message-ID: <07BA42EC-4C6B-11D9-BE18-000A9567635C@redivi.com>


On Dec 12, 2004, at 1:09 PM, Martin v. L?wis wrote:

> Bob Ippolito wrote:
>> Yes, of course, I was talking about the executable, not extensions.  
>> On Mac OS X 10.3+, the linker flag -undefined dynamic_lookup allows 
>> extensions to link to no Python whatsoever.
>
> It's the same on SysV ELF shared libraries, and in most other unices.
>
>> The extensions will just find the symbols it needs from some other 
>> image already loaded into the process at runtime.  If it weren't for 
>> the "forced" ABI incompatibility, we'd already have extensions that 
>> work cross-Python-major-version (assuming they used a safe subset of 
>> functions and structures).
>
> Are you talking about a forced ABI incompatibility, beyond the Windows
> issue of linking with a specific pythonxy.dll?
>
> On Unix, you certainly can have extensions across Python major 
> versions.

I was talking about PYTHON_API_VERSION, hence "forced" in quotes.  
Nobody likes to see an ugly error message.

-bob

From abkhd at hotmail.com  Sun Dec 12 20:21:11 2004
From: abkhd at hotmail.com (A.B., Khalid)
Date: Sun Dec 12 20:23:05 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue   
Message-ID: <BAY23-F336B3D172EA9A1DE45777CABAA0@phx.gbl>

Nick Craig-Wood wrote:

>I'm following this thread with interest because we are considering
>embedding Python into this project, and I'm wondering whether we can
>cross compile python using mingw (almost certainly by the sound of
>it), but probably harder would be to make python module build and
>install system work cross-wise.


I am following this thread with interest too, because I can certainly report 
that it is now indeed possible and quite easy to compile not only Python's 
2.4 extensions but also Python itself in MinGW with the help of pyMinGW[1]. 
Indeed there is no need to buy or even download 400MB+ of code to do so.

After patching the Python 2.4 source with pyMinGW-24, and issuing the 
command:
     make -f python24.mak all

We get Python and all its subprojects (socket, ssl, etc.) ready to run:

$ /c/temp/py24/python/dist/src/MinGW/python -i
Python 2.4.1a0 (#60, Dec  6 2004, 21:05:41)
[GCC 3.4.1 (mingw special)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>import sys
>>>sys.winver
'2.4'
>>>


To compile Armin's Psyco 1.3 in my MinGW compiled Python, I simply issued:

     python setup.py build

and then

     python setup.py bdist_wininst


And I had a /dist/psyco-1.3.win32-py2.4.exe file ready to be installed. It 
works fine at this end (note that I poked myself in the Windows registry to 
insert the pythoncore version information etc., but that should not be the 
case with those running the official Windows distribution). I only wonder if 
extensions created this way will work just as good in the latter. Because if 
they did, then going down the road of extensive downloads or expensive 
investment would be truly and totally unnecessary.


Regards,
Khalid




[1] pyMinGW can be found here:
http://jove.prohosting.com/iwave/ipython/pyMinGW.html

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

From martin at v.loewis.de  Sun Dec 12 23:01:01 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Dec 12 23:01:05 2004
Subject: [Python-Dev] Supporting Third Party Modules (was The other Py2.4
	issue)
In-Reply-To: <07BA42EC-4C6B-11D9-BE18-000A9567635C@redivi.com>
References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au>	<41BAB240.6060101@v.loewis.de>	<3BBC89A1-4B5A-11D9-80F2-000A9567635C@redivi.com>	<41BC6BB1.9010401@v.loewis.de>
	<BA266FBA-4C65-11D9-BE18-000A9567635C@redivi.com>
	<41BC8941.7090605@v.loewis.de>
	<07BA42EC-4C6B-11D9-BE18-000A9567635C@redivi.com>
Message-ID: <41BCBF9D.6060900@v.loewis.de>

Bob Ippolito wrote:
> I was talking about PYTHON_API_VERSION, hence "forced" in quotes.  
> Nobody likes to see an ugly error message.

Ah, that... If PYTHON_API_VERSION changes, it is because there is
a real incompatibility, somewhere. It is not automatically bumped
with each release. So it is safe to ignore the ugly error message
only if you have not used any of the API that became incompatible.

Regards,
Martin
From martin at v.loewis.de  Sun Dec 12 23:07:18 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Dec 12 23:07:21 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
In-Reply-To: <BAY23-F336B3D172EA9A1DE45777CABAA0@phx.gbl>
References: <BAY23-F336B3D172EA9A1DE45777CABAA0@phx.gbl>
Message-ID: <41BCC116.8040202@v.loewis.de>

A.B., Khalid wrote:
> I am following this thread with interest too, because I can certainly 
> report that it is now indeed possible and quite easy to compile not only 
> Python's 2.4 extensions but also Python itself in MinGW with the help of 
> pyMinGW[1]. Indeed there is no need to buy or even download 400MB+ of 
> code to do so.

That's good to know, but somewhat unrelated to the issue under
discussion. While it is clear from your message that you can build
python, as well as python extensions with MingW, it is not that
clear whether extensions build with mingw will work in the standard
2.4 distribution, or whether you can use the standard 2.4 distribution
to build extensions with mingw.

Regards,
Martin
From pje at telecommunity.com  Sun Dec 12 23:21:21 2004
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun Dec 12 23:19:37 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
In-Reply-To: <41BCC116.8040202@v.loewis.de>
References: <BAY23-F336B3D172EA9A1DE45777CABAA0@phx.gbl>
	<BAY23-F336B3D172EA9A1DE45777CABAA0@phx.gbl>
Message-ID: <5.1.1.6.0.20041212171632.03786cb0@mail.telecommunity.com>

At 11:07 PM 12/12/04 +0100, Martin v. L?wis wrote:
>it is not that
>clear whether extensions build with mingw will work in the standard
>2.4 distribution, or whether you can use the standard 2.4 distribution
>to build extensions with mingw.

I thought that it was clarified some time ago, with my success reports for 
the pre-alpha version.  Paul Moore has also independently confirmed this, 
and posted reasons why he doesn't believe that the dangling references to 
msvcrt should cause a problem in practice.

The item that was unclear was whether newer versions of MinGW could get 
away *without* building a libpython24.a file, and Paul indicated that this 
was not possible.

So, using the script that I posted previously to build libpython24.a from 
python24.dll is still a valid path for using MinGW to build extensions for 
Python on Windows.

From adam.jtm30 at gmail.com  Mon Dec 13 00:18:44 2004
From: adam.jtm30 at gmail.com (Adam Bark)
Date: Mon Dec 13 00:18:49 2004
Subject: [Python-Dev] Re: The Other Py2.4 issue?
In-Reply-To: <20041212180927.E24C61E4014@bag.python.org>
References: <20041212180927.E24C61E4014@bag.python.org>
Message-ID: <be4fbf9204121215184b0814b7@mail.gmail.com>

Now this might sound a bit stupid but I've only been programming in
python for about 6 months and before that about the same on VB. Anyway
here goes, as python is built in C & C++ surely every piece of python
code has a corresponding piece of C/C++ albeit more complex. So would
it be possible to somehow make a program to convert the Python to C &
C++ which can then be compiled with a C/C++ compiler.

Adam


On Sun, 12 Dec 2004 19:09:27 +0100 (CET),
python-dev-request@python.org <python-dev-request@python.org> wrote:
> Send Python-Dev mailing list submissions to
>        python-dev@python.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
>        http://mail.python.org/mailman/listinfo/python-dev
> or, via email, send a message with subject or body 'help' to
>        python-dev-request@python.org
> 
> You can reach the person managing the list at
>        python-dev-owner@python.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Python-Dev digest..."
> 
> 
> Today's Topics:
> 
>   1. Re: Re: Re: 2.4 news reaches interesting places (Carlos Ribeiro)
>   2. Re: The other Py2.4 issue (Paul Moore)
>   3. Re: Re: Re: 2.4 news reaches interesting places (Carlos Ribeiro)
>   4. Re: Supporting Third Party Modules (was The other Py2.4
>      issue) (Bob Ippolito)
>   5. Re: The other Py2.4 issue (Carlos Ribeiro)
>   6. Re: Re: 2.4 news reaches interesting places (Fredrik Lundh)
>   7. Re: The other Py2.4 issue (Martin v. L?wis)
>   8. Re: Supporting Third Party Modules (was The other Py2.4
>      issue) (Martin v. L?wis)
> 
> 
> 
> ---------- Forwarded message ----------
> From: Carlos Ribeiro <carribeiro@gmail.com>
> To: Erik Heneryd <erik@heneryd.com>
> Date: Sun, 12 Dec 2004 15:14:09 -0200
> Subject: Re: [Python-Dev] Re: Re: 2.4 news reaches interesting places
> On Sun, 12 Dec 2004 17:10:58 +0100, Erik Heneryd <erik@heneryd.com> wrote:
> > Fredrik Lundh wrote:
> > >>>fwiw, IDG's Computer Sweden,  "sweden's leading IT-newspaper" has a
> > >>>surprisingly big Python article in their most recent issue:
> > >>>
> > >>>    PYTHON FEELS WELL
> > >>>    Better performance biggest news in 2.4
> > >>>
> >
> > >>>and briefly interviews swedish zope-developer Johan Carlsson and Python-
> > >>>Ware co-founder H?kan Karlsson.
> > >>
> >
> > ...
> >
> > >
> > > so I don't think you can blame Johan or H?kan...  the writer simply read the
> > > python.org material, and picked a couple of things that he found interesting
> > > (decorators and generator expressions may be a big thing for an experienced
> > > pythoneer, but they are probably a bit too obscure for a general audience...)
> >
> > I'm a bit puzzled by the last paragraph, where Python is grouped
> > together with PHP and Perl - names starting with p, being popular on
> > Linux and not having big, commercial backers.  The article then
> > concludes "Since Python is copyrighted, it's not truly open.  However,
> > it can be freely used and redistributed, even commercially."
> >
> > Huh?  Where did THAT come from?  You might argue the merits of Python
> > being associated with Perl/PHP, but it's a fact that it is.  But when it
> > is, it's seen as less free?
> 
> The author was probably referring to the old (and as AFAIK already
> solved) CRNI copyright issue that ocurred into the 1.x to 2.x series
> transition. It's amazing how old memes from Python keep being
> remembered and repeated, even years after the fact. It also
> illustrates something very important - the community is not doing a
> good job at spreading the news; perhaps we talk too much between
> ourselves, and too little with the outside market. IMHO the website is
> a great part of this, its message being more important to "sell"
> Python than the standard library or native .exes.
> 
> About the website, a note from my own experience: when I search for
> documentation on Python, I'm usually directed to some of the mirror of
> the main python.org site. To find it inside the main site, I have to
> use "site:python.org", or even "site:docs.python.org". Usually Google
> does a good job at ranking pages, and if it doesn't rank the main
> Python website very highly, it's because they're not being referred
> to. A campaign to ask people to put links back to the canonical
> documentation at the Python website would be nice.
> 
> --
> Carlos Ribeiro
> Consultoria em Projetos
> blog: http://rascunhosrotos.blogspot.com
> blog: http://pythonnotes.blogspot.com
> mail: carribeiro@gmail.com
> mail: carribeiro@yahoo.com
> 
> 
> 
> ---------- Forwarded message ----------
> From: Paul Moore <p.f.moore@gmail.com>
> To: Christian Tismer <tismer@stackless.com>
> Date: Sun, 12 Dec 2004 17:26:49 +0000
> Subject: Re: [Python-Dev] The other Py2.4 issue
> On Sat, 11 Dec 2004 19:57:55 +0100, Christian Tismer
> <tismer@stackless.com> wrote:
> > Armin Rigo wrote:
> 
> > > Hum, this is getting into a Linux-vs-Windows argument.  I don't want to invest
> > > time and money on Windows tools just to compile my extension module for
> > > Windows users...
> 
> First of all, I'm assuming this is a timing issue. If I understood
> your initial posting, your concern was that people wanted Windows
> build of your extension *now*, and it was going to take you time to
> make it available.
> 
> That's a different issue from you not having the facilities to build
> the Windows installers at all, where you rely on 3rd parties making
> builds available.
> 
> As Martin points out, ultimately the provision of Windows binaries is
> an issue for the extension project - is the demand high enough to
> justify the effort, can you find tools and/or a helper, etc etc.
> 
> But the former issue (how quickly you can provide binaries, assuming
> that you will do so ultimately) is more relevant for python-dev.
> Specifically, because lack of binary extensions can be a barrier to
> take-up of the new version. Certainly, in the past, you could pretty
> much guarantee that there would be very few Windows users testing beta
> releases, because pywin32 binaries weren't available. With 2.4, I have
> at least one system I'd upgrade *now* but for the lack of a critical
> extension in binary form (I haven't yet had the time to try to adapt
> the build process to mingw for myself).
> 
> > Maybe we can set this up as a service?
> 
> That sounds like a good idea. What I'd suggest is needed is a website
> where the following can take place:
> 
> 1. People have a way of posting rquests for particular modules.
> 2. Installers can be uploaded to satisfy those requests.
> 3. There is somewhere to put step-by-step "build it yourself"
> instructions, using free components, so that people *without* access
> to VS.NET can make progress themselves. Obviously, if a particular
> extension can't be built with free compilers, then binaries or access
> to VS.NET are the only options.
> 
> The installers should be clearly noted as having no warranty or
> support implied, to encourage people to offer binaries they have built
> without feeling that they are taking on a support burden. Conversely,
> as soon as "official" binaries are available from the extension
> project, the binaries available here should be removed (and replaced
> with a link to the official site) to reinforce the "unofficial" nature
> of the binaries provided here.
> 
> The biggest potential issue with such a site is clearly validation.
> I've no idea how to make something like this work without it being a
> major virus risk. Which may, sadly, be enough to kill the idea :-(
> 
> Paul.
> 
> 
> 
> ---------- Forwarded message ----------
> From: Carlos Ribeiro <carribeiro@gmail.com>
> To: "python-dev@python.org" <python-dev@python.org>
> Date: Sun, 12 Dec 2004 15:32:03 -0200
> Subject: Re: [Python-Dev] Re: Re: 2.4 news reaches interesting places
> Hello all,
> 
> Just to complement my previous remarks, I would like to point out how
> do a competing language defines itself in its own website. The
> perl.org website has a simple faq that is a good piece of marketing.
> What follows are direct quotes, just to point out how ot handle the
> market perception about their quality & speed.
> 
> -- "Perl takes the best features from other languages, such as C, awk,
> sed, sh, and BASIC, among others."
> 
> (A claim can't possibly be any more generic than this. Strangely
> enough, it mentions only older languages -- not Java, C++, or even
> Python (!). This is possibly a sign of an old quote, but anyway: it
> has a good marketing effect, specially for non-techies.)
> 
> -- "Perl can be embedded into web servers to speed up processing by as
> much as 2000%."
> 
> (BTW, this quote is embarrassing misleading -- it probably means that
> Perl is 20x slower when started on a request basis by the web server,
> and that embedding it will accelerate response by a huge factor. I'm
> sure non-techies will read it as "Perl is able to accelerate my server
> 20x!")
> 
> Of course, the point here is not Perl-bashing. The point here is that
> we should be able to "sell" Python better than we do now, even without
> the need to resort to such poor measures. I'm sure the Python
> community does have good & creative people that can write a good
> "selling" FAQ for Python, emphasizing the main points of the language.
> 
> For those who believe that a non-profit project should not do any
> marketing, a reminder. If the perception about Python is one of a slow
> language, it's much more difficult to find places where you can use
> Python. In the long run, many of us may be forced to work with other
> languages & tools, just because that's where the money is. I
> personally take it a matter of personal interest, because I know how
> hard it is to "sell" Python to companies here in Brazil.
> 
> --
> Carlos Ribeiro
> Consultoria em Projetos
> blog: http://rascunhosrotos.blogspot.com
> blog: http://pythonnotes.blogspot.com
> mail: carribeiro@gmail.com
> mail: carribeiro@yahoo.com
> 
> 
> 
> ---------- Forwarded message ----------
> From: Bob Ippolito <bob@redivi.com>
> To:  "Martin v. L?wis" <martin@v.loewis.de>
> Date: Sun, 12 Dec 2004 12:46:18 -0500
> Subject: Re: [Python-Dev] Supporting Third Party Modules (was The other Py2.4 issue)
> 
> On Dec 12, 2004, at 11:02 AM, Martin v. L?wis wrote:
> 
> > Bob Ippolito wrote:
> >>> I believe this is not implementable: How can the DLL know which
> >>> Python
> >>> DLL to use?
> >> Well for py2app on Mac OS X, I wrote an executable stub that chooses
> >> a Python runtime from an XML file, looks up and binds a few symbols
> >> from it dynamically, and then starts doing stuff.
> >
> > While that would work, I think this is inappropriate for this specific
> > issue: we want to write extension modules which are independent of
> > the Python version, and might even be used with multiple Python
> > installations on the same system. In that case, adding configuration
> > files won't work, as each usage of the extension might require a
> > different Python DLL.
> 
> Yes, of course, I was talking about the executable, not extensions.  On
> Mac OS X 10.3+, the linker flag -undefined dynamic_lookup allows
> extensions to link to no Python whatsoever.  The extensions will just
> find the symbols it needs from some other image already loaded into the
> process at runtime.  If it weren't for the "forced" ABI
> incompatibility, we'd already have extensions that work
> cross-Python-major-version (assuming they used a safe subset of
> functions and structures).
> 
> -bob
> 
> 
> 
> ---------- Forwarded message ----------
> From: Carlos Ribeiro <carribeiro@gmail.com>
> To: Martin v. L?wis <martin@v.loewis.de>
> Date: Sun, 12 Dec 2004 15:53:10 -0200
> Subject: Re: [Python-Dev] The other Py2.4 issue
> On Sun, 12 Dec 2004 16:40:22 +0100, Martin v. L?wis <martin@v.loewis.de> wrote:
> > If none of your users volunteers to do the build for you, I would stop
> > worrying about the Windows users.
> 
> Sorry, Martin. I understand your point, but I think you are not being
> realistic. I for myself took the decision to use only free tools for
> my own development, but I still have to suport my Windows customers. I
> can't force them to change to Linux. I don't own a copy of MSVC. Also,
> one of the reasons to choose a third part module is to save time. The
> moment I am required to recompile everything myself I'm losing this
> convenience. This of course impacts my ability to focus on my own
> work, and so the story goes.
> 
> I'm not saying that module authors should work for free just to save
> me some time & hassle. It's fair if an author decides to release a
> Linux-only module. But again -- this is not realistic. The world has a
> lot of Windows users, and I depend on them for my own income. If I
> can't find a good set of Python tools for my projects, what should I
> do? Picking another language is not a choice, mind you :-)
> 
> All in all, I sincerely hope that this discussion end up in a high
> note. I'm not qualified to talk about the particulars of C compilers &
> development environments, but as a Python user, I have hope that a
> good solution will be found to make the process of building Python
> extensions for Windows more convenient. The dream scenario is not to
> require recompiling, at least inside the same major release (2.4 to
> 2.5, for example). That would be really great.
> 
> --
> Carlos Ribeiro
> Consultoria em Projetos
> blog: http://rascunhosrotos.blogspot.com
> blog: http://pythonnotes.blogspot.com
> mail: carribeiro@gmail.com
> mail: carribeiro@yahoo.com
> 
> 
> 
> ---------- Forwarded message ----------
> From: "Fredrik Lundh" <fredrik@pythonware.com>
> To: python-dev@python.org
> Date: Sun, 12 Dec 2004 18:53:27 +0100
> Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
> > . Multiple assignment is slower than individual assignment.  For
> > example "x,y=a,b" is slower than "x=a; y=b".  However, multiple
> > assignment is faster for variable swaps.  For example, "x,y=y,x" is
> > faster than "t=x; x=y; y=t".
> 
> marginally faster in 2.4, a lot slower in earlier versions.  maybe you
> should mark sections that rely on 2.4-specific optimizations, so that
> people who use earlier versions don't end up slowing their programs
> down...
> 
> </F>
> 
> 
> 
> ---------- Forwarded message ----------
> From: "Martin v. L?wis" <martin@v.loewis.de>
> To: Carlos Ribeiro <carribeiro@gmail.com>
> Date: Sun, 12 Dec 2004 19:06:44 +0100
> Subject: Re: [Python-Dev] The other Py2.4 issue
> Carlos Ribeiro wrote:
> >>If none of your users volunteers to do the build for you, I would stop
> >>worrying about the Windows users.
> >
> >
> > Sorry, Martin. I understand your point, but I think you are not being
> > realistic. I for myself took the decision to use only free tools for
> > my own development, but I still have to suport my Windows customers. I
> > can't force them to change to Linux. I don't own a copy of MSVC. Also,
> > one of the reasons to choose a third part module is to save time. The
> > moment I am required to recompile everything myself I'm losing this
> > convenience. This of course impacts my ability to focus on my own
> > work, and so the story goes.
> 
> I did not suggest that *all* your Windows users should recompile
> your module - just a single one would be sufficient.
> 
> > I'm not saying that module authors should work for free just to save
> > me some time & hassle. It's fair if an author decides to release a
> > Linux-only module. But again -- this is not realistic. The world has a
> > lot of Windows users, and I depend on them for my own income. If I
> > can't find a good set of Python tools for my projects, what should I
> > do? Picking another language is not a choice, mind you :-)
> 
> As I said: Find a volunteer that has the necessary build infrastructure,
> and have that volunteer build the extension for you.
> 
> > The dream scenario is not to
> > require recompiling, at least inside the same major release (2.4 to
> > 2.5, for example). That would be really great.
> 
> That is guaranteed. Extensions built for 2.4 will certainly continue
> to work in 2.4.1, and later 2.4.x. They will stop working with 2.5
> (as they are linked with python24.dll).
> 
> Regards,
> Martin
> 
> 
> 
> ---------- Forwarded message ----------
> From: "Martin v. L?wis" <martin@v.loewis.de>
> To: Bob Ippolito <bob@redivi.com>
> Date: Sun, 12 Dec 2004 19:09:05 +0100
> Subject: Re: [Python-Dev] Supporting Third Party Modules (was The other Py2.4 issue)
> Bob Ippolito wrote:
> > Yes, of course, I was talking about the executable, not extensions.  On
> > Mac OS X 10.3+, the linker flag -undefined dynamic_lookup allows
> > extensions to link to no Python whatsoever.
> 
> It's the same on SysV ELF shared libraries, and in most other unices.
> 
> > The extensions will just
> > find the symbols it needs from some other image already loaded into the
> > process at runtime.  If it weren't for the "forced" ABI incompatibility,
> > we'd already have extensions that work cross-Python-major-version
> > (assuming they used a safe subset of functions and structures).
> 
> Are you talking about a forced ABI incompatibility, beyond the Windows
> issue of linking with a specific pythonxy.dll?
> 
> On Unix, you certainly can have extensions across Python major versions.
> 
> Regards,
> Martin
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> 
> 
>
From abkhd at hotmail.com  Mon Dec 13 00:19:55 2004
From: abkhd at hotmail.com (A.B., Khalid)
Date: Mon Dec 13 00:22:58 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
Message-ID: <BAY23-F609728FDC013615101711ABAA0@phx.gbl>

Martin v. Löwis wrote:
>That's good to know, but somewhat unrelated to the issue under
>discussion. While it is clear from your message that you can build
>python, as well as python extensions with MingW, it is not that
>clear whether extensions build with mingw will work in the standard
>2.4 distribution, or whether you can use the standard 2.4 distribution
>to build extensions with mingw.


Martin it is somewhat related. :)

But seriously, I did mention that which you note, and this is partly the 
reason why I am following this thread. But you are right. We must resolve 
that issue as it pertains to the pyMinGW patched & MinGW compiled Python, if 
you would kindly allow me to. And so I propose the following:


     [1] Can someone please email me [*] the officially distributed 
Python24.dll and python.exe,
          (I do not wish to install the full version yet; if this is too 
much to ask, then nevermind, I'll
          download the whole distribution), or alternatively, and more 
conveniently

     [2] Can someone who has the official Python 2.4 download the sample 
extension [**] created
          using the pyMinGW patched & MinGW compiled Python 2.4 and SWIG? 
And see if it works?
          Sources are in the zip file whose details appears bellow.





Regards,
Khalid




[*] Email to this address please: abkhd[-AT-]earth.co.jp
[**] pymingw.zip (5.35 KB):
Location: http://jove.prohosting.com/iwave/misc/pymingw.zip
MD5: b4c1a2ebcb8a00fde89f6efe102d983f
--------------------------
Contents in KB:
--------------------------
_PYMINGW PYD         9.216  13/12/04   1:00 _pymingw.pyd
PYMINGW  PY          1.115  13/12/04   1:00 pymingw.py
PYMINGW  I             278  13/12/04   1:00 pymingw.i
EXAMPLE  C             848  13/12/04   0:23 example.c
SETUP    PY            192  13/12/04   0:59 setup.py

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

From tdelaney at avaya.com  Mon Dec 13 01:11:47 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Mon Dec 13 01:11:53 2004
Subject: [Python-Dev] Re: The Other Py2.4 issue?
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE72121A@au3010avexu1.global.avaya.com>

Adam Bark wrote:

> Now this might sound a bit stupid but I've only been programming in
> python for about 6 months and before that about the same on VB. Anyway
> here goes, as python is built in C & C++ surely every piece of python
> code has a corresponding piece of C/C++ albeit more complex. So would
> it be possible to somehow make a program to convert the Python to C &
> C++ which can then be compiled with a C/C++ compiler.

Yes. And it would run at about the same speed as CPython.

The only way to significantly improve this is to remove much of the
dynamism of Python. As an example, I've been playing around with
Raymond's constant binding - making it do more aggressive binding (I'll
give an example later). By binding constants, I'm able to reduce the
runtimes for psyco-compiled versions of friendly code (lots of things
that can be converted into constant references) from around 2 seconds to
less than 0.001 seconds.

That's a very significant speedup. Unfortunately, it doesn't translate
as well into real-world code - or even benchmarks (parrotbench for
example gets a slight speedup but it's not overly significant).

As a quick example if the changes I'm playing with, 

    def func()
        docutils.parsers.rst.states.struct.__init__

Raymond's generates:

    JUMP_FORWARD   0
    LOAD_CONST    (docutils)
    LOAD_ATTR     (parsers)
    LOAD_ATTR     (rst)
    LOAD_ATTR     (states)
    LOAD_ATTR     (struct)
    LOAD_ATTR     (__init__)

Mine generates

     0 JUMP_ABSOLUTE  15
     3 NOP
     4 NOP
     5 NOP
     6 NOP
     7 NOP
     8 NOP
     9 NOP
    10 NOP
    11 NOP
    12 NOP
    13 NOP
    14 NOP
    15 LOAD_CONST     (<unbound method Struct.__init__)

I've also been doing some peephole optimisations - the above actually
ends up being an immediate jump to the return bytecode, but that's
fairly useless. Using JUMP_ABSOLUTE instead of JUMP_FORWARD simplifies
doing this.

Once I've got the code into a fit state I'll send it to you if you're
interested Raymond ...

Tim Delaney
From barry at python.org  Mon Dec 13 02:36:45 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Dec 13 02:36:48 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <864d3709041212093274becde2@mail.gmail.com>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<cpeh62$ldh$1@sea.gmane.org> <ca471dc204121108225511d43@mail.gmail.com>
	<cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com>
	<864d370904121209147b713f31@mail.gmail.com>
	<864d3709041212093274becde2@mail.gmail.com>
Message-ID: <1102901805.9393.104.camel@geddy.wooz.org>

On Sun, 2004-12-12 at 12:32, Carlos Ribeiro wrote:

> For those who believe that a non-profit project should not do any
> marketing, a reminder. If the perception about Python is one of a slow
> language, it's much more difficult to find places where you can use
> Python. In the long run, many of us may be forced to work with other
> languages & tools, just because that's where the money is. I
> personally take it a matter of personal interest, because I know how
> hard it is to "sell" Python to companies here in Brazil.

Actually, there's another problem in the corporate world that has
nothing to do with Python's performance (at least not directly).  When a
manager has to hire 25 programmers for a project they think to
themselves, "well, Java programmers are a dime a dozen so I'll have no
problem finding warm bodies if we write it in Java.  Can I even /find/
25 Python programmers?"

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20041212/5b10e7ae/attachment.pgp
From carribeiro at gmail.com  Mon Dec 13 03:13:15 2004
From: carribeiro at gmail.com (Carlos Ribeiro)
Date: Mon Dec 13 03:13:18 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <1102901805.9393.104.camel@geddy.wooz.org>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<cpeh62$ldh$1@sea.gmane.org>
	<ca471dc204121108225511d43@mail.gmail.com>
	<cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com>
	<864d370904121209147b713f31@mail.gmail.com>
	<864d3709041212093274becde2@mail.gmail.com>
	<1102901805.9393.104.camel@geddy.wooz.org>
Message-ID: <864d37090412121813379f6aeb@mail.gmail.com>

On Sun, 12 Dec 2004 20:36:45 -0500, Barry Warsaw <barry@python.org> wrote:
> On Sun, 2004-12-12 at 12:32, Carlos Ribeiro wrote:
> 
> > For those who believe that a non-profit project should not do any
> > marketing, a reminder. If the perception about Python is one of a slow
> > language, it's much more difficult to find places where you can use
> > Python. In the long run, many of us may be forced to work with other
> > languages & tools, just because that's where the money is. I
> > personally take it a matter of personal interest, because I know how
> > hard it is to "sell" Python to companies here in Brazil.
> 
> Actually, there's another problem in the corporate world that has
> nothing to do with Python's performance (at least not directly).  When a
> manager has to hire 25 programmers for a project they think to
> themselves, "well, Java programmers are a dime a dozen so I'll have no
> problem finding warm bodies if we write it in Java.  Can I even /find/
> 25 Python programmers?"

You're right, specially for big corporations. But in the end, we're
just running in circles: it's hard to get new programmers to learn
Python, partly because it's in low demand, and partly because the
language has an totally undeserved fame of being slow. That's right -
when I talk to fellow programmers that I'm writing software in Python,
many of them are amazed and ask me, "but isn't it slow?". I've heard
it more than once... having some place that I could point them out so
they could check it for themselves (perhaps "www.pythonspeed.com"?)
would be *great*.

IMHO, Python performance is not an issue 99% of the time. I swear I
can say the same about C++. The difference between C++ and Python is
not how fast they are relatively to each other, but *where* the slow
part is. Anyone who tried to write complex code in any other language
knows that most programmers usually resort to slow but easy to
implement algorithms for things such as sorting, list handling, etc;
these parts of the code are naturally fast in Python, while others may
be faster in C++... so we're just trading "6 for half a dozen", as we
say in portuguese :-)


-- 
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: carribeiro@gmail.com
mail: carribeiro@yahoo.com
From p.f.moore at gmail.com  Mon Dec 13 10:58:49 2004
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon Dec 13 10:58:52 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
In-Reply-To: <BAY23-F609728FDC013615101711ABAA0@phx.gbl>
References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl>
Message-ID: <79990c6b041213015861230c01@mail.gmail.com>

On Sun, 12 Dec 2004 23:19:55 +0000, A.B., Khalid <abkhd@hotmail.com> wrote:
>      [2] Can someone who has the official Python 2.4 download the sample
> extension [**] created using the pyMinGW patched & MinGW compiled Python 2.4
> and SWIG? And see if it works?
> Sources are in the zip file whose details appears bellow.

I tried this out, and from some basic uses, it seems to work OK.
However, the PYD file references msvcrt.dll, which suggests that there
will be issues in more complex cases. The biggest problem with CRT
compatibility issues is that (AFAIK) no-one has actually been able to
trigger a *real* error, all of the problems are still theoretical. I
have done some fairly extensive analysis of what could go wrong, and I
suspect that there are very few genuine cases where data from one CRT
could end up being manipulated by another - but this doesn't mean that
the problems aren't real, just VERY hard to locate :-(

My current technique for checking an extension is compatible with
Python 2.4 is to run objdump -p (from the mingw distribution - use
dumpbin /imports from MSVC) and review the import table. If any
symbols are referenced from msvcrt.dll, you need to convince yourself
that they are used solely by the mingw runtime startup code. For added
certainty, add a dummy function to your C code which references these
symbols, rebuild and confirm that they are now satisfied from msvcr71.
(For total guarantees, you need to get Martin to validate your
analysis, as he seems to be the only person here who *really*
understands the potential issues :-))

If you want to build extensions compatible with the standard Python
2.4 build, you need to add the -lmsvcr71 flag to the build (link
step).

Hope this helps,
Paul.
From theller at python.net  Mon Dec 13 12:36:23 2004
From: theller at python.net (Thomas Heller)
Date: Mon Dec 13 12:35:18 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
In-Reply-To: <79990c6b041213015861230c01@mail.gmail.com> (Paul Moore's
	message of "Mon, 13 Dec 2004 09:58:49 +0000")
References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl>
	<79990c6b041213015861230c01@mail.gmail.com>
Message-ID: <wtvma0qw.fsf@python.net>

Paul Moore <p.f.moore@gmail.com> writes:

> The biggest problem with CRT compatibility issues is that (AFAIK)
> no-one has actually been able to trigger a *real* error, all of the
> problems are still theoretical.

There have been problems with the bdist_wininst exe-stub linking to the
wrong CRT dll, but I don't remember the exact details.

And recently I played with bindings to OpenGL's glut32.dll - glut calls
exit() from internal C code.  If linked with the wrong CRT, this will do
nothing instead of terminating the process.

Thomas

From mal at egenix.com  Mon Dec 13 14:48:36 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon Dec 13 14:48:39 2004
Subject: [Python-Dev] Deprecated xmllib module
In-Reply-To: <41B2C870.5090609@v.loewis.de>
References: <001801c4da72$543801a0$ab2fc797@oemcomputer>
	<41B2C870.5090609@v.loewis.de>
Message-ID: <41BD9DB4.6040501@egenix.com>

Martin v. L?wis wrote:
> As for PEP 4: I don't know whether it needs to be listed there. It
> appears that the PEP is largely unmaintained (I, personally, do not
> really maintain it). So one option would be to just stop using PEP 4
> for recording deprecations, since we now have the warnings module.
> If we want to keep PEP 4, we need to follow the procedures it
> requires (or modify them if we don't like them).

There is a need for a central place where deprecations are
posted. You can't tell people to go hunt the standard lib
for uses of the warning module in order for them to catch
up with the current state of affairs.

One thing that we shouldn't forget is that Python users
do not necessarily upgrade to new Python releases every time
there's a new release.

Production users of Python will often skip a release
or two depending on their needs, time frames and audit
cycles. It is very important for them to know that module
or other technology will be deprecated two releases down
the road, so that they can accomodate for these changes
today.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 13 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2004-12-06: Released eGenix mx Extensions for Python 2.4

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mal at egenix.com  Mon Dec 13 15:13:13 2004
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon Dec 13 15:13:15 2004
Subject: [Python-Dev] Rewriting PEP4
In-Reply-To: <1102370092.9565.30.camel@geddy.wooz.org>
References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer>	<200412060243.01129.anthony@interlink.com.au>	<41B34B62.7040201@v.loewis.de>
	<41B39EA4.9030105@ocf.berkeley.edu>	<41B4CF07.4050108@v.loewis.de>
	<1102370092.9565.30.camel@geddy.wooz.org>
Message-ID: <41BDA379.7030403@egenix.com>

Barry Warsaw wrote:
> On Mon, 2004-12-06 at 16:28, "Martin v. L?wis" wrote:
> 
> Martin, +1 on everything you wrote, with one minor quibble.
> 
> 
>>Removal
>>=======
>>If the module has been deprecated for atleast a year and atleast
>>a version, it can be removed. Removal should move it to old-libs
>>for pure Python modules; a removal procedure for pure C modules
>>has not been defined yet.
> 
> 
> I wonder if the one year/one version rule is too short.  Given that new
> versions come about every 18 months, I'd probably give it a 2 year/one
> version limit.

+1

I'd make that 2 years / 2 versions, though.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 13 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2004-12-06: Released eGenix mx Extensions for Python 2.4

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From sdeibel at wingware.com  Mon Dec 13 17:30:45 2004
From: sdeibel at wingware.com (Stephan Deibel)
Date: Mon Dec 13 17:29:37 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <864d37090412121813379f6aeb@mail.gmail.com>
Message-ID: <Pine.LNX.4.50.0412131050190.3163-100000@hedgehog>

On Mon, 13 Dec 2004, Carlos Ribeiro wrote:
> On Sun, 12 Dec 2004 20:36:45 -0500, Barry Warsaw <barry@python.org> wrote:
> > Actually, there's another problem in the corporate world that has
> > nothing to do with Python's performance (at least not directly).  When a
> > manager has to hire 25 programmers for a project they think to
> > themselves, "well, Java programmers are a dime a dozen so I'll have no
> > problem finding warm bodies if we write it in Java.  Can I even /find/
> > 25 Python programmers?"
> 
> You're right, specially for big corporations. But in the end, we're
> just running in circles: it's hard to get new programmers to learn
> Python, partly because it's in low demand, and partly because the
> language has an totally undeserved fame of being slow.

The perception-of-speed issue is clearly important but we're definately
not running in circles.  There are quite a few signs that the Python 
user base is expanding substantially.  

For example, a September article in InfoWorld said "But the big winner
this time around is the object-oriented scripting language Python, which
saw a 6 percent gain in popularity, almost doubling last year's results."

http://www.infoworld.com/article/04/09/24/39FErrdev_1.html?s=feature

Also, there are companies that have hundreds of Python programmers,
like some of those that have done success stories:

http://www.pythonology.org/success

That doesn't mean the perception that you can't hire 25 at once isn't
a problem, but clearly some companies know that turning someone into
a Python programmer is easy enough to offset the smaller available pool.

To counter speed claims, look at articles like this one:

http://www.pythonology.org/success&story=suzanne

"Python helps AFNIC manage over 10,000 internet domain name registration 
requests per minute in a landrush for the ".fr" top-level internet domain"

Yes it would be nice to have more of these, where performance is mentioned
in the summary!  Please contact me if you can contribute one.

BTW, I can't resist my own favorite speed anecdote:  I once wrote a
one-off script to process down a couple of gigabytes of variously
fragmented web logs into month-by-month files.  I thought I was being
naive doing f.readline() in a for loop with some date parsing code for
each entry.  But I was completely astounded how fast it processed -- and
it just worked the first time around.

- Stephan

From abkhd at hotmail.com  Mon Dec 13 17:43:18 2004
From: abkhd at hotmail.com (A.B., Khalid)
Date: Mon Dec 13 17:44:18 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
Message-ID: <BAY23-F8386F00CD4776C6ACCB87ABAB0@phx.gbl>

Paul Moore wrote:
>I tried this out, and from some basic uses, it seems to work OK.
>However, the PYD file references msvcrt.dll, which suggests that there
>will be issues in more complex cases. The biggest problem with CRT
>compatibility issues is that (AFAIK) no-one has actually been able to
>trigger a *real* error, all of the problems are still theoretical. I
>have done some fairly extensive analysis of what could go wrong, and I
>suspect that there are very few genuine cases where data from one CRT
>could end up being manipulated by another - but this doesn't mean that
>the problems aren't real, just VERY hard to locate :-(



First of all, thank you for your input. It is greatly appreciated. And
yes if Martin, for whom I have much respect, can help here, then that
would be awesome.

Okay. This is partly my own fault. I am not using a standard MinGW
distribution. This is why there is no release date in the sys.version
information of the Python I compiled. I am using GCC RC 3.4.1 but the
linking is still done by the 2002 MinGW 3.2, which is most probably hard-
wired to use msvcrt.dll. Having said that, however, let me quickly add
that it is almost a certainity that the newer standard MinGW
distributions link with the latest libraries.

Nevertheless, I updated pyMinGW's cygwinccompiler.py to always link in
with libmsvcr71, and rebuit the sample pymingw extension[1]. Now
according to objdump we get the following:

[CUT]

00007014       000070b0 00000000 00000000 00007420 00007158

        DLL Name: KERNEL32.dll
        vma:  Hint/Ord Member-Name Bound-To
        72fc        0  AddAtomA
        7308      147  FindAtomA
        7314      191  GetAtomNameA

00007028       000070c4 00000000 00000000 00007468 0000716c

        DLL Name: msvcr71.dll
        vma:  Hint/Ord Member-Name Bound-To
        7324       56  __dllonexit
        7334      189  _errno
        7340      610  ctime
        7348      619  fflush
        7354      630  fputs
        735c      634  free
        7364      642  fwrite
        7370      685  malloc
        737c      701  puts
        7384      724  strcmp
        7390      726  strcpy
        739c      730  strlen
        73a8      733  strncpy
        73b4      748  time

0000703c       00007104 00000000 00000000 00007478 000071ac

        DLL Name: msvcrt.dll
        vma:  Hint/Ord Member-Name Bound-To
        73bc      510  abort

00007050       00000000 00000000 00000000 00000000 00000000

There is an export table in .edata at 0x6000

[CUT]



So what see ye? :) Does it look good?



Best wishes
Khalid




[1] pymingw-01.zip (5.37 KB):
Location: http://jove.prohosting.com/iwave/misc/pymingw-01.zip
MD5: 63b486093b344b0299538c952dea4ce1
--------------------------
Contents in KB:
--------------------------
PYMINGW  PY          1.115  13/12/04  16:12 pymingw.py
PYMINGW  I             278  13/12/04   1:00 pymingw.i
EXAMPLE  C             848  13/12/04   0:23 example.c
SETUP    PY            192  13/12/04   0:59 setup.py
_PYMINGW PYD         9.216  13/12/04  16:12 _pymingw.pyd

_________________________________________________________________
Don't just search. Find. Check out the new MSN Search! 
http://search.msn.click-url.com/go/onm00200636ave/direct/01/

From jhylton at gmail.com  Mon Dec 13 17:49:03 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Mon Dec 13 17:49:08 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <Pine.LNX.4.50.0412131050190.3163-100000@hedgehog>
References: <864d37090412121813379f6aeb@mail.gmail.com>
	<Pine.LNX.4.50.0412131050190.3163-100000@hedgehog>
Message-ID: <e8bf7a53041213084958aba266@mail.gmail.com>

On Mon, 13 Dec 2004 11:30:45 -0500 (EST), Stephan Deibel
<sdeibel@wingware.com> wrote:
> For example, a September article in InfoWorld said "But the big winner
> this time around is the object-oriented scripting language Python, which
> saw a 6 percent gain in popularity, almost doubling last year's results."
> 
> http://www.infoworld.com/article/04/09/24/39FErrdev_1.html?s=feature

Can we extrapolate from the numbers here to get an estimate of how
many Python developers there are?  I was asked for that number at
workshop a few months ago and I didn't have any idea how to answer. 
Is there a good answer?

Two possibilities come to mind.  1) 14% of developers in the survey
work at companies that use Python.  How many developers are there? 
Assume that 14% of them use Python.  But what's a good estimate for
"number of developers."  Pretty rough -- number of survey respondents
at company != number of Python programmers at company, and %age
companies != %age of programmers.  2) 64% of companies use Java, 4.5
times more than Python.  Find out how many Java programmers there are,
divide by 4.5.

Jeremy
From p.f.moore at gmail.com  Mon Dec 13 17:52:45 2004
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon Dec 13 17:52:48 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
In-Reply-To: <BAY23-F8386F00CD4776C6ACCB87ABAB0@phx.gbl>
References: <BAY23-F8386F00CD4776C6ACCB87ABAB0@phx.gbl>
Message-ID: <79990c6b0412130852458e5f0b@mail.gmail.com>

On Mon, 13 Dec 2004 16:43:18 +0000, A.B., Khalid <abkhd@hotmail.com> wrote:
> So what see ye? :) Does it look good?

Looks good to me. The one remaining link to msvcrt, abort, is likely
to be due to the startup code using it, and your code not referencing
that symbol, so that it doesn't force linking from msvcr71. To check
this, you could add a dummy call to abort() somewhere in your code and
this last reference should vanish.

So this now looks just like an extension built using
--compiler=mingw32 with the standard Python 2.4 build, which I guess
means that your build can create extensions that are interoperable
with both itself and the standard Python 2.4.

Paul.
From fredrik at pythonware.com  Mon Dec 13 18:07:04 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon Dec 13 18:07:16 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
References: <864d37090412121813379f6aeb@mail.gmail.com>
	<Pine.LNX.4.50.0412131050190.3163-100000@hedgehog>
Message-ID: <cpki7r$k3p$1@sea.gmane.org>

Stephan Deibel wrote:

> BTW, I can't resist my own favorite speed anecdote:  I once wrote a
> one-off script to process down a couple of gigabytes of variously
> fragmented web logs into month-by-month files.  I thought I was being
> naive doing f.readline() in a for loop with some date parsing code for
> each entry.  But I was completely astounded how fast it processed -- and
> it just worked the first time around.

I was doing the same thing today (well, processing system logs, not web logs)
and was completely baffled when I found that a program that just worked the
first time around on our development machine, crashed with a MemoryError
on the production machine.

oh well.

</F> 



From FBatista at uniFON.com.ar  Mon Dec 13 18:09:03 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Mon Dec 13 18:12:00 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
Message-ID: <A128D751272CD411BC9200508BC2194D053C7CDA@escpl.tcp.com.ar>

[Stephan Deibel]

#- For example, a September article in InfoWorld said "But the 
#- big winner
#- this time around is the object-oriented scripting language 
#- Python, which
#- saw a 6 percent gain in popularity, almost doubling last 
#- year's results."

How big are the chances that SourceForge numbers actually could be
extrapolated to the rest of the universe?

According to them (check software map, and look by programming language),
and showing everything with a "developers choice for their project" share >=
2%: 

C++             18.5
C               18.1
Java            17.5
PHP             12.9
Perl             7.2
Python           4.7
Visual Basic     2.6
C#               2.6
JavaScript       2.6
Delphi/Kylix     2.1
Unix Shell       2.0

It also would nice to see a graph showing tendencies here.

.	Facundo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20041213/08fa6e09/attachment.htm
From sdeibel at wingware.com  Mon Dec 13 19:11:38 2004
From: sdeibel at wingware.com (Stephan Deibel)
Date: Mon Dec 13 19:10:32 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <e8bf7a53041213084958aba266@mail.gmail.com>
References: <864d37090412121813379f6aeb@mail.gmail.com>
	<Pine.LNX.4.50.0412131050190.3163-100000@hedgehog>
	<e8bf7a53041213084958aba266@mail.gmail.com>
Message-ID: <Pine.LNX.4.50.0412131300240.3163-100000@hedgehog>

On Mon, 13 Dec 2004, Jeremy Hylton wrote:
> > http://www.infoworld.com/article/04/09/24/39FErrdev_1.html?s=feature
> 
> Can we extrapolate from the numbers here to get an estimate of how
> many Python developers there are?  I was asked for that number at
> workshop a few months ago and I didn't have any idea how to answer. 
> Is there a good answer?
> 
> Two possibilities come to mind.  1) 14% of developers in the survey
> work at companies that use Python.  How many developers are there? 
> Assume that 14% of them use Python.  But what's a good estimate for
> "number of developers."  Pretty rough -- number of survey respondents
> at company != number of Python programmers at company, and %age
> companies != %age of programmers.  

Supposedly there are 5-6 million developers world-wide (by various
estimates; I've no idea whether to believe them).  If you just multiply
out naively you get 700K-840K Python programmers.  There certainly are
vastly more one person Python projects than large ones, so this may not be
all that far off.

> 2) 64% of companies use Java, 4.5
> times more than Python.  Find out how many Java programmers there are,
> divide by 4.5.

I've seen claims of 3-4 million java programmers so that's 666K-888K
Python programmers.  Interestingly, this matches up with the above abuse
of statistics.

Independently by various other techniques of wildly guesstimating over the
years, I've come to a current figure of about 350K serious Python projects
world-wide.  Many are single-person projects so this does mesh w/in an
order of magnitude with the above.

- Stephan
From sdeibel at wingware.com  Mon Dec 13 19:24:58 2004
From: sdeibel at wingware.com (Stephan Deibel)
Date: Mon Dec 13 19:23:50 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <A128D751272CD411BC9200508BC2194D053C7CDA@escpl.tcp.com.ar>
Message-ID: <Pine.LNX.4.50.0412131315580.3163-100000@hedgehog>

On Mon, 13 Dec 2004, Batista, Facundo wrote:
> [Stephan Deibel]
> 
> #- For example, a September article in InfoWorld said "But the 
> #- big winner
> #- this time around is the object-oriented scripting language 
> #- Python, which
> #- saw a 6 percent gain in popularity, almost doubling last 
> #- year's results."
> 
> How big are the chances that SourceForge numbers actually could be
> extrapolated to the rest of the universe?

Not very good, I think.  I suspect the vast majority of programmers have
never heard of source forge.  

I'm actually surprised that it's only 4.7% on source forge -- that seems
to indicate Python is doing quite a bit better in the non-open source
world than on SF.  Interesting... wouldn't be surprised if this is because
the speed myth has stronger hold among hacker types than business
programmer types.

If people feel this is getting off-topic for python-dev, there is also
the mostly dormant marketing-python list:

http://wingware.com/mailman/listinfo/marketing-python

(the trolls have been removed)

- Stephan
From nidoizo at yahoo.com  Mon Dec 13 20:13:06 2004
From: nidoizo at yahoo.com (Nicolas Fleury)
Date: Mon Dec 13 20:13:19 2004
Subject: [Python-Dev] Re: 2.4 news reaches interesting places
In-Reply-To: <001d01c4df09$87e5caa0$e841fea9@oemcomputer>
References: <e8bf7a530412101128326aea7b@mail.gmail.com>
	<001d01c4df09$87e5caa0$e841fea9@oemcomputer>
Message-ID: <cpkpk0$e0v$1@sea.gmane.org>

Raymond Hettinger wrote:
> I felt the same way when reading it.  Also, it seemed to embody the
> political outlook that optimization is sinful.  The document could be
> much more positive, fact based, and informative.  Also, the wording
> seems somewhat outdated.
> 
> A draft for a new entry is included below.  Review and feedback are
> requested.

I would add a reference to __slots__, particularly for wrappers of objects.

Regards,
Nicolas

From jhylton at gmail.com  Mon Dec 13 21:08:56 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Mon Dec 13 21:09:00 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <Pine.LNX.4.50.0412131300240.3163-100000@hedgehog>
References: <864d37090412121813379f6aeb@mail.gmail.com>
	<Pine.LNX.4.50.0412131050190.3163-100000@hedgehog>
	<e8bf7a53041213084958aba266@mail.gmail.com>
	<Pine.LNX.4.50.0412131300240.3163-100000@hedgehog>
Message-ID: <e8bf7a5304121312086d13839c@mail.gmail.com>

On Mon, 13 Dec 2004 13:11:38 -0500 (EST), Stephan Deibel
<sdeibel@wingware.com> wrote:
> On Mon, 13 Dec 2004, Jeremy Hylton wrote:
> > Two possibilities come to mind.  1) 14% of developers in the survey
> > work at companies that use Python.  How many developers are there?
> > Assume that 14% of them use Python.  But what's a good estimate for
> > "number of developers."  Pretty rough -- number of survey respondents
> > at company != number of Python programmers at company, and %age
> > companies != %age of programmers.
> 
> Supposedly there are 5-6 million developers world-wide (by various
> estimates; I've no idea whether to believe them).  If you just multiply
> out naively you get 700K-840K Python programmers.  There certainly are
> vastly more one person Python projects than large ones, so this may not be
> all that far off.
> 
> > 2) 64% of companies use Java, 4.5
> > times more than Python.  Find out how many Java programmers there are,
> > divide by 4.5.
> 
> I've seen claims of 3-4 million java programmers so that's 666K-888K
> Python programmers.  Interestingly, this matches up with the above abuse
> of statistics.

Thanks!  I didn't know the numbers from which to guestimate, and it's
nice that they are in agreement.  This seems like a satisfactory
back-of-the-envelope answer.

Jeremy
From greg at electricrain.com  Mon Dec 13 21:41:38 2004
From: greg at electricrain.com (Gregory P. Smith)
Date: Mon Dec 13 21:41:42 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
Message-ID: <20041213204138.GH23061@zot.electricrain.com>

On Wed, Dec 08, 2004 at 02:18:48PM -0800, Guido van Rossum wrote:
> I was pleasantly surprised to find a pointer to this article in a news
> digest that the ACM emails me regularly (ACM TechNews).
> 
> http://gcn.com/vol1_no1/daily-updates/28026-1.html
> 
> One thing that bugs me: the article says 3 or 4 times that Python is
> slow, each time with a refutation ("but it's so flexible", "but it's
> fast enough") but still, they sure seem to harp on the point. This is
> a PR issue that Python needs to fight -- any ideas?

All high level dynamic languages need to fight this.  Substitute
python for perl, php, ruby, tcl and the statements in that article
still apply.  They'll all have slower execution times (on avg) their
compiled or JITed bretheren.

I suggest reading that article in a different light:  The point of
view of the people who are likely to believe what they read in
government computer news is that it just confirmed that python is a
useful languange for getting the job done easily.  Not that one
shouldn't use it because "its slow."

--
All new Python 2.4, now with blue crystals!
From janssen at parc.com  Mon Dec 13 21:57:07 2004
From: janssen at parc.com (Bill Janssen)
Date: Mon Dec 13 21:57:39 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places 
In-Reply-To: Your message of "Sun, 12 Dec 2004 18:13:15 PST."
	<864d37090412121813379f6aeb@mail.gmail.com> 
Message-ID: <04Dec13.125717pst."58617"@synergy1.parc.xerox.com>

> That's right -
> when I talk to fellow programmers that I'm writing software in Python,
> many of them are amazed and ask me, "but isn't it slow?". I've heard
> it more than once...

I heard it last month.

In the last couple of months, an acquaintance of mine has been trying
out Python.  He likes the language, particularly the support for
character sets, but has (apparently) given up after benchmarking a
typical (for him) task against Perl.  The task is to use the "re"
package to heavily modify a file by a sequence of re substitutions.
Apparently the Python program, which applies the same re substitutions
in the same order as the Perl program, takes 3 times as long to run.
He thinks it's because of mutable strings in Perl -- that is, he
thinks the string being modified (which is long, a whole file full of
text) is modified in place in Perl, but has to be re-consed in Python.

Bill
From imbaczek at gmail.com  Mon Dec 13 22:03:36 2004
From: imbaczek at gmail.com (=?UTF-8?Q?Marek_=22Baczek=22_Baczy=C5=84ski?=)
Date: Mon Dec 13 22:03:40 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <7143231425898203248@unknownmsgid>
References: <864d37090412121813379f6aeb@mail.gmail.com>
	<7143231425898203248@unknownmsgid>
Message-ID: <5f3d2c31041213130317323b15@mail.gmail.com>

On Mon, 13 Dec 2004 12:57:07 PST, Bill Janssen <janssen@parc.com> wrote:
> Apparently the Python program, which applies the same re substitutions
> in the same order as the Perl program, takes 3 times as long to run.
> He thinks it's because of mutable strings in Perl -- that is, he
> thinks the string being modified (which is long, a whole file full of
> text) is modified in place in Perl, but has to be re-consed in Python.

Did he use re.compile?

-- 
{ Marek Baczy?ski :: UIN 57114871 :: GG 161671 :: JID imbaczek@jabber.gda.pl  }
{ http://www.vlo.ids.gda.pl/ | imbaczek at poczta fm | http://www.promode.org }
.. .. .. .. ... ... ...... evolve or face extinction ...... ... ... .. .. .. ..
From martin at v.loewis.de  Mon Dec 13 23:17:51 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Dec 13 23:17:53 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
In-Reply-To: <79990c6b041213015861230c01@mail.gmail.com>
References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl>
	<79990c6b041213015861230c01@mail.gmail.com>
Message-ID: <41BE150F.9050100@v.loewis.de>

Paul Moore wrote:
> My current technique for checking an extension is compatible with
> Python 2.4 is to run objdump -p (from the mingw distribution - use
> dumpbin /imports from MSVC) and review the import table. If any
> symbols are referenced from msvcrt.dll, you need to convince yourself
> that they are used solely by the mingw runtime startup code. 

I forgot the details of your analysis, but I think you are right.
However, I would feel more comfortable if only a single CRT was used
from an extension module.

As for creating a case that crashes if you mix CRTs: Just try
PyRun_SimpleFile in the extension, with a file that you fopen'ed
in the extension. Satisfy this fopen from, say, msvcrt.dll, and
load the extension into Python 2.4.

Regards,
Martin
From martin at v.loewis.de  Mon Dec 13 23:31:57 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Dec 13 23:31:58 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
In-Reply-To: <wtvma0qw.fsf@python.net>
References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl>	<79990c6b041213015861230c01@mail.gmail.com>
	<wtvma0qw.fsf@python.net>
Message-ID: <41BE185D.6050607@v.loewis.de>

Thomas Heller wrote:
> And recently I played with bindings to OpenGL's glut32.dll - glut calls
> exit() from internal C code.  If linked with the wrong CRT, this will do
> nothing instead of terminating the process.

Interesting. Looking at the code of exit(), it is clear that the wrong
atexit handlers will be called if you use the wrong CRT (i.e. only one
set of atexit handlers will be called, whereas you might have multiple
lists of atexit handlers); see atonexit.c.

It is not so clear why you could possibly get out of exit() without
actually exiting. The only possible code path I see in the msvcr71.dll
is when mscoree.dll is in the address space, in which case the CRT
will invoke CorExitProcess (!). In all other cases, it will invoke
ExitProcess, which, to my understanding, will terminate the process
unconditionally.

Regards,
Martin


From adam.jtm30 at gmail.com  Tue Dec 14 00:11:13 2004
From: adam.jtm30 at gmail.com (Adam Bark)
Date: Tue Dec 14 00:11:17 2004
Subject: [Python-Dev] Re: The Other Py2.4 issue?
In-Reply-To: <20041213110006.ED83E1E4032@bag.python.org>
References: <20041213110006.ED83E1E4032@bag.python.org>
Message-ID: <be4fbf92041213151155de169d@mail.gmail.com>

Even though there would be no significant speed-up I would still be
interested in a proper compiler just to make smaller exe's so I can
give out my programs. I used Py2EXE for my last program that was tiny
but with Tkinter and all the Tcl/Tk gubbins it was over 5Mb.

Also would it perhaps be possible to have some sort of psyco program
that can optimize the compiled code on the fly?
From rsenra at acm.org  Tue Dec 14 04:05:09 2004
From: rsenra at acm.org (Rodrigo Dias Arruda Senra)
Date: Tue Dec 14 04:06:10 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
Message-ID: <20041214010509.4f4ad3ec@Goku>

 [ Guido van Rossum <gvanrossum@gmail.com> ]
 -----------------------------------------------
 | One thing that bugs me: the article says 3 or 4 times that Python is
 | slow, each time with a refutation ("but it's so flexible", "but it's
 | fast enough") but still, they sure seem to harp on the point.

 That reminds me of this joke:

 """Couple of guys find themselves being chased by a lion. The first one
 starts to strecth out preparing to run, the second one finds that ridicule.
 - No way you'll run faster than that lion. While, the other replies:
 - I don't need to run faster than the lion, I just need to run faster than you.
 """

 I love to see Python getting faster and faster. Eventually (if...when PyPy succedes)
 faster-than-C ! But after over 4 years of making-ends-meet with Python, our Brazilian
 customers never complained of our Python solutions being slow. Perhaps because
 they were glad enough with projects delivered 5 to 10 times faster than usual.

 But, but to the joke: I believe Python must strive to run at least as fast as the crowd --
 Java, Perl, Ruby, Lua, Boo, etc

 Maybe we could visit the language shootout sites, translate Python snipets to modern 2.4
 idioms and rerun the test suites:

 http://www.bagley.org/~doug/shootout/ # classic
 http://shootout.alioth.debian.org/    # modern
 
 My 2 cents.

 best regards,
 Senra

-- 
   ,_           
   | )          Rodrigo Senra       <rsenra |at| acm.org>                      
   |(______     -----------------------------------------------
 _(    (|__|]   GPr Sistemas http://www.gpr.com.br                              
_ |    (|___|]  IC - Unicamp http://www.ic.unicamp.br/~921234  
___    (|__|]                       
   L___(|_|]    -----------------------------------------------
From zhangyue99 at tsinghua.org.cn  Tue Dec 14 04:09:58 2004
From: zhangyue99 at tsinghua.org.cn (=?UTF-8?B?WmhhbmdZdWXlvKDlsrM=?=)
Date: Tue Dec 14 04:10:14 2004
Subject: [Python-Dev] httplib timeout patch
Message-ID: <41BE5986.9090904@tsinghua.org.cn>

Hi,

Here is a patch for httplib. I added a timeout value for 
httplib.HTTPConnection, please check.

(diff - CVS 1.94)

Zhang Yue
2004-12-14
-------------- next part --------------
Index: httplib.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/httplib.py,v
retrieving revision 1.94
diff -r1.94 httplib.py
575c575
<     def __init__(self, host, port=None, strict=None):
---
>     def __init__(self, host, port=None, strict=None, timeout=None):
580a581
>         self.timeout = timeout
613a615,616
>                 if self.timeout:
>                     self.sock.settimeout(self.timeout)

From bac at OCF.Berkeley.EDU  Tue Dec 14 08:38:25 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Dec 14 08:38:36 2004
Subject: [Python-Dev] httplib timeout patch
In-Reply-To: <41BE5986.9090904@tsinghua.org.cn>
References: <41BE5986.9090904@tsinghua.org.cn>
Message-ID: <41BE9871.5080408@ocf.berkeley.edu>

ZhangYue?? wrote:
> Hi,
> 
> Here is a patch for httplib. I added a timeout value for 
> httplib.HTTPConnection, please check.
> 

Thanks for the patch, but this is the wrong place for it.  Please create a new 
patch item on SourceForge at http://sourceforge.net/patch/?group_id=5470 .

-Brett
From martin at v.loewis.de  Tue Dec 14 09:19:31 2004
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue Dec 14 09:19:32 2004
Subject: [Python-Dev] httplib timeout patch
In-Reply-To: <41BE9871.5080408@ocf.berkeley.edu>
References: <41BE5986.9090904@tsinghua.org.cn>
	<41BE9871.5080408@ocf.berkeley.edu>
Message-ID: <41BEA213.7070901@v.loewis.de>

Brett C. wrote:
>> Here is a patch for httplib. I added a timeout value for 
>> httplib.HTTPConnection, please check.
>>
> 
> Thanks for the patch, but this is the wrong place for it.  Please create 
> a new patch item on SourceForge at 
> http://sourceforge.net/patch/?group_id=5470 .

In addition, also avoid using plain diffs - use context or unified diffs
instead.

Regards,
Martin
From p.f.moore at gmail.com  Tue Dec 14 11:01:34 2004
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue Dec 14 11:01:37 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
In-Reply-To: <41BE150F.9050100@v.loewis.de>
References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl>
	<79990c6b041213015861230c01@mail.gmail.com>
	<41BE150F.9050100@v.loewis.de>
Message-ID: <79990c6b04121402011d8965bd@mail.gmail.com>

On Mon, 13 Dec 2004 23:17:51 +0100, Martin v. L?wis <martin@v.loewis.de> wrote:
> I forgot the details of your analysis, but I think you are right.
> However, I would feel more comfortable if only a single CRT was used
> from an extension module.

Agreed. But to some extent I'm equally uncomfortable with the idea
that the mingw developers added support for -lmsvcr71 and botched it
badly enough for things like this not to work :-)

> As for creating a case that crashes if you mix CRTs: Just try
> PyRun_SimpleFile in the extension, with a file that you fopen'ed
> in the extension. Satisfy this fopen from, say, msvcrt.dll, and
> load the extension into Python 2.4.

Sorry, I wasn't specific enough. Certainly, if you create a FILE* from
one CRT, then pass it to another CRT, this will fail. But my point is,
if your extension uses fopen() then the link command generated by
distutils will ensure that the symbol is satisfied from msvcr71.dll.
So your scenario cannot occur.

The only symbols which can be satisfied from msvcrt in a
mingw-compiled extension (assuming that distutils is used, or the
-lmsvcr71 flag is otherwise added) are those referenced in the mingw
startup code, but not in user code. As the startup code will never
reference Python APIs, CRT data should never be shared between
incompatible CRTs.

Your point about atexit (from another email) may be an issue. However,
again we have the fact that user C code is *always* satisfied from
msvcr71, so any exit handlers added in C or Python will be registered
with the msvcr71 CRT, and so, as the main python.exe uses msvcr71,
will be executed. I don't see any signs that the mingw runtime for a
DLL uses atexit (this by experiment, I haven't time at the moment to
review the mingw sources) and so I don't believe that the startup code
has issues here, either.

Thanks for your comments. Your support for the viability of building
extensions using mingw is important to me, so if you still have any
concerns, let me know and I will do my best to address them.

Paul.
From p.f.moore at gmail.com  Tue Dec 14 11:13:15 2004
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue Dec 14 11:13:17 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <20041214010509.4f4ad3ec@Goku>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<20041214010509.4f4ad3ec@Goku>
Message-ID: <79990c6b04121402137a577a5b@mail.gmail.com>

On Tue, 14 Dec 2004 01:05:09 -0200, Rodrigo Dias Arruda Senra
<rsenra@acm.org> wrote:
> But, but to the joke: I believe Python must strive to run at least as fast as the crowd --
> Java, Perl, Ruby, Lua, Boo, etc
> 
> Maybe we could visit the language shootout sites, translate Python snipets to modern
> 2.4 idioms and rerun the test suites:

Bill Janssen <janssen@parc.com> wrote:
> In the last couple of months, an acquaintance of mine has been trying
> out Python.  He likes the language, particularly the support for
> character sets, but has (apparently) given up after benchmarking a
> typical (for him) task against Perl.  The task is to use the "re"
> package to heavily modify a file by a sequence of re substitutions.

That might be a good idiom to review/translate/test. It's not a
particularly natural thing to do in Python (REs are used much less in
Python), but it's *very* natural in Perl, and a naive Perl->Python
translation will make Python look slower. So a case study showing how
you might "really" do a task like this in Python would be a useful
resource.

Also, it might point to somewhere where there are opportunities for
optimisation within Python (for some reason this reminds me of the
mutable byte buffer discussions from a while back).

Paul
From amk at amk.ca  Tue Dec 14 15:31:01 2004
From: amk at amk.ca (A.M. Kuchling)
Date: Tue Dec 14 15:31:42 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <864d3709041212093274becde2@mail.gmail.com>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<cpeh62$ldh$1@sea.gmane.org>
	<ca471dc204121108225511d43@mail.gmail.com>
	<cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com>
	<864d370904121209147b713f31@mail.gmail.com>
	<864d3709041212093274becde2@mail.gmail.com>
Message-ID: <20041214143101.GA1404@rogue.amk.ca>

On Sun, Dec 12, 2004 at 03:32:03PM -0200, Carlos Ribeiro wrote:
> Of course, the point here is not Perl-bashing. The point here is that
> we should be able to "sell" Python better than we do now, even without
> the need to resort to such poor measures. I'm sure the Python
> community does have good & creative people that can write a good
> "selling" FAQ for Python, emphasizing the main points of the language.

No one disagrees that Python needs better marketing material.  At the
last PyCon a group of people sat down in a pydotorg BoF and agreed
that yes, we do need a management-friendly marketing site, and that we
could put it on a separate hostname (something.python.org) so that the
current www.python.org wouldn't have to be changed.

However, no one has actually sat down and written such a site, or even
outlined it.  Let me encourage you to go ahead and do that.  You could
draft the outline on a Wiki page, and then later figure out an
attractive design and organization for a new site.

--amk

From martin at v.loewis.de  Tue Dec 14 23:48:41 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Dec 14 23:48:41 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
In-Reply-To: <79990c6b04121402011d8965bd@mail.gmail.com>
References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl>	
	<79990c6b041213015861230c01@mail.gmail.com>	
	<41BE150F.9050100@v.loewis.de>
	<79990c6b04121402011d8965bd@mail.gmail.com>
Message-ID: <41BF6DC9.8030500@v.loewis.de>

Paul Moore wrote:
> Thanks for your comments. Your support for the viability of building
> extensions using mingw is important to me, so if you still have any
> concerns, let me know and I will do my best to address them.

I understand that one still needs to build libpython24.a in order to
use this process. As I have said, I'd happily ship that file with the
2.4.1 MSI, unless the release manager tells me that this would an
unacceptable new feature, and as long as somebody provides a fully
automatic build process integrated into msi.py; for that build process,
it is ok to assume that a cygwin installation is in c:\cygwin.

So if this would be useful (which I don't know for sure), I still need
a volunteer to contribute the appropriate magic.

Regards,
Martin
From tdelaney at avaya.com  Wed Dec 15 00:50:23 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Wed Dec 15 00:50:28 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE721221@au3010avexu1.global.avaya.com>

"Martin v. L?wis" wrote:

> Paul Moore wrote:
>> Thanks for your comments. Your support for the viability of building
>> extensions using mingw is important to me, so if you still have any
>> concerns, let me know and I will do my best to address them.
> 
> I understand that one still needs to build libpython24.a in order to
> use this process. As I have said, I'd happily ship that file with the
> 2.4.1 MSI, unless the release manager tells me that this would an
> unacceptable new feature, and as long as somebody provides a fully
> automatic build process integrated into msi.py; for that build
> process, 
> it is ok to assume that a cygwin installation is in c:\cygwin.

I think we should aim to support MSYS as well as Cygwin, but perhaps not for the first version where this goes in.

OTOH, is it really necessary to have either MSYS or Cygwin? MinGW32 works standalone - distutils should be able to just invoke it.

Tim Delaney
From martin at v.loewis.de  Wed Dec 15 01:08:03 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Dec 15 01:08:01 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE721221@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DE721221@au3010avexu1.global.avaya.com>
Message-ID: <41BF8063.4030602@v.loewis.de>

Delaney, Timothy C (Timothy) wrote:
>> it is ok to assume that a cygwin installation is in c:\cygwin.
> 
> 
> I think we should aim to support MSYS as well as Cygwin, but perhaps
> not for the first version where this goes in.

Not at all. I'm talking about the release process, and prerequisites
required in that process. This is worth mentioning because the list
of prerequisites you need to perform a Python release is already quite
long:
- CVS
- putty (for CVS)
- VC7.1
- current, built versions of all libraries (zlib, bzip2, tcl/tk,
   bsddb, ...)
- Perl (needed to build OpenSSL)
- HTML help workshop
- (the platform SDK)(to build Itanium binaries, if desired)
Now, I would be willing to add Cygwin on top of that - whether the
build process also works with MSYS does not matter to me; I would
not like a build process that *required* MSYS as a matter of
preference.

> OTOH, is it really necessary to have either MSYS or Cygwin? MinGW32
> works standalone - distutils should be able to just invoke it.

I'm not talking about distutils, here.

Regards,
Martin
From tdelaney at avaya.com  Wed Dec 15 01:50:01 2004
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Wed Dec 15 01:50:13 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE721222@au3010avexu1.global.avaya.com>

"Martin v. L?wis" wrote:

> Not at all. I'm talking about the release process, and prerequisites
> required in that process. This is worth mentioning because the list
> of prerequisites you need to perform a Python release is already quite
> long:

Ah - sorry - misinterpreted.

What is the size of the generated libpython24.a? Unfortunately, I don't happen to have binutils installed in my work cygwin (pulling it down now, but it's very slow ...) so I can't check myself (I've built the 2.3 one at home before ...).

Tim Delaney
From greg at electricrain.com  Wed Dec 15 07:21:58 2004
From: greg at electricrain.com (Gregory P. Smith)
Date: Wed Dec 15 07:22:02 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <20041214143101.GA1404@rogue.amk.ca>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<cpeh62$ldh$1@sea.gmane.org>
	<ca471dc204121108225511d43@mail.gmail.com>
	<cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com>
	<864d370904121209147b713f31@mail.gmail.com>
	<864d3709041212093274becde2@mail.gmail.com>
	<20041214143101.GA1404@rogue.amk.ca>
Message-ID: <20041215062158.GA12070@zot.electricrain.com>

> > we should be able to "sell" Python better than we do now, even without
> > the need to resort to such poor measures. I'm sure the Python
> > community does have good & creative people that can write a good
> > "selling" FAQ for Python, emphasizing the main points of the language.
> 
> No one disagrees that Python needs better marketing material.  At the
> last PyCon a group of people sat down in a pydotorg BoF and agreed
> that yes, we do need a management-friendly marketing site, and that we
> could put it on a separate hostname (something.python.org) so that the
> current www.python.org wouldn't have to be changed.
> 
> However, no one has actually sat down and written such a site, or even
> outlined it.  Let me encourage you to go ahead and do that.  You could
> draft the outline on a Wiki page, and then later figure out an
> attractive design and organization for a new site.

suggested hostname: why.python.org


From p.f.moore at gmail.com  Wed Dec 15 10:49:18 2004
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed Dec 15 10:49:20 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
In-Reply-To: <41BF6DC9.8030500@v.loewis.de>
References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl>
	<79990c6b041213015861230c01@mail.gmail.com>
	<41BE150F.9050100@v.loewis.de>
	<79990c6b04121402011d8965bd@mail.gmail.com>
	<41BF6DC9.8030500@v.loewis.de>
Message-ID: <79990c6b041215014944f31097@mail.gmail.com>

On Tue, 14 Dec 2004 23:48:41 +0100, Martin v. L?wis <martin@v.loewis.de> wrote:
> I understand that one still needs to build libpython24.a in order to
> use this process. As I have said, I'd happily ship that file with the
> 2.4.1 MSI, unless the release manager tells me that this would an
> unacceptable new feature, and as long as somebody provides a fully
> automatic build process integrated into msi.py; for that build process,
> it is ok to assume that a cygwin installation is in c:\cygwin.
> 
> So if this would be useful (which I don't know for sure), I still need
> a volunteer to contribute the appropriate magic.

I am willing to work on this, if you wouldn't mind me asking dumb
questions about the build process :-) However, I don't have VS.NET, so
if automation requires integration into the VS project stuff, I can't
help.

For a starter, what steps do you actually take to build a release? I
assume that the first step is to build Python, by clicking on "build"
in VS.NET. Once you have that, is there a single script you run? If
there's a document describing the release process somewhere, feel free
to point me at it.

Thanks,
Paul.
From phd at mail2.phd.pp.ru  Wed Dec 15 11:02:41 2004
From: phd at mail2.phd.pp.ru (Oleg Broytmann)
Date: Wed Dec 15 11:02:44 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <20041215062158.GA12070@zot.electricrain.com>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<cpeh62$ldh$1@sea.gmane.org>
	<ca471dc204121108225511d43@mail.gmail.com>
	<cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com>
	<864d370904121209147b713f31@mail.gmail.com>
	<864d3709041212093274becde2@mail.gmail.com>
	<20041214143101.GA1404@rogue.amk.ca>
	<20041215062158.GA12070@zot.electricrain.com>
Message-ID: <20041215100241.GA8106@phd.pp.ru>

On Tue, Dec 14, 2004 at 10:21:58PM -0800, Gregory P. Smith wrote:
> suggested hostname: why.python.org

   It's only a matter of taste, probably, but that looks a bit ugly for
my eyes. May be use.python.org? corp.python.org?

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd@phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.
From aoshi at OCF.Berkeley.EDU  Wed Dec 15 12:35:01 2004
From: aoshi at OCF.Berkeley.EDU (Randy Chung)
Date: Wed Dec 15 12:35:08 2004
Subject: [Python-Dev] Python in education
Message-ID: <41C02165.2010000@ocf.berkeley.edu>

Hi everyone,

I'm going to be leading a class on Python at the University of 
California, Berkeley next semester (starting in January).  I'm 
interested in using actual bugs in Python as exercises for the class, 
the goal being 1) to give the students something "real" to work with, 
and 2) to (hopefully) contribute back anything we manage to put together 
to the Python dev team.

What I'm looking for are some oustanding bugs which the more experienced 
devs feel would be straightforward to implement (which might not have 
been implemented yet because they're not very interesting, or there are 
simply too many other things to do right now).  I'd appreciate any and 
all suggestions any of you might be able to provide.

I think that Python will have a better chance at wider adoption if 
universities would provide courses for the language.  I'm running this 
class partly because I'd like to give back to the community which 
produced a language I've enjoyed using, and any assistance would be 
greatly appreciated.

If you feel this is off topic, please feel free to reply to me off-list.

Thanks,

Randy Chung
From ncoghlan at iinet.net.au  Wed Dec 15 13:32:39 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed Dec 15 13:32:42 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <20041215100241.GA8106@phd.pp.ru>
References: <ca471dc204120814186f5dc423@mail.gmail.com>	<cpeh62$ldh$1@sea.gmane.org>	<ca471dc204121108225511d43@mail.gmail.com>	<cpfcqh$ejc$1@sea.gmane.org>
	<41BC6D92.5050802@heneryd.com>	<864d370904121209147b713f31@mail.gmail.com>	<864d3709041212093274becde2@mail.gmail.com>	<20041214143101.GA1404@rogue.amk.ca>	<20041215062158.GA12070@zot.electricrain.com>
	<20041215100241.GA8106@phd.pp.ru>
Message-ID: <41C02EE7.3090308@iinet.net.au>

Oleg Broytmann wrote:
> On Tue, Dec 14, 2004 at 10:21:58PM -0800, Gregory P. Smith wrote:
> 
>>suggested hostname: why.python.org
> 
> 
>    It's only a matter of taste, probably, but that looks a bit ugly for
> my eyes. May be use.python.org? corp.python.org?

about.python.org?

And if someone ends up playing with the DNS server, maybe they could add 
wiki.python.org while they're at it :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan@email.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.skystorm.net
From barry at python.org  Wed Dec 15 14:12:12 2004
From: barry at python.org (Barry Warsaw)
Date: Wed Dec 15 14:12:17 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <41C02EE7.3090308@iinet.net.au>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<cpeh62$ldh$1@sea.gmane.org>	<ca471dc204121108225511d43@mail.gmail.com>
	<cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com>
	<864d370904121209147b713f31@mail.gmail.com>
	<864d3709041212093274becde2@mail.gmail.com>
	<20041214143101.GA1404@rogue.amk.ca>
	<20041215062158.GA12070@zot.electricrain.com>
	<20041215100241.GA8106@phd.pp.ru>  <41C02EE7.3090308@iinet.net.au>
Message-ID: <1103116332.9165.69.camel@geddy.wooz.org>

On Wed, 2004-12-15 at 07:32, Nick Coghlan wrote:

> about.python.org?
> 
> And if someone ends up playing with the DNS server, maybe they could add 
> wiki.python.org while they're at it :)

DNS changes have to go through pydotorg at python.org, since Thomas is
the person currently able to add host names.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20041215/2ef3889d/attachment.pgp
From aahz at pythoncraft.com  Wed Dec 15 16:28:24 2004
From: aahz at pythoncraft.com (Aahz)
Date: Wed Dec 15 16:28:33 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <20041215062158.GA12070@zot.electricrain.com>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<cpeh62$ldh$1@sea.gmane.org>
	<ca471dc204121108225511d43@mail.gmail.com>
	<cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com>
	<864d370904121209147b713f31@mail.gmail.com>
	<864d3709041212093274becde2@mail.gmail.com>
	<20041214143101.GA1404@rogue.amk.ca>
	<20041215062158.GA12070@zot.electricrain.com>
Message-ID: <20041215152824.GA24746@panix.com>

On Tue, Dec 14, 2004, Gregory P. Smith wrote:
>Attribution deleted:
>> 
>> No one disagrees that Python needs better marketing material.  At the
>> last PyCon a group of people sat down in a pydotorg BoF and agreed
>> that yes, we do need a management-friendly marketing site, and that we
>> could put it on a separate hostname (something.python.org) so that the
>> current www.python.org wouldn't have to be changed.
>> 
>> However, no one has actually sat down and written such a site, or even
>> outlined it.  Let me encourage you to go ahead and do that.  You could
>> draft the outline on a Wiki page, and then later figure out an
>> attractive design and organization for a new site.
> 
> suggested hostname: why.python.org

This is where the process always gets bogged down.  :-(  Once we have
material, that's the time to start arguing about where it should go.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing."  --Alan Perlis
From tjreedy at udel.edu  Wed Dec 15 20:58:59 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed Dec 15 20:58:55 2004
Subject: [Python-Dev] Re: Python in education
References: <41C02165.2010000@ocf.berkeley.edu>
Message-ID: <cpq51j$bq8$1@sea.gmane.org>


"Randy Chung" <aoshi@OCF.Berkeley.EDU> wrote in message 
news:41C02165.2010000@ocf.berkeley.edu...
> Hi everyone,
>
> I'm going to be leading a class on Python at the University of 
> California, Berkeley next semester (starting in January).

Great.

>  I'm  interested in using actual bugs in Python as exercises

Please consider including review of existing patches.  Besides being 
useful, it will also teach students how to submit good patches of their 
own.

Your primary choice is whether to work on changes in the C code for the 
interpreter and builtin modules or changes in the Python code in standard 
library modules.  The former would be more appropriate for a class on 
application programming in C, so I suggest the latter for learning Python 
programming.  Besides which, improving the library will probably be a focus 
of the just-starting 2.5 cycle.

You can access existing patch and bug lists via 
http://sourceforge.net/tracker/?group_id=5470

For each, you can narrow list by selecting, for instance, Category 'Python 
Library'.

Terry J. Reedy



From tjreedy at udel.edu  Wed Dec 15 21:21:38 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed Dec 15 21:21:48 2004
Subject: [Python-Dev] Re: Re: Re: 2.4 news reaches interesting places
References: <ca471dc204120814186f5dc423@mail.gmail.com>	<cpeh62$ldh$1@sea.gmane.org>	<ca471dc204121108225511d43@mail.gmail.com>	<cpfcqh$ejc$1@sea.gmane.org><41BC6D92.5050802@heneryd.com>	<864d370904121209147b713f31@mail.gmail.com>	<864d3709041212093274becde2@mail.gmail.com>	<20041214143101.GA1404@rogue.amk.ca>	<20041215062158.GA12070@zot.electricrain.com><20041215100241.GA8106@phd.pp.ru>
	<41C02EE7.3090308@iinet.net.au>
Message-ID: <cpq6c2$gjr$1@sea.gmane.org>

For a subsite aimed at businesses, business.python.org is obvious and 
easily remembered.  Not all businesses are corporations.  'about' and 'why' 
are not specific at all.

I think such a subsite, linked from the main site also, would be a good 
idea.  It should explain both why (including success stories) and how 
(including employment/contracting) to use Python in business.  Pieces could 
come from the existing site.

I think the mere existence of such a site would constitute a message to 
businesses that Python is ready for business.  I suspect the existence of 
even a skeletal site would attract contributions to build it further.

Correspondingly, perhaps there should also be a
science(or scitech or learning).python.org with pages on Astronomy, 
Bioscience, Chemistry, Mathematics, Education, etc.  Perhaps the scipy and 
other science application groups would help.

With two subsites for the main application areas, this would leave 
www.python.org to focus on the language itself.

Terry J. Reedy



From allison at sumeru.stanford.EDU  Wed Dec 15 22:04:11 2004
From: allison at sumeru.stanford.EDU (Dennis Allison)
Date: Wed Dec 15 22:04:17 2004
Subject: [Python-Dev] Re: Re: Re: 2.4 news reaches interesting places
In-Reply-To: <cpq6c2$gjr$1@sea.gmane.org>
Message-ID: <Pine.LNX.4.10.10412151302350.19316-100000@sumeru.stanford.EDU>

Ummm... I don't think that anyone looks for information by trolling
subdomain names.  If I am looking for python information, I go to 
www.python.org or python.org.  I would never guess "business.python.org".

Seems to me that what we need is content and let the search engines bring
on the masses.

On Wed, 15 Dec 2004, Terry Reedy wrote:

> For a subsite aimed at businesses, business.python.org is obvious and 
> easily remembered.  Not all businesses are corporations.  'about' and 'why' 
> are not specific at all.
> 
> I think such a subsite, linked from the main site also, would be a good 
> idea.  It should explain both why (including success stories) and how 
> (including employment/contracting) to use Python in business.  Pieces could 
> come from the existing site.
> 
> I think the mere existence of such a site would constitute a message to 
> businesses that Python is ready for business.  I suspect the existence of 
> even a skeletal site would attract contributions to build it further.
> 
> Correspondingly, perhaps there should also be a
> science(or scitech or learning).python.org with pages on Astronomy, 
> Bioscience, Chemistry, Mathematics, Education, etc.  Perhaps the scipy and 
> other science application groups would help.
> 
> With two subsites for the main application areas, this would leave 
> www.python.org to focus on the language itself.
> 
> Terry J. Reedy
> 
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/allison%40sumeru.stanford.edu
> 

From martin at v.loewis.de  Wed Dec 15 22:57:00 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Dec 15 22:56:57 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
In-Reply-To: <79990c6b041215014944f31097@mail.gmail.com>
References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl>	
	<79990c6b041213015861230c01@mail.gmail.com>	
	<41BE150F.9050100@v.loewis.de>	
	<79990c6b04121402011d8965bd@mail.gmail.com>	
	<41BF6DC9.8030500@v.loewis.de>
	<79990c6b041215014944f31097@mail.gmail.com>
Message-ID: <41C0B32C.7040701@v.loewis.de>

Paul Moore wrote:
> For a starter, what steps do you actually take to build a release? I
> assume that the first step is to build Python, by clicking on "build"
> in VS.NET. 

Yes. You can skip this step by just putting all the .pyds, dlls, and
.exes into the PCbuild directory. The packaging will try to pick them
up from there (and proceed if some are missing, like Tcl likely will).

 > Once you have that, is there a single script you run?

Yes. Invoke Tools\msi\msi.py, using a Python that has pythonwin (i.e.
COM interopability). The only tricky part is that you need cabarc.exe,
which is included in VC, and in the platform SDK.

> If
> there's a document describing the release process somewhere, feel free
> to point me at it.

Tools/msi/README.txt

Regards,
Martin
From bac at OCF.Berkeley.EDU  Thu Dec 16 02:18:22 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Dec 16 02:18:28 2004
Subject: [Python-Dev] Re: Python in education
In-Reply-To: <cpq51j$bq8$1@sea.gmane.org>
References: <41C02165.2010000@ocf.berkeley.edu> <cpq51j$bq8$1@sea.gmane.org>
Message-ID: <41C0E25E.1040606@ocf.berkeley.edu>

Terry Reedy wrote:
> "Randy Chung" <aoshi@OCF.Berkeley.EDU> wrote in message 
> news:41C02165.2010000@ocf.berkeley.edu...
[SNIP]
>> I'm  interested in using actual bugs in Python as exercises
> 
> 
> Please consider including review of existing patches.  Besides being 
> useful, it will also teach students how to submit good patches of their 
> own.
> 

To go along with this suggestion, see http://www.python.org/dev/dev_intro.html 
for the basic overview of what a review tends to consist of.

-Brett
From kdart at kdart.com  Thu Dec 16 02:37:44 2004
From: kdart at kdart.com (Keith Dart)
Date: Thu Dec 16 02:37:58 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <20041214143101.GA1404@rogue.amk.ca>
References: <ca471dc204120814186f5dc423@mail.gmail.com>	<cpeh62$ldh$1@sea.gmane.org>	<ca471dc204121108225511d43@mail.gmail.com>	<cpfcqh$ejc$1@sea.gmane.org>
	<41BC6D92.5050802@heneryd.com>	<864d370904121209147b713f31@mail.gmail.com>	<864d3709041212093274becde2@mail.gmail.com>
	<20041214143101.GA1404@rogue.amk.ca>
Message-ID: <41C0E6E8.2000805@kdart.com>

A.M. Kuchling wrote:
> On Sun, Dec 12, 2004 at 03:32:03PM -0200, Carlos Ribeiro wrote:
> 
>>Of course, the point here is not Perl-bashing. The point here is that
>>we should be able to "sell" Python better than we do now, even without
>>the need to resort to such poor measures. I'm sure the Python
>>community does have good & creative people that can write a good
>>"selling" FAQ for Python, emphasizing the main points of the language.
> 
> 
> No one disagrees that Python needs better marketing material.  At the
> last PyCon a group of people sat down in a pydotorg BoF and agreed
> that yes, we do need a management-friendly marketing site, and that we
> could put it on a separate hostname (something.python.org) so that the
> current www.python.org wouldn't have to be changed.
> 
> However, no one has actually sat down and written such a site, or even
> outlined it.  Let me encourage you to go ahead and do that.  You could
> draft the outline on a Wiki page, and then later figure out an
> attractive design and organization for a new site.


Whatever it looks like, it should probably run on Zope plus Plone. 8-) 
You know... eat your own dog food. 8-)

The kind folks over at Zettai! have provided some space for me. Perhaps 
they will be glad to host the main Python site, as well?



-- 
                            \/ \/
                            (O O)
-- --------------------oOOo~(_)~oOOo----------------------------------------
Keith Dart <kdart@kdart.com>
vcard: <http://www.kdart.com/~kdart/kdart.vcf>
public key: ID: F3D288E4       URL: <http://www.kdart.com/~kdart/public.key>
============================================================================
From sdeibel at wingware.com  Thu Dec 16 02:57:29 2004
From: sdeibel at wingware.com (Stephan Deibel)
Date: Thu Dec 16 02:56:18 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <20041215152824.GA24746@panix.com>
Message-ID: <Pine.LNX.4.50.0412152052030.3211-100000@hedgehog>

On Wed, 15 Dec 2004, Aahz wrote:
> On Tue, Dec 14, 2004, Gregory P. Smith wrote:
> >Attribution deleted:
> >> 
> >> No one disagrees that Python needs better marketing material.  At the
> >> last PyCon a group of people sat down in a pydotorg BoF and agreed
> >> that yes, we do need a management-friendly marketing site, and that we
> >> could put it on a separate hostname (something.python.org) so that the
> >> current www.python.org wouldn't have to be changed.
> >> 
> >> However, no one has actually sat down and written such a site, or even
> >> outlined it.  Let me encourage you to go ahead and do that.  You could
> >> draft the outline on a Wiki page, and then later figure out an
> >> attractive design and organization for a new site.
> > 
> > suggested hostname: why.python.org
> 
> This is where the process always gets bogged down.  :-(  Once we have
> material, that's the time to start arguing about where it should go.

Absolutely.  Content first, details later.

Incidentally, I keep offering:  Anyone that actually takes this on should
feel free to rip off content from http://wingware.com/python -- I'm not
saying this is the best content or the most complete but it's a start
anyway (I wrote it in case that wasn't clear ;-)

Incidentally, if someone does get excited about working on this, check
with me or another PSF board member.

- Stephan
From johan at carlstedt.net  Thu Dec 16 13:30:05 2004
From: johan at carlstedt.net (Johan Carlstedt)
Date: Thu Dec 16 13:30:11 2004
Subject: [Python-Dev] Problems compiling Python 2.3.3 on Solaris 10 with gcc
	3.4.1 
Message-ID: <29507.147.114.226.181.1103200205.squirrel@www.flirble.org>

Hello,

I am having problems compiling Python 2.3.3 on Solaris 10 X86 using gcc
3.4.1. The error message below is generated.

I would be grateful for any advice.

With friendly regards,

Johan Carlstedt

# make
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Modules/python.o Modules/python.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Parser/acceler.o Parser/acceler.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Parser/grammar1.o Parser/grammar1.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Parser/listnode.o Parser/listnode.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Parser/node.o Parser/node.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Parser/parser.o Parser/parser.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Parser/parsetok.o Parser/parsetok.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Parser/bitset.o Parser/bitset.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Parser/metagrammar.o Parser/metagrammar.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Parser/firstsets.o Parser/firstsets.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Parser/grammar.o Parser/grammar.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Parser/pgen.o Parser/pgen.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Parser/myreadline.o Parser/myreadline.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Parser/tokenizer.o Parser/tokenizer.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Objects/abstract.o Objects/abstract.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Objects/boolobject.o Objects/boolobject.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Objects/bufferobject.o Objects/bufferobject.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Objects/cellobject.o Objects/cellobject.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Objects/classobject.o Objects/classobject.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Objects/cobject.o Objects/cobject.c
gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I./Include  -DPy_BUILD_CORE -o Objects/complexobject.o
Objects/complexobject.c
Objects/complexobject.c: In function `complex_pow':
Objects/complexobject.c:469: error: invalid operands to binary ==
Objects/complexobject.c:469: error: wrong type argument to unary minus
Objects/complexobject.c:469: error: invalid operands to binary ==
Objects/complexobject.c:469: error: wrong type argument to unary minus
*** Error code 1
make: Fatal error: Command failed for target `Objects/complexobject.o'
# gcc --version
gcc (GCC) 3.4.1
Copyright (C) 2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
# uname -a
SunOS sunbuild1 5.10 s10_71 i86pc i386 i86pc






From eirik.mikkelsen at unix.net  Thu Dec 16 14:03:55 2004
From: eirik.mikkelsen at unix.net (Eirik Mikkelsen)
Date: Thu Dec 16 14:02:32 2004
Subject: [Python-Dev] Problems compiling Python 2.3.3 on Solaris 10	with
	gcc 3.4.1
In-Reply-To: <29507.147.114.226.181.1103200205.squirrel@www.flirble.org>
References: <29507.147.114.226.181.1103200205.squirrel@www.flirble.org>
Message-ID: <1103202235.10436.9.camel@localhost>

On Thu, 2004-12-16 at 12:30 +0000, Johan Carlstedt wrote:
> Hello,
> 
> I am having problems compiling Python 2.3.3 on Solaris 10 X86 using gcc
> 3.4.1. The error message below is generated.
> 
[SNIP]
> Objects/complexobject.c
> Objects/complexobject.c: In function `complex_pow':
> Objects/complexobject.c:469: error: invalid operands to binary ==
> Objects/complexobject.c:469: error: wrong type argument to unary minus
> Objects/complexobject.c:469: error: invalid operands to binary ==
> Objects/complexobject.c:469: error: wrong type argument to unary minus

Solaris 10 doesn't correctly define HUGE_VAL (I think it's called HUGE
in their header files) - so you may want to change the #define in
Include/pyport.h from 

#define Py_HUGE_VAL HUGE_VAL 

to

#define Py_HUGE_VAL HUGE

-- 
Eirik Mikkelsen <eirik.mikkelsen@unix.net>

From gvanrossum at gmail.com  Thu Dec 16 17:15:43 2004
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu Dec 16 17:15:46 2004
Subject: [Python-Dev] Problems compiling Python 2.3.3 on Solaris 10 with
	gcc 3.4.1
In-Reply-To: <1103202235.10436.9.camel@localhost>
References: <29507.147.114.226.181.1103200205.squirrel@www.flirble.org>
	<1103202235.10436.9.camel@localhost>
Message-ID: <ca471dc204121608157f76ef65@mail.gmail.com>

If this still occurs with Python 2.3.4 or 2.4, please file a bug so it
can be fixed (python-dev isn't a good place to remember fixes).


On Thu, 16 Dec 2004 14:03:55 +0100, Eirik Mikkelsen
<eirik.mikkelsen@unix.net> wrote:
> On Thu, 2004-12-16 at 12:30 +0000, Johan Carlstedt wrote:
> > Hello,
> >
> > I am having problems compiling Python 2.3.3 on Solaris 10 X86 using gcc
> > 3.4.1. The error message below is generated.
> > 
> [SNIP]
> > Objects/complexobject.c
> > Objects/complexobject.c: In function `complex_pow':
> > Objects/complexobject.c:469: error: invalid operands to binary ==
> > Objects/complexobject.c:469: error: wrong type argument to unary minus
> > Objects/complexobject.c:469: error: invalid operands to binary ==
> > Objects/complexobject.c:469: error: wrong type argument to unary minus
> 
> Solaris 10 doesn't correctly define HUGE_VAL (I think it's called HUGE
> in their header files) - so you may want to change the #define in
> Include/pyport.h from
> 
> #define Py_HUGE_VAL HUGE_VAL
> 
> to
> 
> #define Py_HUGE_VAL HUGE
> 
> --
> Eirik Mikkelsen <eirik.mikkelsen@unix.net>
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
> 


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
From bsder at mail.allcaps.org  Thu Dec 16 18:12:58 2004
From: bsder at mail.allcaps.org (Andrew P. Lentvorski, Jr.)
Date: Thu Dec 16 18:13:09 2004
Subject: [Python-Dev] Re: Python in education
In-Reply-To: <cpq51j$bq8$1@sea.gmane.org>
References: <41C02165.2010000@ocf.berkeley.edu> <cpq51j$bq8$1@sea.gmane.org>
Message-ID: <BBF201B5-4F85-11D9-B566-000A95C874EE@mail.allcaps.org>


On Dec 15, 2004, at 2:58 PM, Terry Reedy wrote:

> Your primary choice is whether to work on changes in the C code for the
> interpreter and builtin modules or changes in the Python code in 
> standard
> library modules.

The PyPy folks might be useful too given that they are writing Python 
in Python.

http://codespeak.net/pypy/

-a

From stewart.midwinter at gmail.com  Thu Dec 16 14:13:41 2004
From: stewart.midwinter at gmail.com (Stewart Midwinter)
Date: Thu Dec 16 23:57:18 2004
Subject: [Python-Dev] re: 2.4 news reaches interesting places
Message-ID: <d4c0d4800412160513d16f13e@mail.gmail.com>

A number of people commented on the article in GCN, at
http://gcn.com/vol1_no1/daily-updates/28026-1.html, and wondered if
more could not be done to address the perception of speed. The point
is made that although people mention all of the benefits of Python,
like ease of use, flexibility, they always come back to making
references to its speed.  And the question is raised, "what can we do
to counter this perception?". I think the answer lies in a quote by a
user in that same article: "At first, Doak was worried a Python-based
program would not run simulations quickly enough, however he found
performance to be acceptable.

Let's turn this around.  Forget about trying to create a perception
that Python is fast.  Compiled languages will always be faster, at
least for large applications. Or at least they'll be perceived that
way.  So let's acknowledge that upfront, but say "Python is fast
enough for most uses", but  then go on to say "and in addition to its
acceptable speed, it offers many advantages like ease-of-use,
flexibility, easy code maintenance (since the code is still
understandable 6 months later!) etc.

Marketers of other products have used this same technique
successfully.  For example, at one time there was a perception that
Kellogg's Corn Flakes were old and boring.  Sales were slipping. 
Rather than refute that, marketers turned the issue on its head by
emphasizing that the product had been around a long time because it
was good, and good for you.  Hence was born the slogan "taste them
again, for the first time".

Possible slogan for Python: "Fast enough, and better in many ways". 

cheers
S
http://midtoad.homelinux.org/wp/

-- 
Stewart Midwinter
stewart@midwinter.ca
stewart.midwinter@gmail.com
From michael.walter at gmail.com  Fri Dec 17 02:14:25 2004
From: michael.walter at gmail.com (Michael Walter)
Date: Fri Dec 17 02:14:27 2004
Subject: [Python-Dev] re: 2.4 news reaches interesting places
In-Reply-To: <d4c0d4800412160513d16f13e@mail.gmail.com>
References: <d4c0d4800412160513d16f13e@mail.gmail.com>
Message-ID: <877e9a1704121617142070df06@mail.gmail.com>

On Thu, 16 Dec 2004 06:13:41 -0700, Stewart Midwinter
<stewart.midwinter@gmail.com> wrote:
> [...] Possible slogan for Python: "Fast enough, and better in many ways". [...]
Let's make it faster first :-)

Cheers,
Michael
From carribeiro at gmail.com  Fri Dec 17 12:20:18 2004
From: carribeiro at gmail.com (Carlos Ribeiro)
Date: Fri Dec 17 12:20:21 2004
Subject: [Python-Dev] re: 2.4 news reaches interesting places
In-Reply-To: <d4c0d4800412160513d16f13e@mail.gmail.com>
References: <d4c0d4800412160513d16f13e@mail.gmail.com>
Message-ID: <864d37090412170320561ecb8@mail.gmail.com>

On Thu, 16 Dec 2004 06:13:41 -0700, Stewart Midwinter
<stewart.midwinter@gmail.com> wrote:
> A number of people commented on the article in GCN, at
> http://gcn.com/vol1_no1/daily-updates/28026-1.html, and wondered if
> more could not be done to address the perception of speed. The point
> is made that although people mention all of the benefits of Python,
> like ease of use, flexibility, they always come back to making
> references to its speed.  And the question is raised, "what can we do
> to counter this perception?". I think the answer lies in a quote by a
> user in that same article: "At first, Doak was worried a Python-based
> program would not run simulations quickly enough, however he found
> performance to be acceptable.
> 
> Let's turn this around.  Forget about trying to create a perception
> that Python is fast.  Compiled languages will always be faster, at
> least for large applications. Or at least they'll be perceived that
> way.  So let's acknowledge that upfront, but say "Python is fast
> enough for most uses", but  then go on to say "and in addition to its
> acceptable speed, it offers many advantages like ease-of-use,
> flexibility, easy code maintenance (since the code is still
> understandable 6 months later!) etc.
> 
> Marketers of other products have used this same technique
> successfully.  For example, at one time there was a perception that
> Kellogg's Corn Flakes were old and boring.  Sales were slipping.
> Rather than refute that, marketers turned the issue on its head by
> emphasizing that the product had been around a long time because it
> was good, and good for you.  Hence was born the slogan "taste them
> again, for the first time".
> 
> Possible slogan for Python: "Fast enough, and better in many ways".

One possible marketing strategy is to use the adjective "fast" in a
broader sense. The Python slogan could be something like: "Programming
has never been any faster" -- this changes the playing ground, from
raw performance to *programming* performance. And sure, nothing beats
Python (the overall package) in this respect. It can deliver fast code
in a short time. Othere languages are faster to run, but take longer
to code...

BTW, I would move away from the "fast enough" when talking about
performance. It's difficult to qualify what is "enough" in marketing
terms; also, a selling/winning message can't be seen as taking excuses
for any reason. On the other hand, Python never claims to be the
fastest language on raw execution performance, but only to be fast;
but in this sense, being "fast enough" is the same as being "fast".
So, I would never say, "Python allows you to write fast enough code in
a short time"; I would say "Python allows you to write fast code in a
short time". Leave the "fast enough" out of this, please.

-- 
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: carribeiro@gmail.com
mail: carribeiro@yahoo.com
From python at rcn.com  Fri Dec 17 13:47:19 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Dec 17 13:50:20 2004
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Modules
	posixmodule.c, 2.300.8.10, 2.300.8.11
In-Reply-To: <E1CdPE6-0003N9-VL@sc8-pr-cvs1.sourceforge.net>
Message-ID: <000401c4e436$8eabb0c0$e841fea9@oemcomputer>

The Dec 12th check-ins break tests on WinME:


test_glob.py
------------
Traceback (most recent call last):
  File "test_glob.py", line 78, in test_glob_one_directory
    eq(self.glob('a*'), map(self.norm, ['a', 'aab', 'aaa']))
  File "test_glob.py", line 67, in assertSequencesEqual_noorder
    self.assertEqual(set(l1), set(l2))
AssertionError: set(['/tmp\\@test_dir\\aaa', '/tmp\\@test_dir\\aab',
'/tmp\\@tes
t_dir\\a']) != set(['\\tmp\\@test_dir\\aab', '\\tmp\\@test_dir\\aaa',
'\\tmp\\@t
est_dir\\a'])


test_urllib.py
--------------
FAIL: test_basic (__main__.urlretrieve_FileTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_urllib.py", line 142, in test_basic
    self.assertEqual(result[0], test_support.TESTFN)
AssertionError: '\\tmp\\@test' != '/tmp\\@test'



Please fix,


Raymond


> -----Original Message-----
> From: python-checkins-bounces@python.org [mailto:python-checkins-
> bounces@python.org] On Behalf Of aimacintyre@users.sourceforge.net
> Sent: Sunday, December 12, 2004 3:37 AM
> To: python-checkins@python.org
> Subject: [Python-checkins] python/dist/src/Modules posixmodule.c,
> 2.300.8.10, 2.300.8.11
> 
> Update of /cvsroot/python/python/dist/src/Modules
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12893
> 
> Modified Files:
>       Tag: release23-maint
> 	posixmodule.c
> Log Message:
> OS/2 specific fixes related to SF bug # 1003471.
> 
> Also revise a related function to minimise file handle/pipe leakage
> and improve reliability.
> 
> Backported from -HEAD.

From mwh at python.net  Fri Dec 17 14:49:45 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Dec 17 14:49:47 2004
Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places
In-Reply-To: <41C0E6E8.2000805@kdart.com> (Keith Dart's message of "Wed, 15
	Dec 2004 17:37:44 -0800")
References: <ca471dc204120814186f5dc423@mail.gmail.com>
	<cpeh62$ldh$1@sea.gmane.org>
	<ca471dc204121108225511d43@mail.gmail.com>
	<cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com>
	<864d370904121209147b713f31@mail.gmail.com>
	<864d3709041212093274becde2@mail.gmail.com>
	<20041214143101.GA1404@rogue.amk.ca> <41C0E6E8.2000805@kdart.com>
Message-ID: <2mr7lpkpae.fsf@starship.python.net>

Keith Dart <kdart@kdart.com> writes:

> A.M. Kuchling wrote:
>> On Sun, Dec 12, 2004 at 03:32:03PM -0200, Carlos Ribeiro wrote:
>> 
>>>Of course, the point here is not Perl-bashing. The point here is that
>>>we should be able to "sell" Python better than we do now, even without
>>>the need to resort to such poor measures. I'm sure the Python
>>>community does have good & creative people that can write a good
>>>"selling" FAQ for Python, emphasizing the main points of the language.
>> No one disagrees that Python needs better marketing material.  At
>> the
>> last PyCon a group of people sat down in a pydotorg BoF and agreed
>> that yes, we do need a management-friendly marketing site, and that we
>> could put it on a separate hostname (something.python.org) so that the
>> current www.python.org wouldn't have to be changed.
>> However, no one has actually sat down and written such a site, or
>> even
>> outlined it.  Let me encourage you to go ahead and do that.  You could
>> draft the outline on a Wiki page, and then later figure out an
>> attractive design and organization for a new site.
>
>
> Whatever it looks like, it should probably run on Zope plus Plone. 8-) 
> You know... eat your own dog food. 8-)

This is SO not the problem that needs discussion...

> The kind folks over at Zettai! have provided some space for
> me. Perhaps they will be glad to host the main Python site, as well?

Neither is this!

Cheers,
mwh

-- 
  Just point your web browser at http://www.python.org/search/ and
  look for "program", "doesn't", "work", or "my". Whenever you find
  someone else whose program didn't work, don't do what they
  did. Repeat as needed.    -- Tim Peters, on python-help, 16 Jun 1998
From mcherm at mcherm.com  Fri Dec 17 17:24:11 2004
From: mcherm at mcherm.com (Michael Chermside)
Date: Fri Dec 17 17:24:12 2004
Subject: [Python-Dev] re: 2.4 news reaches interesting places
Message-ID: <1103300651.41c3082b5f3d8@mcherm.com>

Carlos Riberio writes:
> One possible marketing strategy is to use the adjective "fast" in a
> broader sense. The Python slogan could be something like: "Programming
> has never been any faster" -- this changes the playing ground, from
> raw performance to *programming* performance.

I think Carlos is onto something here. Guido's original question was
how to fight this meme... in other words, people think of Python as
slow, whether they have measured it or not. Just like they think of
Java as being more portable. Talking about "fast enough" is just another
way of reminding people that we're really quite slow (even if that's not
true).

So how about a slogan like "Code it Fast, with Python", or "Python: Code
Fast" -- one which emphasizes the (easily defended) claim that development
time is shorter with Python, but which at the same time manages to
associate the word "fast" with "Python".

-- Michael Chermside

From python at rcn.com  Fri Dec 17 18:06:34 2004
From: python at rcn.com (Raymond Hettinger)
Date: Fri Dec 17 18:09:34 2004
Subject: [Python-Dev] re: 2.4 news reaches interesting places
In-Reply-To: <1103300651.41c3082b5f3d8@mcherm.com>
Message-ID: <001801c4e45a$c3979fa0$e841fea9@oemcomputer>

> So how about a slogan like "Code it Fast, with Python", or "Python:
Code
> Fast" -- one which emphasizes the (easily defended) claim that
development
> time is shorter with Python, but which at the same time manages to
> associate the word "fast" with "Python".

I always liked:  "Python, the language that wraps tightly around a
problem and swallows it whole."


Raymond

From FBatista at uniFON.com.ar  Fri Dec 17 18:21:54 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Fri Dec 17 18:24:52 2004
Subject: [Python-Dev] re: 2.4 news reaches interesting places
Message-ID: <A128D751272CD411BC9200508BC2194D053C7D31@escpl.tcp.com.ar>

[Raymond Hettinger]

#- > So how about a slogan like "Code it Fast, with Python", or "Python:
#- Code
#- > Fast" -- one which emphasizes the (easily defended) claim that
#- development
#- > time is shorter with Python, but which at the same time manages to
#- > associate the word "fast" with "Python".
#- 
#- I always liked:  "Python, the language that wraps tightly around a
#- problem and swallows it whole."

Yes, but the point is that, even if it swallows the problem not-as-fast as
C, it wraps around the problem very quickly!

.	Facundo

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
ADVERTENCIA.

La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informaci?n
confidencial o propietaria, cuya divulgaci?n es sancionada por la ley.
Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no est?
autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de
ella) contenida en este mensaje. Por favor notif?quenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magn?tico) que pueda haber realizado del mismo.
Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones
Personales S.A. o alguna empresa asociada.
Los mensajes electr?nicos pueden ser alterados, motivo por el cual
Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n
cualquiera sea el resultante de este mensaje.
Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20041217/ed113b44/attachment.html
From carribeiro at gmail.com  Fri Dec 17 19:37:05 2004
From: carribeiro at gmail.com (Carlos Ribeiro)
Date: Fri Dec 17 19:37:08 2004
Subject: [Python-Dev] re: 2.4 news reaches interesting places
In-Reply-To: <1103300651.41c3082b5f3d8@mcherm.com>
References: <1103300651.41c3082b5f3d8@mcherm.com>
Message-ID: <864d37090412171037364834f3@mail.gmail.com>

On Fri, 17 Dec 2004 08:24:11 -0800, Michael Chermside <mcherm@mcherm.com> wrote:
> Carlos Riberio writes:
> > One possible marketing strategy is to use the adjective "fast" in a
> > broader sense. The Python slogan could be something like: "Programming
> > has never been any faster" -- this changes the playing ground, from
> > raw performance to *programming* performance.
> 
> I think Carlos is onto something here. Guido's original question was
> how to fight this meme... in other words, people think of Python as
> slow, whether they have measured it or not. Just like they think of
> Java as being more portable. Talking about "fast enough" is just another
> way of reminding people that we're really quite slow (even if that's not
> true).
> 
> So how about a slogan like "Code it Fast, with Python", or "Python: Code
> Fast" -- one which emphasizes the (easily defended) claim that development
> time is shorter with Python, but which at the same time manages to
> associate the word "fast" with "Python".

Whatever the slogan is, the most important point is: Python (and
Pythonistas in general) should not be ashamed about Python's speed.
I've checked most of the links that popped up as this thread
progressed, and followed other similar links. In many cases, Python
advocates are quoted as saying that speed improvements were due to
recoding in C. That's definitely *not* the way we're going to sell the
perception of speed. It is an unnecessary apology, and it sells the
wrong idea. It says "we know it's slow, but we're moving to a faster
language to improve performance". Not exactly reassuring.

As far as the slogan is concerned - I still stand for my proposal, but
I don't know if anyone has registered it first... I fear it is, it
sounds strangely familiar. But I hope not.

p.s. Can someone imagine a Microsoft senior executive saying that "C#
is now faster because we rewrote the critical parts in C?" Or the same
for Sun & Java? ;-)
-- 
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: carribeiro@gmail.com
mail: carribeiro@yahoo.com
From FBatista at uniFON.com.ar  Fri Dec 17 19:38:55 2004
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Fri Dec 17 19:41:55 2004
Subject: [Python-Dev] re: 2.4 news reaches interesting places
Message-ID: <A128D751272CD411BC9200508BC2194D053C7D37@escpl.tcp.com.ar>

[Carlos Ribeiro]

#- As far as the slogan is concerned - I still stand for my 
#- proposal, but
#- I don't know if anyone has registered it first... I fear it is, it
#- sounds strangely familiar. But I hope not.

Well, Google didn't return any match to the exact phrase "Programming has
never been any faster" (but this is not legal to affirm prior invention
(yet)).

I like your proposal (and slogan) a LOT!

.	Facundo

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
ADVERTENCIA.

La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo,
son para uso exclusivo del destinatario y pueden contener informaci?n
confidencial o propietaria, cuya divulgaci?n es sancionada por la ley.
Si Ud. No es uno de los destinatarios consignados o la persona responsable
de hacer llegar este mensaje a los destinatarios consignados, no est?
autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de
ella) contenida en este mensaje. Por favor notif?quenos respondiendo al
remitente, borre el mensaje original y borre las copias (impresas o grabadas
en cualquier medio magn?tico) que pueda haber realizado del mismo.
Todas las opiniones contenidas en este mail son propias del autor del
mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones
Personales S.A. o alguna empresa asociada.
Los mensajes electr?nicos pueden ser alterados, motivo por el cual
Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n
cualquiera sea el resultante de este mensaje.
Muchas Gracias.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20041217/6dc675ac/attachment.html
From eirik.mikkelsen at unix.net  Fri Dec 17 23:05:08 2004
From: eirik.mikkelsen at unix.net (Eirik Mikkelsen)
Date: Fri Dec 17 23:03:52 2004
Subject: [Python-Dev] re: 2.4 news reaches interesting places
In-Reply-To: <864d37090412170320561ecb8@mail.gmail.com>
References: <d4c0d4800412160513d16f13e@mail.gmail.com>
	<864d37090412170320561ecb8@mail.gmail.com>
Message-ID: <1103321109.10602.28.camel@localhost>

On Fri, 2004-12-17 at 09:20 -0200, Carlos Ribeiro wrote:
> BTW, I would move away from the "fast enough" when talking about
> performance. It's difficult to qualify what is "enough" in marketing
> terms; also, a selling/winning message can't be seen as taking excuses
> for any reason. On the other hand, Python never claims to be the
> fastest language on raw execution performance, but only to be fast;
> but in this sense, being "fast enough" is the same as being "fast".
> So, I would never say, "Python allows you to write fast enough code in
> a short time"; I would say "Python allows you to write fast code in a
> short time". Leave the "fast enough" out of this, please.

I totally agree. Personally, the first thing I think of when I see
"enough" is "640k aught to be enough for anybody" (quote from
you-know-who), like you are defining the needs of the user. Promote the
strong sides, don't excuse the weak ones. "Fast enough" is not a
positive marketing term, it's an excuse for a problem which I fail to
see in Python. Hardware cost is way lower than programmer cost - I am
convinced that in most cases the total expence is lower for a Python
solution compared to an equal performing C, C++, C# or Java solution.

-- 
Eirik Mikkelsen <eirik.mikkelsen@unix.net>

From andymac at bullseye.apana.org.au  Fri Dec 17 23:53:17 2004
From: andymac at bullseye.apana.org.au (Andrew MacIntyre)
Date: Sat Dec 18 01:11:55 2004
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Modules
 posixmodule.c, 2.300.8.10, 2.300.8.11
In-Reply-To: <000401c4e436$8eabb0c0$e841fea9@oemcomputer>
References: <000401c4e436$8eabb0c0$e841fea9@oemcomputer>
Message-ID: <20041218091927.E1961@bullseye.apana.org.au>

On Fri, 17 Dec 2004, Raymond Hettinger wrote:

> The Dec 12th check-ins break tests on WinME:
>
>
> test_glob.py
> ------------
> Traceback (most recent call last):
>   File "test_glob.py", line 78, in test_glob_one_directory
>     eq(self.glob('a*'), map(self.norm, ['a', 'aab', 'aaa']))
>   File "test_glob.py", line 67, in assertSequencesEqual_noorder
>     self.assertEqual(set(l1), set(l2))
> AssertionError: set(['/tmp\\@test_dir\\aaa', '/tmp\\@test_dir\\aab',
> '/tmp\\@tes
> t_dir\\a']) != set(['\\tmp\\@test_dir\\aab', '\\tmp\\@test_dir\\aaa',
> '\\tmp\\@t
> est_dir\\a'])
>
>
> test_urllib.py
> --------------
> FAIL: test_basic (__main__.urlretrieve_FileTests)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "test_urllib.py", line 142, in test_basic
>     self.assertEqual(result[0], test_support.TESTFN)
> AssertionError: '\\tmp\\@test' != '/tmp\\@test'
>
>
>
> Please fix,

????

I don't see any possible way for those checkins to affect any platform
other than OS/2.

2 of the files are platform specific files (PC/os2emx/getpath.c,
PC/os2vacpp/getpath.c), and the checkin to Modules/posixmodule.c is
contained within a platform specific #if/#endif:

...
#ifdef HAVE_POPEN        <<<-- line 3241
...
#if defined(PYOS_OS2)
#if defined(PYCC_VACPP)
...                      <<<-- changes here
#elif defined(PYCC_GCC)
...
#endif /* PYCC_??? */
#elif defined(MS_WINDOWS)
...
#else /* which OS? */
...
#endif /* PYOS_??? */
#endif /* HAVE_POPEN */
...

Note that the posixmodule change affects popen().

The matching PC/getpath.c changes that apply to Windows were checked in by
Tim Peters back in August.

I don't have any Windows development environment, so can't cross check
your report :-(

If you have verified the cause of failure by backing out v2.300.8.11, then
there's a deeper problem about the Windows build picking up explicitly
non-Windows components - which I find hard to believe.

In the absence of more evidence, not guilty your honour.

-------------------------------------------------------------------------
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac@pcug.org.au             (alt) |        Belconnen ACT 2616
Web:    http://www.andymac.org/               |        Australia
From andymac at bullseye.apana.org.au  Sat Dec 18 00:10:35 2004
From: andymac at bullseye.apana.org.au (Andrew MacIntyre)
Date: Sat Dec 18 01:11:57 2004
Subject: [Python-Dev] re: 2.4 news reaches interesting places
In-Reply-To: <001801c4e45a$c3979fa0$e841fea9@oemcomputer>
References: <001801c4e45a$c3979fa0$e841fea9@oemcomputer>
Message-ID: <20041218100407.F1961@bullseye.apana.org.au>

On Fri, 17 Dec 2004, Raymond Hettinger wrote:

> > So how about a slogan like "Code it Fast, with Python", or "Python:
> Code
> > Fast" -- one which emphasizes the (easily defended) claim that
> development
> > time is shorter with Python, but which at the same time manages to
> > associate the word "fast" with "Python".
>
> I always liked:  "Python, the language that wraps tightly around a
> problem and swallows it whole."

The only problem here is the association with the reptilian python, which
is not perceived as being rapid - though it is highly effective as you
describe.  Whereas cobras...

I somewhat suspect a subliminal association linking the reptile and the
language, colouring perceptions without factual basis... :-(

-------------------------------------------------------------------------
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac@pcug.org.au             (alt) |        Belconnen ACT 2616
Web:    http://www.andymac.org/               |        Australia
From kbk at shore.net  Sat Dec 18 04:21:15 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Sat Dec 18 04:21:37 2004
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <200412180321.iBI3LFlA022448@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  259 open ( +0) /  2707 closed ( +2) /  2966 total ( +2)
Bugs    :  822 open (+22) /  4685 closed (+23) /  5507 total (+45)
RFE     :  160 open ( +0) /   139 closed ( +2) /   299 total ( +2)

New / Reopened Patches
______________________

repair typo  (2004-12-08)
CLOSED http://python.org/sf/1080684  opened by  George Yoshida

Description of message_set and command option to store()  (2004-12-13)
       http://python.org/sf/1084092  opened by  Sean Reifschneider

Patches Closed
______________

repair typo  (2004-12-07)
       http://python.org/sf/1080684  closed by  jlgijsbers

fix inspect.getsource breaking with line-continuation & more  (2004-08-19)
       http://python.org/sf/1011890  closed by  jlgijsbers

New / Reopened Bugs
___________________

Inplace set merge produces wrong results  (2004-12-07)
CLOSED http://python.org/sf/1080424  opened by  Matthias Klose

float issue for NaN type in .pyc file  (2004-12-07)
       http://python.org/sf/1080440  opened by  Dileep Nirala

Python Icon in system Tray  (2004-12-07)
CLOSED http://python.org/sf/1080634  opened by  Dileep Nirala

thread.error: release unlocked lock on Queue put  (2004-12-07)
CLOSED http://python.org/sf/1080660  opened by  John Speno

os.ttyname() accepts wrong arguments  (2004-12-07)
       http://python.org/sf/1080713  opened by  Christian H?ltje

full test with all unicode text files  (2004-12-07)
CLOSED http://python.org/sf/1080811  opened by  Grzegorz Makarewicz

locale.py doesn't recognize valid locale setting  (2004-12-07)
CLOSED http://python.org/sf/1080864  opened by  stas Z

readline module doesn't build on MacOSX  (2004-12-07)
       http://python.org/sf/1081045  opened by  Skip Montanaro

Bad reference in whrandom docs  (2004-12-08)
CLOSED http://python.org/sf/1081370  opened by  Lars Marius Garshol

LDAP search segfaults on RedHat FC3  (2004-12-08)
CLOSED http://python.org/sf/1081633  opened by  Tessa Lau

Rewrite of docs for compiler.visitor  (2004-12-09)
       http://python.org/sf/1081824  opened by  Kent Johnson

Pydoc can't find browser (bug+solution!)  (2004-12-08)
       http://python.org/sf/1081879  opened by  Stewart Midwinter

PyString_AsString() segfaults when passed a unicode string  (2004-12-09)
CLOSED http://python.org/sf/1082085  opened by  Andreas Jung

Documentation for PyUnicode_TailMatch incorrrectly says it r  (2004-12-10)
CLOSED http://python.org/sf/1082944  opened by  Jim Fulton

truncated gzip file triggers zlibmodule segfault  (2004-12-10)
       http://python.org/sf/1083110  opened by  Sam Rushing

Change in signal function in the signal module  (2004-12-10)
       http://python.org/sf/1083177  opened by  Gary H. Loechelt

Unable to see Python binary  (2004-12-10)
       http://python.org/sf/1082874  opened by  Prabal Rakshit

UnboundLocalError raised by atexit module  (2004-12-10)
CLOSED http://python.org/sf/1083202  opened by  Gary H. Loechelt

Distutils doesn't pick up all the files it should.  (2004-12-10)
       http://python.org/sf/1083299  opened by  Mike Meyer

font lock keyword regular expressions  (2004-12-09)
       http://python.org/sf/1082487  opened by  Robert Brown

Tests fail instead of skip  (2004-12-12)
       http://python.org/sf/1083645  opened by  Detlef Vollmann

Python 2.4 crashes  (2004-12-12)
       http://python.org/sf/1083793  opened by  Axel Kaiser

functions replaced should point to its docs  (2004-12-12)
       http://python.org/sf/1083895  opened by  Johannes Gijsbers

status of small floats in xml-rpc ?  (2004-12-13)
CLOSED http://python.org/sf/1084279  opened by  Antoine Pitrou

ossaudiodev no longer undocumented  (2004-12-13)
CLOSED http://python.org/sf/1084457  opened by  Gregory H. Ball

sys.stdin segfaults on invalid stdin  (2004-12-13)
       http://python.org/sf/1084766  opened by  Mihai Ibanescu

list initialization bug  (2004-12-14)
CLOSED http://python.org/sf/1084906  opened by  py_py

gethostbyaddr on redhat for multiple hostnames  (2004-12-14)
       http://python.org/sf/1085069  opened by  Dave Kirby

Fix stale link in PEP  (2004-12-14)
CLOSED http://python.org/sf/1085096  opened by  Michael Chermside

Uninstaller unclear about the app it's uninstalling  (2004-12-14)
CLOSED http://python.org/sf/1085168  opened by  Giles Antonio Radford

Uninstaller should restore file associations if possible  (2004-12-14)
       http://python.org/sf/1085172  opened by  Giles Antonio Radford

Installation ends prematurely  (2004-12-14)
       http://python.org/sf/1085208  opened by  anamh0001

binascii.b2a_qp oddities  (2004-12-14)
       http://python.org/sf/1085283  opened by  DSM

Mac Library Modules 1.1.1 Bad Info  (2004-12-14)
       http://python.org/sf/1085300  opened by  Walrus

Make array.array pickle-able  (2004-12-14)
CLOSED http://python.org/sf/1085304  opened by  Nicolas Fleury

Bad interaction between PySequence_Fast and itertools  (2004-12-15)
CLOSED http://python.org/sf/1085744  opened by  Nick Coghlan

subprocess.Popen feature request  (2004-12-15)
       http://python.org/sf/1085861  opened by  Michele Simionato

PySequence_Tuple not as fast as PySequence_List  (2004-12-15)
       http://python.org/sf/1085744  reopened by  rhettinger

two strings holding the same value have different id  (2004-12-15)
CLOSED http://python.org/sf/1086096  opened by  Stefan Seefeld

Typo in module os documentation  (2004-12-15)
CLOSED http://python.org/sf/1086127  opened by  Connelly

refcount problem in syslog  (2004-12-16)
CLOSED http://python.org/sf/1086555  opened by  DSM

segfault in readline  (2004-12-16)
       http://python.org/sf/1086603  opened by  DSM

Compile of _socket fails on 2.4  (2004-12-16)
       http://python.org/sf/1086642  opened by  A. Stocker

optparse docs missing section on Extending  (2004-12-16)
       http://python.org/sf/1086675  opened by  Kent Johnson

"slots" member variable in object.h confuses Qt moc utility  (2004-12-17)
       http://python.org/sf/1086854  opened by  Jeremy Friesner

datetime module documentation missing critical detail  (2004-12-17)
       http://python.org/sf/1087216  opened by  Mike Meyer

Bugs Closed
___________

Inplace set merge produces wrong results  (2004-12-07)
       http://python.org/sf/1080424  closed by  tim_one

thread.error: release unlocked lock on Queue put  (2004-12-07)
       http://python.org/sf/1080660  closed by  rhettinger

full test with all unicode text files  (2004-12-07)
       http://python.org/sf/1080811  closed by  makaron

locale.py doesn't recognize valid locale setting  (2004-12-07)
       http://python.org/sf/1080864  closed by  lemburg

Bad reference in whrandom docs  (2004-12-08)
       http://python.org/sf/1081370  closed by  rhettinger

PythonWin Tray Icon in system Tray on Windows platform  (2004-12-07)
       http://python.org/sf/1080634  closed by  loewis

LDAP search segfaults on RedHat FC3  (2004-12-08)
       http://python.org/sf/1081633  closed by  loewis

PyString_AsString() segfaults when passed a unicode string  (2004-12-09)
       http://python.org/sf/1082085  closed by  dcjim

Documentation for PyUnicode_TailMatch incorrrectly says it r  (2004-12-10)
       http://python.org/sf/1082944  closed by  rhettinger

UnboundLocalError raised by atexit module  (2004-12-10)
       http://python.org/sf/1083202  closed by  rhettinger

Python 1.5.2 security vulnerability still present in 2.3.4   (2004-08-05)
       http://python.org/sf/1003471  closed by  aimacintyre

test_subprocess fails on cygwin  (2004-11-22)
       http://python.org/sf/1071516  closed by  jlt63

Error building _bsddb extension  (2004-12-01)
       http://python.org/sf/1077040  closed by  greg

status of small floats in xml-rpc ?  (2004-12-13)
       http://python.org/sf/1084279  closed by  rhettinger

ossaudiodev no longer undocumented  (2004-12-13)
       http://python.org/sf/1084457  closed by  rhettinger

list initialization bug  (2004-12-14)
       http://python.org/sf/1084906  closed by  rhettinger

Fix stale link in PEP  (2004-12-14)
       http://python.org/sf/1085096  closed by  jlgijsbers

Uninstaller unclear about the app it's uninstalling  (2004-12-14)
       http://python.org/sf/1085168  closed by  loewis

Bad interaction between PySequence_Fast and itertools  (2004-12-15)
       http://python.org/sf/1085744  closed by  rhettinger

Signal reception problem. FreeBSD 4.8 and 4.9 after forkpty  (2004-04-28)
       http://python.org/sf/944119  closed by  levis501

PySequence_Tuple not as fast as PySequence_List  (2004-12-15)
       http://python.org/sf/1085744  closed by  rhettinger

two strings holding the same value have different id  (2004-12-15)
       http://python.org/sf/1086096  closed by  goodger

Typo in module os documentation  (2004-12-16)
       http://python.org/sf/1086127  closed by  doerwalter

refcount problem in syslog  (2004-12-16)
       http://python.org/sf/1086555  closed by  rhettinger

New / Reopened RFE
__________________

Add encoding to DocFileSuite  (2004-12-07)
       http://python.org/sf/1080727  opened by  Bjorn Tillenius

RFE Closed
__________

ignore element format character for PyArg_ParseTuple  (2004-11-30)
       http://python.org/sf/1075902  closed by  rhettinger

Make array.array pickle-able  (2004-12-14)
       http://python.org/sf/1085304  closed by  rhettinger

From bac at OCF.Berkeley.EDU  Sat Dec 18 06:10:21 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sat Dec 18 06:10:27 2004
Subject: [Python-Dev] python-dev Summary for 2004-11-01 through 2004-11-15
	[draft]
Message-ID: <41C3BBBD.9050805@ocf.berkeley.edu>

Now I am only a month behind.

Won't send this out any earlier than Monday night, but probably won't be until 
Tuesday.

------------------------------------

=====================
Summary Announcements
=====================
I am hoping to be caught up with my summary backlog by the end of the month. 
Hear is to hoping.

=========
Summaries
=========
------------
2.4 released
------------
`Python 2.4`_ has been released!  Read the `What's New`_ for 2.4 to see what 
the major changes are since 2.3 .

.. _Python 2.4: http://www.python.org/2.4/
.. _What's New: http://www.python.org/doc/2.4/whatsnew/whatsnew24.html

Contributed threads:
   - `TRUNK FROZEN for 2.4b2 release from 0000UTC 3rd November (tomorrrow) <>`__
   - `RELEASED Python 2.4, beta 2 <>`__
   - `TRUNK UNFROZEN <>`__

-------------------------------------
Code coverage of the regression tests
-------------------------------------
Walter D?rwald has a code coverage tool that runs once a month on Python's 
regression test at http://coverage.livinglogic.de/ .  There was some issue with 
the 'trace' module having partially broken in 2.4 .  The coverage was then run 
using ``Lib/test/regrtest.py -T`` to get more accurate numbers.

Contributing threads:
   - `Code coverage tool updated <>`__

------------------------------------
Garbage collecting weakref callbacks
------------------------------------
Tim Peters discovered that in 2.4 if a weakref's callback is still reachable 
but the referent and weakref itself are unreachable (Tim's example had an 
instance that contained a weakref to itself) that the callback is not called 
during garbage collection.  This seemed inconsistent since if the weakref in 
Tim's example was replaced with an instance that contained a __del__ method 
that the method would have been called.

Tim would like to clean this up so as to be able to ditch __del__ methods in 
Python 3000.  The idea is that one would register a weakref with a callback for 
an object in itself that would be called when it is garbage collected.  This 
would also negate the need for gc.garbage .

Contributing threads:
   - `weakref gc semantics <>`__
   - `patch-finalizer vs Zope3 <>`__

---------
PEP Watch
---------
`PEP 336`_ introduces the idea of having None be a callable object.  The result 
that calling None would return None itself.

.. _PEP 336: http://www.python.org/peps/pep-0336.html

Contributing threads:
   - `PEP 336, "Make None Callable", by Andrew McClelland <>`__


----------------------------------------------
Discussion of including pysqlite in the stdlib
----------------------------------------------
The idea of including pysqlite_ in the stdlib came up once again (not this is 
the wrapper for SQLite_ and not SQLite itself).  The arguments for including 
the module were the SQLite is most likely used more than Sleepycat and thus 
deserved a place in the stdlib if bsddb did.  It also goes along with Python's 
idea of "batteries included".

Arguments against started with the idea of sanctioning pysqlite over other 
SQLite wrappers did not seem proper.  People also thought that including bsddb 
might not be correct anymore and thus should not be used as a basis for 
including pysqlite.

This all then led into a discussion about package management and how to 
simplify extending what can be installed at compile-time.  The idea of pushing 
PyPI_ came up as well as revising `PEP 206`_.

.. _pysqlite: http://pysqlite.org/
.. _SQLite: http://www.sqlite.org/
.. _PyPI: http://www.python.org/pypi
.. _PEP 206: http://www.python.org/peps/pep-0206.html

Contributing threads:
   - `SQLite module for Python 2.5 <>`__


--------------------------------
2.3 maintenance coming to an end
--------------------------------
With Python 2.4 out, maintaining 2.3 is no longer a priority. Anthony Baxter, 
our beloved release manager, has said that 2.3.5 will be most likely coming out 
in January.  After that, though, don't count on another 2.3 release since 2.4 
will take its place as the version to maintain.

All of this came about by the fact that Facundo Batista asked if closing 2.1 
bugs was okay.  As long as they have been fixed in the earliest version being 
maintained, closing them is fine.  This goes for 2.2 as well.  And as soon as 
2.3.5 is out the door will most likely apply to 2.3 bugs.

Contributing threads:
   - `Policy about old Python versions <>`__

-----------------------------------------------------------------
Sync/async servers in the stdlib and whether they should be there
-----------------------------------------------------------------
There are multiple servers in the core for handling communications in both a 
synchronous and asynchronous manner.  The idea came up of developing an API 
that all servers could follow in the core.  While the discussion went back and 
forth with multiple mentions of PEAK, Twisted, and greenlets_, in the end the 
trend of the discussion suggested doing this well would be very difficult and 
not worth the effort.  asyncore and asynchat also seemed to be deemed worth 
deprecation and thus not worth using.

.. _PEAK: http://peak.telecommunity.com/
.. _Twisted: http://www.twistedmatrix.com/
.. _greenlets: http://codespeak.net/svn/user/arigo/greenlet/

Contributing threads:
   - `Synchronous and Asynchronous servers in the standard library <>`__

--------------------------------------------------
Getting the opcode count for rough profile numbers
--------------------------------------------------
The idea was proposed of introducing sys.gettickeraccumulation which would 
contain the number of opcodes executed since the interpreter started.  The idea 
is that you can get a rough estimate of how much time is being spent in Python 
code.  The value of this has wavered back and forth with no clear answer.

Contributing threads:
   - `proposal+patch: sys.gettickeraccumulation() <>`__

-

-
Having syntactic support for variable-sized unpacking came up again

Contributing threads:
   - `syntactic shortcut - unpack to variably sized list <>`__

===============
Skipped Threads
===============
- Deprecate PyRange_New()
- interlocking dependencies on the path	to a release
       see the first email to see what steps are needed for a release
- Overflow in socketmodule.c?
- Int literals and method calls
       To get access to methods on an int literal put a space after the number; 
``1 .__class__``
- os.access versus os.exist
- syntactic shortcut - unpack to variably sized list
       don't expect syntactic support variable-length sequences ain't happenin'
From bac at OCF.Berkeley.EDU  Sat Dec 18 08:24:44 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sat Dec 18 08:24:55 2004
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Modules
	posixmodule.c, 2.300.8.10, 2.300.8.11
In-Reply-To: <20041218091927.E1961@bullseye.apana.org.au>
References: <000401c4e436$8eabb0c0$e841fea9@oemcomputer>
	<20041218091927.E1961@bullseye.apana.org.au>
Message-ID: <41C3DB3C.4030707@ocf.berkeley.edu>

Andrew MacIntyre wrote:

> I don't see any possible way for those checkins to affect any platform
> other than OS/2.
> 
> 2 of the files are platform specific files (PC/os2emx/getpath.c,
> PC/os2vacpp/getpath.c), and the checkin to Modules/posixmodule.c is
> contained within a platform specific #if/#endif:
> 

Perhaps, but if you look at line 3299 you will notice that a comment is started 
but not closed off before the next comment on line 3304.  How is that comment 
supposed to be closed off?

-Brett

From tim.peters at gmail.com  Sat Dec 18 09:11:48 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Sat Dec 18 09:11:51 2004
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Modules
	posixmodule.c, 2.300.8.10, 2.300.8.11
In-Reply-To: <41C3DB3C.4030707@ocf.berkeley.edu>
References: <000401c4e436$8eabb0c0$e841fea9@oemcomputer>
	<20041218091927.E1961@bullseye.apana.org.au>
	<41C3DB3C.4030707@ocf.berkeley.edu>
Message-ID: <1f7befae0412180011484b44d8@mail.gmail.com>

[Raymond, says test_glob and test_urllib fail on WinME now]

[Andrew MacIntyre]
>> I don't see any possible way for those checkins to affect any platform
>> other than OS/2.
>>
>> 2 of the files are platform specific files (PC/os2emx/getpath.c,
>> PC/os2vacpp/getpath.c), and the checkin to Modules/posixmodule.c is
>> contained within a platform specific #if/#endif:

[Brett]
> Perhaps, but if you look at line 3299 you will notice that a comment is started
> but not closed off before the next comment on line 3304.  How is that comment
> supposed to be closed off?

That's obviously a bug, and was introduced by the patch.  Andrew wanted

	/* setup the pipe */

not

	/* setup the pipe

But we don't get there unless PYOS_OS2 and PYCC_VACPP are defined, so
that can't explain Raymond's problem.

FWIW, the  tests at issue pass on WinXP for me today w/ current CVS.
From pedronis at bluewin.ch  Sat Dec 18 16:03:54 2004
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Sat Dec 18 16:02:53 2004
Subject: [Python-Dev] 2.4 news reaches interesting places
In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com>
References: <ca471dc204120814186f5dc423@mail.gmail.com>
Message-ID: <41C446DA.5080905@bluewin.ch>

Guido van Rossum wrote:
> I was pleasantly surprised to find a pointer to this article in a news
> digest that the ACM emails me regularly (ACM TechNews).
> 
> http://gcn.com/vol1_no1/daily-updates/28026-1.html
> 
> One thing that bugs me: the article says 3 or 4 times that Python is
> slow, each time with a refutation ("but it's so flexible", "but it's
> fast enough") but still, they sure seem to harp on the point. This is
> a PR issue that Python needs to fight -- any ideas?
> 

I think that right now there is a promising ongoing happening that
may be useful for this PR problem and Python perception in general,
which is the Chandler project, because it's a piece of software that may 
end up in front of lot of people. It is both an opportunity and a risk.

A not great success of Chandler especially if imputed to Python
shortcomings would be bad.

I think that people with long-time Python experience that feel like to 
contribute should try.

A successful, responsive Chandler, I think, would be a good thing for
Python.


From andymac at bullseye.apana.org.au  Sat Dec 18 10:33:31 2004
From: andymac at bullseye.apana.org.au (Andrew MacIntyre)
Date: Sat Dec 18 19:20:15 2004
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Modules
	posixmodule.c, 2.300.8.10, 2.300.8.11
In-Reply-To: <1f7befae0412180011484b44d8@mail.gmail.com>
References: <000401c4e436$8eabb0c0$e841fea9@oemcomputer> 
	<20041218091927.E1961@bullseye.apana.org.au>
	<41C3DB3C.4030707@ocf.berkeley.edu>
	<1f7befae0412180011484b44d8@mail.gmail.com>
Message-ID: <20041218201510.N3850@bullseye.apana.org.au>

On Sat, 18 Dec 2004, Tim Peters wrote:

> [Raymond, says test_glob and test_urllib fail on WinME now]
>
> [Andrew MacIntyre]
> >> I don't see any possible way for those checkins to affect any platform
> >> other than OS/2.
> >>
> >> 2 of the files are platform specific files (PC/os2emx/getpath.c,
> >> PC/os2vacpp/getpath.c), and the checkin to Modules/posixmodule.c is
> >> contained within a platform specific #if/#endif:
>
> [Brett]
> > Perhaps, but if you look at line 3299 you will notice that a comment is started
> > but not closed off before the next comment on line 3304.  How is that comment
> > supposed to be closed off?
>
> That's obviously a bug, and was introduced by the patch.  Andrew wanted
>
> 	/* setup the pipe */
>
> not
>
> 	/* setup the pipe

Fair cop guv.

Long exposure to a language with comments of the latter form has been a
PITA at times. :-(

> But we don't get there unless PYOS_OS2 and PYCC_VACPP are defined, so
> that can't explain Raymond's problem.
>
> FWIW, the  tests at issue pass on WinXP for me today w/ current CVS.

Good, though Raymond's report appeared to be focussed on the 2.3 branch.

-------------------------------------------------------------------------
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac@pcug.org.au             (alt) |        Belconnen ACT 2616
Web:    http://www.andymac.org/               |        Australia
From python at rcn.com  Sat Dec 18 19:24:07 2004
From: python at rcn.com (Raymond Hettinger)
Date: Sat Dec 18 19:27:03 2004
Subject: [Python-Dev] RE: [Python-checkins]
	python/dist/src/Modulesposixmodule.c, 2.300.8.10, 2.300.8.11
In-Reply-To: <20041218201510.N3850@bullseye.apana.org.au>
Message-ID: <000001c4e52e$c3bfcdc0$7d35c797@oemcomputer>

> > FWIW, the  tests at issue pass on WinXP for me today w/ current CVS.

Tests pass here too.


Raymond

From stewart.midwinter at gmail.com  Fri Dec 17 15:20:53 2004
From: stewart.midwinter at gmail.com (Stewart Midwinter)
Date: Sat Dec 18 19:53:54 2004
Subject: [Python-Dev] re: 2.4 news reaches interesting places
In-Reply-To: <864d37090412170320561ecb8@mail.gmail.com>
References: <d4c0d4800412160513d16f13e@mail.gmail.com>
	<864d37090412170320561ecb8@mail.gmail.com>
Message-ID: <d4c0d480041217062074703419@mail.gmail.com>

On Fri, 17 Dec 2004 09:20:18 -0200, Carlos Ribeiro <carribeiro@gmail.com> wrote:

> One possible marketing strategy is to use the adjective "fast" in a
> broader sense. The Python slogan could be something like: "Programming
> has never been any faster" -- this changes the playing ground, from
> raw performance to *programming* performance. And sure, nothing beats
> Python (the overall package) in this respect. It can deliver fast code
> in a short time. Othere languages are faster to run, but take longer
> to code...

Parabens, Carlos, boa ideia!

Carlos has said it well, Python has the edge in overall performance. 
Evidence seems to indicate a 3-4 times edge over development in Java,
and this is a powerful argument.

cheers,
-- 
Stewart Midwinter
stewart@midwinter.ca
stewart.midwinter@gmail.com
From p.f.moore at gmail.com  Sat Dec 18 21:27:49 2004
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat Dec 18 21:27:52 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
In-Reply-To: <41C0B32C.7040701@v.loewis.de>
References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl>
	<79990c6b041213015861230c01@mail.gmail.com>
	<41BE150F.9050100@v.loewis.de>
	<79990c6b04121402011d8965bd@mail.gmail.com>
	<41BF6DC9.8030500@v.loewis.de>
	<79990c6b041215014944f31097@mail.gmail.com>
	<41C0B32C.7040701@v.loewis.de>
Message-ID: <79990c6b0412181227f5d4959@mail.gmail.com>

On Wed, 15 Dec 2004 22:57:00 +0100, Martin v. L?wis <martin@v.loewis.de> wrote:
> Paul Moore wrote:
> > For a starter, what steps do you actually take to build a release? I
> > assume that the first step is to build Python, by clicking on "build"
> > in VS.NET.
> 
> Yes. You can skip this step by just putting all the .pyds, dlls, and
> .exes into the PCbuild directory. The packaging will try to pick them
> up from there (and proceed if some are missing, like Tcl likely will).
> 
>  > Once you have that, is there a single script you run?
> 
> Yes. Invoke Tools\msi\msi.py, using a Python that has pythonwin (i.e.
> COM interopability). The only tricky part is that you need cabarc.exe,
> which is included in VC, and in the platform SDK.

OK, I've got a copy of the Python sources, and had a look. The change
needed to msi.py to include libpythonXX.a in the installer looks
simple. But I'm less sure as to where to build the file. It seems to
me that msi.py is not the right place - that's focused on building the
installer, not building the files to be installed.

On the other hand, including it in the build process is a nuisance, as
well, as it would add a dependency on mingw (or cygwin) to the MSVC
build process.

My feeling is that building libpythonXX.a should be a separate step,
handled by a dedicated script, which gets run before msi.py.

What do others (particularly Martin) think? Should I keep the steps
separate, and focused on one task each, or should I incorporate the
build of libpythonXX.a into msi.py, so that the installer build still
requires just one step?

Paul.
From jcarlson at uci.edu  Sat Dec 18 22:17:49 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Sat Dec 18 22:37:16 2004
Subject: [Python-Dev] mmap feature or bug?
Message-ID: <20041218130908.9236.JCARLSON@uci.edu>


Quick questions:
Is mmap's inability to be subclassed a feature or bug?
Is one's inability to use a = mmapinstance.__setslice__;a(1,2,'a') (and
others) a feature or bug?

I would imagine they are bugs, but I just wanted to make sure and post
the report/request in the proper location.

Thank you,
 - Josiah

From martin at v.loewis.de  Sat Dec 18 23:34:11 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Dec 18 23:34:12 2004
Subject: [Python-Dev] MinGW And The other Py2.4 issue
In-Reply-To: <79990c6b0412181227f5d4959@mail.gmail.com>
References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl>	
	<79990c6b041213015861230c01@mail.gmail.com>	
	<41BE150F.9050100@v.loewis.de>	
	<79990c6b04121402011d8965bd@mail.gmail.com>	
	<41BF6DC9.8030500@v.loewis.de>	
	<79990c6b041215014944f31097@mail.gmail.com>	
	<41C0B32C.7040701@v.loewis.de>
	<79990c6b0412181227f5d4959@mail.gmail.com>
Message-ID: <41C4B063.3020101@v.loewis.de>

Paul Moore wrote:
> OK, I've got a copy of the Python sources, and had a look. The change
> needed to msi.py to include libpythonXX.a in the installer looks
> simple. But I'm less sure as to where to build the file. It seems to
> me that msi.py is not the right place - that's focused on building the
> installer, not building the files to be installed.

Don't worry about this: there is already quite some building going on
in msi.py. If you look at the CVS copy of Tools/msi, you find that
it now has a nmake makefile to build msisupport.dll, which replaces
the VB scripts. It also extracts msvcr71.dll from the merge module
(.msm) each time it is invoked. So having yet another build process
would be just fine; adding it to the makefile would have the added
advantage that nmake will compare time stamps for us (if it is easier
to do in Python than in nmake, that would be a reason not to use
nmake, though).

> On the other hand, including it in the build process is a nuisance, as
> well, as it would add a dependency on mingw (or cygwin) to the MSVC
> build process.

That is definitely undesirable. Lots of people build Python using the
project files, and only few need the packaging to work.

> My feeling is that building libpythonXX.a should be a separate step,
> handled by a dedicated script, which gets run before msi.py.

Making it separate is fine, as long as msi.py invokes/calls it.

> What do others (particularly Martin) think? Should I keep the steps
> separate, and focused on one task each, or should I incorporate the
> build of libpythonXX.a into msi.py, so that the installer build still
> requires just one step?

Having the entire process involve as few manual steps as possible is
definitely an important goal. Keeping it modular (in terms of splitting
it into several files) is also a good idea, and one which does not
conflict at all with the "fully automatic" goal.

msi.py supports a config.py which allows to add customization. Putting

cygwin_dir = r"C:\cygwin"

into msi.py might be appropriate, with an option to set cygwin_dir to
None, to indicate that cygwin should not be used in the build process.
(similar to the way have_tcl already works).

Regards,
Martin
From martin at v.loewis.de  Sat Dec 18 23:47:29 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat Dec 18 23:47:31 2004
Subject: [Python-Dev] mmap feature or bug?
In-Reply-To: <20041218130908.9236.JCARLSON@uci.edu>
References: <20041218130908.9236.JCARLSON@uci.edu>
Message-ID: <41C4B381.7000808@v.loewis.de>

Josiah Carlson wrote:
> Is mmap's inability to be subclassed a feature or bug?

No.

It's a missing feature: it's not a bug, because nobody says this should
work, and anybody trying will find out that it doesn't work, so nobody
is tricked into believing it should work. The mmap type is not even
documented; mmap.mmap is a function.

It's not a feature, because (atleast IMO) there would be nothing wrong
with making mmap.mmap a subclassable type.

> Is one's inability to use a = mmapinstance.__setslice__;a(1,2,'a') (and
> others) a feature or bug?

That is a bug. Slice assignments are supported, and so should
__setslice__.

Regards,
Martin
From mwh at python.net  Sun Dec 19 17:34:55 2004
From: mwh at python.net (Michael Hudson)
Date: Sun Dec 19 17:34:56 2004
Subject: [Python-Dev] mmap feature or bug?
In-Reply-To: <20041218130908.9236.JCARLSON@uci.edu> (Josiah Carlson's
	message of "Sat, 18 Dec 2004 13:17:49 -0800")
References: <20041218130908.9236.JCARLSON@uci.edu>
Message-ID: <2mis6yl00g.fsf@starship.python.net>

Josiah Carlson <jcarlson@uci.edu> writes:

> Quick questions:
> Is mmap's inability to be subclassed a feature or bug?

Like Martin said, a missing feature.  I'd also quite like the mmap
module to have a C API, but as my use case is especially evil (

http://starship.python.net/crew/mwh/hacks/PPY.html

) I'm not going to push that hard :)

> Is one's inability to use a = mmapinstance.__setslice__;a(1,2,'a') (and
> others) a feature or bug?

I'm not entirely sure about this, though.  Does 

mmapinstance.__setitem__(slice(1,2), 'a')

work?  AIUI __setslice__ and __getslice__ are out of favour these days
(not sure though).

Any patch to fix one of these would probably also fix the other almost
as a sside-effect...

Cheers,
mwh

-- 
  ... with these conditions cam the realisation that ... nothing
  turned a perfectly normal healthy individual into a great political
  or military leader better than irreversible brain damage.
                   -- The Hitch-Hikers Guide to the Galaxy, Episode 11
From martin at v.loewis.de  Sun Dec 19 19:13:55 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Dec 19 19:13:56 2004
Subject: [Python-Dev] Python in education
In-Reply-To: <41C02165.2010000@ocf.berkeley.edu>
References: <41C02165.2010000@ocf.berkeley.edu>
Message-ID: <41C5C4E3.7090001@v.loewis.de>

Randy Chung wrote:
> I'm going to be leading a class on Python at the University of 
> California, Berkeley next semester (starting in January).  I'm 
> interested in using actual bugs in Python as exercises for the class, 
> the goal being 1) to give the students something "real" to work with, 
> and 2) to (hopefully) contribute back anything we manage to put together 
> to the Python dev team.

Welcome to the club! I'm just running a class on development processes
in open source software (i.e. with a somewhat different scope), where
students already have fixed bugs in Mozilla and PHP - unfortunately,
none of them were interested in Python. Nevertheless, this is quite
fun for both the students, and myself - especially when the Mozilla
guys do a review after 3h, the super review after 36h, and explain that
they cannot commit the fix because the code is frozen (and then do
after two weeks, well before the presentation in the class).

> What I'm looking for are some oustanding bugs which the more experienced 
> devs feel would be straightforward to implement

I'd look in the one-to-two-year old range of bugs. One risk in such
a course is that some of the developers fixes the bug while your student
is working on it, which is discouraging. How many bugs do you need?

The *really* easy ones are documentation bugs, but I expect that
students don't see this as a sufficient challenge. There are currently
roughly 100 documentation bugs open.

You can further filter out all assigned bugs; the unassigned ones
are likely resting because nobody cares (yet).

Looking at unassigned bugs starting from offset 300, I think all
of the following bugs might be suitable:
- 840065
- 839151
- 837242 (really trivial)
- 828743
- 813986

> If you feel this is off topic, please feel free to reply to me off-list.

This is clearly python-dev relevant.

Regards,
Martin
From jcarlson at uci.edu  Sun Dec 19 19:19:32 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Sun Dec 19 19:28:01 2004
Subject: [Python-Dev] mmap feature or bug?
In-Reply-To: <2mis6yl00g.fsf@starship.python.net>
References: <20041218130908.9236.JCARLSON@uci.edu>
	<2mis6yl00g.fsf@starship.python.net>
Message-ID: <20041219101536.923B.JCARLSON@uci.edu>


Michael Hudson <mwh@python.net> wrote:
> 
> Josiah Carlson <jcarlson@uci.edu> writes:
> > Is one's inability to use a = mmapinstance.__setslice__;a(1,2,'a') (and
> > others) a feature or bug?
> 
> I'm not entirely sure about this, though.  Does 
> 
> mmapinstance.__setitem__(slice(1,2), 'a')
> 
> work?  AIUI __setslice__ and __getslice__ are out of favour these days
> (not sure though).
> 
> Any patch to fix one of these would probably also fix the other almost
> as a sside-effect...

Accessing mmapi.__setslice__, mmapi.__getslice__, mmapi.__getitem__ and
mmapi.__setitem__ fail with attribute errors.


 - Josiah

From martin at v.loewis.de  Sun Dec 19 23:16:57 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun Dec 19 23:16:58 2004
Subject: [Python-Dev] MS VC compiler versions
In-Reply-To: <41B08186.5070600@egenix.com>
References: <41ADDED2.30707@egenix.com> <41AE1367.5000805@activestate.com>
	<41B08186.5070600@egenix.com>
Message-ID: <41C5FDD9.1030907@v.loewis.de>

M.-A. Lemburg wrote:
> Here's an update of the list I currently have:

I have

* SDK Itanium compiler

Microsoft (R) C/C++ Optimizing Compiler Version 13.10.2240.8 for IA-64
Copyright (C) Microsoft Corporation 1984-2002. All rights reserved.

Regards,
Martin
From martin at v.loewis.de  Mon Dec 20 08:47:37 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon Dec 20 08:47:37 2004
Subject: [Python-Dev] MS VC compiler versions
In-Reply-To: <41C68119.8070405@egenix.com>
References: <41ADDED2.30707@egenix.com>
	<41AE1367.5000805@activestate.com>	<41B08186.5070600@egenix.com>
	<41C5FDD9.1030907@v.loewis.de> <41C68119.8070405@egenix.com>
Message-ID: <41C68399.1030704@v.loewis.de>

M.-A. Lemburg wrote:
> Thanks, Martin. Could you also give me the official version
> number of the compiler or SDK ? 

It is "Windows Server 2003 Platform SDK, February 2003".

Regards,
Martin
From arigo at tunes.org  Mon Dec 20 13:31:32 2004
From: arigo at tunes.org (Armin Rigo)
Date: Mon Dec 20 13:41:14 2004
Subject: [Python-Dev] 
	Re: [Python-checkins] python/dist/src/Python marshal.c, 1.79, 1.80
In-Reply-To: <E1CgMbv-00009V-PD@sc8-pr-cvs1.sourceforge.net>
References: <E1CgMbv-00009V-PD@sc8-pr-cvs1.sourceforge.net>
Message-ID: <20041220123132.GA14073@vicky.ecs.soton.ac.uk>

Hi!

A single-character diff...

On Mon, Dec 20, 2004 at 04:25:59AM -0800, arigo@users.sourceforge.net wrote:
> --- marshal.c	27 Jun 2004 16:51:46 -0000	1.79
> +++ marshal.c	20 Dec 2004 12:25:57 -0000	1.80
> @@ -893,7 +893,7 @@
>  {
>  	PyObject *x;
>  	int version = Py_MARSHAL_VERSION;
> -	if (!PyArg_ParseTuple(args, "O|i:dumps", &x, version))
> +	if (!PyArg_ParseTuple(args, "O|i:dumps", &x, &version))
>  		return NULL;
>  	return PyMarshal_WriteObjectToString(x, version);
>  }

Crash.  Which means that there is no way in 2.4.0 to marshal an object in the
old version format as a string -- you'd have to work around by writing a real
file and reading it back :-(


Armin
From aahz at pythoncraft.com  Mon Dec 20 18:17:53 2004
From: aahz at pythoncraft.com (Aahz)
Date: Mon Dec 20 18:18:30 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	marshal.c, 1.79, 1.80
In-Reply-To: <20041220123132.GA14073@vicky.ecs.soton.ac.uk>
References: <E1CgMbv-00009V-PD@sc8-pr-cvs1.sourceforge.net>
	<20041220123132.GA14073@vicky.ecs.soton.ac.uk>
Message-ID: <20041220171753.GA26855@panix.com>

On Mon, Dec 20, 2004, Armin Rigo wrote:
>
> Hi!
> 
> A single-character diff...
> 
> On Mon, Dec 20, 2004 at 04:25:59AM -0800, arigo@users.sourceforge.net wrote:
>> --- marshal.c	27 Jun 2004 16:51:46 -0000	1.79
>> +++ marshal.c	20 Dec 2004 12:25:57 -0000	1.80
>> @@ -893,7 +893,7 @@
>>  {
>>  	PyObject *x;
>>  	int version = Py_MARSHAL_VERSION;
>> -	if (!PyArg_ParseTuple(args, "O|i:dumps", &x, version))
>> +	if (!PyArg_ParseTuple(args, "O|i:dumps", &x, &version))
>>  		return NULL;
>>  	return PyMarshal_WriteObjectToString(x, version);
>>  }
> 
> Crash.  Which means that there is no way in 2.4.0 to marshal an object in the
> old version format as a string -- you'd have to work around by writing a real
> file and reading it back :-(

Brown bag time?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing."  --Alan Perlis
From raymond.hettinger at verizon.net  Mon Dec 20 18:48:53 2004
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Mon Dec 20 18:52:42 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Pythonmarshal.c, 1.79, 1.80
Message-ID: <000001c4e6bc$483e5060$e0ae958d@oemcomputer>

[Armin]
> Crash.? Which means that there is no way in 2.4.0 to marshal an object
in the
> > old version format as a string -- you'd have to work around by
writing a real
> > file and reading it back :-(

[Aahz]
> Brown bag time?

Perhaps a rather quick Py2.4.1 would be in order.
?
Ideally, it should include other?important fixes:
?
*?Tim's critical thread bug
?? http://www.python.org/sf/1069160?
?
*?Compile of _socket fails on 2.4 (SGI Orion only)
?? http://www.python.org/sf/1086642
?? (waiting for a developer who has this set-up)
?
*?sys.stdin segfaults on invalid stdin
?? http://www.python.org/sf/1084766
?? (Jeff Epler has already isolated the problem)
?
* truncated gzip file triggers zlibmodule segfault
?? http://www.python.org/sf/1083110
?? (assigned to Andrew)
?
* various installer issues (all currently assigned to Martin)
?
* bad arg type to isspace in struct module
?? http://www.python.org/sf/1072182
?? (assigned to me)
?
* fix bug in StringIO.truncate - length not changed
?? http://www.python.org/sf/951915
?? (assigned to me)
?
* Fix for off-by-one bug in urllib.URLopener.retrieve
?? http://www.python.org/sf/810023
?? (assigned to me)
?
?
Raymond


From barry at python.org  Mon Dec 20 21:03:13 2004
From: barry at python.org (Barry Warsaw)
Date: Mon Dec 20 21:03:17 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Pythonmarshal.c, 1.79, 1.80
In-Reply-To: <000001c4e6bc$483e5060$e0ae958d@oemcomputer>
References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer>
Message-ID: <1103572993.6572.145.camel@geddy.wooz.org>

On Mon, 2004-12-20 at 12:48, Raymond Hettinger wrote:

> Perhaps a rather quick Py2.4.1 would be in order.

+1

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20041220/38908ce2/attachment.pgp
From fredrik at pythonware.com  Mon Dec 20 21:07:06 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon Dec 20 21:07:29 2004
Subject: [Python-Dev] Re: Re:
	[Python-checkins]python/dist/src/Pythonmarshal.c, 1.79, 1.80
References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer>
Message-ID: <cq7bd8$bmh$1@sea.gmane.org>

Raymond Hettinger wrote:

> Perhaps a rather quick Py2.4.1 would be in order.

sounds like a good idea.  +1 from here.

</F> 



From jhylton at gmail.com  Mon Dec 20 21:55:26 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Mon Dec 20 21:55:29 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Pythonmarshal.c, 1.79, 1.80
In-Reply-To: <1103572993.6572.145.camel@geddy.wooz.org>
References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer>
	<1103572993.6572.145.camel@geddy.wooz.org>
Message-ID: <e8bf7a530412201255781556b7@mail.gmail.com>

On Mon, 20 Dec 2004 15:03:13 -0500, Barry Warsaw <barry@python.org> wrote:
> On Mon, 2004-12-20 at 12:48, Raymond Hettinger wrote:
> 
> > Perhaps a rather quick Py2.4.1 would be in order.
> 
> +1

Nothing wrong with an incremental release, but none of these sound
like critical bugs to me.

Jeremy
From irmen at xs4all.nl  Mon Dec 20 22:32:57 2004
From: irmen at xs4all.nl (Irmen de Jong)
Date: Mon Dec 20 22:33:00 2004
Subject: [Python-Dev] Latex problem when trying to build documentation
Message-ID: <41C74509.4000109@xs4all.nl>

I wanted to create a doc patch belonging to my patch #1062060,
but got this error when trying to build the documentation from CVS:

....
(/opt/PythonBugDay/python/dist/src/Doc/commontex/reportingbugs.tex
Underfull \hbox (badness 10000) in paragraph at lines 13--17
[]\OT1/ptm/m/n/10 All bug re-ports should be sub-mit-ted via the Python Bug Tra
cker on Source-Forge
) [783] [784]
Appendix C.
(/opt/PythonBugDay/python/dist/src/Doc/commontex/license.tex [785] [786]
[787]
Underfull \hbox (badness 10000) in paragraph at lines 294--297
\OT1/ptm/m/n/10 The []\OT1/pcr/m/n/10 random \OT1/ptm/m/n/10 mod-ule in-cludes
code based on a down-load from

Overfull \hbox (4.8973pt too wide) in paragraph at lines 339--339
[]\OT1/pcr/m/n/9 A PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE CO
PYRIGHT OWNER OR[]
[788]
Overfull \hbox (4.8973pt too wide) in paragraph at lines 376--376
[]\OT1/pcr/m/n/9 FOR GAI_ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
OR CONSEQUENTIAL[]

Overfull \hbox (4.8973pt too wide) in paragraph at lines 376--376
[]\OT1/pcr/m/n/9 HOWEVER CAUSED AND ON GAI_ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT[]
[789] [790] [791] [792] [793] [794])
Underfull \hbox (badness 10000) in paragraph at lines 616--363

(./modlib.ind [795] [796] [797]) (./lib.ind [798] [799])

Package longtable Warning: Table widths have changed. Rerun LaTeX.


! LaTeX Error: \begin{description} on input line 59 ended by \end{document}.

See the LaTeX manual or LaTeX Companion for explanation.
Type  H <return>  for immediate help.
  ...

l.380 \end{document}

?
! Emergency stop.
  ...

l.380 \end{document}

Output written on lib.dvi (805 pages, 2885748 bytes).
Transcript written on lib.log.
*** Session transcript and error messages are in 
/opt/PythonBugDay/python/dist/src/Doc/html/lib/lib.how.
*** Exited with status 1.
make: *** [html/lib/lib.html] Error 1


I dont really understand that latex stuff (used to, but that
was a long time ago). What's the problem?

I'm running this on Mandrake Linux 10.0 with Tetex 2.0.2.

Regards,

Irmen de Jong.
From irmen at xs4all.nl  Mon Dec 20 22:44:22 2004
From: irmen at xs4all.nl (Irmen de Jong)
Date: Mon Dec 20 22:44:24 2004
Subject: [Python-Dev] Re:
	[Python-checkins]	python/dist/src/Pythonmarshal.c, 1.79, 1.80
In-Reply-To: <000001c4e6bc$483e5060$e0ae958d@oemcomputer>
References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer>
Message-ID: <41C747B6.6040500@xs4all.nl>

Raymond Hettinger wrote:
> Perhaps a rather quick Py2.4.1 would be in order.
>  
> Ideally, it should include other important fixes:
[...]
> * Fix for off-by-one bug in urllib.URLopener.retrieve
>    http://www.python.org/sf/810023
>    (assigned to me)

Is http://www.python.org/sf/1062060 perhaps of similar importance?
(fix for urllib.urlretrieve silently truncating download)

--Irmen
From python at rcn.com  Mon Dec 20 23:25:34 2004
From: python at rcn.com (Raymond Hettinger)
Date: Mon Dec 20 23:28:33 2004
Subject: [Python-Dev] Re:[Python-checkins]	python/dist/src/Pythonmarshal.c,
	1.79, 1.80
In-Reply-To: <41C747B6.6040500@xs4all.nl>
Message-ID: <002501c4e6e2$d30787e0$e841fea9@oemcomputer>

> Raymond Hettinger wrote:
> > Perhaps a rather quick Py2.4.1 would be in order.
> >
> > Ideally, it should include other important fixes:
> [...]
> > * Fix for off-by-one bug in urllib.URLopener.retrieve
> >    http://www.python.org/sf/810023
> >    (assigned to me)
> 
> Is http://www.python.org/sf/1062060 perhaps of similar importance?
> (fix for urllib.urlretrieve silently truncating download)

That seems reasonable to me.   There is no point in having the error
pass silently.



Raymond

From irmen at xs4all.nl  Mon Dec 20 23:44:31 2004
From: irmen at xs4all.nl (Irmen de Jong)
Date: Mon Dec 20 23:44:33 2004
Subject: [Python-Dev] Re:[Python-checkins]	python/dist/src/Pythonmarshal.c,
	1.79, 1.80
In-Reply-To: <002501c4e6e2$d30787e0$e841fea9@oemcomputer>
References: <002501c4e6e2$d30787e0$e841fea9@oemcomputer>
Message-ID: <41C755CF.2030305@xs4all.nl>

Raymond Hettinger wrote:

>>>* Fix for off-by-one bug in urllib.URLopener.retrieve
>>>   http://www.python.org/sf/810023
>>>   (assigned to me)
>>
>>Is http://www.python.org/sf/1062060 perhaps of similar importance?
>>(fix for urllib.urlretrieve silently truncating download)
> 
> 
> That seems reasonable to me.   There is no point in having the error
> pass silently.

Well, I wanted to make the patches that Johannes suggested, but
ran into trouble when building the Python docs from CVS source
(see my other message about this).

Also, I'm not sure how a test-case should be constructed
for this patch? Can the Python regression test download stuff
as part of a test? Or is there some other way to make a
testcase for this.


--Irmen
From tcdelaney at optusnet.com.au  Tue Dec 21 00:00:09 2004
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Tue Dec 21 00:00:24 2004
Subject: [Python-Dev] Re:[Python-checkins]	python/dist/src/Pythonmarshal.c,
	1.79, 1.80
References: <002501c4e6e2$d30787e0$e841fea9@oemcomputer>
	<41C755CF.2030305@xs4all.nl>
Message-ID: <005b01c4e6e7$a819bf30$4c00a8c0@ryoko>

Irmen de Jong wrote:

> Also, I'm not sure how a test-case should be constructed
> for this patch? Can the Python regression test download stuff
> as part of a test? Or is there some other way to make a
> testcase for this.

Hmm - perhaps start a server on the local machine at the start of the test, 
and tear it down at the end? you've then got full control of that server and 
can make it do whatever you want.

Or replace the socket objects with mock objects?

Tim Delaney 



From aahz at pythoncraft.com  Tue Dec 21 00:25:39 2004
From: aahz at pythoncraft.com (Aahz)
Date: Tue Dec 21 00:25:41 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Pythonmarshal.c, 1.79, 1.80
In-Reply-To: <e8bf7a530412201255781556b7@mail.gmail.com>
References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer>
	<1103572993.6572.145.camel@geddy.wooz.org>
	<e8bf7a530412201255781556b7@mail.gmail.com>
Message-ID: <20041220232539.GA10918@panix.com>

On Mon, Dec 20, 2004, Jeremy Hylton wrote:
> On Mon, 20 Dec 2004 15:03:13 -0500, Barry Warsaw <barry@python.org> wrote:
>> On Mon, 2004-12-20 at 12:48, Raymond Hettinger wrote:
>>> 
>>> Perhaps a rather quick Py2.4.1 would be in order.
>> 
>> +1
> 
> Nothing wrong with an incremental release, but none of these sound
> like critical bugs to me.

You don't think a blowup in marshal is critical?  Mind expanding on
that?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing."  --Alan Perlis
From jhylton at gmail.com  Tue Dec 21 01:05:05 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Tue Dec 21 01:05:09 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Pythonmarshal.c, 1.79, 1.80
In-Reply-To: <20041220232539.GA10918@panix.com>
References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer>
	<1103572993.6572.145.camel@geddy.wooz.org>
	<e8bf7a530412201255781556b7@mail.gmail.com>
	<20041220232539.GA10918@panix.com>
Message-ID: <e8bf7a53041220160573d0e99e@mail.gmail.com>

On Mon, 20 Dec 2004 18:25:39 -0500, Aahz <aahz@pythoncraft.com> wrote:
> On Mon, Dec 20, 2004, Jeremy Hylton wrote:
> > On Mon, 20 Dec 2004 15:03:13 -0500, Barry Warsaw <barry@python.org> wrote:
> >> On Mon, 2004-12-20 at 12:48, Raymond Hettinger wrote:
> >>>
> >>> Perhaps a rather quick Py2.4.1 would be in order.
> >>
> >> +1
> >
> > Nothing wrong with an incremental release, but none of these sound
> > like critical bugs to me.
> 
> You don't think a blowup in marshal is critical?  Mind expanding on
> that?

An undocumented extension to marshal causes a segfault.  It's
certainly a bug worth fixing.  It doesn't sound like a critical bug to
me.

Jeremy
From tim.peters at gmail.com  Tue Dec 21 01:26:25 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Dec 21 01:26:29 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Pythonmarshal.c, 1.79, 1.80
In-Reply-To: <e8bf7a53041220160573d0e99e@mail.gmail.com>
References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer>
	<1103572993.6572.145.camel@geddy.wooz.org>
	<e8bf7a530412201255781556b7@mail.gmail.com>
	<20041220232539.GA10918@panix.com>
	<e8bf7a53041220160573d0e99e@mail.gmail.com>
Message-ID: <1f7befae04122016264c6a624d@mail.gmail.com>

[Jeremy Hylton on a quick 2.4.1]
>>> Nothing wrong with an incremental release, but none of these sound
>>> like critical bugs to me.

[Aahz]
>> You don't think a blowup in marshal is critical?  Mind expanding on
>> that?

[Jeremy]
> An undocumented extension to marshal causes a segfault.  It's
> certainly a bug worth fixing.  It doesn't sound like a critical bug to
> me.

The new optional ``version`` argument to marshal.dumps() is
documented.  The easiest way to see that is to look at 2.4's
marshal.dumps() docs <wink>.  Unfortunately, it was wholly untested.

Still, it's a new-in-2.4 gimmick, and no pre-2.4 code could be using
it.  I suppose Armin found a use for it in 2.4, but I'm still
scratching my head.  If ZODB doesn't already depend on it, how useful
can it be?  QED

WRT "my" critical thread bug, I asked that everyone pretend I hadn't
submitted it until a month after 2.4 was released.  That hasn't
happened yet, so I refuse to admit it exists.

FWIW, I'd press on with 2.3.5 first, while it can still attract some
volunteer effort.
From andrewm at object-craft.com.au  Tue Dec 21 09:21:44 2004
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Tue Dec 21 09:21:39 2004
Subject: [Python-Dev] Py2.4 _sre uses uninitialised memory (Bug 1088891)
Message-ID: <20041221082144.A8EE13C8E5@coffee.object-craft.com.au>

_sre.c, data_stack_grow() in Py2.4 uses realloc()'ed memory without
initialising the newly allocated memory. For complex regexps that require
additional sre stack space, this ultimately results in a core dump or
corrupted heap. Filling the newly allocated memory with 0x55 makes the
problem more obvious (dies on a reference to 0x55555558) for me.

See bug ID 1088891:

    http://sourceforge.net/tracker/index.php?func=detail&aid=1088891&group_id=5470&atid=105470

Can I be the only person who crafts diabolical regexps? Here, have a
lend of my brown paper bag...

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/
From python at dynkin.com  Tue Dec 21 13:13:12 2004
From: python at dynkin.com (George Yoshida)
Date: Tue Dec 21 13:15:51 2004
Subject: [Python-Dev] Latex problem when trying to build documentation
In-Reply-To: <41C74509.4000109@xs4all.nl>
References: <41C74509.4000109@xs4all.nl>
Message-ID: <41C81358.3080601@dynkin.com>

Irmen de Jong wrote:

 > I wanted to create a doc patch belonging to my patch #1062060,
 > but got this error when trying to build the documentation from CVS:
 > [snip]
 > I dont really understand that latex stuff (used to, but that
 > was a long time ago). What's the problem?
 >

Looks like the end tag of {description} has disappeared with this
check-in:

rhettinger@users.sourceforge.net wrote:
 > Update of /cvsroot/python/python/dist/src/Doc/lib
 > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23423
 >
 > Modified Files:
 > 	libundoc.tex
 > Log Message:
 > SF bug #1084457:  ossaudiodev no longer undocumented
 >
 >
 >
 > Index: libundoc.tex
 > ===================================================================
 > RCS file: /cvsroot/python/python/dist/src/Doc/lib/libundoc.tex,v
 > retrieving revision 1.88
 > retrieving revision 1.89
 > diff -u -d -r1.88 -r1.89
 > --- libundoc.tex	7 Aug 2004 19:21:59 -0000	1.88
 > +++ libundoc.tex	14 Dec 2004 07:19:22 -0000	1.89
 > @@ -71,11 +71,6 @@
 >  --- Convert "arbitrary" sound files to AIFF files; should probably
 >  become a tool or demo.  Requires the external program \program{sox}.
 >
 > -\item[\module{ossaudiodev}]
 > ---- Play audio data via the Open Sound System API.  This is 
usable on
 > -Linux, some flavors of BSD, and some commercial \UNIX{} platforms.
 > -\end{description}
 > -
 >
 >  \section{Obsolete \label{obsolete-modules}}


George

From niemeyer at conectiva.com  Tue Dec 21 13:43:13 2004
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Tue Dec 21 13:43:32 2004
Subject: [Python-Dev] Py2.4 _sre uses uninitialised memory (Bug 1088891)
In-Reply-To: <20041221082144.A8EE13C8E5@coffee.object-craft.com.au>
References: <20041221082144.A8EE13C8E5@coffee.object-craft.com.au>
Message-ID: <20041221124313.GA7759@burma.localdomain>

Hello Andrew,

> _sre.c, data_stack_grow() in Py2.4 uses realloc()'ed memory without
> initialising the newly allocated memory. For complex regexps that
> require additional sre stack space, this ultimately results in a core
> dump or corrupted heap. Filling the newly allocated memory with 0x55
> makes the problem more obvious (dies on a reference to 0x55555558) for
> me.

As I just reported in the bug, the problem is not initializing the
allocated memory, but acknowledging memory reallocation in certain
situations where it's reallocated outside of the main matching
function.

Have a look at the bug at http://python.org/sf/1072259 for more
information and for a patch fixing the problem.

Thanks for the report,

-- 
Gustavo Niemeyer
http://niemeyer.net
From arigo at tunes.org  Tue Dec 21 15:01:35 2004
From: arigo at tunes.org (Armin Rigo)
Date: Tue Dec 21 15:11:28 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Pythonmarshal.c, 1.79, 1.80
In-Reply-To: <1f7befae04122016264c6a624d@mail.gmail.com>
References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer>
	<1103572993.6572.145.camel@geddy.wooz.org>
	<e8bf7a530412201255781556b7@mail.gmail.com>
	<20041220232539.GA10918@panix.com>
	<e8bf7a53041220160573d0e99e@mail.gmail.com>
	<1f7befae04122016264c6a624d@mail.gmail.com>
Message-ID: <20041221140135.GA18577@vicky.ecs.soton.ac.uk>

Hi Tim,

On Mon, Dec 20, 2004 at 07:26:25PM -0500, Tim Peters wrote:
> Still, it's a new-in-2.4 gimmick, and no pre-2.4 code could be using
> it.  I suppose Armin found a use for it in 2.4, but I'm still
> scratching my head.  If ZODB doesn't already depend on it, how useful
> can it be?  QED

Some code in the 'py' lib used to use marshal to send simple objects between
the main process and a subprocess.  We ran into trouble when we extended the
idea to a subprocess that would actually run via ssh on a remote machine, and
the remote machine's Python version didn't match the local one.  The obvious
quick fix was to set the 'version' argument to 0 and pretend that it would be
fine forever.

Now that we essentially can't use this trick any more because of the 2.4.0
bug, we reverted to repr/eval, which is quite slower (and actually not
guaranteed to work across Python versions either: string escapes sometimes
change).  We avoid to use cPickle because we want to be sure that only "simple
enough" objects are sent over this way -- essentially nothing that depends on
a precise Python module to be installed and identical on both machines.


Armin
From pycon at python.org  Tue Dec 21 15:57:18 2004
From: pycon at python.org (Steve Holden)
Date: Tue Dec 21 15:57:19 2004
Subject: [Python-Dev] PyCon is coming - we need your help
Message-ID: <20041221145718.34B591E4002@bag.python.org>


Dear Python User:

I wonder if you would be kind enough to take the time to
read this email and help us to publicize PyCon DC 2005,
being held March 23-26 at the Cafritz Conference Center of
George Washington University.

The Call for Participation went out some time ago, but it
is a good time to remind people that the deadline for
submissions is December 31. If you personally are thinking
of submitting a paper then this can be a reminder to do so
soon!

We already have acceptances from two keynote speakers, and
hope to announce them when a third is finalised shortly.

As always you can find out about the conference at

    http://www.pycon.org/
    http://www.python.org/pycon/

This year we are going to be able to accept credit card
payments for the first time, which we hope will be more
convenient for delegates.

The registration fees this year are the same as for 2004:

    Early Bird (to Jan 22)   $175 ($125 student)
    Regular (to Mar 19)      $250 ($175 student)
    On-Site                  $300 ($225 student)

A further announcment will be made when the registration
site opens for business. In the meantime I would appreciate
your assistance in posting this message via any channels you
know of that have an interest in the Python language and its
applications - publicity is the key to getting the most
diverse group of people at PyCon.


regards
Steve Holden
Chairman, PyCON DC 2005
-- 
PyCon DC 2005: The third Python Community Conference
http://www.pycon.org/   http://www.python.org/pycon/
The scoop on Python implementations and applications
From fredrik at pythonware.com  Tue Dec 21 16:21:28 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue Dec 21 16:21:41 2004
Subject: [Python-Dev] Re: [Python-checkins]python/dist/src/Pythonmarshal.c,
	1.79, 1.80
References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer><1103572993.6572.145.camel@geddy.wooz.org><e8bf7a530412201255781556b7@mail.gmail.com><20041220232539.GA10918@panix.com><e8bf7a53041220160573d0e99e@mail.gmail.com><1f7befae04122016264c6a624d@mail.gmail.com>
	<20041221140135.GA18577@vicky.ecs.soton.ac.uk>
Message-ID: <cq9f1l$2rs$1@sea.gmane.org>

Armin Rigo wrote:

> Some code in the 'py' lib used to use marshal to send simple objects between
> the main process and a subprocess.  We ran into trouble when we extended the
> idea to a subprocess that would actually run via ssh on a remote machine, and
> the remote machine's Python version didn't match the local one.  The obvious
> quick fix was to set the 'version' argument to 0 and pretend that it would be
> fine forever.

this is a rather common use case.

</F> 



From irmen at xs4all.nl  Tue Dec 21 19:57:43 2004
From: irmen at xs4all.nl (Irmen de Jong)
Date: Tue Dec 21 19:57:44 2004
Subject: [Python-Dev] Latex problem when trying to build documentation
In-Reply-To: <41C81358.3080601@dynkin.com>
References: <41C74509.4000109@xs4all.nl> <41C81358.3080601@dynkin.com>
Message-ID: <41C87227.1070107@xs4all.nl>

George Yoshida wrote:
> Irmen de Jong wrote:
> 
>  > I wanted to create a doc patch belonging to my patch #1062060,
>  > but got this error when trying to build the documentation from CVS:
>  > [snip]
>  > I dont really understand that latex stuff (used to, but that
>  > was a long time ago). What's the problem?
>  >
> 
> Looks like the end tag of {description} has disappeared with this
> check-in:

[...]

Out of curiousity, how did you find this?

--Irmen
From irmen at xs4all.nl  Tue Dec 21 20:00:22 2004
From: irmen at xs4all.nl (Irmen de Jong)
Date: Tue Dec 21 20:00:25 2004
Subject: [Python-Dev] Re:[Python-checkins]	python/dist/src/Pythonmarshal.c,
	1.79, 1.80
In-Reply-To: <005b01c4e6e7$a819bf30$4c00a8c0@ryoko>
References: <002501c4e6e2$d30787e0$e841fea9@oemcomputer>	<41C755CF.2030305@xs4all.nl>
	<005b01c4e6e7$a819bf30$4c00a8c0@ryoko>
Message-ID: <41C872C6.8040505@xs4all.nl>

Tim Delaney wrote:
> Irmen de Jong wrote:
> 
>> Also, I'm not sure how a test-case should be constructed
>> for this patch? Can the Python regression test download stuff
>> as part of a test? Or is there some other way to make a
>> testcase for this.
> 
> 
> Hmm - perhaps start a server on the local machine at the start of the 
> test, and tear it down at the end? you've then got full control of that 
> server and can make it do whatever you want.
> 
> Or replace the socket objects with mock objects?

Thanks for those suggestions.
Let's see what I can concoct.

Never made (or studied in detail) a python regression test case
before so now is a good time :)

--Irmen
From jlg at dds.nl  Tue Dec 21 22:06:45 2004
From: jlg at dds.nl (Johannes Gijsbers)
Date: Tue Dec 21 22:03:38 2004
Subject: [Python-Dev] Re:[Python-checkins]	python/dist/src/Pythonmarshal.c,
	1.79, 1.80
In-Reply-To: <41C872C6.8040505@xs4all.nl>
References: <002501c4e6e2$d30787e0$e841fea9@oemcomputer>
	<41C755CF.2030305@xs4all.nl> <005b01c4e6e7$a819bf30$4c00a8c0@ryoko>
	<41C872C6.8040505@xs4all.nl>
Message-ID: <20041221210645.GB9105@authsmtp.dds.nl>

On Tue, Dec 21, 2004 at 08:00:22PM +0100, Irmen de Jong wrote:
> Thanks for those suggestions.
> Let's see what I can concoct.
> 
> Never made (or studied in detail) a python regression test case
> before so now is a good time :)

You might be especially interested in 'class urlopen_HttpTests' in
test_urllib.py and all the Mock* classes in test_urllib2.py. BTW, don't
hesitate to ask for help in the bug/patch. I'm happy to help, though I might be
a bit slow to answer.

Cheers,

Johannes
From jlg at dds.nl  Tue Dec 21 22:20:15 2004
From: jlg at dds.nl (Johannes Gijsbers)
Date: Tue Dec 21 22:17:08 2004
Subject: [Python-Dev] Latex problem when trying to build documentation
In-Reply-To: <41C87227.1070107@xs4all.nl>
References: <41C74509.4000109@xs4all.nl> <41C81358.3080601@dynkin.com>
	<41C87227.1070107@xs4all.nl>
Message-ID: <20041221212015.GC9105@authsmtp.dds.nl>

On Tue, Dec 21, 2004 at 07:57:43PM +0100, Irmen de Jong wrote:
> >Looks like the end tag of {description} has disappeared with this
> >check-in:
> 
> [...]
> 
> Out of curiousity, how did you find this?

Once you filter through all the noise LaTeX generates there's actually a pretty
clear error message:

"\begin{description} on input line 59 ended by \end{document}."

That's obviously not right. I've just fixed it, by the way.

Cheers,

Johannes
From tim.peters at gmail.com  Tue Dec 21 23:21:29 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Dec 21 23:26:46 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Pythonmarshal.c, 1.79, 1.80
In-Reply-To: <20041221140135.GA18577@vicky.ecs.soton.ac.uk>
References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer>
	<1103572993.6572.145.camel@geddy.wooz.org>
	<e8bf7a530412201255781556b7@mail.gmail.com>
	<20041220232539.GA10918@panix.com>
	<e8bf7a53041220160573d0e99e@mail.gmail.com>
	<1f7befae04122016264c6a624d@mail.gmail.com>
	<20041221140135.GA18577@vicky.ecs.soton.ac.uk>
Message-ID: <1f7befae041221142115d3fd04@mail.gmail.com>

[Armin Rigo]
> Some code in the 'py' lib used to use marshal to send simple objects between
> the main process and a subprocess.  We ran into trouble when we extended the
> idea to a subprocess that would actually run via ssh on a remote machine, and
> the remote machine's Python version didn't match the local one.  The obvious
> quick fix was to set the 'version' argument to 0 and pretend that it would be
> fine forever.

Yes, I believed you had *some* use for it <wink>.

> Now that we essentially can't use this trick any more because of the 2.4.0
> bug,

Well, you can still use 2.3.4 -- or wait for 2.4.1 -- or use your
patched 2.4.  Or use the stock 2.4, but set up a "marshal server"
running 2.3.4 <heh>.

> we reverted to repr/eval, which is quite slower (and actually not guaranteed to
> work across Python versions either: string escapes sometimes change).

Really?  The precise rules str's __repr__ uses for which escapes to
produce certainly change, but I don't recall any case outside
Unicodeland where a new string escape was ever introduced.  So, e.g.,
current Python str.__repr__() produces '\n' for a newline, while
long-ago Pythons produced '\012', but all versions of Python *accept*
either form of escape.  The biggest change of this kind was moving
from octal escapes to hex escapes in Python 2.1, but hex escapes have
always been accepted -- repr() just didn't produce them before 2.1.

>  We avoid to use cPickle because we want to be sure that only "simple
> enough" objects are sent over this way -- essentially nothing that depends on
> a precise Python module to be installed and identical on both machines.

It's possible (but irksome) to subclass pickle.py's Pickler class, and
override its save_global() and save_inst() methods.  For example,
replace them with one-liners that just raise an exception.  Then any
attempt to pickle an object requiring a specific module will raise
that exception.  But if you're worried about speed, going thru
pickle.py is significantly slower than going thru repr().
From tim.peters at gmail.com  Tue Dec 21 23:21:29 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Tue Dec 21 23:52:17 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Pythonmarshal.c, 1.79, 1.80
In-Reply-To: <20041221140135.GA18577@vicky.ecs.soton.ac.uk>
References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer>
	<1103572993.6572.145.camel@geddy.wooz.org>
	<e8bf7a530412201255781556b7@mail.gmail.com>
	<20041220232539.GA10918@panix.com>
	<e8bf7a53041220160573d0e99e@mail.gmail.com>
	<1f7befae04122016264c6a624d@mail.gmail.com>
	<20041221140135.GA18577@vicky.ecs.soton.ac.uk>
Message-ID: <1f7befae041221142115d3fd04@mail.gmail.com>

[Armin Rigo]
> Some code in the 'py' lib used to use marshal to send simple objects between
> the main process and a subprocess.  We ran into trouble when we extended the
> idea to a subprocess that would actually run via ssh on a remote machine, and
> the remote machine's Python version didn't match the local one.  The obvious
> quick fix was to set the 'version' argument to 0 and pretend that it would be
> fine forever.

Yes, I believed you had *some* use for it <wink>.

> Now that we essentially can't use this trick any more because of the 2.4.0
> bug,

Well, you can still use 2.3.4 -- or wait for 2.4.1 -- or use your
patched 2.4.  Or use the stock 2.4, but set up a "marshal server"
running 2.3.4 <heh>.

> we reverted to repr/eval, which is quite slower (and actually not guaranteed to
> work across Python versions either: string escapes sometimes change).

Really?  The precise rules str's __repr__ uses for which escapes to
produce certainly change, but I don't recall any case outside
Unicodeland where a new string escape was ever introduced.  So, e.g.,
current Python str.__repr__() produces '\n' for a newline, while
long-ago Pythons produced '\012', but all versions of Python *accept*
either form of escape.  The biggest change of this kind was moving
from octal escapes to hex escapes in Python 2.1, but hex escapes have
always been accepted -- repr() just didn't produce them before 2.1.

>  We avoid to use cPickle because we want to be sure that only "simple
> enough" objects are sent over this way -- essentially nothing that depends on
> a precise Python module to be installed and identical on both machines.

It's possible (but irksome) to subclass pickle.py's Pickler class, and
override its save_global() and save_inst() methods.  For example,
replace them with one-liners that just raise an exception.  Then any
attempt to pickle an object requiring a specific module will raise
that exception.  But if you're worried about speed, going thru
pickle.py is significantly slower than going thru repr().
From tim.peters at gmail.com  Tue Dec 21 23:21:29 2004
From: tim.peters at gmail.com (Tim Peters)
Date: Wed Dec 22 00:11:30 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Pythonmarshal.c, 1.79, 1.80
In-Reply-To: <20041221140135.GA18577@vicky.ecs.soton.ac.uk>
References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer>
	<1103572993.6572.145.camel@geddy.wooz.org>
	<e8bf7a530412201255781556b7@mail.gmail.com>
	<20041220232539.GA10918@panix.com>
	<e8bf7a53041220160573d0e99e@mail.gmail.com>
	<1f7befae04122016264c6a624d@mail.gmail.com>
	<20041221140135.GA18577@vicky.ecs.soton.ac.uk>
Message-ID: <1f7befae041221142115d3fd04@mail.gmail.com>

[Armin Rigo]
> Some code in the 'py' lib used to use marshal to send simple objects between
> the main process and a subprocess.  We ran into trouble when we extended the
> idea to a subprocess that would actually run via ssh on a remote machine, and
> the remote machine's Python version didn't match the local one.  The obvious
> quick fix was to set the 'version' argument to 0 and pretend that it would be
> fine forever.

Yes, I believed you had *some* use for it <wink>.

> Now that we essentially can't use this trick any more because of the 2.4.0
> bug,

Well, you can still use 2.3.4 -- or wait for 2.4.1 -- or use your
patched 2.4.  Or use the stock 2.4, but set up a "marshal server"
running 2.3.4 <heh>.

> we reverted to repr/eval, which is quite slower (and actually not guaranteed to
> work across Python versions either: string escapes sometimes change).

Really?  The precise rules str's __repr__ uses for which escapes to
produce certainly change, but I don't recall any case outside
Unicodeland where a new string escape was ever introduced.  So, e.g.,
current Python str.__repr__() produces '\n' for a newline, while
long-ago Pythons produced '\012', but all versions of Python *accept*
either form of escape.  The biggest change of this kind was moving
from octal escapes to hex escapes in Python 2.1, but hex escapes have
always been accepted -- repr() just didn't produce them before 2.1.

>  We avoid to use cPickle because we want to be sure that only "simple
> enough" objects are sent over this way -- essentially nothing that depends on
> a precise Python module to be installed and identical on both machines.

It's possible (but irksome) to subclass pickle.py's Pickler class, and
override its save_global() and save_inst() methods.  For example,
replace them with one-liners that just raise an exception.  Then any
attempt to pickle an object requiring a specific module will raise
that exception.  But if you're worried about speed, going thru
pickle.py is significantly slower than going thru repr().
From python at dynkin.com  Wed Dec 22 00:22:42 2004
From: python at dynkin.com (George Yoshida)
Date: Wed Dec 22 00:22:25 2004
Subject: [Python-Dev] Latex problem when trying to build documentation
In-Reply-To: <41C87227.1070107@xs4all.nl>
References: <41C74509.4000109@xs4all.nl> <41C81358.3080601@dynkin.com>
	<41C87227.1070107@xs4all.nl>
Message-ID: <41C8B042.40102@dynkin.com>

Irmen de Jong wrote:

>> Looks like the end tag of {description} has disappeared with this
>> check-in:
> 
> 
> [...]
> 
> Out of curiousity, how did you find this?
> 

Well, I periodically download daily snapshots, so I compiled TeX
files with them. The snapshot from Dec-11 was OK, but with the one
from Dec-17 I couldn't compile.

Once you're sure in what period the bug crept into the source, all
you need to do is to check every commit applied to files under
src/Doc directory in that period.

Anyway, I don't know TeX well enough to narrow down the bug from
the error log. It's too verbose for me.


George
From titus at caltech.edu  Sun Dec 19 11:13:24 2004
From: titus at caltech.edu (Titus Brown)
Date: Wed Dec 22 06:17:49 2004
Subject: [Python-Dev] Patches: 1 for the price of 10.
Message-ID: <20041219101324.GA17049@caltech.edu>

Hello all,

per previous discussion,

	http://mail.python.org/pipermail/python-dev/2004-October/049495.html

I'd like to push a trivial little patch to sgmllib (#1087808) on you
gents, in exchange for my reviews & effort etc. on 10 other patches.

Without further ado:

No-brainers:

1055159 -- a docstring+docs update to CGIHTTPServer describing already-
	existing behavior.  Recommend apply.

1037974 -- fix HTTP digest auth for broken servers, e.g. LiveJournal.
	Trivial code fix, should break nothing.  Recommend apply +
	backport.

1028908 -- JJ Lee's updates to urllib2.  Passes regr tests, by an
	original author of much of the code (I think).  Recommend apply.

901480 -- patch to urllib2.parse_http_list (bug 735248).  Works.
	Updated patch.  Recommend apply + backport.

827559 -- SimpleHTTPServer redirects to 'dir/' when asked for 'dir'.
	This behavior mimics common behavior online and fixes a problem
	with relative URLs when clicking on files within 'dir'.
	Recommend apply.

810023 -- fixes off-by-one bug in urllib reporthook.  regr tests & all.
	Good stuff.  Recommend apply + backport.

893642 -- adds allow_none option to SimpleXMLRPCServer & associated
	classes.  Doesn't change default behavior.  Recommend apply.

755670 -- modify HTMLParser to accept clearly broken HTML.
	Recommend reject.

Slightly more complicated:

1067760 -- float-->long conversion on fileobj.seek calls, rather than
	float-->int.  Permits larger floats (2.0**62) to match large
	int (2**62) arguments.  rhettinger marked as "won't fix" in
	the original bug report; this seems like a clean solution,
	tho.  Recommend apply.

755660 -- should HTMLParser fail on all bad input, or do best effort?
	I'd recommend more sweeping changes where must-fail situations
	are distinguished from fails-by-default situations.
	Alternatively take a stand and say "nein!" once and for all.
	(See my comment for more information.)

--

For no particularly good reason, all of these were tested against
the current CVS HEAD rather than 2.4.  All of them should be trivial
to backport, although I think only a few are real problems worthy
of the effort.

--

I'm kind of curious to see how this goes, I must admit ;).  Please CC
me on replies so I can listen in...

One comment to Martin: it clearly isn't worth the effort of reviewing 10
patches to push a patch the size of my sgmllib patch.  On the other
hand, it's nice to have a guarantee & it's an educational experience,
that's for sure.

A 5:1 ratio might be more reasonable, since that in practice will
mean 1 serious patch, 2 or 3 updates, and 1 drop-dead easy patch.

cheers,
--titus
From arigo at tunes.org  Wed Dec 22 13:53:37 2004
From: arigo at tunes.org (Armin Rigo)
Date: Wed Dec 22 14:03:26 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Pythonmarshal.c, 1.79, 1.80
In-Reply-To: <1f7befae041221142115d3fd04@mail.gmail.com>
References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer>
	<1103572993.6572.145.camel@geddy.wooz.org>
	<e8bf7a530412201255781556b7@mail.gmail.com>
	<20041220232539.GA10918@panix.com>
	<e8bf7a53041220160573d0e99e@mail.gmail.com>
	<1f7befae04122016264c6a624d@mail.gmail.com>
	<20041221140135.GA18577@vicky.ecs.soton.ac.uk>
	<1f7befae041221142115d3fd04@mail.gmail.com>
Message-ID: <20041222125337.GA11426@vicky.ecs.soton.ac.uk>

Hi Tim,

On Tue, Dec 21, 2004 at 05:21:29PM -0500, Tim Peters wrote:
> > we reverted to repr/eval, which is quite slower (and actually not guaranteed to
> > work across Python versions either: string escapes sometimes change).
> 
> Really?  The precise rules str's __repr__ uses for which escapes to
> produce certainly change, but I don't recall any case outside
> Unicodeland where a new string escape was ever introduced.

My mistake, I seemed to remember something along these lines but you are
obviously right.  Python 1.5.2 reads all escapes of a recent Python just fine.


Armin
From andre.duque at varig.com  Wed Dec 22 16:34:23 2004
From: andre.duque at varig.com (=?ISO-8859-1?Q?Andr=E9_Amram_Duque?=)
Date: Wed Dec 22 16:36:14 2004
Subject: [Python-Dev] Problem with ./configure (py2.3.4)
Message-ID: <41C993FF.2030907@varig.com>

I have problem with re-install python 2.3.4, when I execute ./configure 
is appear one message in config.log, follow below :
configure:1710: gcc    conftest.cc  >&5
gcc: installation problem, cannot exec `cc1plus': No such file or directory
configure:1713: $? = 1

My gnu/linux is 2.6.8-1-386(debian/sarge)


Somebody could  help me?



Regards,

-- 
Andre Amram Duque
Consultor de Sistemas
VARIG Brazilian Airlines - RIONS
GGTI - Gerencia Geral de Tecnologia da Informacao
Tel.: + 55-0XX-21-3814-5920
Fax:: + 55-0XX-21-3814-5796
e-mail: andre.duque@varig.com
 

From fredrik at pythonware.com  Wed Dec 22 17:04:04 2004
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed Dec 22 17:04:08 2004
Subject: [Python-Dev] Re: Problem with ./configure (py2.3.4)
References: <41C993FF.2030907@varig.com>
Message-ID: <cqc5ti$bov$1@sea.gmane.org>

hi andre,

>I have problem with re-install python 2.3.4, when I execute ./configure is appear one message in 
>config.log, follow below :
> configure:1710: gcc    conftest.cc  >&5
> gcc: installation problem, cannot exec `cc1plus': No such file or directory
> configure:1713: $? = 1
>
> My gnu/linux is 2.6.8-1-386(debian/sarge)
>
> Somebody could  help me?

Since this is most likely a problem with your local configuration, it's pretty much
off-topic for python-dev.  I suggest asking for help on comp.lang.python instead.

(but feel free to return if you can confirm that this really is a problem with the
./configure file, and that the problem is still there in Python 2.4).

</F> 



From jhylton at gmail.com  Wed Dec 22 23:34:20 2004
From: jhylton at gmail.com (Jeremy Hylton)
Date: Wed Dec 22 23:34:24 2004
Subject: [Python-Dev] Patches: 1 for the price of 10.
In-Reply-To: <20041219101324.GA17049@caltech.edu>
References: <20041219101324.GA17049@caltech.edu>
Message-ID: <e8bf7a5304122214347a9dd294@mail.gmail.com>

On Sun, 19 Dec 2004 02:13:24 -0800, Titus Brown <titus@caltech.edu> wrote:
> Hello all,
> 
> per previous discussion,
> 
>         http://mail.python.org/pipermail/python-dev/2004-October/049495.html
> 
> I'd like to push a trivial little patch to sgmllib (#1087808) on you
> gents, in exchange for my reviews & effort etc. on 10 other patches.

I got started on these this morning, will likely finish them tomorrow.
 It would be perverse to apply your patch last, wouldn't it?

Jeremy
From martin at v.loewis.de  Wed Dec 22 23:40:26 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed Dec 22 23:40:22 2004
Subject: [Python-Dev] Patches: 1 for the price of 10.
In-Reply-To: <e8bf7a5304122214347a9dd294@mail.gmail.com>
References: <20041219101324.GA17049@caltech.edu>
	<e8bf7a5304122214347a9dd294@mail.gmail.com>
Message-ID: <41C9F7DA.1040408@v.loewis.de>

Jeremy Hylton wrote:
> I got started on these this morning, will likely finish them tomorrow.
>  It would be perverse to apply your patch last, wouldn't it?

It turns out that Titus' patch might be more involved than he thought
it would be.

In any case, the review itself is a highly appreciated contribution.

In the hope that this progress can trigger more contributions like this,
I'll happily reduce the requirements to 5 reviews.

Regards,
Martin
From bac at ocf.berkeley.edu  Thu Dec 23 02:15:11 2004
From: bac at ocf.berkeley.edu (Brett C.)
Date: Thu Dec 23 02:15:19 2004
Subject: [Python-Dev] Patches: 1 for the price of 10.
In-Reply-To: <41C9F7DA.1040408@v.loewis.de>
References: <20041219101324.GA17049@caltech.edu>	<e8bf7a5304122214347a9dd294@mail.gmail.com>
	<41C9F7DA.1040408@v.loewis.de>
Message-ID: <41CA1C1F.8080803@ocf.berkeley.edu>

Martin v. L?wis wrote:
> Jeremy Hylton wrote:
> 
>> I got started on these this morning, will likely finish them tomorrow.
>>  It would be perverse to apply your patch last, wouldn't it?
> 
> 
> It turns out that Titus' patch might be more involved than he thought
> it would be.
> 
> In any case, the review itself is a highly appreciated contribution.
> 
> In the hope that this progress can trigger more contributions like this,
> I'll happily reduce the requirements to 5 reviews.
> 

And to help this along I am willing to officially toss my hat into the fray by 
being another person who will review a patch if someone does 5 reviews (sans 
anything that doesn't work on OS X, which leaves out Tkinter).

-Brett
From titus at caltech.edu  Thu Dec 23 02:34:23 2004
From: titus at caltech.edu (Titus Brown)
Date: Thu Dec 23 03:09:46 2004
Subject: [Python-Dev] Patches: 1 for the price of 10.
In-Reply-To: <41C9F7DA.1040408@v.loewis.de>
References: <20041219101324.GA17049@caltech.edu>
	<e8bf7a5304122214347a9dd294@mail.gmail.com>
	<41C9F7DA.1040408@v.loewis.de>
Message-ID: <20041223013423.GA24543@caltech.edu>

-> Jeremy Hylton wrote:
-> >I got started on these this morning, will likely finish them tomorrow.
-> > It would be perverse to apply your patch last, wouldn't it?
-> 
-> It turns out that Titus' patch might be more involved than he thought
-> it would be.

*shrug* that's life ;).  I stole my patch from the other HTMLParser &
thought that would be sufficient; now I'll have to fix both!

-> In any case, the review itself is a highly appreciated contribution.

It was very educational; just wish I could remember to always submit
context diffs!  <sigh>

The only patch that I think deserves some actual discussion - here or on
c.l.p, not sure which -- is patch 755660, which deals with
HTMLParser.HTMLParser.  The goal of the original submitter was to allow
subclasses of HTMLParser deal with bad HTML in a more robust way;
essentially this comes down to allowing returns from self.error() calls.

I have now come across the same problem in my work with PBP
(pbp.berlios.de): it turns out that many Web pages (such as the
SourceForge mailman admindb page...) contain errors that cause
HTMLParser to raise an exception.  It's simply not possible to reliably
change this behavior within either htmllib.HTMLParser or
HTMLParser.HTMLParser as they're currently written.  This is a big
problem for people basing packages on either HTMLParser class.

An additional problem is that both HTMLParser.HTMLParser and
htmllib.HTMLParser are based on other classes that call self.error(), so 
those base classes would have to altered to fit the new behavior.

What I proposed doing in my comment on patch 755660 was changing
HTMLParser.HTMLParser (and its base class markupbase, too) to call
_fail() when a hard failure was called for, and otherwise to call
error() and proceed parsing on an as-best-possible basis.  This wouldn't
change the *behavior* of the existing code, but would allow for it to be
overridden when necessary.

Right now the error() call is undocumented and so it's probably
ok to change what happens upon return.  As is it can leave the parser
in a borked state upon return, and that's the behavior I propose to
fix.

I'd of course be willing to do the work & submit the more involved
patch.

Your opinions are not only welcome but (as I understand it) necessary ;).

cheers,
--titus
From gvanrossum at gmail.com  Thu Dec 23 03:19:52 2004
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu Dec 23 03:19:55 2004
Subject: [Python-Dev] Python for Series 60 released
In-Reply-To: <AA11E1E902CA54468B88D06A86A81A4A0B4DA6@esebe056.ntc.nokia.com>
References: <AA11E1E902CA54468B88D06A86A81A4A0B4DA6@esebe056.ntc.nokia.com>
Message-ID: <ca471dc2041222181945d20394@mail.gmail.com>

Python runs on Nokia cell phones (the high-end ones, anyway) and has
support from Nokia!

Pretty cool all around.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

---------- Forwarded message ----------
From: Michele.Marchetti@nokia.com <Michele.Marchetti@nokia.com>
Date: Wed, 22 Dec 2004 11:59:55 +0200
Subject: python for Series 60 released
To: guido@python.org

Hello,

like promised during EuroPython I let you know about the fact that
Python for Symbian Series 60 is released.
You may find it in www.forum.nokia.com by choosing under "resources"
"Tools and SDK's" main page.

Have a nice Christmas time.
Best Regards,
Michele Marchetti
From bob at redivi.com  Thu Dec 23 03:59:21 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Dec 23 03:59:27 2004
Subject: [Python-Dev] Python for Series 60 released
In-Reply-To: <ca471dc2041222181945d20394@mail.gmail.com>
References: <AA11E1E902CA54468B88D06A86A81A4A0B4DA6@esebe056.ntc.nokia.com>
	<ca471dc2041222181945d20394@mail.gmail.com>
Message-ID: <A544CCC8-548E-11D9-B983-000A9567635C@redivi.com>

Merry Christmas to me!  I actually purchased a Nokia Series 60 phone 
when this was first announced, in hopes that this would be available 
soon.  A littler later than I'd have liked, but better than never :)

-bob

On Dec 22, 2004, at 9:19 PM, Guido van Rossum wrote:

> Python runs on Nokia cell phones (the high-end ones, anyway) and has
> support from Nokia!
>
> Pretty cool all around.
>
> -- 
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>
> ---------- Forwarded message ----------
> From: Michele.Marchetti@nokia.com <Michele.Marchetti@nokia.com>
> Date: Wed, 22 Dec 2004 11:59:55 +0200
> Subject: python for Series 60 released
> To: guido@python.org
>
> Hello,
>
> like promised during EuroPython I let you know about the fact that
> Python for Symbian Series 60 is released.
> You may find it in www.forum.nokia.com by choosing under "resources"
> "Tools and SDK's" main page.
>
> Have a nice Christmas time.
> Best Regards,
> Michele Marchetti
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/bob%40redivi.com

From kbk at shore.net  Thu Dec 23 05:09:12 2004
From: kbk at shore.net (Kurt B. Kaiser)
Date: Thu Dec 23 05:09:42 2004
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <200412230409.iBN49ClA021597@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  257 open ( -2) /  2715 closed ( +8) /  2972 total ( +6)
Bugs    :  807 open (-15) /  4717 closed (+32) /  5524 total (+17)
RFE     :  163 open ( +3) /   139 closed ( +0) /   302 total ( +3)

New / Reopened Patches
______________________

sgmllib.SGMLParser does not unescape attribute values; patch  (2004-12-18)
       http://python.org/sf/1087808  opened by  Titus Brown

Patch for bug 1088077  (2004-12-19)
       http://python.org/sf/1088078  opened by  Mike Meyer

msi.py patch to build mingw library  (2004-12-20)
CLOSED http://python.org/sf/1088716  opened by  Paul Moore

acknowledge signals in non-main threads  (2004-12-21)
       http://python.org/sf/1088832  opened by  Andrew Langmead

Patches Closed
______________

CGIHTTPServer can't redirect  (2004-10-27)
       http://python.org/sf/1055159  closed by  jhylton

urllib2 HTTP digest authentication fix  (2004-09-30)
       http://python.org/sf/1037974  closed by  jhylton

fix bsddb memory leaks  (2004-06-06)
       http://python.org/sf/967763  closed by  greg

fast dictionary lookup by name  (2002-09-07)
       http://python.org/sf/606098  closed by  rhettinger

add time elapsed to gc debug output  (2003-06-26)
       http://python.org/sf/760990  closed by  rhettinger

Argument passing from /usr/bin/idle2.3 to idle.py  (2003-11-30)
       http://python.org/sf/851459  closed by  jafo

fix bug in StringIO.truncate - length not changed  (2004-05-11)
       http://python.org/sf/951915  closed by  rhettinger

msi.py patch to build mingw library  (2004-12-21)
       http://python.org/sf/1088716  closed by  loewis

New / Reopened Bugs
___________________

inspect.py module  (2004-12-18)
CLOSED http://python.org/sf/1087551  opened by  sprasanna199

mmap instance method access bug  (2004-12-18)
       http://python.org/sf/1087735  opened by  Josiah Carlson

Mac: make frameworkinstall skips docs, scripts  (2004-12-19)
       http://python.org/sf/1087737  opened by  Jack Jansen

example code not working  (2004-12-19)
CLOSED http://python.org/sf/1087975  opened by  perica

CGIHTTPServer: directories/scripts with spaces in their name  (2004-12-19)
       http://python.org/sf/1088039  opened by  LT

[PATCH] tty needs a way to restore the terminal mode.  (2004-12-19)
       http://python.org/sf/1088077  opened by  Mike Meyer

Comments regarding 'macintosh' behaviour wrong for MacOS X  (2004-12-20)
CLOSED http://python.org/sf/1088119  opened by  James Matthew Farrow

zlib decompressobj documentation typo  (2004-12-20)
CLOSED http://python.org/sf/1088206  opened by  Scott David Daniels

calculation wrong rounding  (2004-12-20)
CLOSED http://python.org/sf/1088563  opened by  Sebastian Rockel

_sre.c references uninitialised memory  (2004-12-21)
CLOSED http://python.org/sf/1088891  opened by  Andrew McNamara

need siginterrupt()  on Linux - impossible to do timeouts  (2004-12-21)
       http://python.org/sf/1089358  opened by  Jason

segfault/assert in tokenizer  (2004-12-21)
       http://python.org/sf/1089395  opened by  Walter D?rwald

Carbon.Res misses GetIndString  (2004-12-21)
       http://python.org/sf/1089399  opened by  Jack Jansen

Carbon.File.FSCatalogInfo.createDate implementation  (2004-12-22)
       http://python.org/sf/1089624  opened by  Ronald Oussoren

_DummyThread() objects not freed from threading._active map  (2004-12-22)
       http://python.org/sf/1089632  opened by  saravanand

Carbon.File.FSCatalogInfo.createDate implementation  (2004-12-22)
CLOSED http://python.org/sf/1089643  opened by  Ronald Oussoren

special methods become static  (2004-11-14)
       http://python.org/sf/1066490  reopened by  kquick

mmap missing offset parameter  (2004-12-22)
       http://python.org/sf/1089974  opened by  James Y Knight

exec scoping problem  (2004-12-22)
       http://python.org/sf/1089978  opened by  Kevin Quick

Defaults in ConfigParser.get overrides section values  (2004-12-22)
       http://python.org/sf/1090076  opened by  Gabriel Genellina

presentation typo in lib: 6.21.4.2 How callbacks are called  (2004-12-22)
       http://python.org/sf/1090139  opened by  Jesse Weinstein

Bugs Closed
___________

Tests fail instead of skip  (2004-12-11)
       http://python.org/sf/1083645  closed by  bcannon

readline module doesn't build on MacOSX  (2004-12-07)
       http://python.org/sf/1081045  closed by  bcannon

Python 2.4 crashes  (2004-12-12)
       http://python.org/sf/1083793  closed by  axel_kaiser

inspect.py module  (2004-12-18)
       http://python.org/sf/1087551  closed by  jlgijsbers

Possible error during LINKCC check in Configure.in  (2004-06-25)
       http://python.org/sf/980127  closed by  bcannon

example code not working  (2004-12-19)
       http://python.org/sf/1087975  closed by  rhettinger

Shared object modules in Windows have no __file__.  (2003-10-05)
       http://python.org/sf/818315  closed by  loewis

Easier-to-create alternative Python installer for Windows  (2003-08-22)
       http://python.org/sf/793078  closed by  loewis

datetime module documentation missing critical detail  (2004-12-17)
       http://python.org/sf/1087216  closed by  rhettinger

"Limitations" section of profiler docs is incorrect  (2004-11-15)
       http://python.org/sf/1066607  closed by  rhettinger

Incorrect error message (somewhat)  (2004-12-04)
       http://python.org/sf/1079011  closed by  rhettinger

test_shelve failures  (2004-12-01)
       http://python.org/sf/1076819  closed by  nijel

input from numeric pad always dropped when numlock off  (2004-11-27)
       http://python.org/sf/1074333  closed by  kbk

mkarg undocumented  (2004-11-24)
       http://python.org/sf/1072410  closed by  rhettinger

C:\Python24\Lib\compileall.py returns False  (2004-11-19)
       http://python.org/sf/1069409  closed by  kbk

Comments regarding 'macintosh' behaviour wrong for MacOS X  (2004-12-19)
       http://python.org/sf/1088119  closed by  bcannon

zlib decompressobj documentation typo  (2004-12-19)
       http://python.org/sf/1088206  closed by  rhettinger

socket object method &quot;makefile&quot; has wrong doc  (2003-11-03)
       http://python.org/sf/835300  closed by  rhettinger

calculation wrong rounding  (2004-12-20)
       http://python.org/sf/1088563  closed by  tim_one

_sre.c references uninitialised memory  (2004-12-21)
       http://python.org/sf/1088891  closed by  niemeyer

Incorrect behaviour of StreamReader.readline leads to crash  (2004-12-01)
       http://python.org/sf/1076985  closed by  doerwalter

SSL-ed sockets don't close correct?  (2004-06-24)
       http://python.org/sf/978833  closed by  bcannon

Carbon.File.FSCatalogInfo.createDate implementation  (2004-12-22)
       http://python.org/sf/1089643  closed by  ronaldoussoren

Installation ends prematurely  (2004-12-14)
       http://python.org/sf/1085208  closed by  loewis

python-2.4.msi install error  (2004-12-05)
       http://python.org/sf/1079545  closed by  loewis

Windows 2.4c1 installer default location issues  (2004-11-28)
       http://python.org/sf/1074873  closed by  loewis

Bad IDLE shortcut by 2.4 installer on XP  (2004-12-01)
       http://python.org/sf/1076861  closed by  loewis

Uninstaller should restore file associations if possible  (2004-12-14)
       http://python.org/sf/1085172  closed by  loewis

urllib2's HTTPPasswordMgr and uri's with port numbers  (2004-06-17)
       http://python.org/sf/974757  closed by  jhylton

wininst --install-script leaves residual files on C:  (2004-11-17)
       http://python.org/sf/1067732  closed by  theller

bdist_wininst improvements  (2004-07-09)
       http://python.org/sf/988022  closed by  theller

Installing w/o admin generates key error  (2002-08-27)
       http://python.org/sf/600952  closed by  theller

Windows installer halts  (2003-11-25)
       http://python.org/sf/848871  closed by  theller

New / Reopened RFE
__________________

long int bitwise ops speedup (patch included)  (2004-12-18)
       http://python.org/sf/1087418  opened by  Gregory Smith

subclassable mmap  (2004-12-18)
       http://python.org/sf/1087741  opened by  Josiah Carlson

optparse .error() should print options list  (2004-12-22)
       http://python.org/sf/1089955  opened by  Mike Orr

From tcdelaney at optusnet.com.au  Thu Dec 23 05:22:01 2004
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Thu Dec 23 05:22:09 2004
Subject: [Python-Dev] Python for Series 60 released
References: <AA11E1E902CA54468B88D06A86A81A4A0B4DA6@esebe056.ntc.nokia.com>
	<ca471dc2041222181945d20394@mail.gmail.com>
Message-ID: <00ab01c4e8a6$f3bab050$4c00a8c0@ryoko>

Guido van Rossum wrote:

> Python runs on Nokia cell phones (the high-end ones, anyway) and has
> support from Nokia!
>
> Pretty cool all around.

I couldn't find out which version of Python is supported - have they told 
you?

Cheers.

Tim Delaney 



From bob at redivi.com  Thu Dec 23 05:32:35 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Dec 23 05:32:41 2004
Subject: [Python-Dev] Python for Series 60 released
In-Reply-To: <00ab01c4e8a6$f3bab050$4c00a8c0@ryoko>
References: <AA11E1E902CA54468B88D06A86A81A4A0B4DA6@esebe056.ntc.nokia.com>
	<ca471dc2041222181945d20394@mail.gmail.com>
	<00ab01c4e8a6$f3bab050$4c00a8c0@ryoko>
Message-ID: <AB94CAFC-549B-11D9-B983-000A9567635C@redivi.com>


On Dec 22, 2004, at 11:22 PM, Tim Delaney wrote:

> Guido van Rossum wrote:
>
>> Python runs on Nokia cell phones (the high-end ones, anyway) and has
>> support from Nokia!
>>
>> Pretty cool all around.
>
> I couldn't find out which version of Python is supported - have they 
> told you?

Python 2.2.2 (#0, Dec  2 2004, 18:12:32)
[GCC 2.9-psion-98r2 (Symbian build 540)] on symbian_s60
Type "copyright", "credits" or "license" for more information.
Type "commands" to see the commands available in this simple line 
editor.

screenshot of a bluetooth serial session w/ the device: 
http://redivi.com/~bob/s60_python.?pdf

This is on a Series 60 v1.0 device (3650, IIRC).

-bob

From firemoth at gmail.com  Thu Dec 23 06:36:49 2004
From: firemoth at gmail.com (Timothy Fitz)
Date: Thu Dec 23 06:38:44 2004
Subject: [Python-Dev] Patches: 1 for the price of 10.
In-Reply-To: <20041219101324.GA17049@caltech.edu>
References: <20041219101324.GA17049@caltech.edu>
Message-ID: <972ec5bd04122221362a95fad7@mail.gmail.com>

> 1067760 -- float-->long conversion on fileobj.seek calls, rather than
>        float-->int.  Permits larger floats (2.0**62) to match large
>        int (2**62) arguments.  rhettinger marked as "won't fix" in
>        the original bug report; this seems like a clean solution,
>        tho.  Recommend apply.

Wouldn't this cause subtle errors when the float -> long conversion is
no longer precise? Or is this a non issue because it could only happen
when seeking on impossibly large files?
From bob at redivi.com  Thu Dec 23 07:18:33 2004
From: bob at redivi.com (Bob Ippolito)
Date: Thu Dec 23 07:18:44 2004
Subject: [Python-Dev] Patches: 1 for the price of 10.
In-Reply-To: <972ec5bd04122221362a95fad7@mail.gmail.com>
References: <20041219101324.GA17049@caltech.edu>
	<972ec5bd04122221362a95fad7@mail.gmail.com>
Message-ID: <78D0DFF7-54AA-11D9-B983-000A9567635C@redivi.com>

On Dec 23, 2004, at 12:36 AM, Timothy Fitz wrote:

>> 1067760 -- float-->long conversion on fileobj.seek calls, rather than
>>        float-->int.  Permits larger floats (2.0**62) to match large
>>        int (2**62) arguments.  rhettinger marked as "won't fix" in
>>        the original bug report; this seems like a clean solution,
>>        tho.  Recommend apply.
>
> Wouldn't this cause subtle errors when the float -> long conversion is
> no longer precise? Or is this a non issue because it could only happen
> when seeking on impossibly large files?

I think that Raymond marked as "won't fix" because automatic float -> 
integer conversion has been deprecated since Python 2.3.0 (if not 
earlier), for exactly the reason you state.  It's dumb.

 >>> range(2.7)
__main__:1: DeprecationWarning: integer argument expected, got float
[0, 1]

Apparently file.seek doesn't have this DeprecationWarning though..  
Strange, that.
 >>> f.seek(3.6)
 >>> f.tell()
3L

-bob

From gvanrossum at gmail.com  Thu Dec 23 07:39:57 2004
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu Dec 23 07:40:00 2004
Subject: [Python-Dev] Patches: 1 for the price of 10.
In-Reply-To: <78D0DFF7-54AA-11D9-B983-000A9567635C@redivi.com>
References: <20041219101324.GA17049@caltech.edu>
	<972ec5bd04122221362a95fad7@mail.gmail.com>
	<78D0DFF7-54AA-11D9-B983-000A9567635C@redivi.com>
Message-ID: <ca471dc20412222239326d0dda@mail.gmail.com>

> Apparently file.seek doesn't have this DeprecationWarning though..
> Strange, that.
>  >>> f.seek(3.6)
>  >>> f.tell()
> 3L

That's a bug. Who'll fix it?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
From ncoghlan at iinet.net.au  Thu Dec 23 12:41:20 2004
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Thu Dec 23 12:41:25 2004
Subject: [Python-Dev] Website documentation - link to descriptor information
Message-ID: <41CAAEE0.70301@iinet.net.au>

I just spent 10 minutes hunting through the Python website for this link:
http://www.python.org/doc/newstyle.html

I knew it was there somewhere, I just couldn't find the darn thing.

It turns out the major mistake I made was to start from "docs.python.org" 
instead of "www.python.org/doc".

Would it be possible for the "New-style classes" link to be added to the sidebar 
menu for the individual version's documentation pages? Or else given its own 
link on the Topic Guides page?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan@email.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.skystorm.net
From titus at caltech.edu  Thu Dec 23 06:51:33 2004
From: titus at caltech.edu (Titus Brown)
Date: Thu Dec 23 16:37:38 2004
Subject: [Python-Dev] Patches: 1 for the price of 10.
In-Reply-To: <972ec5bd04122221362a95fad7@mail.gmail.com>
References: <20041219101324.GA17049@caltech.edu>
	<972ec5bd04122221362a95fad7@mail.gmail.com>
Message-ID: <20041223055133.GB4353@caltech.edu>

-> > 1067760 -- float-->long conversion on fileobj.seek calls, rather than
-> >        float-->int.  Permits larger floats (2.0**62) to match large
-> >        int (2**62) arguments.  rhettinger marked as "won't fix" in
-> >        the original bug report; this seems like a clean solution,
-> >        tho.  Recommend apply.
-> 
-> Wouldn't this cause subtle errors when the float -> long conversion is
-> no longer precise? Or is this a non issue because it could only happen
-> when seeking on impossibly large files?

When would the float --> long conversion not be precise?  The docs
say PyLong_FromDouble takes the integer part, so it's like doing
a floor(), yes?

The current behavior is to convert directly from float to int, even
though seek() can take longs as an argument.  Thus 2.0**31 works,
2**31 works, 2**62 works, but 2.0**62 doesn't.  This seems mildly
inconsistent and prompted the guy to submit a bug report, and then
a patch.

The patch (with a bit more code context) is:

--
 if (!PyArg_ParseTuple(args, "O|i:seek", &offobj, &whence))
          return NULL;

 #if !defined(HAVE_LARGEFILE_SUPPORT)
  	offset = PyInt_AsLong(offobj);
  #else
+ 	if(PyFloat_Check(offobj)) {
+ 	  offobj = PyLong_FromDouble(PyFloat_AsDouble(offobj));
+ 	}
  	offset = PyLong_Check(offobj) ?
  		PyLong_AsLongLong(offobj) : PyInt_AsLong(offobj);
  #endif
--

so the issue only comes into play with large file support.

Heh, and I just noticed that PyLong_FromDouble returns a new reference,
so this is a memory leak, I believe.  Whoooops.  I'll fix that.

cheers,
--titus
From titus at caltech.edu  Thu Dec 23 09:28:19 2004
From: titus at caltech.edu (Titus Brown)
Date: Thu Dec 23 16:37:40 2004
Subject: [Python-Dev] Patches: 1 for the price of 10.
In-Reply-To: <ca471dc20412222239326d0dda@mail.gmail.com>
References: <20041219101324.GA17049@caltech.edu>
	<972ec5bd04122221362a95fad7@mail.gmail.com>
	<78D0DFF7-54AA-11D9-B983-000A9567635C@redivi.com>
	<ca471dc20412222239326d0dda@mail.gmail.com>
Message-ID: <20041223082819.GA2455@caltech.edu>

-> > Apparently file.seek doesn't have this DeprecationWarning though..
-> > Strange, that.
-> >  >>> f.seek(3.6)
-> >  >>> f.tell()
-> > 3L
-> 
-> That's a bug. Who'll fix it?

Added:

+         if (PyFloat_Check(offobj))
+             PyErr_Warn(PyExc_DeprecationWarning,
+                        "integer argument expected, got float" );

see attached diff.  I also attached it to the patch at SourceForge,
with a comment, just to keep the record straight.

The DeprecationWarning wasn't given because a manual PyObject -->
int/long conversion was being done, rather than the usual conversion
implicit in ParseTuple.

Regression tests run w/o problem on RH 9.0/i386.  Now you get the
correct behavior:

--
>>> f = open('/dev/zero')
>>> f.seek(5)
>>> f.tell()
5L
>>> f.seek(5.2)
__main__:1: DeprecationWarning: integer argument expected, got float
>>> f.tell()
5L
--

This seems like a reasonable resolution to patch #1067760, to me...

cheers,
--titus
-------------- next part --------------
? Objects/.fileobject.c.swp
Index: Objects/fileobject.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Objects/fileobject.c,v
retrieving revision 2.193
diff -c -c -r2.193 fileobject.c
*** Objects/fileobject.c	7 Nov 2004 14:15:28 -0000	2.193
--- Objects/fileobject.c	23 Dec 2004 08:19:20 -0000
***************
*** 462,467 ****
--- 462,472 ----
  	whence = 0;
  	if (!PyArg_ParseTuple(args, "O|i:seek", &offobj, &whence))
  		return NULL;
+ 
+         if (PyFloat_Check(offobj))
+             PyErr_Warn(PyExc_DeprecationWarning,
+                        "integer argument expected, got float" );
+         
  #if !defined(HAVE_LARGEFILE_SUPPORT)
  	offset = PyInt_AsLong(offobj);
  #else
From mwh at python.net  Thu Dec 23 17:48:17 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Dec 23 17:48:19 2004
Subject: [Python-Dev] Patches: 1 for the price of 10.
In-Reply-To: <20041223055133.GB4353@caltech.edu> (Titus Brown's message of
	"Wed, 22 Dec 2004 21:51:33 -0800")
References: <20041219101324.GA17049@caltech.edu>
	<972ec5bd04122221362a95fad7@mail.gmail.com>
	<20041223055133.GB4353@caltech.edu>
Message-ID: <2mllbpj6zy.fsf@starship.python.net>

Titus Brown <titus@caltech.edu> writes:

> -> > 1067760 -- float-->long conversion on fileobj.seek calls, rather than
> -> >        float-->int.  Permits larger floats (2.0**62) to match large
> -> >        int (2**62) arguments.  rhettinger marked as "won't fix" in
> -> >        the original bug report; this seems like a clean solution,
> -> >        tho.  Recommend apply.
> -> 
> -> Wouldn't this cause subtle errors when the float -> long conversion is
> -> no longer precise? Or is this a non issue because it could only happen
> -> when seeking on impossibly large files?
>
> When would the float --> long conversion not be precise?

When the float is so large that it is the closest approximation to
more than one integer? (i.e. larger than 2**53 for 754 doubles).

Cheers,
mwh

-- 
  #ifndef P_tmpdir
  printf( "Go buy a better computer" );
  exit( ETHESKYISFALLINGANDIWANTMYMAMA );
                         -- Dimitri Maziuk on writing secure code, asr
From mwh at python.net  Thu Dec 23 17:50:35 2004
From: mwh at python.net (Michael Hudson)
Date: Thu Dec 23 17:50:38 2004
Subject: [Python-Dev] Patches: 1 for the price of 10.
In-Reply-To: <20041223082819.GA2455@caltech.edu> (Titus Brown's message of
	"Thu, 23 Dec 2004 00:28:19 -0800")
References: <20041219101324.GA17049@caltech.edu>
	<972ec5bd04122221362a95fad7@mail.gmail.com>
	<78D0DFF7-54AA-11D9-B983-000A9567635C@redivi.com>
	<ca471dc20412222239326d0dda@mail.gmail.com>
	<20041223082819.GA2455@caltech.edu>
Message-ID: <2mhdmdj6w4.fsf@starship.python.net>

Titus Brown <titus@caltech.edu> writes:

> -> > Apparently file.seek doesn't have this DeprecationWarning though..
> -> > Strange, that.
> -> >  >>> f.seek(3.6)
> -> >  >>> f.tell()
> -> > 3L
> -> 
> -> That's a bug. Who'll fix it?
>
> Added:
>
> +         if (PyFloat_Check(offobj))
> +             PyErr_Warn(PyExc_DeprecationWarning,
> +                        "integer argument expected, got float" );
>
> see attached diff.  I also attached it to the patch at SourceForge,
> with a comment, just to keep the record straight.

You should check the return value of PyErr_Warn in case the user
passed -Werror on the command line.

> This seems like a reasonable resolution to patch #1067760, to me...

I guess I could look at that -- but when I'm not on dialup at my
parents' house...

Cheers,
mwh

-- 
  I love the way Microsoft follows standards.  In much the same
  manner that fish follow migrating caribou.           -- Paul Tomblin
               -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html
From jcarlson at uci.edu  Thu Dec 23 17:43:53 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Thu Dec 23 17:52:27 2004
Subject: [Python-Dev] Patches: 1 for the price of 10.
In-Reply-To: <20041223055133.GB4353@caltech.edu>
References: <972ec5bd04122221362a95fad7@mail.gmail.com>
	<20041223055133.GB4353@caltech.edu>
Message-ID: <20041223084029.9246.JCARLSON@uci.edu>


Titus Brown <titus@caltech.edu> wrote:
> 
> -> > 1067760 -- float-->long conversion on fileobj.seek calls, rather than
> -> >        float-->int.  Permits larger floats (2.0**62) to match large
> -> >        int (2**62) arguments.  rhettinger marked as "won't fix" in
> -> >        the original bug report; this seems like a clean solution,
> -> >        tho.  Recommend apply.
> -> 
> -> Wouldn't this cause subtle errors when the float -> long conversion is
> -> no longer precise? Or is this a non issue because it could only happen
> -> when seeking on impossibly large files?
> 
> When would the float --> long conversion not be precise?  The docs
> say PyLong_FromDouble takes the integer part, so it's like doing
> a floor(), yes?

Precision.
>>> 2.0**52
4503599627370496.0
>>> 2.0**52+1
4503599627370497.0
>>> 2.0**53
9007199254740992.0
>>> 2.0**53+1
9007199254740992.0

Anything above float(2**53-1) (one must be careful about the order of
float conversions) are basically useless for offsets into files.


> The current behavior is to convert directly from float to int, even
> though seek() can take longs as an argument.  Thus 2.0**31 works,
> 2**31 works, 2**62 works, but 2.0**62 doesn't.  This seems mildly
> inconsistent and prompted the guy to submit a bug report, and then
> a patch.

"works" and "doesn't work" aren't precise.  "doesn't raise an exception"
and "does raise an exception" are precise.  Note the above float
precision precision example about why even non-exceptions may not "work".


 - Josiah

From gvanrossum at gmail.com  Thu Dec 23 18:53:47 2004
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Thu Dec 23 18:53:50 2004
Subject: [Python-Dev] Website documentation - link to descriptor
	information
In-Reply-To: <41CAAEE0.70301@iinet.net.au>
References: <41CAAEE0.70301@iinet.net.au>
Message-ID: <ca471dc20412230953440f8eaf@mail.gmail.com>

> It turns out the major mistake I made was to start from "docs.python.org"
> instead of "www.python.org/doc".

If you ask me that wasn't your mistake but the mistake of whoever
decided to create a subdomain for docs (or for anything else).


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
From pycon at python.org  Thu Dec 23 22:25:55 2004
From: pycon at python.org (Steve Holden)
Date: Thu Dec 23 22:25:56 2004
Subject: [Python-Dev] PyCon Registration Opens Today!
Message-ID: <20041223212555.AAD4B1E400F@bag.python.org>


Dear Python User:

Following my last message, I am pleased to be able to
announce that you can register for PyCon DC 2005 on the
web at

    http://www.python.org/pycon/2005/register.html

starting at 1700 EST today (December 23). Thanks to
George Belotsky of Open Light Software for assistance
in preparing the credit card processing software.

As always, further information about PyCon is available
in the usual places:

    http://www.pycon.org/
    http://www.python.org/pycon/

I look forward to meeting you at PyCON DC 2005. In the
meantime please let me offer my best wishes for a
happy and peaceful holiday season.


regards
Steve Holden
Chairman, PyCON DC 2005
-- 
PyCon DC 2005: The third Python Community Conference
http://www.pycon.org/   http://www.python.org/pycon/
The scoop on Python implementations and applications
From barry at barrys-emacs.org  Thu Dec 23 23:20:33 2004
From: barry at barrys-emacs.org (Barry Scott)
Date: Thu Dec 23 23:21:47 2004
Subject: [Python-Dev] Build extensions for windows python 2.4 what are the
	compiler rules?
Message-ID: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org>

I see that, as expected, windows python 2.4 was built with MSVC 7.1 
rather then msvc 6.0.

It seems that I can build extensions with msvc 6.0 that work with the 
python 2.4 windows
binary kit.

Is this safe?

I recall warning a while ago about mixing msvc 6.0 and msvc 7.1 runtime 
DLL's. Is this
an issue with python 2.4?

I'm also surprised that the python 2.4 source kit only mentions MSVC 
6.0 and not the
compiler that you actually built python 2.4 with.

Barry

From martin at v.loewis.de  Fri Dec 24 00:12:42 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Dec 24 00:12:38 2004
Subject: [Python-Dev] Build extensions for windows python 2.4 what are
	the	compiler rules?
In-Reply-To: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org>
References: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org>
Message-ID: <41CB50EA.9070100@v.loewis.de>

Barry Scott wrote:
> It seems that I can build extensions with msvc 6.0 that work with the 
> python 2.4 windows
> binary kit.
> 
> Is this safe?

No, it isn't. This emerges as a Python 2.4 FAQ.

> I recall warning a while ago about mixing msvc 6.0 and msvc 7.1 runtime 
> DLL's. Is this
> an issue with python 2.4?

Yes, it is.

> I'm also surprised that the python 2.4 source kit only mentions MSVC 6.0 
> and not the compiler that you actually built python 2.4 with.

Why do you say that? PCbuild/readme.txt starts with

Building Python using VC++ 7.1
-------------------------------------
This directory is used to build Python for Win32 platforms, e.g. Windows
95, 98 and NT.  It requires Microsoft Visual C++ 7.1
(a.k.a. Visual Studio .NET 2003).
(For other Windows platforms and compilers, see ../PC/readme.txt.)

Regards,
Martin
From barry at barrys-emacs.org  Fri Dec 24 01:17:49 2004
From: barry at barrys-emacs.org (Barry Scott)
Date: Fri Dec 24 01:19:06 2004
Subject: [Python-Dev] Build extensions for windows python 2.4 what are
	the	compiler rules?
In-Reply-To: <41CB50EA.9070100@v.loewis.de>
References: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org>
	<41CB50EA.9070100@v.loewis.de>
Message-ID: <3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org>


On Dec 23, 2004, at 23:12, Martin v. L?wis wrote:

> Barry Scott wrote:
>> It seems that I can build extensions with msvc 6.0 that work with the 
>> python 2.4 windows
>> binary kit.
>> Is this safe?
>
> No, it isn't. This emerges as a Python 2.4 FAQ.
>
>> I recall warning a while ago about mixing msvc 6.0 and msvc 7.1 
>> runtime DLL's. Is this
>> an issue with python 2.4?
>
> Yes, it is.
>
>> I'm also surprised that the python 2.4 source kit only mentions MSVC 
>> 6.0 and not the compiler that you actually built python 2.4 with.
>
> Why do you say that? PCbuild/readme.txt starts with

I recursive grep'ed and missed this ref. However I did read this in 
README.TXT:

 > Building on non-UNIX systems
 > ----------------------------

 > For Windows (2000/NT/ME/98/95), assuming you have MS VC++ 6.0, the
 > project files are in PCbuild, the workspace is pcbuild.dsw.  See
 > PCbuild\readme.txt for detailed instructions.

Which says that PCbuild/readme.txt is about VC++ 6.0

> Building Python using VC++ 7.1
> -------------------------------------
> This directory is used to build Python for Win32 platforms, e.g. 
> Windows
> 95, 98 and NT.  It requires Microsoft Visual C++ 7.1
> (a.k.a. Visual Studio .NET 2003).
> (For other Windows platforms and compilers, see ../PC/readme.txt.)
>
> Regards,
> Martin

Barry

From martin at v.loewis.de  Fri Dec 24 09:29:13 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri Dec 24 09:29:08 2004
Subject: [Python-Dev] Build extensions for windows python 2.4 what are
	the	compiler rules?
In-Reply-To: <3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org>
References: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org>
	<41CB50EA.9070100@v.loewis.de>
	<3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org>
Message-ID: <41CBD359.4020007@v.loewis.de>

Barry Scott wrote:
> I recursive grep'ed and missed this ref. However I did read this in 
> README.TXT:
> 
>  > Building on non-UNIX systems
>  > ----------------------------
> 
>  > For Windows (2000/NT/ME/98/95), assuming you have MS VC++ 6.0, the
>  > project files are in PCbuild, the workspace is pcbuild.dsw.  See
>  > PCbuild\readme.txt for detailed instructions.
> 
> Which says that PCbuild/readme.txt is about VC++ 6.0

I see. This is now corrected in CVS.

Regards,
Martin
From jlg at dds.nl  Fri Dec 24 12:08:44 2004
From: jlg at dds.nl (Johannes Gijsbers)
Date: Fri Dec 24 12:05:34 2004
Subject: [Python-Dev] Website documentation - link to descriptor
	information
In-Reply-To: <41CAAEE0.70301@iinet.net.au>
References: <41CAAEE0.70301@iinet.net.au>
Message-ID: <20041224110844.GE9338@authsmtp.dds.nl>

On Thu, Dec 23, 2004 at 09:41:20PM +1000, Nick Coghlan wrote:
> I just spent 10 minutes hunting through the Python website for this link:
> http://www.python.org/doc/newstyle.html
> 
> I knew it was there somewhere, I just couldn't find the darn thing.
> 
> It turns out the major mistake I made was to start from "docs.python.org" 
> instead of "www.python.org/doc".

It's on docs.python.org now.

Cheers,

Johannes
From mwh at python.net  Fri Dec 24 12:37:23 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Dec 24 12:37:25 2004
Subject: [Python-Dev] Build extensions for windows python 2.4 what are
	the	compiler rules?
In-Reply-To: <3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org> (Barry
	Scott's message of "Fri, 24 Dec 2004 00:17:49 +0000")
References: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org>
	<41CB50EA.9070100@v.loewis.de>
	<3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org>
Message-ID: <2m1xdgj5ak.fsf@starship.python.net>

Barry Scott <barry@barrys-emacs.org> writes:

> I recursive grep'ed and missed this ref. However I did read this in
> README.TXT:

The top-level README file is hilariously out-of-date, in some ways.  I
meant to do something about this before 2.4 final, but didn't get
around to it...

Cheers,
mwh

-- 
  <exarkun> speak of the devil
  <moshez> exarkun: froor
  <exarkun> not you                             -- from Twisted.Quotes
From arigo at tunes.org  Fri Dec 24 13:32:27 2004
From: arigo at tunes.org (Armin Rigo)
Date: Fri Dec 24 13:42:30 2004
Subject: [Python-Dev] Build extensions for windows python 2.4 what are
	the	compiler rules?
In-Reply-To: <3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org>
References: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org>
	<41CB50EA.9070100@v.loewis.de>
	<3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org>
Message-ID: <20041224123227.GA12925@vicky.ecs.soton.ac.uk>

Hi,

On Fri, Dec 24, 2004 at 12:17:49AM +0000, Barry Scott wrote:
> I recursive grep'ed and missed this ref. However I did read this in 
> README.TXT:

The "extending and embedding" tutorial is similarily out-of-date.


Armin
From mwh at python.net  Fri Dec 24 14:05:55 2004
From: mwh at python.net (Michael Hudson)
Date: Fri Dec 24 14:05:57 2004
Subject: [Python-Dev] Build extensions for windows python 2.4 what are
	the	compiler rules?
In-Reply-To: <20041224123227.GA12925@vicky.ecs.soton.ac.uk> (Armin Rigo's
	message of "Fri, 24 Dec 2004 12:32:27 +0000")
References: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org>
	<41CB50EA.9070100@v.loewis.de>
	<3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org>
	<20041224123227.GA12925@vicky.ecs.soton.ac.uk>
Message-ID: <2mwtv7j170.fsf@starship.python.net>

Armin Rigo <arigo@tunes.org> writes:

> Hi,
>
> On Fri, Dec 24, 2004 at 12:17:49AM +0000, Barry Scott wrote:
>> I recursive grep'ed and missed this ref. However I did read this in 
>> README.TXT:
>
> The "extending and embedding" tutorial is similarily out-of-date.

Well, I've half re-written that (the entire thing, see doc-sig passim)
but this hasn't attracted a great deal of interest.  I've just
uploaded my most recent version to:

    http://starship.python.net/crew/mwh/toext/

I haven't done any substantial work on it in a while, and probably
won't have time to do any for some time.  It would still be nice to
have it ready in the 2.5 timeframe.

Cheers,
mwh

-- 
  A difference which makes no difference is no difference at all.
                        -- William James (I think.  Reference anyone?)
From gvanrossum at gmail.com  Fri Dec 24 16:39:06 2004
From: gvanrossum at gmail.com (Guido van Rossum)
Date: Fri Dec 24 16:39:09 2004
Subject: [Python-Dev] Merry Christmas and a Happy New Year to all Python
	developers!
Message-ID: <ca471dc2041224073912c09188@mail.gmail.com>

I'm about to travel to a place where I don't expect to have internet
access for over a week, so this is my last message in 2004. I'll have
more time next year for Python that I had this year, so I'm looking
forward to working again with this great community. I wish everyone
happy celebrations of the dark season, and I hope to see many of you
at the upcoming PyCon conference. Cheers!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
From bac at ocf.berkeley.edu  Fri Dec 24 21:43:19 2004
From: bac at ocf.berkeley.edu (Brett C.)
Date: Fri Dec 24 21:43:20 2004
Subject: [Python-Dev] proto-pep: How to change Python's bytecode
Message-ID: <41CC7F67.9070009@ocf.berkeley.edu>

After implementing over 10 new opcodes for my thesis I figured I should write 
down the basic steps in an info PEP so that there is enough guidelines with 
this PEP and PEP 306 to cover the bases on changes to the language itself.

To go along with this I also plan to write some benchmarks for individual 
opcodes that could possibly lead to a testing suite for the opcodes themselves 
(will probably do this piece-meal and put it up on SF initially since there are 
a lot of opcodes).

Anyway, let me know if I seem to be missing anything or have something to add. 
  After a reasonable time of non-response to this I will request a PEP number 
(assuming people don't think this PEP is stupid).

------------------------------------------

PEP: XXX
Title: How to change Python's bytecode
Version: $Revision: 1.4 $
Last-Modified: $Date: 2003/09/22 04:51:50 $
Author: Brett Cannoon <brett@python.org>
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: XX-XXX-XXXX
Post-History: XX-XXX-XXXX


Abstract
========

Python source code is compiled down to something called bytecode.  This
bytecode (which can be viewed as sequences of opcodes) defines what Python is
capable of.  As such, knowing how to add, remove, or change the bytecode is
important to do properly when changing the abilities of the Python language.


Rationale
=========

While changing Python's bytecode is not a frequent occurence, it still happens.
Having the required steps documented in a single location should make
experimentation with the bytecode easier since it is not necessarily obvious
what the steps are to change the bytecode.

This PEP, paired with PEP 306 [#PEP-306]_, should provide enough basic
guidelines for handling any changes performed to the Python language itself in
terms of syntactic changes that introduce new semantics.


Checklist
=========

This is a rough checklist of what files need to change and how they are
involved with the bytecode.  All paths are given from the viewpoint of
``/cvsroot/python/dist/src`` from CVS).  This list should not be considered
exhaustive nor to cover all possible situations.

- ``Include/opcode.h``
	This include file lists all known opcodes and associates each opcode
	name with
	a unique number.  When adding a new opcode it is important to take note
	of the ``HAVE_ARGUMENT`` value.  This ``#define``'s value specifies the
	value at which all opcodes that have a value greater than
	``HAVE_ARGUMENT`` are expected to take an argument to the opcode.

- ``Lib/opcode.py``
	Lists all of the opcodes and their associated value.  Used by the dis
	module [#dis]_ to map bytecode values to their names.

- ``Python/ceval.c``
	Contains the main interpreter loop.  Code to handle the evalution of an
	opcode here.

- ``Python/compile.c``
	To make sure an opcode is actually used, this file must be altered.
	The emitting of all bytecode occurs here.

- ``Lib/compiler/pyassem.py``, ``Lib/compiler/pycodegen.py``
	The 'compiler' package [#compiler]_ needs to be altered to also reflect
	any changes to the bytecode.

- ``Doc/lib/libdis.tex``
	The documentation [#dis-docs] for the dis module contains a complete
	list of all the opcodes.

- ``Python/import.c``
	Defines the magic word (named ``MAGIC``) used in .pyc files to detect if
	the bytecode used matches the one used by the version of Python running.
	This number needs to be changed to make sure that the running
	interpreter does not try to execute bytecode that it does not know
	about.


Suggestions for bytecode development
====================================

A few things can be done to make sure that development goes smoothly when
experimenting with Python's bytecode.  One is to delete all .py(c|o|w) files
after each semantic change to Python/compile.c .  That way all files will use
any bytecode changes.

Make sure to run the entire testing suite [#test-suite]_.  Since the
``regrtest.py`` driver recompiles all source code before a test is run it acts
a good test to make sure that no existing semantics are broken.

Running parrotbench [#parrotbench]_ is also a good way to make sure existing
semantics are not broken; this benchmark is practically a compliance test.


References
==========

.. [#PEP-306] PEP 306, How to Change Python's Grammar, Hudson
    (http://www.python.org/peps/pep-0306.html)

.. [#dis] XXX

.. [#test-suite] XXX

.. [#parrotbench] XXX

.. [#dis-docs] XXX


Copyright
=========

This document has been placed in the public domain.



..
    Local Variables:
    mode: indented-text
    indent-tabs-mode: nil
    sentence-end-double-space: t
    fill-column: 70
    End:
From Scott.Daniels at Acm.Org  Sat Dec 25 02:04:38 2004
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Sat Dec 25 02:03:15 2004
Subject: [Python-Dev] Re: proto-pep: How to change Python's bytecode
In-Reply-To: <41CC7F67.9070009@ocf.berkeley.edu>
References: <41CC7F67.9070009@ocf.berkeley.edu>
Message-ID: <cqie8c$jv$1@sea.gmane.org>

Brett C. wrote:
> A few things can be done to make sure that development goes smoothly when
> experimenting with Python's bytecode.  One is to delete all .py(c|o|w) 
Don't you mean ".pyc or .pyo (remember such files in zips as well)"
.pyw is normal python source.

-- Scott David Daniels
Scott.Daniels@Acm.Org

From tjreedy at udel.edu  Sat Dec 25 02:54:32 2004
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat Dec 25 02:54:37 2004
Subject: [Python-Dev] Re: proto-pep: How to change Python's bytecode
References: <41CC7F67.9070009@ocf.berkeley.edu>
Message-ID: <cqih8n$4du$1@sea.gmane.org>


"Brett C." <bac@ocf.berkeley.edu> wrote in message 
news:41CC7F67.9070009@ocf.berkeley.edu...

At to the title, bytecodes are a property of the CPython implementation, 
not of Python itself.  Since I think the distinction is quite important to 
maintain, I would insert the missing 'C' and everywhere else as 
appropriate.

> After implementing over 10 new opcodes for my thesis I figured I should 
> write down the basic steps in an info PEP so that there is enough 
> guidelines with this PEP and PEP 306 to cover the bases on changes to the 
> language itself.

Over the last several years, various people have reported experimenting 
with CPython's bytecodes.  I wonder if it would be helpful to have a 
respository of the results, in one place, for new experimenters and curious 
people to peruse.

> Anyway, let me know if I seem to be missing anything or have something to 
> add.

As I said, the importanct 'C' qualifier.

> PEP: XXX
> Title: How to change Python's bytecode

/P/CP/

> Python source code is compiled down to something called bytecode.

Suggested replacements in quotes:
"CPython compiles Python source code to something called bytecode."

> This  bytecode (which can be viewed as sequences of opcodes)
> defines what Python is capable of.

This is backwards.  "The language, as defined in the Reference Manual, 
determines what bytecodes are needed (collectively, not one by one) for a 
bytecode implementation."

>  As such, knowing how to add, remove, or change the bytecode is
> important to do properly when changing the abilities of the Python 
> language.

"Therefore, changes in the language may require changes in the set of 
bytecodes.  In addition, changing the bytecode set for a given definition 
may result in desireable changes in the interpreter behavior.  This 
document describes how to do so for either reason."

> Rationale
> =========
>
> While changing Python's bytecode is not a frequent occurence, it still 
> happens.

/P/CP/   Experiments are much more frequent than committed changes -- all 
of which start as experiments.
...

Terry J. Reedy



From bac at ocf.berkeley.edu  Sat Dec 25 20:31:06 2004
From: bac at ocf.berkeley.edu (Brett C.)
Date: Sat Dec 25 20:31:13 2004
Subject: [Python-Dev] Re: proto-pep: How to change Python's bytecode
In-Reply-To: <cqie8c$jv$1@sea.gmane.org>
References: <41CC7F67.9070009@ocf.berkeley.edu> <cqie8c$jv$1@sea.gmane.org>
Message-ID: <41CDBFFA.1060405@ocf.berkeley.edu>

Scott David Daniels wrote:
> Brett C. wrote:
> 
>> A few things can be done to make sure that development goes smoothly when
>> experimenting with Python's bytecode.  One is to delete all .py(c|o|w) 
> 
> Don't you mean ".pyc or .pyo (remember such files in zips as well)"
> .pyw is normal python source.
> 

Yeah, my bad.  Been a while since I have written a .pyw file.

-Brett
From bac at ocf.berkeley.edu  Sat Dec 25 20:36:17 2004
From: bac at ocf.berkeley.edu (Brett C.)
Date: Sat Dec 25 20:36:17 2004
Subject: [Python-Dev] Re: proto-pep: How to change Python's bytecode
In-Reply-To: <cqih8n$4du$1@sea.gmane.org>
References: <41CC7F67.9070009@ocf.berkeley.edu> <cqih8n$4du$1@sea.gmane.org>
Message-ID: <41CDC131.70702@ocf.berkeley.edu>

Terry Reedy wrote:
> "Brett C." <bac@ocf.berkeley.edu> wrote in message 
> news:41CC7F67.9070009@ocf.berkeley.edu...
> 
> At to the title, bytecodes are a property of the CPython implementation, 
> not of Python itself.  Since I think the distinction is quite important to 
> maintain, I would insert the missing 'C' and everywhere else as 
> appropriate.
> 

Then the same could be said for PEP 306.  But I don't mind changing the title.

> 
>>After implementing over 10 new opcodes for my thesis I figured I should 
>>write down the basic steps in an info PEP so that there is enough 
>>guidelines with this PEP and PEP 306 to cover the bases on changes to the 
>>language itself.
> 
> 
> Over the last several years, various people have reported experimenting 
> with CPython's bytecodes.  I wonder if it would be helpful to have a 
> respository of the results, in one place, for new experimenters and curious 
> people to peruse.
> 

Wouldn't hurt.  Adding that section would not be bad, but I don't have the 
inclination to hunt them down.  What do others think about having this section?

-Brett
From arigo at tunes.org  Sat Dec 25 21:23:03 2004
From: arigo at tunes.org (Armin Rigo)
Date: Sat Dec 25 21:33:07 2004
Subject: [Python-Dev] Upcoming PyPy sprint
Message-ID: <20041225202303.GA16023@vicky.ecs.soton.ac.uk>

Hi Python people,

The next sprint for PyPy, the Python-in-Python interpreter, will take 
place in Leysin, in the lower mountains of Switzerland,

    22nd - 29th January 2005 (travel days: 22nd and 29-30th)
    http://codespeak.net/moin/pypy/moin.cgi/LeysinSprint

The technical goals will be two-fold: we want to continue the hard work 
started at the previous Vilnius sprint (a first generated C version of our 
interpreter); but the upcoming sprint will also be a "learning PyPy" 
sprint, covering all aspects of PyPy that are easier to start with.  
Some examples are described below.  For more information about PyPy, see 
http://codespeak.net/pypy/index.cgi?doc .

The "learning PyPy" focus comes from the fact that it will be our first 
sprint as a European Union sponsored group, and not all members of this 
EU/PyPy project are already familiar with PyPy.  This is a good occasion 
for newcomers that want to look at PyPy more closely too, in a "fun and 
somewhat mind-altering" sprint event.  Note that as part of our funding we 
want to be able to support some of the travel and lodging costs for a 
number of outside people as well, but although the corresponding money is 
in the budgets, not all administrative issues have been solved yet.  We 
don't know yet if we will be able to do so for the January sprint.

However, if you would like to come to the sprint but can't afford 
travel and accomodation costs then please contact us.  Even if the
administrative issues have not been sorted out, we may be able to 
help with private money.  Just contact us or me personally for this
sprint. 

About Leysin
------------

The place where the sprint will take place is located in the pre-Alps of 
the french-speaking (west) region of Switzerland.  Leysin is a village at 
an altitude of 1200 to 1400m.  It is a skiing station, and also the place 
where I (Armin) lived for 20 years.  There are 2000 people in summer and 
10000 in winter :-) but the sprint will be just before the most crowded 
periods of February.  Of course one full day will be dedicated to winter 
sports!

Both the sprint venue and the logding will be in a very spacious pair of 
chalets built specifically for bed&breakfast: http://www.ermina.ch/.  The 
place has a baseline ADSL Internet connexion (600Kb); we need to arrange 
between ourselves to bring a wireless hub.  You can of course arrange your 
own lodging anywhere (you cannot be more than a 15 minutes walk away from 
the sprint venue), but I definitely recommend to lodge there too -- you 
won't find better sights anywhere else (though you probably won't get much 
worse ones easily, either :-)


Subscription
------------

Please subscribe at

    http://codespeak.net/moin/pypy/moin.cgi/LeysinSprintAttendants

and mention if you would like to participate in the group reservation, and
which size of room you would prefer (2 to 6 beds available).  If you wish, you
can extend your stay for some days after the 29th of January -- please mention
it if you want to book with the group (but you cannot arrive there earier
than the 22nd: fully booked).

If you have any question don't hesitate to contact 
pypy-sprint@codespeak.net or one of us personally.


Learning PyPy
-------------

Here are some goals that are quite reachable without in-depth prior 
knowledge of PyPy:

 * systematic testing: the interpreter and type implementations work 
   generally well, but there are quite some areas that need refactoring 
   and refinement.  A good approach is to run as many of CPython's tests 
   as possible and fix stuff accordingly.  This is a good way to learn 
   random parts of PyPy until you know them all!

 * extension modules and extension types: some can be just rewritten as a 
   plain Python equivalent, but others need to be more integrated into
   PyPy's internals.

 * the bytecode compiler: we don't have any compiler integrated with PyPy 
   so far, but there exists a complete set of Python code out there 
   (tokenizer, parser, st->ast, ast->bytecode) that could be used.  We
   could also discuss ideas like syntax extensions generating new
   bytecodes, etc.

 * tracing and other wrappers: e.g. how to write a fully debugging Object 
   Space that records the history of all changes to objects, etc.

 * more bits and pieces: small things missing from our interpreter to be fully 
   compliant.  Tracing/profiling hooks come to mind; more small things 
   are listed in http://codespeak.net/svn/pypy/trunk/src/pypy/TODO .

In addition to the above examples, there are a number of more involved tasks
that nevertheless don't require a complete grasp of PyPy -- whose parts are
relatively independent from each other.  The "hard" work currently going on in
PyPy is on the translation part, which is needed to make PyPy run faster
and/or in other environments.  This work is however mostly independent.  For
people that prefer to focus on cross-language stuff rather than Python
internals we can discuss about tasks like writing the basics of an interpreter
for another language -- for example, it would be great to have a decent
error-checking Javascript interpreter, but we can discuss any other language,
Python-like or not.  In parallel, there is always pending work to allow the
code generator to target other languages and platforms (currently there is C,
Lisp, Pyrex and Java) or generate code differently (Stackless-style,
Psyco-style).


A bientot,

Armin Rigo
From Scott.Daniels at Acm.Org  Sun Dec 26 01:37:40 2004
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Sun Dec 26 01:36:19 2004
Subject: [Python-Dev] Re: proto-pep: How to change Python's bytecode
In-Reply-To: <41CDC131.70702@ocf.berkeley.edu>
References: <41CC7F67.9070009@ocf.berkeley.edu> <cqih8n$4du$1@sea.gmane.org>
	<41CDC131.70702@ocf.berkeley.edu>
Message-ID: <cql11r$ot$1@sea.gmane.org>

Brett C. wrote:
> Terry Reedy wrote:
>> "Brett C." <bac@ocf.berkeley.edu> wrote....
>> Over the last several years, various people have reported 
>> experimenting with CPython's bytecodes.  I wonder if it would be 
>> helpful to have a respository of the results, in one place, for new 
>> experimenters and curious people to peruse.
> 
> Wouldn't hurt.  Adding that section would not be bad, but I don't have 
> the inclination to hunt them down.  What do others think about having 
> this section?
What you could do is declare a wiki page which anyone could go to; less
work for you, and forms a new place to pool knowledge.

-- Scott David Daniels
Scott.Daniels@Acm.Org

From skip at pobox.com  Sun Dec 26 01:43:31 2004
From: skip at pobox.com (Skip Montanaro)
Date: Sun Dec 26 01:43:48 2004
Subject: [Python-Dev] Re: proto-pep: How to change Python's bytecode
In-Reply-To: <41CDC131.70702@ocf.berkeley.edu>
References: <41CC7F67.9070009@ocf.berkeley.edu> <cqih8n$4du$1@sea.gmane.org>
	<41CDC131.70702@ocf.berkeley.edu>
Message-ID: <16846.2355.592620.198686@montanaro.dyndns.org>


    >> Over the last several years, various people have reported
    >> experimenting with CPython's bytecodes.  I wonder if it would be
    >> helpful to have a respository of the results, in one place, for new
    >> experimenters and curious people to peruse.

    Brett> Wouldn't hurt.  Adding that section would not be bad, but I don't
    Brett> have the inclination to hunt them down.  What do others think
    Brett> about having this section?

How about just references in the PEP?  I presented a paper several years ago
at a Python workshop on a peephole optimizer for Python.  Also, Michael
Hudson has his no-longer-active bytecodehacks stuff:

    http://www.foretec.com/python/workshops/1998-11/proceedings/papers/montanaro/montanaro.html
    http://bytecodehacks.sourceforge.net/bch-docs/bch/index.html

I imagine there's other stuff as well.

Skip
From Jack.Jansen at cwi.nl  Mon Dec 27 15:15:26 2004
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Mon Dec 27 15:15:32 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src setup.py, 1.208,
	1.209
In-Reply-To: <E1CflUs-0005gG-6Q@sc8-pr-cvs1.sourceforge.net>
References: <E1CflUs-0005gG-6Q@sc8-pr-cvs1.sourceforge.net>
Message-ID: <C16E5FE9-5811-11D9-81BB-000D934FF6B4@cwi.nl>


On 18-dec-04, at 21:48, bcannon@users.sourceforge.net wrote:

> Update of /cvsroot/python/python/dist/src
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21493
>
> Modified Files:
> 	setup.py
> Log Message:
> Switch from getting LDFLAGS and CPPFLAGS from the environment to the 
> Makefile.
> This is to avoid a problem that inconsistently comes up where the 
> environment
> variable is unset while the Makefile clearly has the values set and 
> are used
> during ``make``.

Brett,
I'm building MacPython binary distributions with an LDFLAGS configure 
option, as in
	$PYTHONSRC/configure -C --enable-framework LDFLAGS=-Wl,-x

This has suddenly started failing with a very mysterious error message. 
When the
make tries to build the extension modules I get
	running build
	running build_ext
	usage: setup.py [options]
	setup.py: error: no such option: -W

Could this somehow be caused by your fix?
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman

From eric.nieuwland at xs4all.nl  Mon Dec 27 22:30:32 2004
From: eric.nieuwland at xs4all.nl (Eric Nieuwland)
Date: Mon Dec 27 22:30:36 2004
Subject: [Python-Dev] Re: [Pythonmac-SIG] The versioning question...
In-Reply-To: <41D0542E.9000404@noaa.gov>
References: <a05200f02bdec9add0045@[81.107.210.110]>	<8128F38C-686F-45F7-90C4-CB181B992AD1@mac.com>	<a05200f01bdf0b1aae35e@[81.107.210.110]>	<C804CB62-5522-11D9-B983-000A9567635C@redivi.com>	<a05200f01bdf486f9ef33@[82.3.232.103]>
	<4520D194-5773-11D9-97BB-000A9567635C@redivi.com>
	<41D0542E.9000404@noaa.gov>
Message-ID: <897942DB-584E-11D9-8852-000393894CEA@xs4all.nl>

Hi all,

On 27 dec 2004, at 19:27, Chris Barker wrote:
> Robin has added versioning to the latest wxPython, and I. for one am 
> ecstatic. It works great for me. I am generally using 2.5.3, but have 
> 2.4.2 installed, and a number of my apps depend  on it (on Linux 
> anyway,  it's pretty useless on OS-X)
>
> It's great to be able to put:
>
> import wxversion
> wxversion.select("2.4")
>
> At the top of my apps, and know that they'll use the version of 
> wxPython I tested against.

Would it be an idea to submit a PEP for extending the 'import' keyword?

I can imagine it to be like:

import spam version 1
import foo version 2, 4
import bar version 7, 1, 6

with version numbers consisting of some fixed maximum of parts and the 
version number parts not given might be wildcards.

We could then have a per module version system like:
.../
	spam/
		1.0.1/
			...			[version 1.0.1 stuff in here]
		1.5.7/
			...			[version 1.5.7 stuff in here]

there should be nothing else in the module directory, except for a 
mechanism to further support versioning.
Default behaviour of 'import spam' would be to select the most recent 
version. We might want to override that by pointing to some other 
version. This might be achieved using a symbolic link/alias/shortcut to 
the directory of that version (spam/current -> spam/1.0.1).
More like the directory/module method would be to allow for a 
__version__.py file in 'spam'. Content of this file is to be 
determined, but I think it should at least something like __current__ 
or __version__ to act as the pointer to the current version.

--eric

From martin at v.loewis.de  Tue Dec 28 00:34:54 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Dec 28 00:34:57 2004
Subject: [Python-Dev] Re: [Pythonmac-SIG] The versioning question...
In-Reply-To: <897942DB-584E-11D9-8852-000393894CEA@xs4all.nl>
References: <a05200f02bdec9add0045@[81.107.210.110]>	<8128F38C-686F-45F7-90C4-CB181B992AD1@mac.com>	<a05200f01bdf0b1aae35e@[81.107.210.110]>	<C804CB62-5522-11D9-B983-000A9567635C@redivi.com>	<a05200f01bdf486f9ef33@[82.3.232.103]>	<4520D194-5773-11D9-97BB-000A9567635C@redivi.com>	<41D0542E.9000404@noaa.gov>
	<897942DB-584E-11D9-8852-000393894CEA@xs4all.nl>
Message-ID: <41D09C1E.1060009@v.loewis.de>

Eric Nieuwland wrote:
> Would it be an idea to submit a PEP for extending the 'import' keyword?

No. Normally, packages should aim for backwards compatibility, so that
applications would only want to specify a minimum version, such as

import xml
assert xml.version_info > (0,8,2)

If you really want side-by-side installation of different versions,
and a mechanism to select between them, the package could support

import xml_0_8_2 as xml

IOW, "import-as" should be sufficient for what you want to achieve.

Regards,
Martin
From Scott.Daniels at Acm.Org  Tue Dec 28 01:18:46 2004
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Tue Dec 28 01:17:22 2004
Subject: [Python-Dev] Zipfile needs?
Message-ID: <cqq8mc$3g9$1@sea.gmane.org>

I'm hoping to add BZIP2 compression to zipfile for 2.5.  My primary
motivation is that Project Gutenberg seems to be starting to use BZIP2
compression for some of its zips.  What other wish list things do
people around here have for zipfile?  I thought I'd collect input here
and make a PEP.

I can open a pseudo-file for STORED files in binary read mode, for
example, to allow reading zip-in-zip files without fully occupying
memory.

-- Scott David Daniels
Scott.Daniels@Acm.Org

From jcarlson at uci.edu  Tue Dec 28 02:43:15 2004
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue Dec 28 03:11:59 2004
Subject: [Python-Dev] Zipfile needs?
In-Reply-To: <cqq8mc$3g9$1@sea.gmane.org>
References: <cqq8mc$3g9$1@sea.gmane.org>
Message-ID: <20041227174248.9252.JCARLSON@uci.edu>


Scott David Daniels <Scott.Daniels@Acm.Org> wrote:
> 
> I'm hoping to add BZIP2 compression to zipfile for 2.5.  My primary
> motivation is that Project Gutenberg seems to be starting to use BZIP2
> compression for some of its zips.  What other wish list things do
> people around here have for zipfile?  I thought I'd collect input here
> and make a PEP.
> 
> I can open a pseudo-file for STORED files in binary read mode, for
> example, to allow reading zip-in-zip files without fully occupying
> memory.

I'm not sure that zipfile needs BZIP2 support...being that there is a
bzip2 module.

 - Josiah

From bob at redivi.com  Tue Dec 28 03:37:42 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Dec 28 03:37:53 2004
Subject: [Python-Dev] Re: [Pythonmac-SIG] The versioning question...
In-Reply-To: <41D0BAB4.70903@noaa.gov>
References: <a05200f02bdec9add0045@[81.107.210.110]>	<8128F38C-686F-45F7-90C4-CB181B992AD1@mac.com>	<a05200f01bdf0b1aae35e@[81.107.210.110]>	<C804CB62-5522-11D9-B983-000A9567635C@redivi.com>	<a05200f01bdf486f9ef33@[82.3.232.103]>	<4520D194-5773-11D9-97BB-000A9567635C@redivi.com>	<41D0542E.9000404@noaa.gov>
	<897942DB-584E-11D9-8852-000393894CEA@xs4all.nl>
	<41D09C1E.1060009@v.loewis.de> <41D0BAB4.70903@noaa.gov>
Message-ID: <729AC65F-5879-11D9-AA13-000A9567635C@redivi.com>


On Dec 27, 2004, at 8:45 PM, Chris Barker wrote:

> The versioning system that wxPython now has is quite nice, and seems 
> to fit most people's needs well. However, it's also quite new, and who 
> know what problems will arise. For those interested, here's a 
> synopsis.
>
> http://wiki.wxpython.org/index.cgi/MultiVersionInstalls

I just cleaned up a bunch of spelling on that wiki page and rewrote the 
"what about py2exe" section.  It now mentions py2app, is (hopefully) 
written more clearly, and includes an easier alternative for bundling 
the desired version of wxPython (wxversion can be used from setup.py).

-bob

From bob at redivi.com  Tue Dec 28 03:41:23 2004
From: bob at redivi.com (Bob Ippolito)
Date: Tue Dec 28 03:41:29 2004
Subject: [Python-Dev] Zipfile needs?
In-Reply-To: <20041227174248.9252.JCARLSON@uci.edu>
References: <cqq8mc$3g9$1@sea.gmane.org> <20041227174248.9252.JCARLSON@uci.edu>
Message-ID: <F6A8BE19-5879-11D9-AA13-000A9567635C@redivi.com>


On Dec 27, 2004, at 8:43 PM, Josiah Carlson wrote:

> Scott David Daniels <Scott.Daniels@Acm.Org> wrote:
>>
>> I'm hoping to add BZIP2 compression to zipfile for 2.5.  My primary
>> motivation is that Project Gutenberg seems to be starting to use BZIP2
>> compression for some of its zips.  What other wish list things do
>> people around here have for zipfile?  I thought I'd collect input here
>> and make a PEP.
>>
>> I can open a pseudo-file for STORED files in binary read mode, for
>> example, to allow reading zip-in-zip files without fully occupying
>> memory.
>
> I'm not sure that zipfile needs BZIP2 support...being that there is a
> bzip2 module.

Note that the bzip2 module is named bz2 and does provide a 
file-like-interface.  Also, it is implemented entirely as a C extension 
(ick).

-bob

From skip at pobox.com  Tue Dec 28 05:26:51 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Dec 28 05:27:16 2004
Subject: [Python-Dev] Re: [Pythonmac-SIG] The versioning question...
In-Reply-To: <41D09C1E.1060009@v.loewis.de>
References: <a05200f02bdec9add0045@[81.107.210.110]>
	<8128F38C-686F-45F7-90C4-CB181B992AD1@mac.com>
	<a05200f01bdf0b1aae35e@[81.107.210.110]>
	<C804CB62-5522-11D9-B983-000A9567635C@redivi.com>
	<a05200f01bdf486f9ef33@[82.3.232.103]>
	<4520D194-5773-11D9-97BB-000A9567635C@redivi.com>
	<41D0542E.9000404@noaa.gov>
	<897942DB-584E-11D9-8852-000393894CEA@xs4all.nl>
	<41D09C1E.1060009@v.loewis.de>
Message-ID: <16848.57483.204434.666107@montanaro.dyndns.org>


    Martin> If you really want side-by-side installation of different
    Martin> versions, and a mechanism to select between them, the package
    Martin> could support

    Martin> import xml_0_8_2 as xml

    Martin> IOW, "import-as" should be sufficient for what you want to achieve.

That's more-or-less the scheme adopted where I work.  If we have two
versions of a sybase module installed side-by-side, the imports might look
like:

    import local.db.sybase.v1 as sybase

or

    import local.db.sybase.v2 as sybase

It's a bit cumbersome, but it's worked okay for us so far.

Skip
From Chris.Barker at noaa.gov  Tue Dec 28 02:45:24 2004
From: Chris.Barker at noaa.gov (Chris Barker)
Date: Tue Dec 28 05:27:35 2004
Subject: [Python-Dev] Re: [Pythonmac-SIG] The versioning question...
In-Reply-To: <41D09C1E.1060009@v.loewis.de>
References: <a05200f02bdec9add0045@[81.107.210.110]>	<8128F38C-686F-45F7-90C4-CB181B992AD1@mac.com>	<a05200f01bdf0b1aae35e@[81.107.210.110]>	<C804CB62-5522-11D9-B983-000A9567635C@redivi.com>	<a05200f01bdf486f9ef33@[82.3.232.103]>	<4520D194-5773-11D9-97BB-000A9567635C@redivi.com>	<41D0542E.9000404@noaa.gov>
	<897942DB-584E-11D9-8852-000393894CEA@xs4all.nl>
	<41D09C1E.1060009@v.loewis.de>
Message-ID: <41D0BAB4.70903@noaa.gov>

Martin v. L?wis wrote:
> No. Normally, packages should aim for backwards compatibility, so that
> applications would only want to specify a minimum version, such as
> 
> import xml
> assert xml.version_info > (0,8,2)

Well, yes, but life is not always so simple, and while, as a rule, 
version 2.3 should be backward compatible with 2.2, I see no reason to 
expect that version 5.0 and version 2.0 will be compatible. Frankly, 
backward compatibility can be a real impediment to progress. (when will 
"true division" be default?)

> If you really want side-by-side installation of different versions,
> and a mechanism to select between them, the package could support
> 
> import xml_0_8_2 as xml
> 
> IOW, "import-as" should be sufficient for what you want to achieve.

This really doesn't work well for complex packages. I was quite involved 
with the debate about versioning for wxPython (and helped drive it 
happening) and that was what I originally proposed. The problem is that 
you have a whole pile of modules and libs and user code that all imports 
the package. There are a LOT of "import wx" lines in the wxPython 
library, and a whole bunch more in a sizable wxPython app. As you 
upgrade, every one of those would have to be changed to

import wx_x_y_z as wx

This was not considered a reasonable solution. Among other things, it's 
really nice to be able to make a small change in just one file, and be 
able to test your app against a new version.

Other approaches were suggested and were used in the past:

- A script that you run to change what version is installed
- An environment variable that you set.
- Whatever else I can't recall.

Personally, I was only going to be really happy with an approach that 
worked at the python level, in one place.

The versioning system that wxPython now has is quite nice, and seems to 
fit most people's needs well. However, it's also quite new, and who know 
what problems will arise. For those interested, here's a synopsis.

http://wiki.wxpython.org/index.cgi/MultiVersionInstalls

Skip Montanaro wrote:
> There's more to it than that:
>     1. Most people simply don't need it.  My first brush with versioning
>        Python modules occurred in the past few months at my current job, and
>        that was mostly so our SWIG'd modules could be released in step with
>        our locally written C++ libraries.  I used Python for ten years
>        before that without ever feeling the need for versioning.

I'm surprised, but I think maturity and complexity has something to do 
with it. The more mature Python and its add-on packages become, the more 
this will be an issue. There was certainly a lot of uproar about 
backward compatible changes in Python itself, another issue that could 
be lessoned by a bit easier accommodation of multiple versions of 
python. (or why, or why did Redhat not put a version on their #! lines? 
  have they yet?)

>     2. I think it will be challenging to come up with a versioning scheme
>        that works for everyone.

This is very true we had a hard enough time coming to a consensus among 
a small group of wxPython developers. I'm not sure we even did anyway, 
Robin just decided on one to implement.

 > To do versioning at work we had to solve
>        issues related to module naming, directory structure, source code
>        control and local build environment tools to make it work.

Wow! that sounds a lot more complex that it has to be, but I'm sure 
there's a lot to what you've done. Note that you've kind of contradicted 
yourself (or at least contradicted Martin). I suspect your versioning 
issues would have been much simpler if Python itself had supported it.

> Write a PEP and put it out for review.  I don't recall seeing this raised in
> the Python community before.

That I'm surprised about. I've seen it (and brought it up) a number of 
times. I know there are a few major packages with roll your own 
versioning systems, wxPython, PyGTK, and PMW to mention a few.

 > I certainly don't think it's something the
> core developers worry about,

This is quite true, and why I haven't bothered with a PEP, but maybe 
I've got the thinking backwards, we need the PEP to get the "important" 
people thinking about it.

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@noaa.gov
From Scott.Daniels at Acm.Org  Tue Dec 28 06:36:07 2004
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Tue Dec 28 06:34:43 2004
Subject: [Python-Dev] Re: Zipfile needs?
In-Reply-To: <20041227174248.9252.JCARLSON@uci.edu>
References: <cqq8mc$3g9$1@sea.gmane.org> <20041227174248.9252.JCARLSON@uci.edu>
Message-ID: <cqqr9d$vrk$1@sea.gmane.org>

Josiah Carlson wrote:
> Scott David Daniels <Scott.Daniels@Acm.Org> wrote:
> 
>>I'm hoping to add BZIP2 compression to zipfile for 2.5.  My primary
>>motivation is that Project Gutenberg seems to be starting to use BZIP2
>>compression for some of its zips.  What other wish list things do
>>people around here have for zipfile?  I thought I'd collect input here
>>and make a PEP.
>>
>>I can open a pseudo-file for STORED files in binary read mode, for
>>example, to allow reading zip-in-zip files without fully occupying
>>memory.
> 
> 
> I'm not sure that zipfile needs BZIP2 support...being that there is a
> bzip2 module.
> 
>  - Josiah
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
> 
But if you look at the zipfile document, BZIP2 is a compression
technique you can use (per file) in a zip archive.  In fact, I
use bz2 to compress/decompress, but the data still needs to inhabit
the archive.

-- 
-- Scott David Daniels
Scott.Daniels@Acm.Org

From bac at ocf.berkeley.edu  Tue Dec 28 08:03:44 2004
From: bac at ocf.berkeley.edu (Brett C.)
Date: Tue Dec 28 08:03:40 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src setup.py,
	1.208,	1.209
In-Reply-To: <C16E5FE9-5811-11D9-81BB-000D934FF6B4@cwi.nl>
References: <E1CflUs-0005gG-6Q@sc8-pr-cvs1.sourceforge.net>
	<C16E5FE9-5811-11D9-81BB-000D934FF6B4@cwi.nl>
Message-ID: <41D10550.5000105@ocf.berkeley.edu>

Jack Jansen wrote:
> 
> On 18-dec-04, at 21:48, bcannon@users.sourceforge.net wrote:
> 
>> Update of /cvsroot/python/python/dist/src
>> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21493
>>
>> Modified Files:
>>     setup.py
>> Log Message:
>> Switch from getting LDFLAGS and CPPFLAGS from the environment to the 
>> Makefile.
>> This is to avoid a problem that inconsistently comes up where the 
>> environment
>> variable is unset while the Makefile clearly has the values set and 
>> are used
>> during ``make``.
> 
> 
> Brett,
> I'm building MacPython binary distributions with an LDFLAGS configure 
> option, as in
>     $PYTHONSRC/configure -C --enable-framework LDFLAGS=-Wl,-x
> 
> This has suddenly started failing with a very mysterious error message. 
> When the
> make tries to build the extension modules I get
>     running build
>     running build_ext
>     usage: setup.py [options]
>     setup.py: error: no such option: -W
> 
> Could this somehow be caused by your fix?

Yep, it's my doing.  I bet optparse is raising an error since it doesn't have 
options for -W1 or -x registered with the parser.  Anyone off the top of their 
head know how to cause optparse to just skip over options it doesn't know about?

I am on vacation at the moment so I will deal with this when I get back this 
weekend.

-Brett
From eric.nieuwland at xs4all.nl  Tue Dec 28 09:22:08 2004
From: eric.nieuwland at xs4all.nl (Eric Nieuwland)
Date: Tue Dec 28 09:22:11 2004
Subject: [Python-Dev] Re: [Pythonmac-SIG] The versioning question...
In-Reply-To: <41D09C1E.1060009@v.loewis.de>
References: <a05200f02bdec9add0045@[81.107.210.110]>	<8128F38C-686F-45F7-90C4-CB181B992AD1@mac.com>	<a05200f01bdf0b1aae35e@[81.107.210.110]>	<C804CB62-5522-11D9-B983-000A9567635C@redivi.com>	<a05200f01bdf486f9ef33@[82.3.232.103]>	<4520D194-5773-11D9-97BB-000A9567635C@redivi.com>	<41D0542E.9000404@noaa.gov>
	<897942DB-584E-11D9-8852-000393894CEA@xs4all.nl>
	<41D09C1E.1060009@v.loewis.de>
Message-ID: <90D320DB-58A9-11D9-8852-000393894CEA@xs4all.nl>

Martin v. L?wis wrote:
> Eric Nieuwland wrote:
>> Would it be an idea to submit a PEP for extending the 'import' 
>> keyword?
>
> No. Normally, packages should aim for backwards compatibility, so that
> applications would only want to specify a minimum version, such as
>
> import xml
> assert xml.version_info > (0,8,2)
>
> If you really want side-by-side installation of different versions,
> and a mechanism to select between them, the package could support
>
> import xml_0_8_2 as xml
>
> IOW, "import-as" should be sufficient for what you want to achieve.

Unless you are doing comparison tests, where it would be nice to be 
able to state in a generic way that the new implementation should not 
change answers. May be something like:

import spam[1] as spamnext		# next version
import spam[0]	as spamnow		# current version

assert spamnow.Ni() == spamnext.Ni()

--eric

From skip at pobox.com  Tue Dec 28 14:23:59 2004
From: skip at pobox.com (Skip Montanaro)
Date: Tue Dec 28 14:24:04 2004
Subject: [Python-Dev] Re: [Pythonmac-SIG] The versioning question...
In-Reply-To: <90D320DB-58A9-11D9-8852-000393894CEA@xs4all.nl>
References: <a05200f02bdec9add0045@[81.107.210.110]>
	<8128F38C-686F-45F7-90C4-CB181B992AD1@mac.com>
	<a05200f01bdf0b1aae35e@[81.107.210.110]>
	<C804CB62-5522-11D9-B983-000A9567635C@redivi.com>
	<a05200f01bdf486f9ef33@[82.3.232.103]>
	<4520D194-5773-11D9-97BB-000A9567635C@redivi.com>
	<41D0542E.9000404@noaa.gov>
	<897942DB-584E-11D9-8852-000393894CEA@xs4all.nl>
	<41D09C1E.1060009@v.loewis.de>
	<90D320DB-58A9-11D9-8852-000393894CEA@xs4all.nl>
Message-ID: <16849.24175.124239.931764@montanaro.dyndns.org>


    Eric> Unless you are doing comparison tests, where it would be nice to
    Eric> be able to state in a generic way that the new implementation
    Eric> should not change answers. May be something like:

    Eric> import spam[1] as spamnext            # next version
    Eric> import spam[0]        as spamnow              # current version

    Eric> assert spamnow.Ni() == spamnext.Ni()

>From the Zen of Python I quote:

    Namespaces are one honking great idea -- let's do more of those!

import spam.v1 as spamnext
import spam.v0 as spamnow

assert spamnow.Ni() == spamnext.Ni()


Skip
From martin at v.loewis.de  Tue Dec 28 14:30:46 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Dec 28 14:30:49 2004
Subject: [Python-Dev] Re: [Pythonmac-SIG] The versioning question...
In-Reply-To: <41D0BAB4.70903@noaa.gov>
References: <a05200f02bdec9add0045@[81.107.210.110]>	<8128F38C-686F-45F7-90C4-CB181B992AD1@mac.com>	<a05200f01bdf0b1aae35e@[81.107.210.110]>	<C804CB62-5522-11D9-B983-000A9567635C@redivi.com>	<a05200f01bdf486f9ef33@[82.3.232.103]>	<4520D194-5773-11D9-97BB-000A9567635C@redivi.com>	<41D0542E.9000404@noaa.gov>	<897942DB-584E-11D9-8852-000393894CEA@xs4all.nl>	<41D09C1E.1060009@v.loewis.de>
	<41D0BAB4.70903@noaa.gov>
Message-ID: <41D16006.8020103@v.loewis.de>

Chris Barker wrote:
> This really doesn't work well for complex packages. I was quite involved 
> with the debate about versioning for wxPython (and helped drive it 
> happening) and that was what I originally proposed. The problem is that 
> you have a whole pile of modules and libs and user code that all imports 
> the package. There are a LOT of "import wx" lines in the wxPython 
> library, and a whole bunch more in a sizable wxPython app. As you 
> upgrade, every one of those would have to be changed to
> 
> import wx_x_y_z as wx

Not at all. Import wx_x_y_z could do

sys.modules["wx"] = wx_x_y_z

and then all references to

import wx

would automatically resolve to the version that was imported first
(or last, depending on your implementation strategy).

> This was not considered a reasonable solution. Among other things, it's 
> really nice to be able to make a small change in just one file, and be 
> able to test your app against a new version.

Even if the package is not prepared to register itself under a different
name also, this one file could modify sys.modules. Or you can have a 
wx.py that reads

from wx_x_y_z import *

Then, wx.py would be that single file.

> The versioning system that wxPython now has is quite nice, and seems to 
> fit most people's needs well. 

Very good! Then I don't see a need to change anything in Python.

> This is quite true, and why I haven't bothered with a PEP, but maybe 
> I've got the thinking backwards, we need the PEP to get the "important" 
> people thinking about it.

I don't think a PEP is needed - Python already appears to have
everything you need to come up with your own versioning rules.

Regards,
Martin
From martin at v.loewis.de  Tue Dec 28 14:41:52 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue Dec 28 14:41:55 2004
Subject: [Python-Dev] Zipfile needs?
In-Reply-To: <cqq8mc$3g9$1@sea.gmane.org>
References: <cqq8mc$3g9$1@sea.gmane.org>
Message-ID: <41D162A0.9000502@v.loewis.de>

Scott David Daniels wrote:
> I'm hoping to add BZIP2 compression to zipfile for 2.5.  My primary
> motivation is that Project Gutenberg seems to be starting to use BZIP2
> compression for some of its zips.  What other wish list things do
> people around here have for zipfile?  I thought I'd collect input here
> and make a PEP.

AFAIR, compression mechanisms are defined by numbers in the zip file.
So you should not bother with such a change unless there is some
"official" specification that explains how bzip2 is used in zipfiles.

IOW, looking at

http://www.pkware.com/company/standards/appnote/

you'll see that PKWARE has assigned algorithm 12 for bzip2. You might
want to take a look at the spec to see what else the Python
implementation lacks, and either document these features as deliberately
missing, TODO, or just implement them right away.

Regards,
Martin
From bac at ocf.berkeley.edu  Tue Dec 28 20:15:18 2004
From: bac at ocf.berkeley.edu (Brett C.)
Date: Tue Dec 28 20:15:19 2004
Subject: [Python-Dev] Zipfile needs?
In-Reply-To: <cqq8mc$3g9$1@sea.gmane.org>
References: <cqq8mc$3g9$1@sea.gmane.org>
Message-ID: <41D1B0C6.8040208@ocf.berkeley.edu>

Scott David Daniels wrote:
> I'm hoping to add BZIP2 compression to zipfile for 2.5.  My primary
> motivation is that Project Gutenberg seems to be starting to use BZIP2
> compression for some of its zips.  What other wish list things do
> people around here have for zipfile?  I thought I'd collect input here
> and make a PEP.
> 

Encryption/decryption support.  Will most likely require a C extension since 
the algorithm relies on ints (or longs, don't remember) wrapping around when 
the value becomes too large.

-Brett
From bob at redivi.com  Wed Dec 29 04:35:15 2004
From: bob at redivi.com (Bob Ippolito)
Date: Wed Dec 29 04:35:21 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Mac/OSX
	fixapplepython23.py, 1.1, 1.2
In-Reply-To: <E1CjOvO-0002UH-EH@sc8-pr-cvs1.sourceforge.net>
References: <E1CjOvO-0002UH-EH@sc8-pr-cvs1.sourceforge.net>
Message-ID: <A79A1AF6-594A-11D9-A28B-000A9567635C@redivi.com>

On Dec 28, 2004, at 4:30 PM, jackjansen@users.sourceforge.net wrote:

> Update of /cvsroot/python/python/dist/src/Mac/OSX
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9229
>
> Modified Files:
> 	fixapplepython23.py
> Log Message:
> Just passing -undefined dynamic_lookup isn't enough: we also need to 
> set
> the MACOSX_DEPLOYMENT_TARGET environment variable to 10.3 when calling 
> the
> loader. And we do this with "env" because distutils apparently doesn't
> understand environment variable assignments before command names.

This is the wrong fix.  Distutils is dumber than you think.  This patch 
just breaks C++ compilation in a different way.  The correct solution 
is a patch to distutils of some kind.

from distutils/unixccompiler.py:174

                 if target_lang == "c++" and self.compiler_cxx:
                     linker[0] = self.compiler_cxx[0]
                 self.spawn(linker + ld_args)

"linker" is the variable expanded LDSHARED (or whatever comes out of 
sysconfig.customize_compiler).

-bob

From Jack.Jansen at cwi.nl  Wed Dec 29 11:23:52 2004
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Wed Dec 29 11:23:51 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Mac/OSX
	fixapplepython23.py, 1.1, 1.2
In-Reply-To: <A79A1AF6-594A-11D9-A28B-000A9567635C@redivi.com>
References: <E1CjOvO-0002UH-EH@sc8-pr-cvs1.sourceforge.net>
	<A79A1AF6-594A-11D9-A28B-000A9567635C@redivi.com>
Message-ID: <BCDC0AB4-5983-11D9-81BB-000D934FF6B4@cwi.nl>


On 29-dec-04, at 4:35, Bob Ippolito wrote:

> This is the wrong fix.  Distutils is dumber than you think.  This 
> patch just breaks C++ compilation in a different way.  The correct 
> solution is a patch to distutils of some kind.
>
> from distutils/unixccompiler.py:174
>
>                 if target_lang == "c++" and self.compiler_cxx:
>                     linker[0] = self.compiler_cxx[0]
>                 self.spawn(linker + ld_args)
>
> "linker" is the variable expanded LDSHARED (or whatever comes out of 
> sysconfig.customize_compiler).

Bah!

Any suggestions as to what to do to get c++ compilation fixed?

Also, could you point me to a readily available extension package that 
uses c++?
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman

From bob at redivi.com  Wed Dec 29 11:40:30 2004
From: bob at redivi.com (Bob Ippolito)
Date: Wed Dec 29 11:40:38 2004
Subject: [Pythonmac-SIG] Re: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Mac/OSX fixapplepython23.py, 1.1, 1.2
In-Reply-To: <BCDC0AB4-5983-11D9-81BB-000D934FF6B4@cwi.nl>
References: <E1CjOvO-0002UH-EH@sc8-pr-cvs1.sourceforge.net>
	<A79A1AF6-594A-11D9-A28B-000A9567635C@redivi.com>
	<BCDC0AB4-5983-11D9-81BB-000D934FF6B4@cwi.nl>
Message-ID: <0F7FB55E-5986-11D9-9660-000A9567635C@redivi.com>

On Dec 29, 2004, at 5:23 AM, Jack Jansen wrote:

> On 29-dec-04, at 4:35, Bob Ippolito wrote:
>
>> This is the wrong fix.  Distutils is dumber than you think.  This 
>> patch just breaks C++ compilation in a different way.  The correct 
>> solution is a patch to distutils of some kind.
>>
>> from distutils/unixccompiler.py:174
>>
>>                 if target_lang == "c++" and self.compiler_cxx:
>>                     linker[0] = self.compiler_cxx[0]
>>                 self.spawn(linker + ld_args)
>>
>> "linker" is the variable expanded LDSHARED (or whatever comes out of 
>> sysconfig.customize_compiler).
>
> Bah!
>
> Any suggestions as to what to do to get c++ compilation fixed?

I can think of two ways:

1. Patch distutils to actually do 
os.environ['MACOSX_DEPLOYMENT_TARGET'] = '10.3' because CCompiler.spawn 
doesn't take an environment dict.
2. Patch distutils to skip over environment variables (basically change 
that "0" into something smarter).   This is probably less desirable 
because who knows where this happens, and who knows what third party 
compiler subclasses used this original stupid code as a template.

In either case, setting the environment variable should only be done if 
it's not already set, and it should raise if is set to anything lower 
than 10.3.  For example, an existing MACOSX_DEPLOYMENT_TARGET variable 
of 10.4 should be allowed.  Values smaller than 10.3 should not be 
allowed because they are incompatible with the linker feature we're 
trying to enable.

> Also, could you point me to a readily available extension package that 
> uses c++?

I have no idea, but I remember this issue was brought up at some point 
during one of the many discussions of this issue.  I think it was also 
mentioned that SciPy's distutils extensions (or is it a fork?) does 
something similarly stupid when frobbing in a Fortran compiler.

-bob

From mdehoon at ims.u-tokyo.ac.jp  Wed Dec 29 13:13:24 2004
From: mdehoon at ims.u-tokyo.ac.jp (Michiel Jan Laurens de Hoon)
Date: Wed Dec 29 13:10:17 2004
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Mac/OSX	fixapplepython23.py, 1.1, 1.2
In-Reply-To: <BCDC0AB4-5983-11D9-81BB-000D934FF6B4@cwi.nl>
References: <E1CjOvO-0002UH-EH@sc8-pr-cvs1.sourceforge.net>	<A79A1AF6-594A-11D9-A28B-000A9567635C@redivi.com>
	<BCDC0AB4-5983-11D9-81BB-000D934FF6B4@cwi.nl>
Message-ID: <41D29F64.5040308@ims.u-tokyo.ac.jp>

Jack Jansen wrote:
> Also, could you point me to a readily available extension package that 
> uses c++?

The Bio.Affy and Bio.KDTree extensions in Biopython use c++. See www.biopython.org.

--Michiel.

-- 
Michiel de Hoon, Assistant Professor
University of Tokyo, Institute of Medical Science
Human Genome Center
4-6-1 Shirokane-dai, Minato-ku
Tokyo 108-8639
Japan
http://bonsai.ims.u-tokyo.ac.jp/~mdehoon
From rkern at ucsd.edu  Thu Dec 30 02:58:22 2004
From: rkern at ucsd.edu (Robert Kern)
Date: Thu Dec 30 02:58:36 2004
Subject: [Python-Dev] Re: [Pythonmac-SIG] Re: Re: [Python-checkins]
 python/dist/src/Mac/OSX fixapplepython23.py, 1.1, 1.2
In-Reply-To: <0F7FB55E-5986-11D9-9660-000A9567635C@redivi.com>
References: <E1CjOvO-0002UH-EH@sc8-pr-cvs1.sourceforge.net>	<A79A1AF6-594A-11D9-A28B-000A9567635C@redivi.com>	<BCDC0AB4-5983-11D9-81BB-000D934FF6B4@cwi.nl>
	<0F7FB55E-5986-11D9-9660-000A9567635C@redivi.com>
Message-ID: <cqvnbt$f3r$1@sea.gmane.org>

Bob Ippolito wrote:

> I have no idea, but I remember this issue was brought up at some point 
> during one of the many discussions of this issue.  I think it was also 
> mentioned that SciPy's distutils extensions (or is it a fork?) does 
> something similarly stupid when frobbing in a Fortran compiler.

Extension, and yes it does something similarly stupid.

-- 
Robert Kern
rkern@ucsd.edu

"In the fields of hell where the grass grows high
  Are the graves of dreams allowed to die."
   -- Richard Harter

From martin at v.loewis.de  Thu Dec 30 11:46:38 2004
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Dec 30 11:46:38 2004
Subject: [Python-Dev] Build extensions for windows python 2.4 what are
	the	compiler rules?
In-Reply-To: <20041224123227.GA12925@vicky.ecs.soton.ac.uk>
References: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org>
	<41CB50EA.9070100@v.loewis.de>
	<3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org>
	<20041224123227.GA12925@vicky.ecs.soton.ac.uk>
Message-ID: <41D3DC8E.8020100@v.loewis.de>

Armin Rigo wrote:
> The "extending and embedding" tutorial is similarily out-of-date.

This is now fixed in the CVS, both for PC/example_nt, and for
Doc/ext/windows.tex. These files still could stand further
correction. I forgot to update the output of the examples to 2.4,
and the .def file stuff should be removed from the examples,
replacing them with the proper macros.

Regards,
Martin
From bac at OCF.Berkeley.EDU  Fri Dec 31 09:14:04 2004
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Fri Dec 31 09:14:22 2004
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src setup.py,
	1.208,	1.209
In-Reply-To: <41D10550.5000105@ocf.berkeley.edu>
References: <E1CflUs-0005gG-6Q@sc8-pr-cvs1.sourceforge.net>
	<C16E5FE9-5811-11D9-81BB-000D934FF6B4@cwi.nl>
	<41D10550.5000105@ocf.berkeley.edu>
Message-ID: <41D50A4C.4070508@ocf.berkeley.edu>

Brett C. wrote:
> Jack Jansen wrote:
> 
>>
>> On 18-dec-04, at 21:48, bcannon@users.sourceforge.net wrote:
>>
>>> Update of /cvsroot/python/python/dist/src
>>> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21493
>>>
>>> Modified Files:
>>>     setup.py
>>> Log Message:
>>> Switch from getting LDFLAGS and CPPFLAGS from the environment to the 
>>> Makefile.
>>> This is to avoid a problem that inconsistently comes up where the 
>>> environment
>>> variable is unset while the Makefile clearly has the values set and 
>>> are used
>>> during ``make``.
>>
>>
>>
>> Brett,
>> I'm building MacPython binary distributions with an LDFLAGS configure 
>> option, as in
>>     $PYTHONSRC/configure -C --enable-framework LDFLAGS=-Wl,-x
>>
>> This has suddenly started failing with a very mysterious error 
>> message. When the
>> make tries to build the extension modules I get
>>     running build
>>     running build_ext
>>     usage: setup.py [options]
>>     setup.py: error: no such option: -W
>>
>> Could this somehow be caused by your fix?
> 
> 
> Yep, it's my doing.  I bet optparse is raising an error since it doesn't 
> have options for -W1 or -x registered with the parser.  Anyone off the 
> top of their head know how to cause optparse to just skip over options 
> it doesn't know about?
> 

OK, rev. 1.210 has my fix.  Now I strip out double dashes (for long options) 
and any dashes preceding any options that setup.py doesn't care about before 
passing the string to optparse.  That way they just look like interspersed 
arguments to optparse and it just ignores them.

Tested it with your LDFLAGS value and it worked fine for me.

> I am on vacation at the moment so I will deal with this when I get back 
> this weekend.
> 

So I lied and got to it sooner.  =)

Happy New Year to everyone.

-Brett
From python at dynkin.com  Fri Dec 31 11:33:10 2004
From: python at dynkin.com (George Yoshida)
Date: Fri Dec 31 11:33:13 2004
Subject: [Python-Dev] Re: python/dist/src/Doc/ext windows.tex,1.9,1.10
In-Reply-To: <E1CjxnH-0004X8-HB@sc8-pr-cvs1.sourceforge.net>
References: <E1CjxnH-0004X8-HB@sc8-pr-cvs1.sourceforge.net>
Message-ID: <41D52AE6.2030107@dynkin.com>

loewis@users.sourceforge.net wrote:
> Update of /cvsroot/python/python/dist/src/Doc/ext
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17396
> Index: windows.tex
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/Doc/ext/windows.tex,v
> retrieving revision 1.9
> retrieving revision 1.10
> diff -u -d -r1.9 -r1.10
> --- windows.tex	23 Jan 2004 09:01:56 -0000	1.9
> +++ windows.tex	30 Dec 2004 10:44:32 -0000	1.10
> @@ -142,62 +142,62 @@
[snip]
> +    Now open the \menuselection{Project \sub spam properties} dialog.
> +    You only need to change a few settings.  Make sure \guilable{All
> +    Configurations} is selected from the \guilable{Settings for:}

Macro names are wrong, which results in a compile error.
\guilable should read \guilabel.


George